Archive for August, 2009

Simpson’s Paradox

Posted in Bad Statistics with tags , , on August 30, 2009 by telescoper

 I haven’t put anything in the Bad Statistics  file for a while, so I thought I’d put this interesting little example up for your perusal.

Although my own field of modern cosmology requires a great deal of complicated statistical reasoning, cosmologists have it relatively easy because there is not much chance that any errors we make will actually end up harming anyone. Speculations about the Anthropic Principle or Theories of Everything are sometimes  reported in the mass media but, if they are, and are garbled, the resulting confusion is unlikely to be fatal. The same can not be said of the field of medical statistics. I can think of scores of examples where poor statistical reasoning has been responsible for shambles in the domain of public health.

Here’s an example of how a relatively simple statistical test can lead to total confusion. In this version, it is known as Simpson’s Paradox.

 A standard thing to do in a medical trial is to take a set of patients suffering from some condition and divide them into two groups. One group is given a treatment (T) and the other group is given a placebo; this latter group is called the control and I will denote it T* (no treatment).

To make things specific suppose we have 100 patients, of whom 50 are actively treated and 50 form the control.  Suppose that at the end of the trial for the treatment, patients can be classified as recovered (“R”) or not recovered (“R*”).  Consider the following outcome, displayed in a contingency table:

 

  R R* Total Recovery
T 20 30 50 40%
T* 16 34 50 32%
Totals 36 64 100  

 

 Clearly the recovery rate for those actively treated (40%) exceeds that for the control group, so the treatment seems at first sight to produce some benefit.

 Now let us divide the group into older and younger patients: the young group Y contains those under 50 years old (carefully defined so that I would belong to it) and Y* is those over 50.

 The following results are obtained for the young patients.

 

  R R* Total Recovery
T 19 21 40 47.5%
T* 5 5 10 50%
Totals 24 26 50  

The older group returns the following data: 

  R R* Total Recovery
T 1 9 10 10%
T* 11 29 40 27.5%
Totals 12 38 50  

 For each of the two groups separately, the recovery rate for the control exceeds that of the treated patients. The placebo works better than the treatment for the young and the old separately, but for the population as a whole the treatment seems to work better than the placebo!

This seems very confusing, and just think how many medical reports in newspapers contain results of this type: drinking red wine is good for you, eating meat is bad for you, and so on. What has gone wrong?

 The key to this paradox is to note that many more of the younger patients are actually in the treatment group than in the non-treatment group, while the situation is reversed for the older patients. The result is to confuse the effect of the treatment with a perfectly possible dependence of recovery on the age of the recipient. In essence this is a badly designed trial, but there is no doubting that it is a subtle effect and not one that most people could understand without a great deal of careful explanation which it is unlikely to get in the pages of a newspaper.

Happy Birthday Bird!

Posted in Jazz with tags , , , , , , on August 29, 2009 by telescoper

I was listening to Jazz Record Requests on BBC Radio 3 this afternoon, which reminded me that today is the 89th anniversary of the birth of the great Charlie Parker, who was known to his friends as “Bird”. Looking for something to celebrate with, I was delighted to find on Youtube this version of the classic bebop tune Anthropology, which appeared on another blog post of mine about Bud Powell (who also plays on this track). This clip (inevitably without video I’m afraid) is in fact taken from the first ever Charlie Parker LP I bought when I was about 15 and which I still have. Sadly, it has never been released on CD so I’m very glad I held onto the LP for so long.

No information is provided on Youtube, but referring to the sleeve note reveals that the track was recorded from a radio broadcast live from  Birdland in New York City on March 31st 1951 using a primitive disc recording machine by an amateur recording buff called Boris Rose. The sound quality isn’t great, but he deserves much greater recognition for capturing this and so many other classic performances and preserving them for posterity.

The personnels consist of Charlie Parker (alto saxophone), Dizzy Gillespie (trumpet), Bud Powell (piano), Tommy Potter (bass) and Roy Haynes (drums).

Here’s what the sleevenote (written by Gary Giddens) says about this track:

“Anthropology is an “I Got Rhythm” variation which originally appeared, in a slightly different form, as “Thriving on a Riff” on Parker’s first session as leader. The tempo is insanely fast; the performance is stunning. Bird has plenty of ideas in his first chorus, but he builds the second and third around a succession of quotations: “Tenderly”, “High Society”, “Temptation.” Gillespie’s second chorus is especially fine – only Fats Navarro had comparable control among the trumpeters who worked with Bird. His blazing high notes tend to set his lyrical phrases in bold relief. Bud, the ultimate bop pianist (and much more), jumps in for two note-gobbling choruses: no quotes, though, it’s all Powell. The four bar exchanges that follow demonstrate Hayne’s precision.

Spot on, but words aren’t really enough to describe this scintillating music, so listen!

Anonymity

Posted in Uncategorized with tags , , , on August 29, 2009 by telescoper

It’s not often that I blog about celebrity tittle-tattle – I have no idea who most “celebrities” are these days anyway – but a little story in last week’s Guardian online caught my eye. I thought I’d mention it here because it raises some interesting issues.

The story is of a fashion model, Liskula Cohen (whom I’d obviously never heard of). It appears that an anonymous blogger (with the charming pseudonym “Skanks”) wrote some derogatory remarks about said Ms Cohen on a website. The latter decided to sue for defamation, but that was difficult because the identity of the blogger wasn’t known. Cohen therefore went to court in an attempt to compel Google to identify the person responsible. She won that case, and duly found out that the blogger was a person called Rosemary Port (who I’d never heard of either). Anyway, to cut a long story short, Ms Cohen dropped her original lawsuit but Ms Port is now suing Google for handing over her real identity…

Of course the story is all a bit childish, but there is a serious question behind it, namely to what extent one has a right to anonymity. I’m not at all sure what the law says on this or what it should say, in fact, especially when it comes to the internet.

In Britain we don’t have identity cards (not yet anyway), so there’s a sort of de facto right to anonymity there. However, with the increasing levels of surveillance and state intrusion into people’s lives, that is changing. The  issue generated by the story above, however, is how the right to anonymity extends into the blogosphere (or the internet generally) rather than how it applies in real life.

Some blogs I know are anonymous but I happen also to know who writes them. I presume the authors have reasons for wishing to conceal their identities so I wouldn’t dream of revealing them myself. However, these are all sites run by reasonably civilised people and it’s very unlikely that any of them would use their anonymity to engage in abusive or defamatory activities. If one of them did, I wouldn’t have any qualms at all about exposing their identity, but I’m not sure whether that would be a legally acceptable course of action.

But anonymity still makes me a  bit uncomfortable. In academic life we come across it in the context of refereeing grant applications and papers submitted to journals for consideration. Usually the default is for referees to remain anonymous is such situations. Most referees are conscientious and if they have criticisms they are usually presented politely and constructively. There are, however, some exceptions. Fortunately these are few and far between, but there are some individuals who take the opportunity provided by anonymity to be downright abusive. Us old hands have sufficiently thick skins to brush such attacks off, but vitriolic comments made on papers written by inexperienced scientists (perhaps even research students) are completely out of order. This probably wouldn’t happen if referees didn’t have the right to remain anonymous. On the other hand, having your identity known might make it difficult for some  to write critically of, say, the work of more senior scientists. Perhaps the answer is to retain anonymity but for the journal editor, for instance, to monitor the reports produced by referees and reprimand any who transgress.

Going back to the original subject of blogs, provides me with an opportunity to describe some of the behind-the-scenes issues with running this blog. In the beginning I decided to have an open comment policy so that anyone and everyone could comment without any form of intervention. That turned out to be a disaster because of the numbers of automatically generated  SPAM comments that clogged up the boxes. I therefore switched on a SPAM filter so it could veto obvious garbage, but otherwise kept an open policy. The alternatives offered by WordPress include one that requires all comments to be from people registered at the site (which I thought would probably be a deterrent to people only wanting to comment on the odd post). Another option is to maintain a blacklist which treats all messages from persons on the list as SPAM. It’s also possible to block all comments entirely, of course, but I enjoy reading most of them so I think it would be a shame to do that just because of a few breaches of netiquette.

All went fairly well and I only had to ban a couple of individuals for abuse. However, over the course of the year I have received a steadily increasing number of crudely abusive comments (of a personal nature) from various anonymous sources. These are mostly depressingly puerile and they don’t affect me much but I find it very disconcerting to think that there are people sitting out there with nothing better to do.

 Since WordPress notifies me every time a  comment is posted, it is quite easy to remove this junk but I found it very tiresome (when there were several per day) and eventually decided to change my policy and automatically block comments from all anonymous sources. Since this requires a manual check into whether the identity information given with the comment is bona fide, comments from people who haven’t commented on this blog before may take a little while to get approved.

There are still comments on here which appear anonymous (or with a pseudonym) on here, but these are from people who have identified themselves to me with a proper email address or who the software has identified through their IP address or information revealed by their web browser (which is probably more than you think…). I’m happy for people to comment without requiring they release their name to the world, and will do my best to ensure their confidentiality, but I’m not happy to publish comments from people whose identity I don’t know.

If you’re interested, as of today this blog has received 4105 comments in total, but only 1747 have been published. The rest were either SPAM or abuse.

Am I denying freedom of speech by rejecting anonymous comments? I don’t think so. If you want freedom of speech that much, you can write your own blog (anonymous or otherwise). And if every sight of this blog makes you want to write abusive comments, perhaps you should exercise your freedom not to read it.

I’d be interested to know from any fellow bloggers if they have the same problems with abusive comments. If not, perhaps I should start taking it personally!

More generally, I will not accept anonymous comments on the subject of the anonymity of comments, but any other contributions are welcome via the box.

Unless you’re banned.

The Heldentenor

Posted in Opera with tags , , , , , , on August 28, 2009 by telescoper

Last week I was listening one of this summer’s Promenade concerts on the radio. The one in question featured the West-Eastern Divan Orchestra, a group of young Arab and Israeli players conducted by Daniel Barenboim. Before the music actually started there was a lengthy discussion by the radio pundits and members of the orchestra about the decision to include in their programme a piece by Richard Wagner, the Prelude and Liebestod from Tristan und Isolde. The orchestra had actually done this as an encore piece previously, but had never had in on their published programme. The problem was that Wagner was a notorious anti-semitic bastard and his music is considered by many to be emblematic of German Nazism. Many members of the orchestra – not only those who happened to be Jewish, in fact – did not feel at all comfortable playing music that carried such distressing overtones. After much discussion, however, they had decided to reclaim Wagner’s music from its awful past and treat it as their own. The performance they produced last week was really excellent, I should add.

For reasons which should become obvious fairly soon, this spurred me on to put something up here by the great Danish singer  Lauritz Melchior. Born in Copenhagen in 1890, Melchior was probably the greatest of all the Heldentenors. If you don’t know what a Heldentenor is, it’s a term used to describe the heroic lead in most of Wagner’s operas. In the words of Anna Russell, he

… is very big, very strong, very brave, very stupid. He carries a spear and wears a helmet. He talks to birds, laughs at dragons, and travels by swan.

Melchior had an immensely powerful voice, which is obligatory if you have to cut through a huge Wagnerian orchestra, but, unlike many other singers who can sing very loud, he was also extremely accurate and his voice had a very rich texture. Other dramatic tenors of his day had purer voices – but if Richard Tauber‘s was polished silver, Melchior’s was more like wrought iron. It’s a matter of taste of course, but I haven’t heard any modern singers anywhere near as good as him in Wagnerian roles. For me, Melchior is the Heldentenor.

The other thing I should mention about Melchior was that he was Jewish (NOTE: This appears to be incorrect – see comments). Although he performed frequently in German opera houses – including  Bayreuth – during the 1920s, he stopped doing so in 1931 when Hitler and his cronies started systematically persecuting Jewish musicians. He spent much of the rest of his life as a star performer at the Metropolitan Opera in New York, and eventually took up American citizenship. Despite his ancestry and his hatred for the Nazis, however, he never stopped performing Wagner’s music.

When he  died in 1973,  Melchior’s body was transported back to his native Denmark and he was buried in the famous Assistens Cemetery in his home town of Copenhagen. His grave, in fact, is not far from that of the jazz musician Ben Webster.

I don’t know the date of this particular clip, but it was made for American television, I guess sometime during the 1950s. Melchior was already an old man by then, but I love the way he sets himself for this performance and you don’t have to make any allowance for his years. His diction is superb, there’s a wonderful timbre to his voice and when he unleashes the fortissimo his power is almost shocking. This is the narration In Fernem Land, from Wagner’s Lohengrin that also provides the theme for the exquisite instrumental Prelude to Act I of the opera, which also provided Anna Russell with her reference to travelling by swan..

Consummation

Posted in Music, Poetry, The Universe and Stuff with tags , , on August 27, 2009 by telescoper

Not long ago I put up an item containing a  poem by John Wilmot, Earl of Rochester. Since I’ve been reading a collection which contains poems by another of the metaphysical poets, Thomas Traherne, I thought I’d pick one of his to put up too.

I was also reminded of Traherne’s poetry when John Peacock commented on another recent post because it was he that introduced me to the truly wonderful musical settings of some of Traherne’s poetry made by Gerald Finzi in his cantata Dies Natalis, and pointed me in the direction of the stunning recording of that work made by Wilfred Brown with the English Chamber Orchestra, conducted by Christopher Finzi (son of Gerald). Why Finzi –  and especially that work – is not better known is something I’ll never understand. But that’s another story…

The story of Thomas Traherne’s poetry is strange and fascinating. The son of  a cobbler, he was a devoutly religious man who lived most of his short life (1637-1674) in relative obscurity as a clergyman and theologian. He was a prolific writer of both prose and poetry, but very little of his work was published during his lifetime. Vast number of handwritten manuscripts survived his death, however, and many of these remained in the safekeeping of a local family in his native Herefordshire. However, in 1888 the estate of this family was wound up, sold, and the manuscripts became dispersed. Eventually, in 1897, one set of papers was  accidentally discovered in a bookstall. Traherne’s first volume of verse was published in 1903 and a second collection followed in 1908.

When these poems finally found their way into the literary world they were greeted with astonishment as well as deep appreciation and they were widely  influential: TS Eliot was a great admirer of Traherne, as was Dorothy L Sayers. The timing of their publication probably explains why Finzi’s music teacher, Ernest Farrar, suggested them to his young student; Finzi was born in 1901 and Farrar taught him as a young boy before he was called up for service in the First World War and killed in action in 1918.

Over the years further manuscripts  have also come to light – literally, in one case, because in 1967 another lost Traherne manuscript was found, on fire, in a  rubbish dump and rescued in the nick of time.

Traherne is sometimes described as the last metaphysical poet and, indeed, the last poems in the collection I have been reading are by him. However, it seems to me he might equally be described as the first romantic poet. The themes he tackles – love of nature and loss of childhood innocence – and his visionary, rhapsodic style have as much in common with William Blake and, especially, William Wordsworth as they do with better known metaphysical poets such as John Donne.

Traherne’s most famous poem is probably Shadows in the Water, but I decided to pick a relatively obscure one, primarily because it deals with matters close to the concerns of a cosmologist! The central theme is the inadequacy of human thought processes in finding a true description of reality or, if you like, full intercourse with nature. For the poet, this can only be achieved through God. This is the consummation referred to in the title.

He’s particularly good at capturing  how we tend to gloss over difficulties with our conceptual framework and how we invent things to plug the gaps. I particularly like the lines from the fourth verse “Wherein because we no//Object distinctly find or know,//We sundry things invent,//That may our fancy give content.” Dark matter?

Consummation

The thoughts of men appear
Freely to move within a sphere
Of endless reach; and run
Though in the soul, beyond the sun.
The ground on which they acted be
Is unobserved infinity.

Extended through the sky,
Though here, beyond it far they fly:
Abiding in the mind
An endless liberty they find:
Throughout all spaces can extend,
Nor ever meet or know an end.

They, in their native sphere,
At boundless distances appear:
Eternity can measure;
Its no beginning see with pleasure.
Thus in the mind an endless space
Doth naturally display its face.

Wherein because we no
Object distinctly find or know,
We sundry things invent,
That may our fancy give content;
See points of space beyond the sky,
And in those points see creatures lie;

Spy fishes in the seas,
Conceit them swimming there with ease;
The dolphins and the whales,
Their very fins, their very scales,
As there within the briny deep
Their tails the flowing waters sweep.

Can see the very skies,
As if the same were in our eyes;
The sun, though in the night,
As if it moved within our sight;
One space beyond another still
Discovered; think while ye will.

Which though we don’t descry,
(Much like by night an idle eye,
Not shaded with a lid,
But in a darksome dungeon hid)
At last shall in a glorious day
Be made its objects to display,

And then shall ages be,
Within its wide eternity;
All kingdoms stand
Howe’er remote, yet nigh at hand;
The skies, and what beyond them lie,
Exposed unto every eye.

Nor shall we then invent
Not alter things; but with content
All in their places see,
As doth the glorious deity;
Within the scope of whose great mind,
We all in their true nature find.

Audio Video Disco

Posted in Biographical, The Universe and Stuff with tags , , on August 26, 2009 by telescoper

2656852604_d321d54df1

This scary picture is taken from an interactive exhibit in the Weller Galleries of the Royal Observatory at Greenwich, which opened in 2007. The exhibit, I mean, not the Royal Observatory. I remember going down there to record the video segments, but had forgotten all about it until somebody found this image on the net and drew my attention to it.

The exhibit consists of a series of display screens with various astronomical and cosmological concepts and questions on them, along with appropriate images. Visitors touch the screens to bring up the video segments in which distinguished astronomers (or me) attempt to provide explanations.

The lady to the bottom right is probably providing a sign language translation of my contribution. Or she could simply be screaming and waving her hands in terror. Wouldn’t you?

PS. If you want an explanation of the title of this blog post, I’ll translate Audio Video Disco from the latin for you. It means “I hear, I see, I learn”. Since they have to touch the screen, I might have added “I touch” which would be Tango….

The Athenian Option

Posted in Politics with tags , , on August 25, 2009 by telescoper

I’ve just finished reading a very interesting an provocative little book called The Athenian Option, which offers a radical vision of how to renew Britain’s democracy.

The context within which this book was written was the need to reform Britain’s unelected second chamber, the House of Lords. The authors of the book, Anthony Barnett and Peter Carty, were proposing a way to do this even before Tony Blair’s New Labour party came to power in 1997, promising to reform  the House of Lords in its manifesto. Despite being well into it’s third Parliament, New Labour hasn’t done much about it yet, and has even failed to offer any real proposals. Although it has removed voting rights from the hereditary peers, the result of this is that the House of Lords is still stuffed full of people appointed by the government.

The need for reform is now greater than ever.

Over the past year or so, we have seen dramatically increasing disillusionment with the political establishment, which has been handing out billions of pounds of tax payers’ money to the profligate banking sector causing a ballooning public debt and the imminent prospect of savage cuts in public spending with consequent reductions in jobs and services.

Meanwhile, under New Labour, the culture of cronyism has led to the creation of a myriad pointless quangos doing their best to strangle the entire country with red tape. Although Gordon Brown stated in 2004 that he was going to reduce  bureaucracy, the number of civil servants in the UK has grown by about 12% (from 465,7000 to 522,930) between then and now. If the amount of bureaucracy within the British university system is anything to go by, the burden of the constant processes of evaluation, assessment and justification is out of all proportion to what useful stuff actually gets done. This isn’t all the fault of New Labour . It started with previous Conservative governments who viewed the public services as a kind of enemy within, to be suspected, regulated and subdued. However, there’s no denying that it has got worse in recent years.

There is an even more sinister side to all this, in the steady erosion of civil liberties through increased clandestine surveillance, detention without trial and the rest of the paraphernalia of paranoid goverment. Big Brother isn’t as far off as we’d all like to think.

The recent furore over MP’s expenses has led to further disgust with the behaviour of our elected representatives, many of whom seem to be more interested in lining their own pockets than in carrying out their duties as our elected representatives.

The fact is that the political establishment has become so remote from its original goal of serving the people that it is now regarded with near-total contempt by a large fraction of the population. Politics now primarily serves itself and, of course, big business. It needs to be forced to become more accountable to ordinary people. This is why I think the suggestion of radical reform along the lines suggested by Barnett and Carty is not only interesting, but something like it is essential if we are to survive as a democracy.

What they propose is to abolish the House of Lords as the Second Chamber, and replace it with a kind of jury selected by lottery from the population in much the same way that juries are selected for the crown courts except that they would be much larger, of order a thousand people or so.  This is called the Athenian Option because in ancient Athens all citizens could vote (although I should add that there were about 5000 citizens and about 100,000 slaves, and women couldn’t vote even if they weren’t slaves, so the name isn’t at all that appropriate).

Selection of representatives from the electoral roll would be quite straightforward to achieve.  Service should be mandatory, but the composition of the Second Chamber could be refreshed sufficiently frequently that participation should not be too onerous for any individual. It may even be possible for the jury not to have to attend a physical `house’ anyway. They could vote by telephone or internet, although safeguards would be needed to prevent fraud or coercion. It would indeed probably be better if each member of the panel voted independently and in secret anyway.

The central body of government would continue to be a representative Parliament similar to the current House of Commons. The role of the jury would be  limited to voting on legislation sent to it by the House of Commons, which would continue to be elected by a General Election as it is at present. Laws passed by the Commons could not become law unless approved by the juries.

Turnout at British general elections has been falling steadily over the past two decades. Apathy has increased  because the parliamentary machine has become detached from its roots. If nothing is done to bring it back under popular control, extremist parties like the British National Party will thrive and the threat to our democracy will grow further.

Living in Wales, I would add another element to the argument. The creation of regional assemblies in Wales, Scotland and Northern Ireland has not been as successful as it might have been because it has resulted not in more democracy, but in more politicians. The Welsh Assembly has little real power, but has fancy offices and big salaries for its members and we have it as well as Westminster and the local Councils.

We also have a European Parliament, again with very little real power but with its own stock of overpaid and self-important politicians elected by the tiny fraction of the electorate that bothers to vote.

My solution to this mess would be to disband the regional assemblies and create regional juries in their place. No legislation would be enacted in Wales unless passed by the Welsh jury, likewise elsewhere.

To be consistent, the replacement House of Lords should be an English jury, although perhaps there could be regional structures within England too. We would therefore have one representative house, The House of Commons, and regional juries for Wales, Scotland, England and Northern Ireland. This would create a much more symmetrical structure for the governance of the United Kingdom, putting an end to such idiocies as the West Lothian Question.

Of course many details would need to be worked out, but it seems to me that this proposal makes a lot of sense. It retains the political party system in the House of Commons where legislation would be debated and amended before being sent to the popular juries. The new system would, however, be vastly cheaper than our current system. It would be much fairer and more democratic. It would make the system of government more accountable, and it would give citizens a greater sense of participation in and responsibility for the United Kingdom’s political culture. Politics is too important to be left to politicians.

On the other hand, in order to set it up we would need entire sections of the current political structure to vote themselves out of existence. Since they’re doing very nicely out of the current arrangements, I think change is unlikely to be forthcoming through the usual channels.

Anyone care for a revolution?

Test Odds

Posted in Cricket with tags , , on August 24, 2009 by telescoper

I’m very grateful to Daniel Mortlock for sending me this fascinating plot. It comes from the cricket pages of The Times Online and it shows how the probability of the various possible outcomes of the Final Ashes Test at the Oval evolved with time according to their “Hawk-Eye Analysis”.

pastedGraphic

 I think I should mention that Daniel is an Australian supporter, so this graph must make painful viewing for him! Anyway, it’s a fascinating plot, which I read as an application of Bayesian probability.

At the beginning of the match, a prior probability is assigned to each of the three possible outcomes: England win (blue); Australia win (yellow); and Draw (grey). It looks like these are roughly in the ratio 1:2:2. No details are given as to how these were arrived at, but it must have taken into account the fact that Australia thrashed England in the previous match at Headingley. Information from previous Tests at the Oval was presumably also included.I don’t know if the fact that England won the toss and decided to bat first altered the prior odds significantly, but it should have.

Anyway, what happens next depends on how sophisticated a model is used to determine the subsequent evolution of the  probabilities. In good Bayesian fashion, information is incorporate in a likelihood function determined by the model and this is used to update the  prior  to produce a posterior probability. This is passed on as a prior for  the next time step. And so it goes on until the end of the match where, regardless of what prior is chosen, the data force the model to the correct conclusion.

The red dots show the fall of wickets, but the odds fluctuate continually in accord with variables such as scoring rate, number of wickets,  and, presumably, the weather. Some form of difference equation is clearly being used, but we don’t know the details.

England got off to a pretty good start, so their probability to win started to creep up, but not by all that much, presumably because the model didn’t think their first-innings total of 332 was enough against a good batting side like Australia. However, the odds of a draw fell more significantly as a result of fairly quick scoring and the lack of any rain delays.

When the Australians batted they were going well at the start so England’s probability to win started to fall and theirs to rise. But when they started to lose quick wickets (largely to Stuart Broad), the blue and yellow trajectories swap over and England became favourites by a large margin. Despite a wobble when they lost 3 early wickets and some jitters when Australia’s batsmen put healthy partnerships together, England remained the more probable to win from that point to the end.

Although it all basically makes some sense, there are some curiosities.  Daniel Mortlock asked, for example, whether Australia were  really as likely to win at about 200 for 2 on the fourth day as  England were when Australia were 70 without loss in the first innings?  That’s what the graph seems to say. His reading of this is that too much stock is placed in the difficulty of   breaking a big (100+ runs) parnership, as the curves seem to   “accelerate” when the batsmen seem to be doing well.

I wonder how new information is included in general terms. Australia’s poor first innings batting (160 all out) in any case only reduced their win probability to about the level that England started at. How was their batting in the first innings balanced against their performance in the last match?

I’d love to know more about the algorithm used in this analysis, but I suspect it is copyright. There may be a good reason for not disclosing it. I have noticed in recent years that bookmakers have been setting extremely parsimonious odds for cricket outcomes. Gone are the days (Headingley 1981) when bookmakers offered 500-1 against England to beat Australia, which they then proceeded to do. In those days the bookmakers relied on expert advisors to fix their odds. I believe it was the late Godfrey Evans who persuaded them to offer 500-1. I’m not sure if they ever asked him again!

The system on which Hawkeye is based is much more conservative. Even on the last day of the test, odds against an Australian victory remained around 4-1 until they were down to their last few wickets. Notice also that the odds on a draw were never as long against as they should have been either, when that outcome was clearly virtually impossible. On the morning of the final day I could only find 10-1 against the draw which I think is remarkably ungenerous. However, even with an England victory a near certainty you could still find odds like 1-4. It seems like the system doesn’t like to produce extremely long or extremely short odds.

Perhaps the bookies are now using analyses like this to set their odds, which explains why betting on cricket isn’t as much fun as it used to be. On the other hand, if the system is predisposed against very short odds then maybe that’s the kind of bet to make in order to win. Things like this may be why the algorithm behind Hawkeye isn’t published…

The Ashes Return

Posted in Cricket with tags on August 23, 2009 by telescoper

Just a short note, for posterity more than anything else. England have won back The Ashes from Australia!!

The final test match at the Oval ended this evening, with England finally knocking over the last Australian wicket, the gallant Mike Hussey who scored a defiant 121 today. Set to make a total of 546 to win in the second innings, Australia got to a creditable 348 all out but it was always going to be too stiff a target on a pitch that has been visibly deteriorating since the first day.

I’ve actually been struggling since last Friday with a nasty stomach bug, otherwise I might have had time to blog a bit more about this match. I have to say the tension as England’s fortunes have ebbed and flowed hasn’t done much for my convalescence either! Earlier this afternoon Australia looked very comfortable, in fact. Two run outs set them back a bit , but even so England were making heavy work of bowling them out. Cue Steve Harmison (who  had a mediocre match until that point) and Graham Swann (who had an excellent game) , both of them weighing in with wickets, as Australia finally keeled over, from 327-5 to 348 all out. Indeed Harmison took two wickets with consecutive balls and could have finished the match and won the Ashes with a hat-trick.

Although the game finished (just) inside four days instead of carrying on into the fifth, I was struck by how much the pattern of this match followed that of the Lord’s Test  that I was lucky enough to see a part of. In particular, both games turned on poor first innings with the bat for the Australians.

As for the series, we now all know just how important that last day at Cardiff was! With England winning at Lord’s, a draw at Edgbaston and Australia trouncing England at Headingley, it was only still 1-1 because of that staunch rearguard with the bat by England’s bowlers, Anderson and Panesar. Without them the series would have been 2-1 to Australia. If the series is tied the Ashes stay with whoever held them before the series started, so  a draw in this match would have meant Australia retaining them,  but England won and took the series 2-1. Although Monty Panesar didn’t play at the Oval, let’s not forget how important his contribution was.

 I don’t think this series produced as much quality cricket as the epic struggle of 2005. In that series the games were closer and on most occasions the game went into the final day with any result possible, which is actually quite a rare occurence in Test cricket.  But the result is the same and the celebrations will be similar.

If England can sort out their batting problems (especially at No. 3) then they could become a really good side. They’ll need to be when they travel to South Africa later this year. How about bringing Monty in at No. 3?

The football season is also under way and that’s the last Test cricket of the summer. Time passes. I enjoy football , especially when Newcastle United are winning – an all too rare experience but one which is happening these days – but I have to say that nothing can match Test cricket for drama and entertainment. If football is like a rock concert, then cricket is grand opera.

Giant Steps

Posted in Jazz with tags , on August 21, 2009 by telescoper

After all the griping about musical taste  in two of my earlier posts this week (here and here), it’s probably good to put something up which I think is a masterpiece. You may, of course, disagree….

I came across this on Youtube a while ago. It made me think of the hours I spent trying to transcribe a Johnny Dodds clarinet solo from an old record, and that came out as a single page of music!

Here’s what the incredible virtuosity of John Coltrane‘s tenor sax playing looks like when written down. Or not quite. For some reason, the transcription is done as if the instrument is in Concert Pitch (C) whereas the tenor saxophone is a transposing instrument (B-flat). This means when you play what is written as C on the stave what actually comes out as B-flat, etc. Music for such instruments has to be written taking this into account, but this transcription doesn’t do so. There used to be (and probably still are, here and there) C-melody saxophones but they’re not very popular, and John Coltrane certainly wasn’t playing one on this track!

Neverthless, the speed and inventiveness of his playing is just amazing to behold. The tune is a Coltrane original which involves an unusual (and difficult to play) chord progression based on three keys shifted by major thirds.

It’s  called Giant Steps