Author Archive

« Older Entries
Newer Entries »

The Distribution of Cauchy

Posted in Bad Statistics, The Universe and Stuff with tags , , , , , on April 6, 2016 by telescoper

Back into the swing of teaching after a short break, I have been doing some lectures this week about complex analysis to theoretical physics students. The name of a brilliant French mathematician called Augustin Louis Cauchy (1789-1857) crops up very regularly in this branch of mathematics, e.g. in the Cauchy integral formula and the Cauchy-Riemann conditions, which reminded me of some old jottings aI made about the Cauchy distribution, which I never used in the publication to which they related, so I thought I’d just quickly pop the main idea on here in the hope that some amongst you might find it interesting and/or amusing.

What sparked this off is that the simplest cosmological models (including the particular one we now call the standard model) assume that the primordial density fluctuations we see imprinted in the pattern of temperature fluctuations in the cosmic microwave background and which we think gave rise to the large-scale structure of the Universe through the action of gravitational instability, were distributed according to Gaussian statistics (as predicted by the simplest versions of the inflationary universe theory).  Departures from Gaussianity would therefore, if found, yield important clues about physics beyond the standard model.

Cosmology isn’t the only place where Gaussian (normal) statistics apply. In fact they arise  fairly generically,  in circumstances where variation results from the linear superposition of independent influences, by virtue of the Central Limit Theorem. Thermal noise in experimental detectors is often treated as following Gaussian statistics, for example.

The Gaussian distribution has some nice properties that make it possible to place meaningful bounds on the statistical accuracy of measurements made in the presence of Gaussian fluctuations. For example, we all know that the margin of error of the determination of the mean value of a quantity from a sample of size n independent Gaussian-dsitributed varies as 1/\sqrt{n}; the larger the sample, the more accurately the global mean can be known. In the cosmological context this is basically why mapping a larger volume of space can lead, for instance, to a more accurate determination of the overall mean density of matter in the Universe.

However, although the Gaussian assumption often applies it doesn’t always apply, so if we want to think about non-Gaussian effects we have to think also about how well we can do statistical inference if we don’t have Gaussianity to rely on.

That’s why I was playing around with the peculiarities of the Cauchy distribution. This distribution comes up in a variety of real physics problems so it isn’t an artificially pathological case. Imagine you have two independent variables X and Y each of which has a Gaussian distribution with zero mean and unit variance. The ratio Z=X/Y has a probability density function of the form

p(z)=\frac{1}{\pi(1+z^2)},

which is a Cauchy distribution. There’s nothing at all wrong with this as a distribution – it’s not singular anywhere and integrates to unity as a pdf should. However, it does have a peculiar property that none of its moments is finite, not even the mean value!

Following on from this property is the fact that Cauchy-distributed quantities violate the Central Limit Theorem. If we take n independent Gaussian variables then the distribution of sum X_1+X_2 + \ldots X_n has the normal form, but this is also true (for large enough n) for the sum of n independent variables having any distribution as long as it has finite variance.

The Cauchy distribution has infinite variance so the distribution of the sum of independent Cauchy-distributed quantities Z_1+Z_2 + \ldots Z_n doesn’t tend to a Gaussian. In fact the distribution of the sum of any number of  independent Cauchy variates is itself a Cauchy distribution. Moreover the distribution of the mean of a sample of size n does not depend on n for Cauchy variates. This means that making a larger sample doesn’t reduce the margin of error on the mean value!

This was essentially the point I made in a previous post about the dangers of using standard statistical techniques – which usually involve the Gaussian assumption – to distributions of quantities formed as ratios.

We cosmologists should be grateful that we don’t seem to live in a Universe whose fluctuations are governed by Cauchy, rather than (nearly) Gaussian, statistics. Measuring more of the Universe wouldn’t be any use in determining its global properties as we’d always be dominated by cosmic variance…

8 Comments »

The Insignificance of ORB

Posted in Bad Statistics with tags , , , on April 5, 2016 by telescoper

A piece about opinion polls ahead of the EU Referendum which appeared in today’s Daily Torygraph has spurred me on to make a quick contribution to my bad statistics folder.

The piece concerned includes the following statement:

David Cameron’s campaign to warn voters about the dangers of leaving the European Union is beginning to win the argument ahead of the referendum, a new Telegraph poll has found.

The exclusive poll found that the “Remain” campaign now has a narrow lead after trailing last month, in a sign that Downing Street’s tactic – which has been described as “Project Fear” by its critics – is working.

The piece goes on to explain

The poll finds that 51 per cent of voters now support Remain – an increase of 4 per cent from last month. Leave’s support has decreased five points to 44 per cent.

This conclusion is based on the results of a survey by ORB in which the number of participants was 800. Yes, eight hundred.

How much can we trust this result on statistical grounds?

Suppose the fraction of the population having the intention to vote in a particular way in the EU referendum is p. For a sample of size n with x respondents indicating that they hen one can straightforwardly estimate p \simeq x/n. So far so good, as long as there is no bias induced by the form of the question asked nor in the selection of the sample which, given the fact that such polls have been all over the place seems rather unlikely.

A little bit of mathematics involving the binomial distribution yields an answer for the uncertainty in this estimate of p in terms of the sampling error:

\sigma = \sqrt{\frac{p(1-p)}{n}}

For the sample size of 800 given, and an actual value p \simeq 0.5 this amounts to a standard error of about 2%. About 95% of samples drawn from a population in which the true fraction is p will yield an estimate within p \pm 2\sigma, i.e. within about 4% of the true figure. In other words the typical variation between two samples drawn from the same underlying population is about 4%. In other other words, the change reported between the two ORB polls mentioned above can be entirely explained by sampling variation and does not at all imply any systematic change of public opinion between the two surveys.

I need hardly point out that in a two-horse race (between “Remain” and “Leave”) an increase of 4% in the Remain vote corresponds to a decrease in the Leave vote by the same 4% so a 50-50 population vote can easily generate a margin as large as  54-46 in such a small sample.

Why do pollsters bother with such tiny samples? With such a large margin error they are basically meaningless.

I object to the characterization of the Remain campaign as “Project Fear” in any case. I think it’s entirely sensible to point out the serious risks that an exit from the European Union would generate for the UK in loss of trade, science funding, financial instability, and indeed the near-inevitable secession of Scotland. But in any case this poll doesn’t indicate that anything is succeeding in changing anything other than statistical noise.

Statistical illiteracy is as widespread amongst politicians as it is amongst journalists, but the fact that silly reports like this are commonplace doesn’t make them any less annoying. After all, the idea of sampling uncertainty isn’t all that difficult to understand. Is it?

And with so many more important things going on in the world that deserve better press coverage than they are getting, why does a “quality” newspaper waste its valuable column inches on this sort of twaddle?

2 Comments »

Sonnet No. 98

Posted in History, Poetry with tags , , , on April 5, 2016 by telescoper

It’s been a while since I posted any of Shakespeare’s sonnets. A brief mention on the radio this morning that William Shakespeare died 400 years ago this month convinced me to rectify that omission and, since it is April, I thought I’d put up this one, No. 98. As with the rest of the first 126 of these poems, it is addressed by the poet to a “fair youth”, i.e. from an older man to a younger one. These sonnets deal with such themes as love, beauty, mortality, absence and longing, framed by the affectionate relationship between two men of very different ages:

From you have I been absent in the spring,
When proud-pied April, dressed in all his trim,
Hath put a spirit of youth in every thing,
That heavy Saturn laughed and leaped with him.
Yet nor the lays of birds, nor the sweet smell
Of different flowers in odour and in hue
Could make me any summer’s story tell,
Or from their proud lap pluck them where they grew.
Nor did I wonder at the lily’s white,
Nor praise the deep vermilion in the rose;
They were but sweet, but figures of delight
Drawn after you, you pattern of all those.
Yet seemed it winter still, and you away,
As with your shadow I with these did play.

 

Leave a comment »

Panama

Posted in History, Jazz with tags , , , on April 4, 2016 by telescoper

For some reason today’s news made me think of this famous old record by Bunk Johnson.

Bunk Johnson was born in New Orleans way back in 1879 and he made his name playing trumpet in the very early days of jazz, including – if his own account can be believed – a stint with the legendary Buddy Bolden.  He was regarded by many, including Louis Armstrong no less, as one of the top trumpeters in New Orleans in the period between 1905 and 1915. Jazz had begun with the  marching bands that performed in New Orleans but then largely moved into the bordellos of Storyville, the biggest (legal) red light district in the history of the United States. When Storyville was closed down in 1917 most professional jazz musicians lost their only source of regular income. However, a few years later, in 1919, the United States Senate proposed the 18th Amendment to the Constitution which prohibited the manufacture, distribution and sale of alcohol for human consumption and ushered in the era of Prohibition. This turned Chicago into a bootlegger’s paradise and jazz musicians flocked there to perform in the numerous speakeasies. That’s why the great New Orleans Jazz records of the 1920s were all made in Chicago and it also caused the music to evolve in new directions.  Bunk Johnson did not join the mass exodus to Chicago and his career faded into obscurity, ending entirely in 1931 when he had his front teeth knocked out in a brawl and could no longer play the trumpet.

However, in 1942, Bunk Johnson was rediscovered as a very old man by some young jazz fans who travelled to New Orleans and recorded him playing with a band of local musicians in the basement of a house courtesy of a new set of dentures. Despite the poor sound quality of the recording, the resulting tracks proved incredibly popular, ushering in the New Orleans Revival that began in the United States and then propagated across to Europe after the war to the extent that many revivalist bands even sedulously acquired the “recorded-in-a-garage” sound. Bunk Johnson passed away in 1947 but George Lewis, who plays clarinet on this track, carried the flag for “authentic” New Orleans jazz for many years after that, visiting Europe on many occasions. My Dad played the drums with him a few times..

P.S. I’ve always felt particularly sorry for  Walter Decou, who played piano on the famous Bunk Johnson records of 1942, because apparently he was pounding away like a good ‘un but you can barely hear a note from him on any of them!

 

4 Comments »

The End of Hitomi..

Posted in The Universe and Stuff with tags , , , on April 4, 2016 by telescoper

Time for a gloomy Monday update to my recent post about the Japanese X-ray satellite Hitomi.

First here’s a new plot of the debris (via Jonathan McDowell):

Hitomi

This shows more pieces of debris than the one I showed previously, and also demonstrates that some of the pieces are in rapidly-decaying orbits. A rough estimate suggests that some of these – those in the lower right of the diagram- will burn up in the atmosphere within a week or so. This behaviour is consistent with them being rather light fragments, on which the effect of drag is greater, and consequently possibly rather small.  Their behaviour does not therefore necessarily imply anything too catastrophic about the main spacecraft.

However, there is now strong evidence that the main spacecraft actually did break up fairly completely rather than shedding a few pieces of casing or whatever. Two of the brightest pieces are of roughly equal size and, ominously, the original identification of one of them with the main part of the spacecraft has been shown to be wrong. Furthermore, no signals have been received from the onboard beacon for six days now. It all sounds very terminal to me.

2-hitomispacec

So what happened? Of course I don’t know for sure, but the above picture suggests the possibility of an explosion (possibly violent outgassing of cryogens needed for the instruments near the rear of the main body of the vehicle). The structure to the rear of the vehicle is a deployable optical bench used to increase the focal length of the telescope for hard X-ray work. This could well have broken off during such an explosion, as could all or part of the solar panels used to supply power to the satellite.

The Japanese Space Agency JAXA has not officially given up on Hitomi (formerly known as ASTRO-H) but I think the hopes of most commenters I’m aware of have now faded away.

It’s all very sad.

 

 

 

7 Comments »

Have you been threatened with legal action by “Scientists for Britain”?

Posted in Politics, Science Politics with tags , , , , , on April 3, 2016 by telescoper

Back in circulation after a short break I hope to write a few pieces about why I support the case for the United Kingdom to remain in the European Union, partly because it’s good for science, but also because it’s good for many other reasons.

But before I do that, I feel I have to do a quick post about the extremely unpleasant antics of an organization called “Scientists for Britain“, or rather the anonymous person or persons operating their Twitter feed.

Last Saturday I found myself in receipt of a message, apparently sent by this outfit, that explicitly threatened legal action on grounds of libel because of a comment I had made on one of their posts on Twitter which was alleged to be “disparaging”. I was refrained from referring the sender of this intentionally intimidatory message to the response given in Arkell versus Pressdram but it soon became clear that a number of other scientists on Twitter had received similar threats.Then, fortunately for us, in stepped renowned legal journalist David Allen Green, who blogs as Jack of Kent and is something of a specialist in libel law. He made it quite clear that the threats sent out by Scientists for Britain had no basis whatsoever in law, not least because you can’t libel an anonymous person. I hadn’t said anything even remotely actionable anyway.

Within hours, all the threatening messages had been deleted by Scientists for Britain, and they also blocked those of us to whom they had sent them in the first place, including myself. There are such things as screen grabs, however…

This social media car crash would be very funny were there not something very sinister behind it. I’m all for healthy robust and vigorous debate on the issue of the United Kingdom’s membership of the European Union ahead of the forthcoming referendum, but bullying those you disagree with by means of threats of legal action is no way to make your case. Also, for the record, I will point out that I have seen no evidence that the anonymous operator of the Scientists for Britain Twitter feed, who delights in issuing unwarranted libel threats, is a actually scientist at all. I very much doubt that is the case, in fact. Why else would Scientists for Britain be so obsessive about their anonymity? Even their response to a letter signed by 150 Fellows of the Royal Society is unsigned….

I am posting this information here in an attempt to find out how many other scientists  Scientists for Britain have tried to silence through legal threats.  If this has happened to you, please let me know via email, Twitter, or via the comments box  of this blog (below).

If Scientists for Britain wish to comment they are welcome to do so below, although please note my comments policy: I do not accept postings from anonymous individuals.

 

 

4 Comments »

A Bridge Question

Posted in Uncategorized with tags , on April 3, 2016 by telescoper

image

Catching up on all the important journals after a short holiday I found the above deal in the Bridge column of The Oldie.

I’m not sure I would have the courage to bid to 7NT as South – 7♦ seems a more likely contract – but in the game described that’s what happened.

West led the 8♦, which in my view plays directly into South’s hands. Declarer already has 12 tricks guaranteed, and the 13th needed for a Grand Slam comes easily if West holds the Q♣, which is indeed the case. However, South didn’t even need the club finesse to succeed in taking all 13 because East discarded two clubs as South played off the obvious Diamond and Spade winners, so the Queen fell to either the A♣ or the K♣ on a club lead.

My question, though, as a mediocre player, is why did West lead a diamond?  South had opened with 1♦, and North had jumped to 3♦  before South’s slam try with 4NT (Blackwood). Against 7♦ West’s would have been a standard defensive lead of a trump, but in No Trumps surely it is better here to follow convention and lead the 4th highest of the longest suit, ie the 8♥? South would then have to play the Ace from dummy, which at least destroys one line of communication as the A♦ can no longer provide an entry to dummy.

I think Declarer still makes the contract but my question is, against this lead, does South have to try the club finesse?

2 Comments »

Wrote for Luck

Posted in Uncategorized on April 1, 2016 by telescoper

Some interesting comments on frequencies in the National Lottery. I stopped playing when it went up to £2. I figured a pound wasn’t too high a price for a small frisson of excitement when the draw was taking place, but two pounds was too much…

quantixed's avatarquantixed

Fans of probability love random processes. And lotteries are a great example of random number generation.

The UK National Lottery ran in one format from 19/11/1994 until 7/10/2015. I was talking to somebody who had played the same set of numbers in all of these lottery draws and I wondered what the net gain or loss has been for them over this period.

The basic format is that people buy a line of numbers (6 numbers, from 1-49) and try to match the six numbers (from 49 balls numbered 1-49) drawn from a machine. The aim is to match all six balls and win the jackpot. The odds of this are fantastically small (1 in ~14 million), but if they are the only person matching these numbers they can take away £3-5 million. There are prizes for matching three numbers (1 in ~56 chance), four numbers (1 in ~1,032),  five numbers (1…

View original post 577 more words

Leave a comment »

The Trouble with Hitomi

Posted in The Universe and Stuff with tags , , on March 31, 2016 by telescoper

One of the stories I’ve been following a bit while taking a break from blogging has been that of the Japanese X-ray satellite Hitomi (formerly known as ASTRO-H), which was launched into a low-Earth orbit on February 17th 2016, experienced a “communication anomaly” on Saturday March 26th. It has now become clear that this was more than a simple communications glitch. Astronomer Jonathan McDowell posted this diagram on Twitter that showed a sudden decrease in the orbital period of the satellite:CekOyLxXEAAeNxF

Students of orbital dynamics will know that a decrease in orbital period corresponds to a decrease in the semi-major axis of the orbit, so Hitomi actually fell during this episode. It dropped only slightly – look at the % change on the graph – but by enough to be very worrying.

The plot thickened still further when radar detected five pieces of debris near the satellite and visual observations indicated the spacecraft to be tumbling rapidly. That suggested a very grim picture.

Putting the evidence together it seems that some kind of explosive event – possibly connected with out-gassing of cryogenic material from one of the on-board experiments – had damaged the satellite, changed its orbit and set it spinning uncontrollably.

Since then ground stations have picked up some signals from Hitomi, which is good news,  but these broadcasts are just from the on-board beacon. It has not yet proved possible to communicate with the attitude control system which is the only way to get it back into a stable state.

Obviously it’s touch and go as to whether the Japanese Space Agency JAXA will be able to regain control of Hitomi, but at least there’s more hope than on Saturday when many of us thought the vehicle had fallen apart. In fact the pieces of debris reported may be rather small (ten cm or so is detectable) and the main body of the telescoper may be intact. Maybe.

Update: April 1st. Tracking facilities are now reporting 11 pieces of debris, and also suggesting the object whose period is plotted in the above graph may not be the main part of the spacecraft. This does not sound good.

Update: April 2nd. The debris from Hitomi has now spread out due to different orbital speeds. The two largest pieces are both spinning out of control. I would say at this point that hope of a recovery has now disappeared. It’s very sad.

5 Comments »

How to cite social media in academic writing

Posted in Uncategorized on March 30, 2016 by telescoper

I have often wondered how to do this properly. Now I know!

I am told my blog has been cited in the literature a few times but I have been unable to find any evidence for this as I don’t think there’s any mechanism for tracking citations to blogs. Or is there?

Sue Beckingham's avatarSocial Media for Learning

Image source: https://en.wikipedia.org/wiki/Wikipedia:Citation_needed Image source: https://en.wikipedia.org/wiki/Wikipedia:Citation_needed

Referencing and citation is an important part of any writing. This post looks at some recommendations and consideration when citing social media. Citations have several important purposes:

  • to uphold intellectual honesty (or avoiding plagiarism),
  • to attribute prior or unoriginal work and ideas to the correct sources, to allow the reader to determine independently whether the referenced material supports the author’s argument in the claimed way,
  • and to help the reader gauge the strength and validity of the material the author has used. (Wikipedia)

At Sheffield Hallam University (SHU) the Guide to Referencing offers detailed guidance for producing citations and references according to the Harvard method in the Harvard-SHU style recommended by the library. You may be asked to use another method, or a variation of the Harvard style. If this is the case, you may wish to refer to guidance that matches this style. However the recommendations below…

View original post 834 more words

2 Comments »

« Older Entries
Newer Entries »