Archive for the Bad Statistics Category

« Older Entries
Newer Entries »

The Impact X-Factor

Posted in Bad Statistics, Open Access with tags , , on August 14, 2012 by telescoper

Just time for a quick (yet still rather tardy) post to direct your attention to an excellent polemical piece by Stephen Curry pointing out the pointlessness of Journal Impact Factors. For those of you in blissful ignorance about the statistical aberration that is the JIF, it’s basically a measure of the average number of citations attracted by a paper published in a given journal. The idea is that if you publish a paper in a journal with a large JIF then it’s in among a number of papers that are highly cited and therefore presumably high quality. Using a form of Proof by Association, your paper must therefore be excellent too, hanging around with tall people being a tried-and-tested way of becoming tall.

I won’t repeat all Stephen Curry’s arguments as to why this is bollocks – read the piece for yourself – but one of the most important is that the distribution of citations per paper is extremely skewed, so the average is dragged upwards by a few papers with huge numbers of citations. As a consequence most papers published in a journal with a large JIF attract many fewer citations than the average. Moreover, modern bibliometric databases make it quite easy to extract citation information for individual papers, which is what is relevant if you’re trying to judge the quality impact of a particular piece of work, so why bother with the JIF at all?

I will however copy the summary, which is to the point:

So consider all that we know of impact factors and think on this: if you use impact factors you are statistically illiterate.

  • If you include journal impact factors in the list of publications in your cv, you are statistically illiterate.
  • If you are judging grant or promotion applications and find yourself scanning the applicant’s publications, checking off the impact factors, you are statistically illiterate.
  • If you publish a journal that trumpets its impact factor in adverts or emails, you are statistically illiterate. (If you trumpet that impact factor to three decimal places, there is little hope for you.)
  • If you see someone else using impact factors and make no attempt at correction, you connive at statistical illiteracy.

Statistical illiteracy is by no means as rare among scientists as we’d like to think, but at least I can say that I pay no attention whatsoever to Journal Impact Factors. In fact I don’t think many people in in astronomy or astrophysics use them at all. I’d be interested to hear from anyone who does.

I’d like to add a little coda to Stephen Curry’s argument. I’d say that if you publish a paper in a journal with a large JIF (e.g. Nature) but the paper turns out to attract very few citations then the paper should be penalised in a bibliometric analysis, rather like the handicap system used in horse racing or golf. If, despite the press hype and other tedious trumpetings associated with the publication of a Nature paper, the work still attracts negligible interest then it must really be a stinker and should be rated as such by grant panels, etc. Likewise if you publish a paper in a less impactful journal which nevertheless becomes a citation hit then it should be given extra kudos because it has gained recognition by quality alone.

Of course citation numbers don’t necessarily mean quality. Many excellent papers are slow burners from a bibliometric point of view. However, if a journal markets itself as being a vehicle for papers that are intended to attract large citation counts and a paper published there flops then I think it should attract a black mark. Hoist it on its own petard, as it were.

So I suggest papers be awarded an Impact X-Factor, based on the difference between its citation count and the JIF for the journal. For most papers this will of course be negative, which would serve their authors right for mentioning the Impact Factor in the first place.

PS. I chose the name “X-factor” as in the TV show precisely for its negative connotations.

18 Comments »

Nevaeh ot Yawriats

Posted in Bad Statistics, Music, The Universe and Stuff with tags , , , on July 12, 2012 by telescoper

I just remembered hearing this a while ago at a public talk given by Simon Singh. I guess many of you will have come across it before, but there’s no harm in repeating it. I don’t know why it popped into my head at this particular moment, but perhaps it’s because I’ve been reading some stuff about how my colleagues in gravitational wave research use templates to try to detect specific patterns in noisy data. The method involves cross-correlating a simulated signal against the data until a match is obtained; the problem is often how to assess the probability of  a “chance” coincidence correctly and thus avoid spurious detections. The following might perhaps be a useful warning that unless you do this carefully, you only get out what you put in!

This is an excerpt from the classic  track Stairway to Heaven, by the popular beat combo Led Zeppelin, played backwards. I suggest that you listen to it once without looking at the words on the video, and then again with the words in front of you. If you haven’t heard/seen  it before, I think you’ll find it surprising…

Of course the proper way to interpret (or dismiss) matches like this is to use tools based on  Bayesian inference….

9 Comments »

Clusters, Splines and Peer Review

Posted in Bad Statistics, Open Access, The Universe and Stuff with tags , , , , , on June 26, 2012 by telescoper

Time for a grumpy early morning post while I drink my tea.

There’s an interesting post on the New Scientist blog site by that young chap Andrew Pontzen who works at Oxford University (in the Midlands). It’s on a topic that’s very pertinent to the ongoing debate about Open Access. One of the points the academic publishing lobby always makes is that Peer Review is essential to assure the quality of research. The publishers also often try to claim that they actually do Peer Review, which they don’t. That’s usual done, for free, by academics.

But the point Andrew makes is that we should also think about whether the form of Peer Review that journals undertake is any good anyway.  Currently we submit our paper to a journal, the editors of which select one (or perhaps two or three) referees to decide whether it merits publication. We then wait – often many months – for a report and a decision by the Editorial Board.

But there’s also a free online repository called the arXiv which all astrophysics papers eventually appear on. Some researchers like to wait for the paper to be refereed and accepted before putting it on the arXiv, while others, myself included, just put it on the arXiv straight away when we submit it to the journal. In most cases one gets prompter and more helpful comments by email from people who read the paper on arXiv than from the referee(s).

Andrew questions why we trust the reviewing of a paper to one or two individuals chosen by the journal when the whole community could do the job quicker and better. I made essentially the same point in a post a few years ago:

I’m not saying the arXiv is perfect but, unlike traditional journals, it is, in my field anyway, indispensable. A little more investment, adding a comment facilities or a rating system along the lines of, e.g. reddit, and it would be better than anything we get academic publishers at a fraction of the cost. Reddit, in case you don’t know the site, allows readers to vote articles up or down according to their reaction to it. Restrict voting to registered users only and you have the core of a peer review system that involves en entire community rather than relying on the whim of one or two referees. Citations provide another measure in the longer term. Nowadays astronomical papers attract citations on the arXiv even before they appear in journals, but it still takes time for new research to incorporate older ideas.

In any case I don’t think the current system of Peer Review provides the Gold Standard that publishers claim it does. It’s probably a bit harsh to single out one example, but then I said I was feeling grumpy, so here’s something from a paper that we’ve been discussing recently in the cosmology group at Cardiff. The paper is by Gonzalez et al. and is called IDCS J1426.5+3508: Cosmological implications of a massive, strong lensing cluster at Z = 1.75. The abstract reads

The galaxy cluster IDCS J1426.5+3508 at z = 1.75 is the most massive galaxy cluster yet discovered at z > 1.4 and the first cluster at this epoch for which the Sunyaev-Zel’Dovich effect has been observed. In this paper we report on the discovery with HST imaging of a giant arc associated with this cluster. The curvature of the arc suggests that the lensing mass is nearly coincident with the brightest cluster galaxy, and the color is consistent with the arc being a star-forming galaxy. We compare the constraint on M200 based upon strong lensing with Sunyaev-Zel’Dovich results, finding that the two are consistent if the redshift of the arc is  z > 3. Finally, we explore the cosmological implications of this system, considering the likelihood of the existence of a strongly lensing galaxy cluster at this epoch in an LCDM universe. While the existence of the cluster itself can potentially be accomodated if one considers the entire volume covered at this redshift by all current high-redshift cluster surveys, the existence of this strongly lensed galaxy greatly exacerbates the long-standing giant arc problem. For standard LCDM structure formation and observed background field galaxy counts this lens system should not exist. Specifically, there should be no giant arcs in the entire sky as bright in F814W as the observed arc for clusters at  z \geq 1.75, and only \sim 0.3 as bright in F160W as the observed arc. If we relax the redshift constraint to consider all clusters at z \geq 1.5, the expected number of giant arcs rises to \sim 15 in F160W, but the number of giant arcs of this brightness in F814W remains zero. These arc statistic results are independent of the mass of IDCS J1426.5+3508. We consider possible explanations for this discrepancy.

Interesting stuff indeed. The paper has been accepted for publication by the Astrophysical Journal too.

Now look at the key result, Figure 3:

I’ll leave aside the fact that there aren’t any error bars on the points, and instead draw your attention to the phrase “The curves are spline interpolations between the data points”. For the red curve only two “data points” are shown; actually the points are from simulations, so aren’t strictly data, but that’s not the point. I would have expected an alert referee to ask for all the points needed to form the curve to be shown, and it takes more than two points to make a spline.  Without the other point(s) – hopefully there is at least one more! – the reader can’t reproduce the analysis, which is what the scientific method requires, especially when a paper makes such a strong claim as this.

I’m guessing that the third point is at zero (which is at – ∞ on the log scale shown in the graph), but surely that must have an error bar on it, deriving from the limited simulation size?

If this paper had been put on a system like the one I discussed above, I think this would have been raised…

12 Comments »

Power versus Pattern

Posted in Bad Statistics, The Universe and Stuff with tags , , , , , on June 15, 2012 by telescoper

One of the challenges we cosmologists face is how to quantify the patterns we see in galaxy redshift surveys. In the relatively recent past the small size of the available data sets meant that only relatively crude descriptors could be used; anything sophisticated would be rendered useless by noise. For that reason, statistical analysis of galaxy clustering tended to be limited to the measurement of autocorrelation functions, usually constructed in Fourier space in the form of power spectra; you can find a nice review here.

Because it is so robust and contains a great deal of important information, the power spectrum has become ubiquitous in cosmology. But I think it’s important to realise its limitations.

Take a look at these two N-body computer simulations of large-scale structure:

The one on the left is a proper simulation of the “cosmic web” which is at least qualitatively realistic, in that in contains filaments, clusters and voids pretty much like what is observed in galaxy surveys.

To make the picture on the right I first  took the Fourier transform of the original  simulation. This approach follows the best advice I ever got from my thesis supervisor: “if you can’t think of anything else to do, try Fourier-transforming everything.”

Anyway each Fourier mode is complex and can therefore be characterized by an amplitude and a phase (the modulus and argument of the complex quantity). What I did next was to randomly reshuffle all the phases while leaving the amplitudes alone. I then performed the inverse Fourier transform to construct the image shown on the right.

What this procedure does is to produce a new image which has exactly the same power spectrum as the first. You might be surprised by how little the pattern on the right resembles that on the left, given that they share this property; the distribution on the right is much fuzzier. In fact, the sharply delineated features  are produced by mode-mode correlations and are therefore not well described by the power spectrum, which involves only the amplitude of each separate mode.

If you’re confused by this, consider the Fourier transforms of (a) white noise and (b) a Dirac delta-function. Both produce flat power-spectra, but they look very different in real space because in (b) all the Fourier modes are correlated in such away that they are in phase at the one location where the pattern is not zero; everywhere else they interfere destructively. In (a) the phases are distributed randomly.

The moral of this is that there is much more to the pattern of galaxy clustering than meets the power spectrum…

8 Comments »

Milky Way Satellites and Dark Matter

Posted in Astrohype, Bad Statistics, The Universe and Stuff with tags , , , , on May 4, 2012 by telescoper

I found a strange paper on the ArXiv last week, and was interested to see that it had been deemed to merit a press release from the Royal Astronomical Society that had been picked up by various sites across the interwebs.

The paper, to appear in due course in Monthly Notices of the Royal Astronomical Society, describes a study of the positions and velocities of small satellite galaxies and other object around the Milky Way, which suggest the existence of a flattened structure orientated at right angles to the Galactic plane. They call this the “Vast Polar Structure”. There’s even a nifty video showing this arrangement:

They argue that this is is evidence that these structures have a tidal origin, having been thrown out   in the collision between two smaller galaxies during the formation of the Milky Way. One would naively expect a much more isotropic distribution of material around our Galaxy if matter had fallen into it in the relatively quiescent way envisaged by more standard theoretical models.

Definitely Quite Interesting.

However, I was rather taken aback by this quotation by one of the authors, Pavel Kroupa, which ends the press release.

Our model appears to rule out the presence of dark matter in the universe, threatening a central pillar of current cosmological theory. We see this as the beginning of a paradigm shift, one that will ultimately lead us to a new understanding of the universe we inhabit.

Hang on a minute!

One would infer from this rather bold statement that the paper concerned contained a systematic comparison between the observations – allowing for selection effects, such as incomplete sky coverage – and detailed theoretical calculations of what is predicted in the standard theory of galaxy formation involving dark matter.

But it doesn’t.

What it does contain is a simple statistical calculation of the probability that the observed distribution of satellite galaxies would have arisen in an exactly isotropic distribution function, which they conclude to be around 0.2 per cent.

However, we already know that galaxies like the Milky Way are not exactly isotropic, so this isn’t really a test of the dark matter hypothesis. It’s a test of an idealised unrealistic model. And even if it were a more general test of the dark matter hypothesis, the probability of this hypothesis being correct is not what has been calculated. The probability of a model given the data is not the same as the probability of the data given the model. To get that you need Bayes’ theorem.

What needs to be done is to calculate the degree of anisotropy expected in the dark matter theory and in the tidal theory and then do a proper (i.e. Bayesian) comparison with the observations to see which model gives the better account of the data. This is not any easy thing to do because it necessitates doing detailed dynamical calculations at very high resolution of what galaxy like the Milky Way should look like according to both theories.

Until that’s done, these observations by no means “rule out” the dark matter theory.

31 Comments »

The H-index is Redundant…

Posted in Bad Statistics, Science Politics with tags , , , , , on January 28, 2012 by telescoper

An interesting paper appeared on the arXiv last week by astrophysicist Henk Spruit on the subject of bibliometric indicators, and specifically the Hirsch index (or H-index) which has been the subject of a number of previous blog posts on here. The author’s surname is pronounced “sprout”, by the way.

The H-index is defined to be the largest number H such that the author has written at least H papers having H citations. It can easily be calculated by looking up all papers by a given author on a database such as NASA/ADS, sorting them by (decreasing) number of citations, and working down the list to the point where the number of citations of a paper falls below the number representing position in the list. Normalized quantities – obtained by dividing the number of citations a paper receives by the number of authors of that paper for each paper – can be used to form an alternative measure.

Here is the abstract of the paper:

Here are a couple of graphs which back up the claim of a near-perfect correlation between H-index and total citations:

The figure shows both total citations (right) and normalized citations (left); the latter, in my view, a much more sensible measure of individual contributions. The basic problem of course is that people don’t get citations, papers do. Apportioning appropriate credit for a multi-author paper is therefore extremely difficult. Does each author of a 100-author paper that gets 100 citations really deserve the same credit as a single author of a paper that also gets 100 citations? Clearly not, yet that’s what happens if you count total citations.

The correlation between H index and the square root of total citation numbers has been remarked upon before, but it is good to see it confirmed for the particular field of astrophysics.

Although I’m a bit unclear as to how the “sample” was selected I think this paper is a valuable contribution to the discussion, and I hope it helps counter the growing, and in my opinion already excessive, reliance on the H-index by grants panels and the like. Trying to condense all the available information about an applicant into a single number is clearly a futile task, and this paper shows that using H-index and total numbers doesn’t add anything as they are both measuring exactly the same thing.

A very interesting question emerges from this, however, which is why the relationship between total citation numbers and h-index has the form it does: the latter is always roughly half of the square-root of the former. This suggests to me that there might be some sort of scaling law describing onto which the distribution of cites-per-paper can be mapped for any individual. It would be interesting to construct a mathematical model of citation behaviour that could reproduce this apparently universal property….

15 Comments »

Bayes in the Dock

Posted in Bad Statistics with tags , , , , on October 6, 2011 by telescoper

A few days ago John Peacock sent me a link to an interesting story about the use of Bayes’ theorem in legal proceedings and I’ve been meaning to post about it but haven’t had the time. I get the distinct feeling that John, who is of the frequentist persuasion,  feels a certain amount of delight that the beastly Bayesians have got their comeuppance at last.

The story in question concerns an erroneous argument given during a trial about the significance of a match found between a footprint found at a crime scene and footwear belonging to a suspect.  The judge took exception to the fact that the figures being used were not known sufficiently accurately to make a reliable assessment, and thus decided that Bayes’ theorem shouldn’t be used in court unless the data involved in its application were “firm”.

If you read the Guardian article you will see that there’s a lot of reaction from the legal establishment and statisticians about this, focussing on the forensic use of probabilistic reasoning. This all reminds me of the tragedy of the Sally Clark case and what a disgrace it is that nothing has been done since then to improve the misrepresentation of statistical arguments in trials. Some of my Bayesian colleagues have expressed dismay at the judge’s opinion, which no doubt pleases Professor Peacock no end.

My reaction to this affair is more muted than you would probably expect. First thing to say is that this is really not an issue relating to the Bayesian versus frequentist debate at all. It’s about a straightforward application of Bayes’ theorem which, as its name suggests, is a theorem; actually it’s just a straightforward consequence of the sum and product laws of the calculus of probabilities. No-one, not even the most die-hard frequentist, would argue that Bayes’ theorem is false. What happened in this case is that an “expert” applied Bayes’ theorem to unreliable data and by so doing obtained misleading results. The  issue is not Bayes’ theorem per se, but the application of it to inaccurate data. Garbage in, garbage out. There’s no place for garbage in the courtroom, so in my opinion the judge was quite right to throw this particular argument out.

But while I’m on the subject of using Bayesian logic in the courts, let me add a few wider comments. First, I think that Bayesian reasoning provides a rigorous mathematical foundation for the process of assessing quantitatively the extent to which evidence supports a given theory or interpretation. As such it describes accurately how scientific investigations proceed by updating probabilities in the light of new data. It also describes how a criminal investigation works too.

What Bayesian inference is not good at is achieving closure in the form of a definite verdict. There are two sides to this. One is that the maxim “innocent until proven guilty” cannot be incorporated in Bayesian reasoning. If one assigns a zero prior probability of guilt then no amount of evidence will be able to change this into a non-zero posterior probability; the required burden is infinite. On the other hand, there is the problem that the jury must decide guilt in a criminal trial “beyond reasonable doubt”. But how much doubt is reasonable, exactly? And will a jury understand a probabilistic argument anyway?

In pure science we never really need to achieve this kind of closure, collapsing the broad range of probability into a simple “true” or “false”, because this is a process of continual investigation. It’s a reasonable inference, for example, based on Supernovae and other observations that the Universe is accelerating. But is it proven that this is so? I’d say “no”,  and don’t think my doubts are at all unreasonable…

So what I’d say is that while statistical arguments are extremely important for investigating crimes – narrowing down the field of suspects, assessing the reliability of evidence, establishing lines of inquiry, and so on – I don’t think they should ever play a central role once the case has been brought to court unless there’s much clearer guidance given to juries on how to use it and stricter monitoring of so-called “expert” witnesses.

I’m sure various readers will wish to express diverse opinions on this case so, as usual, please feel free to contribute through the box below!

19 Comments »

The Laws of Extremely Improbable Things

Posted in Bad Statistics, The Universe and Stuff with tags , , , , , , , , on June 9, 2011 by telescoper

After a couple of boozy nights in Copenhagen during the workshop which has just finished, I thought I’d take things easy this evening and make use of the free internet connection in my hotel to post a short item about something I talked about at the workshop here.

Actually I’ve been meaning to mention a nice bit of statistical theory called Extreme Value Theory on here for some time, because not so many people seem to be aware of it, but somehow I never got around to writing about it. People generally assume that statistical analysis of data revolves around “typical” quantities, such as averages or root-mean-square fluctuations (i.e. “standard” deviations). Sometimes, however, it’s not the typical points that are interesting, but those that appear to be drawn from the extreme tails of a probability distribution. This is particularly the case in planning for floods and other natural disasters, but this field also finds a number of interesting applications in astrophysics and cosmology. What should be the mass of the most massive cluster in my galaxy survey? How bright the brightest galaxy? How hot the hottest hotspot in the distribution of temperature fluctuations on the cosmic microwave background sky? And how cold the coldest? Sometimes just one anomalous event can be enormously useful in testing a theory.

I’m not going to go into the theory in any great depth here. Instead I’ll just give you a simple idea of how things work. First imagine you have a set of n observations labelled X_i. Assume that these are independent and identically distributed with a distribution function F(x), i.e.

\Pr(X_i\leq x)=F(x)

Now suppose you locate the largest value in the sample, X_{\rm max}. What is the distribution of this value? The answer is not F(x), but it is quite easy to work out because the probability that the largest value is less than or equal to, say, z is just the probability that each one is less than or equal to that value, i.e.

F_{\rm max}(z) = \Pr \left(X_{\rm max}\leq z\right)= \Pr \left(X_1\leq z, X_2\leq z\ldots, X_n\leq z\right)

Because the variables are independent and identically distributed, this means that

F_{\rm max} (z) = \left[ F(z) \right]^n

The probability density function associated with this is then just

f_{\rm max}(z) = n f(z) \left[ F(z) \right]^{n-1}

In a situation in which F(x) is known and in which the other assumptions apply, then this simple result offers the best way to proceed in analysing extreme values.

The mathematical interest in extreme values however derives from a paper in 1928 by Fisher \& Tippett which paved the way towards a general theory of extreme value distributions. I don’t want to go too much into details about that, but I will give a flavour by mentioning a historically important, perhaps surprising, and in any case rather illuminating example.

It turns out that for any distribution F(x) of exponential type, which means that

\lim_{x\rightarrow\infty} \frac{1-F(x)}{f(x)} = 0

then there is a stable asymptotic distribution of extreme values, as n \rightarrow \infty which is independent of the underlying distribution, F(x), and which has the form

G(z) = \exp \left(-\exp \left( -\frac{(z-a_n)}{b_n} \right)\right)

where a_n and b_n are location and scale parameters; this is called the Gumbel distribution. It’s not often you come across functions of the form e^{-e^{-y}}!

This result, and others, has established a robust and powerful framework for modelling extreme events. One of course has to be particularly careful if the variables involved are not independent (e.g. part of correlated sequences) or if there are not identically distributed (e.g. if the distribution is changing with time). One also has to be aware of the possibility that an extreme data point may simply be some sort of glitch (e.g. a cosmic ray hit on a pixel, to give an astronomical example). It should also be mentioned that the asymptotic theory is what it says on the tin – asymptotic. Some distributions of exponential type converge extremely slowly to the asymptotic form. A notable example is the Gaussian, which converges at the pathetically slow rate of \sqrt{\ln(n)}! This is why I advocate using the exact distribution resulting from a fully specified model whenever this is possible.

The pitfalls are dangerous and have no doubt led to numerous misapplications of this theory, but, done properly, it’s an approach that has enormous potential.

I’ve been interested in this branch of statistical theory for a long time, since I was introduced to it while I was a graduate student by a classic paper written by my supervisor. In fact I myself contributed to the classic old literature on this topic myself, with a paper on extreme temperature fluctuations in the cosmic microwave background way back in 1988..

Of course there weren’t any CMB maps back in 1988, and if I had thought more about it at the time I should have realised that since this was all done using Gaussian statistics, there was a 50% chance that the most interesting feature would actually be a negative rather than positive fluctuation. It turns out that twenty-odd years on, people are actually discussing an anomalous cold spot in the data from WMAP, proving that Murphy’s law applies to extreme events…

5 Comments »

Guest Post – Bayesian Book Review

Posted in Bad Statistics, Books, Talks and Reviews with tags , , , on May 30, 2011 by telescoper

My regular commenter Anton circulated this book review by email yesterday and it stimulated quite a lot of reaction. I haven’t read the book myself, but I thought it would be fun to post his review on here to see whether it provokes similar responses. You can find the book on Amazon here (UK) or here ( USA). If you’re not completely au fait with Bayesian probability and the controversy around it, you might try reading one of my earlier posts about it, e.g. this one. I hope I can persuade some of the email commenters to upload their contributions through the box below!

-0-

The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy

by Sharon Bertsch Mcgrayne

I found reading this book, which is a history of Bayes’ theorem written for the layman, to be deeply frustrating. The author does not really understand what probability IS – which is the key to all cogent writing on the subject. She never mentions the sum and product rules, or that Bayes’ theorem is an easy consequence of them. She notes, correctly, that Bayesian methods or something equivalent to them have been rediscovered advantageously again and again in an amazing variety of practical applications, and says that this is because they are pragmatically better than frequentist sampling theory – ie, she never asks the question: Why do they work better and what deeper rationale explains this? RT Cox is not mentioned. Ed Jaynes is mentioned only in passing as someone whose Bayesian fervour supposedly put people off.

The author is correct that computer applications have catalysed the Bayesian revolution, but in the pages on image processing and other general inverse problems (p218-21) she manages to miss the key work through the 1980s of Steve Gull and John Skilling, and you will not find “Maximum entropy” in the index. She does get the key role of Markov Chain Monte Carlo methods in computer implementation of Bayesian methods, however. But I can’t find Dave Mackay either, who deserves to be in the relevant section about modern applications.

On the other hand, as a historian of Bayesianism from Bayes himself to about 1960, she is full of superb anecdotes and information about
people who are to us merely names on the top of papers, or whose personalities are mentioned tantalisingly briefly in Jaynes’ writing.
For this material alone I recommend the book to Bayesians of our sort and am glad that I bought it.

7 Comments »

Shooting at the Cosmic Circles

Posted in Astrohype, Bad Statistics, The Universe and Stuff with tags , , , , on May 11, 2011 by telescoper

Another brief update post of something that whizzed past while I was away and thought I’d mention now that I’m back.

Remember the (now infamous) paper by Gurzadyan and Penrose about evidence for the Conformal Cyclic Cosmology that I blogged about last year?

The original analysis was comprehensively dissected and refuted by a number of papers within a few days of its appearance – see here, here and here – only for Gurzadyan and Penrose to dig an even bigger hole for themselves with a nonsensical reply.

Undaunted, the dynamic duo of Gurzadyan and Penrose have produced yet another paper on the same subject which came out just as I was heading off on my hols.

There has subsequently been another riposte, by Eriksen and Wehus, although I suspect most cosmologists ceased to care about this whole story some time ago. Although it’s a pretty easy target, the Eriksen-Wehus reply does another comprehensive demolition job. The phrase “shooting fish in a barrel” sprang to my mind, but from facebook I learned that the equivalent idiomatic expression in Italian is sparare sulla Croce Rossa (i.e. shooting on the Red Cross). Perhaps we can add a brand new phrase for “taking aim at an easy target” – shooting at the cosmic circles!

I was struck, however, by the closing sentences of the abstract of Eriksen-Wehus reply:

Still, while this story is of little physical interest, it may have some important implications in terms of scienctific sociology: Looking back at the background papers leading up to the present series by Gurzadyan and Penrose, in particular one introducing the Kolmogorov statistic, we believe one can find evidence that a community based and open access referee process may be more efficient at rejecting incorrect results and claims than a traditional journal based approach.

I wholeheartedly agree. I’ve blogged already to the effect that academic journals are a waste of time and money and we’d be much better off with open access and vigorous internet scrutiny. It may be that this episode has just given us a glimpse of the future of scientific publishing.

Share/Bookmark

11 Comments »

« Older Entries
Newer Entries »