In the Dark

The other day, via Twitter, I came across an interesting blog post about the relatively recent resurgence of Bayesian reasoning in science. That piece had triggered a discussion about why cosmologists seem to be largely Bayesian in outlook, so I thought I’d share a few thoughts about that. You can find a lot of posts about various aspects of Bayesian reasoning on this blog, e.g. here.

When I was an undergraduate student I didn’t think very much about statistics at all, so when I started my DPhil studies I realized I had a great deal to learn. However, at least to start with, I mainly used frequentist methods. Looking back I think that’s probably because I was working on cosmic microwave background statistics and we didn’t really have any data back in 1985. Or actually we had data, but no firm detections. I was therefore taking models and calculating things in what I would call the forward direction, indicated by the up arrow. What I was trying to do was find statistical descriptors that looked likely to be able to discriminate between different models but I didn’t have the data.

Once measurements started to become available the inverse-reasoning part of the diagram indicated by the downward arrow came to the fore. It was only then that it started to become necessary to make firm statements about which models were favoured by the data and which weren’t. That is what Bayesian methods do best, especially when you have to combine different data sets.

By the early 1990s I was pretty much a confirmed Bayesian – as were quite a few fellow theorists -but I noticed that most observational cosmologists I knew were confirmed frequentists. I put that down to the fact that they preferred to think in “measurement space” rather than “theory space”, the latter requiring the inductive step furnished by Bayesian reasoning indicated by the downward arrow. As cosmology has evolved the separation between theorists and observers in some areas – especially CMB studies – has all but vanished and there’s a huge activity at the interface between theory and measurement.

But my first exposure to Bayesian reasoning came long before that change. I wasn’t aware of its usefulness until 1987, when I returned to Cambridge for a conference called The Post-Recombination Universe organized by Nick Kaiser and Anthony Lasenby. There was an interesting discussion in one session about how to properly state the upper limit on CMB fluctuations arising from a particular experiment, which had been given incorrectly in a paper using a frequentist argument. During the discussion, Nick described Anthony as a “Born-again Bayesian”, a phrase that stuck in my memory though I’m still not sure whether or not it was meant as an insult.

It may be the case for many people that a relatively simple example convinces them of the superiority of a particular method or approach. I had previously found statistical methods – especially frequentist hypothesis-testing – muddled and confusing, but once I’d figured out what Bayesian reasoning was I found it logically compelling. It’s not always easy to do a Bayesian analysis for reasons discussed in the paper to which I linked above, but it least you have a clear idea in your mind what question it is that you are trying to answer!

Anyway, it was only later that I became aware that there were many researchers who had been at Cambridge while I was there as a student who knew all about Bayesian methods: people such as Steve Gull, John Skilling, Mike Hobson, Anthony Lasenby and, of course, one Anthony Garrett. It was only later in my career that I actually got to talk to any of them about any of it!

So I think the resurgence of Bayesian ideas in cosmology owes a very great deal to the Cambridge group which had the expertise necessary to exploit the wave of high quality data that started to come in during the 1990s and the availability of the computing resources needed to handle it.

But looking a bit further back I think there’s an important Cambridge (but not cosmological) figure that preceded them, Sir Harold Jeffreys whose book The Theory of Probability was first published in 1939. I think that book began to turn the tide, and it still makes for interesting reading.

P.S. I have to say I’ve come across more than one scientist who has argued that you can’t apply statistical reasoning in cosmology because there is only one Universe and you can’t use probability theory for unique events. That erroneous point of view has led to many otherwise sensible people embracing the idea of a multiverse, but that’s the subject for another rant.

Follow @telescoper