In the Dark

Just time for a quickie today because tomorrow is the first day of teaching (in what we optimistically call the “Spring Semester”) and I’ve decided to head into the department this afternoon to prepare some handouts and concoct some appropriately fiendish examples for my first problem set.

I thought I’d take the opportunity to add a little postscript to some comments I made in a post earlier this week on the subject of misguided criticisms of science. Where I (sometimes) tend to agree with some such attacks is when they are aimed at scientists who have exaggerated levels of confidence in the certainty of their results. The point is that scientific results are always conditional, which is to say that they are of the form “IF we assume this theoretical framework and have accounted for all sources of error THEN we can say this”.

To give an example from my own field of cosmology we could say “IF we assume the general theory of relativity applies and the Universe is homogeneous and isotropic on large scales and we have dealt with all the instrumental uncertainties involved etc etc THEN 74% of the energy density in the Universe is in a form we don’t understand (i.e. dark energy).” We don’t know for sure that dark energy exists, although it’s a pretty solid inference, because it’s by no means certain that our assumptions – and there are a lot of them – are all correct.

Similar statements are made in the literature across the entire spectrum of science. We don’t deal with absolute truths, but always work within a given theoretical framework which we should always be aware might be wrong. Uncertainty also derives from measurement error and statistical noise. A scientist’s job is to deal with all these ifs buts and don’t-knows in as hard-nosed a way as possible.

The big problem is that, for a variety of reasons, many people out there don’t understand that this is the way science works. They think of science in terms of a collection of yes or no answers to well-posed questions, not the difficult and gradual process of gathering understanding from partial clues and (occasionally inspired) guesswork.

Why is this? There are several reasons. One is that our system of science education does not place sufficient emphasis on science-as-method as opposed to science-as-facts. Another is that the media don’t have time for scientists to explain the uncertainties. With only a two-minute slot on the news to explain cosmology to a viewer waiting for the football results all you can do is deliver a soundbite.
This is what I wrote in my book From Cosmos to Chaos:

Very few journalists or television producers know enough about science to report sensibly on the latest discoveries or controversies. As a result, important matters that the public needs to know about do not appear at all in the media, or if they do it is in such a garbled fashion that they do more harm than good. I have listened many times to radio interviews with scientists on the Today programme on BBC Radio 4. I even did such an interview once. It is a deeply frustrating experience. The scientist usually starts by explaining what the discovery is about in the way a scientist should, with careful statements of what is assumed, how the data is interpreted, and what other possible interpretations might be. The interviewer then loses patience and asks for a yes or no answer. The scientist tries to continue, but is badgered. Either the interview ends as a row, or the scientist ends up stating a grossly oversimplified version of the story.

Here’s another, more recent, example. A couple of weeks ago, a clutch of early release papers from the Planck satellite came out; I blogged about them here. Among these results were some interesting new insights concerning the nature of the Anomalous Microwave Emission (AME) from the Milky Way; the subject of an excellent presentation by Clive Dickinson at the conference where the results were announced.

The title of a story in National Geographic is typical of the coverage this result received:

Fastest Spinning Dust Found; Solves Cosmic “Fog” Puzzle

Now look at the actual result. The little bump in the middle is the contribution from the anomalous emission, and the curve underneath it shows the corresponding “spinning dust” model:

There’s certainly evidence that supports this interpretation, but it’s clearly nowhere near the level of “proof”. In fact, in Clive’s talk he stated the result as follows:

Plausible physical models appear to fit the data

OK, so that would never do for a headline in a popular magazine, but I hope I’ve made my point. There’s a big difference between what this particular scientist said and what was presented through the media.

I hope you’re not thinking that I’m criticising this bit of work. Having read the papers I think it’s excellent science.

But it’s not just the fault of the educationalists and the media. Certain scientists play this dangerous game themselves. Some enjoy their 15 minutes – or, more likely, two minutes – of fame so much that they will happily give the journalists what they want regardless of the consequences. Worse still, even in the refereed scientific literature you can find examples of scientists clearly overstating the confidence that should be placed in their results. We’re all human, of course, but my point is that a proper statement of the caveats is at least as much a part of good science as theoretical calculation, clever instrument design or accurate observation and experiment.

We can complain all we like about non-scientists making ill-informed criticisms of science, but we need to do a much better job at being honest about what little we really know and resist the temptation to be too certain.