In the Dark

I have a short gap in my schedule today so I thought I would use it to post a short note about the BICEP2 results announced to great excitement on Monday.

There has been a great deal of coverage in the popular media about a “Spectacular Cosmic Discovery” and this is mirrored by excitement at a more technical level about the theoretical implications of the BICEP2 results. Having taken a bit of time out last night to go through the discovery paper, I think I should say that I think all this excitement is very premature. In that respect I agree with the result of my straw poll.

First of all let me make it clear that the BICEP2 experiment is absolutely superb. It was designed and built by top-class scientists and has clearly functioned brilliantly to improve its sensitivity so much that it has gone so far ahead of so many rivals:

Notice that the only other detection of the elusive B-mode signal is by POLARBEAR, but that is actually accounted for by gravitational lensing effects rather than being evidence of a primordial gravitational wave contribution.

The B-mode signal is so weak that it is to mind absolutely amazing that an experiment can get anywhere near measuring it. There’s no denying the fact that BICEP2 team have done heroic work.

But – and it’s a big “but” – we have to ask the question “How confident can we be that the signal detected by BICEP2 is, in fact, the imprint of primordial gravitational waves on the cosmic microwave background that cosmologists were hoping for?”

The answer to this question will depend on the individual, but I would say that to convince me the absolute minimum would be a detection of the signal in more than one frequency band. A primordial signal should not vary as a function of frequency, whereas foreground emission (likely to be from dust) would be frequency dependent.

Now BICEP2 only operates at one frequency, 150GHz, so the experiment on its own can’t satisfy this criterion but it could through cross-correlation with the original BICEP1 instrument which worked at 100 GHz and 150 GHz. In the discovery paper we find the

Additionally, cross-correlating BICEP2 against 100GHz maps from the BICEP1 experiment, the excess signal is confirmed with 3sigma significance and its spectral index is found to be consistent with that of the CMB.

Here is the relevant plot, Figure 7 from the paper,

Well, the correct though the statement in the paper might be,  it is clear from this (rather ratty) cross-correlation that there is actually no firm detection of the B-modes at all at 100GHz. In other words, the 100 GHz BICEP1 data may be consistent with BICEP2 but they are also consistent with zero. (NOTE ADDED: I am ready to rescind this statement when I see a full analysis of these cross-correlations; at face value the scatter looks strange and certainly consistent with a null detection). In any case a positive cross-correlation does not exclude the possibility that the signal in common across the two channels is dust. If we only have a detection at one frequency we have no compelling evidence at all that the signal is cosmological.

When asked on Tuesday about this by Physics World I stated that I wasn’t convinced:

It seems to me though that there’s a significant possibility of some of the polarization signal in E and B [modes] not being cosmological. This is a very interesting result, but I’d prefer to reserve judgement until it is confirmed by other experiments. If it is genuine, then the spectrum is a bit strange and may indicate something added to the normal inflationary recipe.

My scepticism was then derived primarily from the distribution of the points around l=200 in the first figure: they look too high compared to the expected gravitational lensing contribution (which seems to have been pinned down by the POLARBEAR measurements to the right of the plot):

I’ve taken this plot from the post I reblogged yesterday. The errors in the measurements ringed in blue are probably correlated so the fact that all three lie well above the red curve may not be as significant as it first seems, but note that the vertical scale is logarithmic. If some sort of systematic error has indeed bumped these points up then the amount of power involved could easily account for all the signal in the points to the left; the fit to the primordial B-mode (red dashed) part of the curve could then be fortuitous.

One possible systematic, apart from foreground contamination by dust, is leakage between E and B modes in the spherical harmonic decomposition. This arises because the spherical harmonic modes are only orthogonal over a complete sphere; BICEP2 does not map the whole sky, so the modes get mixed and separating them becomes extremely messy. Since the E-mode signal is so much larger, the worry is that some of it might leak into the B-mode.

UPDATE: 20/3/2014

I noticed a post on the BICEP2 Facebook Page from Hans Kristian Eriksen pointing another oddity:

The above plot is one of many showing jackknife estimates relating to various aspects of the polarization signal. What is strange is that all the blue dots lie so close to zero. Statistically speaking this is extremely unlikely and it may suggest that the noise levels have been over-estimated underestimated; roughly one in three data points should be further away than one sigma from zero if sigma is estimated correctly.

Taking all this together I have to say that I stick to the point of view I took when I first saw the results. They are very  interesting, but it is far too earlier to even claim that they are cosmological, let alone to start talking about providing evidence for or against particular models of the early Universe. No doubt I’ll be criticized for trying to put a wet blanket over the whole affair, but this is a measurement of such potential importance that I think we have to set the bar very high indeed when it comes to evidence. If I were running a book on this, I would put it at no better than even money that this is a cosmological signal.

Of course the rush to embrace these results as “definitive proof” of something is a product of human nature and the general level of excitement this amazing experiment has generated. That’s entirely understandable and basically a very good thing. It reminds those of us working in cosmology how lucky we are that we work in a field in which such momentous discoveries do actually happen. This is no doubt why so many budding scientists are drawn into cosmology in the first place. Let’s not forget, however, that there is a thing called the scientific method and often after years of hard work there remain more questions than answers. For the time being, that’s where we are with gravitational waves.