The ‘Danish Paper’ and How Science Works
I’m off work today but couldn’t resist posting a very quick update on the controversial claims of inconsistencies in the recent detection of gravitational waves by LIGO.
If you’re following the story you will know that it started with a paper on the arXiv by Cresswell et al., a group mainly based in Denmark, which is why the paper is now frequently referred to as ‘The Danish Paper’ although its authors actually come from all round the world.
Well the same group have now written a rejoinder to the LIGO critique of their analysis. They’re clearly sticking to their guns, at least on their claim that the residuals left after removing the gravitational wave events from the two time series are correlated, which they should not be if they are simply noise.
Hopefully the public airing this controversy had received will lead to other independent groups downloading and analysing the data, which is all in the public domain, and we’ll eventually arrive at the truth.
Contrary to the opinion of one of my Cardiff colleagues I think this is how science works and, importantly, how it should be seen to work. Science is a process of investigation, and it doesn’t come to an end when results have been published in refereed journals.
The more the public see how science really works – warts and all – the better they will understand its strengths as well as its limitations.
Whatever the eventual outcome of this discussion I think we will find that the ‘Danish Paper’ has helped advance our understanding, and for that the authors deserve a great deal of credit.
Follow @telescoper
In the Dark 
June 27, 2017 at 4:10 pm
I don’t think they’re completely sticking to their guns, as their rejoinder makes no mention of their previous claim about the Fourier phases, which was shown to be wrong.
They do however stick to their guns on the question of whether correlations exist between the two cleaned (template-subtracted) signals. But the LIGO response by Ian Harry argued that whether or not such correlations exist doesn’t affect the detection claim.
June 27, 2017 at 4:22 pm
Yes, I’ve edited to clarify this.
It may indeed turn out not to affect the detection claims but it is nevertheless a property of the data that needs to be understood.
June 27, 2017 at 5:48 pm
It did strike me that the Danish paper was questioning only the accuracy of the time delay between the observations at the two detectors, and that the excellent match of the signal profile to one of the theoretical templates, together with the fact that both detectors saw it, was decisive evidence that it was gravitational radiation. All that the Danish paper therefore throws into renewed question is what part of the sky it came from (which anyway can’t be answered exactly with only two detectors even if the time delay is known with certainty).
June 28, 2017 at 12:59 pm
The simplest explanation for the observed correlations is just that the model for the signal is not perfect which means that when it is subtracted there is a residual present in both time series.
This is perfectly possible – indeed, likely – because the model templates are derived from a grid constructed using simulations using different model parameters.
I’m not sure however whether and how this information is used in determining the posterior distribution of model parameters.
June 28, 2017 at 5:33 pm
That sounds like a crap way to do the data analysis! Subtract (ie, add the negative of) the wrong amplitude of the model and you will have superlative correlations! You have to estimate the amplitude somehow (and other parameters, of course, such as the masses of the merging black holes). Has to be Bayesian…
June 28, 2017 at 6:36 pm
You can use the LIGO data to get a posterior probability defined over the parameters of the models in a well-defined way, but you will inevitably find that the `best fit’ model leaves residuals when it is subtracted.
A Bayesian approach of this sort would give a probability but would assume that the set of templates is exhaustive. I guess looking at the residuals is a kind of sanity check to see if there is any evidence that even the best-fitting template(s) can’t fit the data.
June 28, 2017 at 8:16 pm
It’s an Ockham analysis. Suppose temporarily that there is only one detector. You throw into the ring (1) zero signal vs (2) black hole merger with parameters unknown, and calculate the probabilities of the two hypotheses, giving a prior to and marginalising over the parameters. Supposing that (2) is the clear winner, you then do parameter estimation.
That should settle whether gravitational waves really have been detected. That the second detector saw something of similar profile at almost exactly the same time obviously firms it a good deal, and you can make that formal too with more work.
June 28, 2017 at 8:20 pm
I don’t think there’s any question that there’s been a detection. The question is whether there’s an effect on the parameter estimates..
June 28, 2017 at 9:01 pm
I agree. But the Danish Abstract ended, “A clear distinction between signal and noise therefore remains to be established in order to determine the contribution of gravitational waves to the detected signals.” And the paper itself concluded, “The purpose in having two independent detectors is precisely to ensure that, after sufficient cleaning, the only genuine correlations between them will be due to gravitational wave effects. The results presented here suggest this level of cleaning has not yet been obtained and that the detection of the GW events needs to be re-evaluated with more careful consideration of noise properties.” If nobody is suggesting that gravitational waves have not been detected then these statements written as punchlines are, frankly, misleading.
June 29, 2017 at 4:30 am
The problem here is you cannot “throw into the ring zero signal”.
I think that’s why you need a third detector to characterize the correlated noise between the other two detectors. Unless you can identify all possible sources of noise and distinguish it from a GW signal to establish a clear signal to noise ratio.
I still think Ligo measured GWs though.
June 29, 2017 at 5:42 am
Peter: What you say is possible. However this is in conflict with the claim in the tests of GR paper arxiv:1602.03841 in the section titled “Residuals after subtracting the most-probable waveform model. ” It says “Our analysis reveals that the GW150914 residual favors the instrumental noise hypothesis over the presence of a coherent signal as well as the presence of glitches in either detectors;” It maybe that in this paper a more sensitive analysis has been done and traces of residual signal has been found.
June 27, 2017 at 7:12 pm
One thing I would like really to see is if there are similar peaks in cross-correlation (after the 7 ms offset) in data not containing the signal. AFAIk, the Danes haven’t shown this in their paper.
June 28, 2017 at 6:07 am
Also Harry hasn’t addressed the issue of correlations in the calibration lines between the two detectors after shifting the data by 7 milliseconds. Of course this could also be due to some trivial mistake or an artifact of how they have band-pass filtered the data/
June 28, 2017 at 11:47 am
In the LIGO response, Harry said
“Nevertheless, if such a correlation were present it would suggest that we have not perfectly subtracted the real signal from the data, which would not invalidate any detection claim. There could be any number of reasons for this, for example the fact that our best-fit waveform will not exactly match what is in the data as we cannot measure all parameters with infinite precision. There might also be some deviations because the waveform models we used, while very good, are only approximations to the real signal (LIGO put out a paper quantifying this possibility). Such deviations might also be indicative of a subtle deviation from general relativity. These are of course things that LIGO is very interested in pursuing, and we have published a paper exploring potential deviations from general relativity (finding no evidence for that), which includes looking for a residual signal in the data after subtraction of the waveform (and again finding no evidence for that).”
This seems to directly answer your point.
June 28, 2017 at 11:49 am
Oops, sorry – I see you are talking about correlation of the calibration lines, so the following quote is more relevant: “The claimed correlations between the two detectors due to resonance and calibration lines in the data would be present also in the time-shifted analyses—The calibration lines are repetitive lines, and so if correlated in the non-time shift analyses, they will also be correlated in the time-shift analyses as well.”
June 28, 2017 at 2:29 pm
Yes, but again not obvious to me why the correlation is maximized at 6.9 milliseconds. Maybe there is some residual signal present when they band-passed the data around the calibration lines?
June 28, 2017 at 12:51 pm
Absolutely, this is science working as it should!
August 1, 2017 at 8:10 pm
[…] from LIGO and there’s been a lot of serious – but good-natured – discussion of `The Danish Paper‘ that came out some time ago and which questioned some aspects of the data analysis of the […]
August 8, 2017 at 1:10 pm
[…] Bohr Institute between some members of the LIGO Scientific Collaboration and the authors of the `Danish Paper‘. As with the other one I attended last week it was both interesting and informative. […]
August 10, 2017 at 4:22 pm
[…] Niels Bohr Institute and of the LIGO scientific collaboration concerning matters arising from the `Danish Paper‘. The most prominent among these appears to be the LIGO team and the Danish team have agreed […]
July 11, 2018 at 11:06 pm
Residual correlations, phase-locked noise and indistinguishable models are to be the least of concerns for LIGO in the future given the scope of excess correlations found between LIGO event parameters and environmental background variations, plausibly driven by orbital resonances in the interplanetary magnetic field through coronal hole dynamics.
Intermittency and strong solar wind-IMF epsilon coupling are common to all seven GW triggers, prominent in ground magnetometer field data, and in the coordinated almost-periodic global CG lightning triggering for periods identical to noise-lag correlations in LIGO strain.
Even underlying spectral components (ordinary broadband, upchirped, log-normal ELF) in pre-whitened LIGO transient time series are related by a single scaling constant and show amplitude and partition related by the same cutoffs as expected for signal for thunderstorms in the same topographic-ionospheric waveguide.