In the Dark

I was thinking just the other day that I haven’t posted much in either the Astrohype or the Bad Statistics folders on this blog. Well today I found an item that belongs in both categories. Many people will have seen the widespread press coverage of a misleading claim of the discovery of alien life; see, e.g., here. This misleading press coverage is based on a misleading press release from the University of Cambridge which you can find here.

The story is based on a paper in the pay-to-publish Astrophysical Journal Letters with the title “New Constraints on DMS and DMDS in the Atmosphere of K2-18 b from JWST MIRI“. The DMS and DMDS in the title refer to Dimethyl Sulphide and Dimethyl Disulphide respectively. These are interpreted by the authors as biosignatures.

There are two main problems with this claim. One is that DMS and DMDS are not necessarily biosignatures in the first place; see here for the reasons. The other is that there isn’t even any evidence for the detection of DMS or DMDS anyway. Here is the spectrum of which the lead author of the paper, Prof. Nikku Madhusudhan, has claimed “the signal came through loud and clear”.

Yeah, right. In statistical terms this is a non-detection. The Bayes Factor used in the paper to quantify the evidence for a model with DMS and/or DMDS over one without is just 2.62 in the logarithm. That’s not a detection by any stretch of the imagination; to be anywhere near convincing a Bayes Factor has to be at least 100. The subsequent cherry-picking of the data to improve the apparent probability of a detection is just statistical flummery.

Notice that the use of the phrase “Constraints on” in the title of the paper does not indicate that the article presents evidence that a detection has been made. That the claim has somehow morphed into the “the strongest evidence for life beyond our solar system” is absurd. The most charitable thing I can say is that Prof. Madhusudhan must have been carried away by enthusiasm. This doesn’t reflect very well on Cambridge University either.

This episode worries me greatly. This is a time of increasing hostility towards science and this sort of thing can only make matters worse. Scientists need to be much more careful in communicating the uncertainties in their results.

UPDATE: There’s a now paper on arXiv here that argues that a straight line is a better fit to the data, in other words that there is no strong statistical evidence for spectral features at all.

This entry was posted on April 17, 2025 at 8:12 pm and is filed under Astrohype, Bad Statistics, The Universe and Stuff with tags Bayes Factor, Biosignatures, Dimethyl Disulphide, Dimethyl Sulphide, Prof. Nikku Madhusudhan. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.