Archive for the The Universe and Stuff Category

Straw Poll on Statistical Computing

Posted in Bad Statistics, The Universe and Stuff with tags , , on December 20, 2016 by telescoper

The abstract of my previous (reblogged) post claims that R is “the premier language of statistical computing”. That may be true for the wider world of statistics, and I like R very much, but in my experience astronomers and cosmologists are much more likely to do their coding in Python.  It’s certainly the case that astronomers and physicists are much more likely to be taught Python than R. There may well even be some oldies out there still using other languages like Fortran, or perhaps  relying on books of statistical tables!

Out of interest therefore I’ve decided to run the following totally biased and statistically meaningless poll of my immense readership:

 

If you choose “something else”, please let me know through the comments box what your alternative is. I can then add additional options.

 

Everything we'd like to do with LSST data, but we don't know (yet) how [IMA]

Posted in The Universe and Stuff with tags , on December 16, 2016 by telescoper

Here’s a nice little summary paper (via arXiver) covering the data challenges posed by the Large Synoptic Survey Telescope (LSST) which will come online in 2020 or thereabouts.

To give a bit of a perspective, when the Sloan Digital Sky Survey (SDSS) began to collect astronomical data in 2000, it amassed more in its first few weeks than all data collected in the entire history of astronomy to that date.

Continuing at a rate of about 200 GB per night, SDSS has subsequently amassed more than 140 terabytes of information.

When LSST, which is in many ways a successor to SDSS, comes online it is expected to acquire that amount of data every five days…

arxiver's avatararXiver

http://arxiv.org/abs/1612.04772

The Large Synoptic Survey Telescope (LSST), the next-generation optical imaging survey sited at Cerro Pachon in Chile, will provide an unprecedented database of astronomical measurements. The LSST design, with an 8.4m (6.7m effective) primary mirror, a 9.6 sq. deg. field of view, and a 3.2 Gigapixel camera, will allow about 10,000 sq. deg. of sky to be covered twice per night, every three to four nights on average, with typical 5-sigma depth for point sources of $r$=24.5 (AB). With over 800 observations in $ugrizy$ bands over a 10-year period, these data will enable a deep stack reaching $r$=27.5 (about 5 magnitudes deeper than SDSS) and faint time-domain astronomy. The measured properties of newly discovered and known astrometric and photometric transients will be publicly reported within 60 sec after observation. The vast database of about 30 trillion observations of 40 billion objects will be mined for the unexpected and used…

View original post 61 more words

LIGO Echoes, P-values and the False Discovery Rate

Posted in Astrohype, Bad Statistics, The Universe and Stuff with tags , , , , on December 12, 2016 by telescoper

Today is our staff Christmas lunch so I thought I’d get into the spirit by posting a grumbly article about a paper I found on the arXiv. In fact I came to this piece via a News item in Nature. Anyway, here is the abstract of the paper – which hasn’t been refereed yet:

In classical General Relativity (GR), an observer falling into an astrophysical black hole is not expected to experience anything dramatic as she crosses the event horizon. However, tentative resolutions to problems in quantum gravity, such as the cosmological constant problem, or the black hole information paradox, invoke significant departures from classicality in the vicinity of the horizon. It was recently pointed out that such near-horizon structures can lead to late-time echoes in the black hole merger gravitational wave signals that are otherwise indistinguishable from GR. We search for observational signatures of these echoes in the gravitational wave data released by advanced Laser Interferometer Gravitational-Wave Observatory (LIGO), following the three black hole merger events GW150914, GW151226, and LVT151012. In particular, we look for repeating damped echoes with time-delays of 8MlogM (+spin corrections, in Planck units), corresponding to Planck-scale departures from GR near their respective horizons. Accounting for the “look elsewhere” effect due to uncertainty in the echo template, we find tentative evidence for Planck-scale structure near black hole horizons at 2.9σ significance level (corresponding to false detection probability of 1 in 270). Future data releases from LIGO collaboration, along with more physical echo templates, will definitively confirm (or rule out) this finding, providing possible empirical evidence for alternatives to classical black holes, such as in firewall or fuzzball paradigms.

I’ve highlighted some of the text in bold. I’ve highlighted this because as written its wrong.

I’ve blogged many times before about this type of thing. The “significance level” quoted corresponds to a “p-value” of 0.0037 (or about 1/270). If I had my way we’d ban p-values and significance levels altogether because they are so often presented in a misleading fashion, as it is here.

What is wrong is that the significance level is not the same as the false detection probability.  While it is usually the case that the false detection probability (which is often called the false discovery rate) will decrease the lower your p-value is, these two quantities are not the same thing at all. Usually the false detection probability is much higher than the p-value. The physicist John Bahcall summed this up when he said, based on his experience, “about half of all 3σ  detections are false”. You can find a nice (and relatively simple) explanation of why this is the case here (which includes various references that are worth reading), but basically it’s because the p-value relates to the probability of seeing a signal at least as large as that observed under a null hypothesis (e.g.  detector noise) but says nothing directly about the probability of it being produced by an actual signal. To answer this latter question properly one really needs to use a Bayesian approach, but if you’re not keen on that I refer you to this (from David Colquhoun’s blog):

One problem with all of the approaches mentioned above was the need to guess at the prevalence of real effects (that’s what a Bayesian would call the prior probability). James Berger and colleagues (Sellke et al., 2001) have proposed a way round this problem by looking at all possible prior distributions and so coming up with a minimum false discovery rate that holds universally. The conclusions are much the same as before. If you claim to have found an effects whenever you observe a P value just less than 0.05, you will come to the wrong conclusion in at least 29% of the tests that you do. If, on the other hand, you use P = 0.001, you’ll be wrong in only 1.8% of cases.

Of course the actual false detection probability can be much higher than these limits, but they provide a useful rule of thumb,

To be fair the Nature item puts it more accurately:

The echoes could be a statistical fluke, and if random noise is behind the patterns, says Afshordi, then the chance of seeing such echoes is about 1 in 270, or 2.9 sigma. To be sure that they are not noise, such echoes will have to be spotted in future black-hole mergers. “The good thing is that new LIGO data with improved sensitivity will be coming in, so we should be able to confirm this or rule it out within the next two years.

Unfortunately, however, the LIGO background noise is rather complicated so it’s not even clear to me that this calculation based on “random noise”  is meaningful anyway.

The idea that the authors are trying to test is of course interesting, but it needs a more rigorous approach before any evidence (even “tentative” can be claimed). This is rather reminiscent of the problems interpreting apparent “anomalies” in the Cosmic Microwave Background, which is something I’ve been interested in over the years.

In summary, I’m not convinced. Merry Christmas.

 

 

Galaxy Formation in the EAGLE Project

Posted in The Universe and Stuff with tags , , , on December 8, 2016 by telescoper

Yesterday I went to a nice Colloquium by Rob Crain of Liverpool John Moores University (which is in the Midlands). Here’s the abstract of his talk which was entitled
Cosmological hydrodynamical simulations of the galaxy population:

I will briefly recap the motivation for, and progress towards, numerical modelling of the formation and evolution of the galaxy population – from cosmological initial conditions at early epochs through to the present day. I will introduce the EAGLE project, a flagship program of such simulations conducted by the Virgo Consortium. These simulations represent a major development in the discipline, since they are the first to broadly reproduce the key properties of the evolving galaxy population, and do so using energetically-feasible feedback mechanisms. I shall present a broad range of results from analyses of the EAGLE simulation, concerning the evolution of galaxy masses, their luminosities and colours, and their atomic and molecular gas content, to convey some of the strengths and limitations of the current generation of numerical models.

I added the link to the EAGLE project so you can find more information. As one of the oldies in the audience I can’t help remembering the old days of the galaxy formation simulation game. When I started my PhD back in 1985 the state of the art was a gravity-only simulation of 323 particles in a box. Nowadays one can manage about 20003 particles at the same time aas having a good go at dealing not only with gravity but also the complex hydrodynamical processes involved in assembling a galaxy of stars, gas, dust and dark matter from a set of primordial fluctuations present in the early Universe. In these modern simulations one does not just track the mass distribution but also various themrmodynamic properties such as temperature, pressure, internal energy and entropy, which means that they require large supercomputers. This certainly isn’t a solved problem – different groups get results that differ by an order of magnitude in some key predictions – but the game has certainly moved on dramatically in the past thirty years or so.

Another thing that has certainly improved a lot is data visualization: here is a video of one of the EAGLE simulations, showing a region of the Universe about 25 MegaParsecs across. The gas is colour-coded for temperature. As the simulation evolves you can see the gas first condense into the filaments of the Cosmic Web, thereafter forming denser knots in which stars form and become galaxies, experiencing in some cases explosive events which expel the gas. It’s quite a messy business, which is why one has to do these things numerically rather than analytically, but it’s certainly fun to watch!

Absorbed in a Quasar Spectrum

Posted in The Universe and Stuff with tags , , , , , , on December 5, 2016 by telescoper

Many people seem to think that astronomers spend all their time looking at pretty pictures of stars and galaxies. Actually a large part of observational astronomy isn’t about making images of things but doing spectroscopy. In fact the rise of astronomical spectroscopy is what turned astronomy into astrophysics. But that’s not to say that spectra can’t be pretty either. Here is an example (from here) which shows the light from the quasar HE0940-1050 taken by the UVES instrument mounted on ESO’s Very Large Telescope in Chile.

This quasar is an interesting object, at a redshift of z= 3.0932 (which converts to a look-back time of about 11.6 billion years). The dark bands and lines you can see in the spectrum are caused by absorption of the light from the quasar by clouds of hydrogen gas between the quasar and the observer; the strength of the absorption indicates how much gas the light from the quasar has travelled through.  The absorption occurs at a particular wavelength corresponding to the Lyman-α transition but, because the clouds are all at different redshifts, each produces a line at a different observed wavelength in the quasar spectrum. There are many lines, which is why the collection of clouds responsible for them is often called the Lyman-α Forest. In effect the quasar sample is very much like a core sample, as if we were able to drill back in time to the quasar through the material that lies along the line of sight.

This spectrum is particularly remarkable because of the number of faint lines that can be seen: it’s like a detailed DNA Fingerprint of cosmic structure. It’s also very pretty.

 

Nailing Cosmological Jelly to the Wall

Posted in The Universe and Stuff with tags , , on November 28, 2016 by telescoper

When asked to provide comments for a recent piece about cosmology in New Scientist, all I could come up with was the quote in the following excerpt:

But no measurement will rule out inflation entirely, because it doesn’t make specific predictions. “There is a huge space of possible inflationary theories, which makes testing the basic idea very difficult,” says Peter Coles at Cardiff University, UK. “It’s like nailing jelly to the wall.”

Certain of my colleagues have cast doubt on whether I am qualified to comment on the nailing of jelly to the wall, so I feel obliged to share the results of my highly successful research into this in the form of the following photograph:
orange_jelly_nailed_to_wall

I regret that I was unable to find any Dark Jelly, so had to settle for the more familiar baryonic type. Also, for the record, I should point out that what is shown is actually jelly concentrate. A similar experiment with the more normal diluted form of jelly was somewhat less successful.

I hope this clarifies the situation.

Infinite LIGO Dreams

Posted in Art, The Universe and Stuff with tags , , , , , on November 28, 2016 by telescoper

There was a special event in the School of Physics & Astronomy here at Cardiff University on Friday afternoon – the unveiling of a new work of art in our coffee area. The work, a large oil painting, called Infinite LIGO Dreams by local artist Penelope Rose Cowley was inspired by the detection of gravitational waves earlier this year:

 

gravitational-wave-artwork-copyright-penelope-cowley-16x9

You can read more about this work, and the circumstances behind its creation, at the Cardiff University website and via the Physics World blog. If you like the piece you can order a poster-sized print from Penelope Cowleys’s own website here.

The unveiling of this artwork was preceded by a drinks reception, which probably accounts for the errors that crept into the blog post I wrote on Friday after the party!

 

R.I.P. John M Stewart (1943-2016)

Posted in Biographical, Education, The Universe and Stuff with tags , , , on November 23, 2016 by telescoper

john-stewartI was very sad this morning to hear of the death of distinguished mathematical physicist Dr John M. Stewart (left). Apart from a few years in Munich in the 1970s John Stewart spent most of his working life in Cambridge, having studied there as an undergraduate and postgraduate and then returning from his spell at the Max Planck Institute to the Department of Applied Mathematics and Theoretical Physics for forty years.

John’s research mostly concerned relativistic fluid dynamics. Indeed, he was one of the pioneers of numerical relativity in the United Kingdom, and he applied his knowledge to a number of problems in early Universe cosmology and structure formation. I think it is fair to say that he wasn’t the most prolific researcher in terms of publications, which is perhaps why he only got promoted to Senior Lecturer in 2000 and never made it to a Chair, retiring as Reader in Gravitational Physics in 2010. However, his work was always of a very high technical standard and presented with great clarity and he was held in a very high regard by those who knew him and worked with him.

The tributes paid to John Stewart by King’s College (of which he was a Life Fellow) here and his colleagues in the Centre for Theoretical Cosmology here give a detailed account of his research achievements, so I refer you to them for more information about that aspect of his career.

I just wanted to add a personal note not about John Stewart’s research, but about something else mentioned in the obituaries linked to above: his teaching. I was fortunate enough to have him as a lecturer when I was studying Natural Sciences at Cambridge during the early 1980s. In the second year (Part IB) I specialised in Physics and Mathematics, and John taught part of the Mathematics syllabus. He was an absolutely superb teacher. For a start he was superbly well organized and had clearly thought very deeply about how best to present some quite difficult material. But it wasn’t just that. He projected a very engaging personality, with nice touches of humour, that made him easy to listen. His lectures were also very well paced for taking notes. In fact he was one of the few lecturers I had whose material I didn’t have to transcribe into a neat form from rough notes.

I have kept all the notes from that course for over thirty years. Here are a couple of pages as an example:

wp-1479909754508.jpeg

Anyone who has ever seen my handwriting will know that this is about as neat as I ever get!

When I was called upon to teach similar material at Cardiff and Sussex I drew on them heavily, so anyone who has learned anything from me about complex analysis, contour integration, Green’s functions and a host of other things actually owes a huge debt to John Stewart. Anything they didn’t understand was of course my fault, not his..

I also remember that John came to Queen Mary to give a seminar when I worked there in the early 90s as a postdoc. I was still a bit in awe of him because of my experience of him in Cambridge. His talk was about a method for handling the evolution of cosmological matter perturbations based on an approach based on the Hamilton-Jacobi formalism. His visit was timely, as I’d been struggling to understand the papers that had been coming out at the time on this topic. In the bar after his talk I plucked up the courage to explain to him what it was that I was struggling to understand. He saw immediately where I was going wrong and put me right on my misconceptions straight away, plucking a simple illustrative example apparently out of thin air. I was deeply impressed, not only by his ability to identify the issue but also with his friendly and helpful demeanour.

Rest in Peace, Dr John M. Stewart (1943-2016).

Dunes on Mars

Posted in Art, The Universe and Stuff with tags , , , , on November 21, 2016 by telescoper

dunes

This isn’t a new picture, but I hadn’t seen it before a friend put in on their Facebook page at the weekend. It isn’t what I first thought it was – a wonderful piece of abstract art – but is, in fact, an equally wonderful photograph of the inside of the Bunge crater on Mars, where a complex pattern of dunes has formed through wind action. The area covered by the image is about 14 kilometers wide.

According to the official NASA webpage: “This image was taken in January 2006 by the Thermal Emission Imaging System instrument on NASA’s Mars Odyssey orbiter and posted in a special December 2010 set marking the occasion of Odyssey becoming the longest-working Mars spacecraft in history.”

Magnets, Data Science and the Intelligent Pig

Posted in Biographical, The Universe and Stuff with tags , , , , , on November 18, 2016 by telescoper

The other day I was talking to some colleagues in the pub (as one does). At one point the subject of conversation turned to the pressure we academics are under these days to collaborate more with the world of industry and commerce. That’s one of the things that the Cardiff University Data Innovation Research Institute – which currently pays half my wages  – is supposed to do, but there was general consternation when I mentioned that I have in the past spent quite a long time working in industry. I am, after all, Professor of Theoretical Astrophysics. Of what possible interest could that be to industry?

My time in industry was spent at one of the research stations of British Gas, called the On-Line Inspection Centre (“OLIC”) which was situated in Cramlington, Northumberland. I started work there in 1981, just after I’d finished my A-levels and the Cambridge Entrance Examination and I worked there for about 9 months, before leaving to start my undergraduate course in 1982. At that time British Gas was still state-owned, and one of the consequences of that was that I had to sign the Official Secrets Act when I joined the staff. Among other things that forbade me from making “unauthorized disclosures” of what I was working on for thirty years. I feel comfortable discussing that work now, partly because the thirty years passed some time ago and partly because OLIC no longer exists. I’m not sure exactly what happened to it, but I presume it got flogged off on the cheap when British Gas was privatized during the Thatcher regime.

The main activity of the On-Line Inspection Centre was developing and exploiting techniques for inspecting gas pipelines for various forms of faults. The UK’s gas transmission network comprises thousands of kilometres of pipelines, made from steel in sections joined together by seam welds. I always thought of it as like a road network: the motorways which were made of 36″ diameter pipes; the A-roads were of smaller, 24″, diameter; and the minor roads were generally made of 12″ pipes. It’s interesting that despite the many failings of my memory now that I’ve reached middle age, I can still remember the names of some of the routes: “Huddersfield to Hopton Top” and “Seabank to Frampton Cotterell” spring immediately to mind.

Anyway, as part of the Mathematics Group at OLIC my job was to work on algorithms to analyse data from various magnetic inspection vehicles. These vehicles – known as “pigs” – were of different sizes to fit snugly  in the various pipes. The term “pig” had originally been applied to simple devices used to clean the gunk from inside of a pipe. They were just put in one end of the line and  gas pressure would push them all the way to the other end, often tens of kilometres away. The pipeline could thus be cleaned without taking it out of service.

This basic idea was modified to produce the much more sophisticated “intelligent pig” which produced the data I worked on. You can read much more about this here. This looked very similar to the cleaning pig, but had a complicated assembly of magnets and sensors, shown schematically here:

pig

The two sets  of magnets are connected to the pipe wall by steel brushes to maintain good contact. The magnetic field applied by the front set of magnets is contained within the pipe wall forms a kind of circuit with the rear set as shown, unless there is a variation in the thickness of the material. In that case magnetic flux leaks out and is detected by the sensors. The magnets and sensors are deployed in rings to cover the whole circumference of the pipe. A 24″ diameter pig would have 240 sensors, each recorded as a separate channel on the vehicle.

The actual system is fairly complicated so some of the work was experimental. Sections of pipe were made with defects of various sizes machined into them. The pig would then be pulled through these sections and the signals studied to build up an understanding of how the magnetic field would respond in different situations.

The actual pig (which could be several metres long and weighing a couple of tonnes) looks like this:

pig2

I always thought they looked a bit like spacecraft.

The pig usually travels at something like walking pace along the pipeline, and the sampling rate of the sensors was such that a reading would be taken every few millimetres. That sampling rate was necessary because corrosion pits as small as 1cm across could be dangerous.  The larger vehicles had “on-board thresholding” so that recordings of quiescent sections were discarded. Even so pipe surfaces (especially those of smaller bore) could be uneven for various reasons to do with their production rather than the effects of corrosion. Moreover, every few metres there would be a circumferential seam weld where two sections of pipe were joined together; these features would produce a large signal on all channels which the thresholding algorithm did not suppress.  The net result was that a lot of data had to be stored on the vehicle. When I say “a lot”, I mean for that time. A full run might produce about 5 × 107 readings. That seems like nothing now, but it was “Big Data” in those days!

So how was all this data processed back at the station? You probably won’t believe this, but it was printed out on Versatec printers in the form of a chart recording for each channel. Operators then identified funny-looking signals by eye and we then pulled down the data from tape and had a further look, usually comparing the patterns visually with those obtained from “pull-through” experiments.

Among the things I worked on was an algorithm to recognize seam weld signals automatically. That was quite easy actually – because it just requires looking for simultaneous activity on all channels – although it had to be made robust enough to deal with the odd dead channel and other instrumental glitches. This algorithm proved to be useful because sometimes the on-board telemetry would go wrong and we had to locate the pig by counting the number of welds it had passed since the start of the run.

A far more difficult challenge was dealing with data from 12″ diameter pipe. These are manufactured in a way that’s completely different from that used to make pipes of larger diameter, which are made of rolled steel. The 12″ pipes were made from a solid plug of molten steel, the centre of which is bored out by a device that rotates as it goes along. The effect of this is that it imposes a peculiar form of variation on the pipe wall, in the form of a spirally modulated “noise”. Annoyingly, the pitch and amplitude of the spiral varied from one section of pipe to another. After many failed attempts, the group finally came up with an algorithm that used the weld detector as a starting point to establish the vehicle had entered a new section of pipe. It then used data from the start of each section to estimate the parameters of the spiral pattern for that section, and then applied a filter to remove it from the rest of the section. It wasn’t particularly elegant, but it certainly cleaned up the data massively and made it much easier to spot significant features.

You might ask why I’ve written at such length about this when it’s got nothing to do with my current research (or indeed, anything else I’ve done since I graduated from Cambridge in 1985). One reason is that, although I didn’t know it at the time, my time at OLIC was going to prepare me very well for when I started my PhD. That was the case because all the programming I did used VAX computers, which turned out to be the computers used by STARLINK.  When I started my life as a research student I was already fluent in the command language (DCL) as well as the database software DATATRIEVE, which was a great advantage. Another reason is that working in this environment I had to learn to make my code (which, incidentally, was all in Fortran-77) conform to various very strict standards. I didn’t like some of the things we were forced to do, but I was shouted at sufficiently often that I gave up and did what I was told. I have never been particularly good at doing that in general, but in the context of software it is a lesson I’m glad I learned. Above all, though, I think working outside academia gave me a different perspective on research.  As academics were are very lucky to be able- at least some of the time – to choose our own research problems, but I believe that in the long run it can be very for your intellectual development to do something completely different every now and then.

We’re currently discussing a scheme whereby Physics and Astrophysics research students can interrupt their PhD for up to 6 months to undertake a (paid) work placement outside academia. I suspect many graduate students will not be keen on this, as they’ll see it as a distraction from their PhD topic, but I think it has many potential advantages as I hope I’ve explained.