Author Archive

On the Edgeworth Series…

Posted in Bad Statistics, The Universe and Stuff with tags , , on September 12, 2017 by telescoper

There’s a nice paper on the arXiv today by Elena Sellentin, Andrew Jaffe and Alan Heavens about the use of the Edgeworth series in statistical cosmology; it is evidently the first in a series about the Edgeworth series.

Here is the abstract:

Non-linear gravitational collapse introduces non-Gaussian statistics into the matter fields of the late Universe. As the large-scale structure is the target of current and future observational campaigns, one would ideally like to have the full probability density function of these non-Gaussian fields. The only viable way we see to achieve this analytically, at least approximately and in the near future, is via the Edgeworth expansion. We hence rederive this expansion for Fourier modes of non-Gaussian fields and then continue by putting it into a wider statistical context than previously done. We show that in its original form, the Edgeworth expansion only works if the non-Gaussian signal is averaged away. This is counterproductive, since we target the parameter-dependent non-Gaussianities as a signal of interest. We hence alter the analysis at the decisive step and now provide a roadmap towards a controlled and unadulterated analysis of non-Gaussianities in structure formation (with the Edgeworth expansion). Our central result is that, although the Edgeworth expansion has pathological properties, these can be predicted and avoided in a careful manner. We also show that, despite the non-Gaussianity coupling all modes, the Edgeworth series may be applied to any desired subset of modes, since this is equivalent (to the level of the approximation) to marginalising over the exlcuded modes. In this first paper of a series, we restrict ourselves to the sampling properties of the Edgeworth expansion, i.e.~how faithfully it reproduces the distribution of non-Gaussian data. A follow-up paper will detail its Bayesian use, when parameters are to be inferred.

The Edgeworth series – a method of approximating a probability distribution in terms of a series determined by its cumulants – has found a number of cosmological applications over the years, but it does suffer from a number of issues, one of the most important being that it is not guaranteed to be a proper probability distribution, in that the resulting probabilities can be negative…

I’ve been thinking about how to avoid this issue myself, and mentioned a possibility in the talk I gave at South Kensington Technical Imperial College earlier this summer. The idea is to represent the cosmological density field (usually denoted δ) in terms of the square of the modulus of a (complex) wave function ψ i.e. |ψψ*|. It then turns out that the evolution equations for cosmic fluid can be rewritten as a kind of Schrodinger equation. One powerful advantage of this approach is that whatever you do in terms of approximating ψ, the resulting density ψψ* is bound to be positive. This finesses the problem of negative probabilities but at the price of introducing more complexity (geddit?) into the fluid equations. On the other hand, it does mean that even first-order perturbative evolution of ψ guarantees a sensible probability distribution whereas first-order evolution of δ does not.

Summer’s Ending

Posted in Bad Statistics, Biographical, Cricket with tags , , , , , on September 11, 2017 by telescoper

There’s no escaping the signs that summer is drawing to a close. The weather took a decidedly autumnal turn  at the end of last week, and though I resisted the temptation to turn the central heating on at Chateau Coles I fear it won’t be long before I have to face reality and take that step. I hope I can hold out at least until the conventional end of summer, the autumnal equinox, which this year happens at 21.02 BST on Friday, 22 September.

Saturday saw the Last Night of the BBC Proms season. I’ve enjoyed a great many of the concerts but I only listened to a bit of the first half of the Last Night as I find the jingoism of the second half rather hard to stomach. I did catch Nina Stemme on the wireless giving it some welly in the Liebestod from Tristan und Insolde, though.  Pretty good, but difficult to compare with my favourite version by Kirsten Flagstad.

One of the highlights of the season, just a few days ago, was Sir András Schiff’s late-night performance of Book I of The Well Tempered Clavier which had me captivated for two hours, until well past my usual bedtime…

However, as the Proms season ends in London the music-making continues in Cardiff with a new series of international concerts at St David’s Hall and Welsh National Opera’s new season at the Wales Millennium Centre (which starts on 23rd September). I notice also that, having finished his complete Beethoven cycle,  Llŷr Williams is embarking on a series of recitals of music by Schubert, starting on November 9th at the Royal Welsh College of Music and Drama.

Another sign that summer is over is that the last Test Match of the summer has ended. Excellent bowling by Jimmy Anderson (and, in the first innings, by Ben Stokes) meant that England had only a small total to chase, which they managed comfortably. Victory at Lord’s gives England a 2-1 win for the series over West Indies. That outcome is welcome for England fans, but it doesn’t do much to build confidence for the forthcoming Ashes series in Australia. England’s pace bowlers have shown they can prosper in English conditions, when the Duke ball can be made to swing, but in Australia with the Kookaburra they may find success much harder to come by. More importantly, however, only two of England’s five top-order batsmen are of proven international class, making their batting lineup extremely fragile. So much depends on Cook and Root, as I don’t think it is at all obvious who should take the other three positions, despite a whole summer of experimentation.

There are a few one-day internationals and Twenty20 matches coming up as well as three full weeks of County Championship fixtures. In particular, there are two home games for Glamorgan in the next two weeks (one against Northants, starting tomorrow, and one next week against Gloucestershire). Their last match (away against Derbyshire) was drawn because three of the four days were lost to rain, but weather permitting there should still be a few opportunities to see cricket at Sophia Gardens this year.

And of course it will soon be time to for the start of the new academic year, welcoming new students (including the first intake on our MSc courses in Data-Intensive Physics and Astrophysics and new PhD students in Data-Intensive Science who form the first intake of our new Centre for Doctoral Training). All that happens just a couple of weeks from today, and we’re having a big launch event on 25th-26th September to welcome the new intake and introduce them to our industrial and academic partners.

Anyway, that reminds me that I have quite a lot to do before term starts so I’d better get on with it, especially if I’m going to make time to watch a few days of cricket between now and the end of the month!

Random Image

Posted in Bad Statistics on September 10, 2017 by telescoper

No time for a proper post today so here’s a random* picture made by my student, Will..

*In some sense…

Blowers.

Posted in Cricket on September 10, 2017 by telescoper

A warm tribute to Henry Blofeld who today did his last commentary for Test Match Special, as England beat the West Indies by 9 wickets at Lord’s.

Jimmy Anderson took 7 for 42 in the West Indies 2nd innings and passed 500 Test wickets in the process.

bowlingatvincent's avatarsportslaureate

Pre- the final curtain there was the inevitable falling over backwards; too many forced ‘dear old things, too much cranking up of the emotional. Vaughan and Tufnell maybe slightly nervously fawning.

The Old Pro, though, ploughed on, admirably briskly – because that’s what his generation do, right? – because he had a game to commentate on; because ‘tremendous fun’ loometh.

TMS, of course. Blowers. Dressed to kill; impossibly vital and irretrievably amiable, as always. No doubt aware of the doe-eyes around him but impressively focused on that uniquely distracted world-view, drawn in to Lords and to his cricket.

Throughout he remained seemingly unaffected by the smiley furore around him. Doing his thing – scene-setting, describing. All of it via that, yaknow… voice.

There have been contenders, overs the decades, for the Voice of Cricket moniker. Few if any (Arlott, Benaud, perhaps?) have breached that bubble of national/international consciousness in quite…

View original post 937 more words

The People’s March for the European Union

Posted in Politics on September 9, 2017 by telescoper

I’m very sorry I couldn’t be in London today for the People’s March for the EU

I hope it went well and applaud all those who took part!

Jazz, Icarus and Henri Matisse

Posted in Art, Biographical with tags , , , on September 8, 2017 by telescoper

I forgot to mention that while I was in London last weekend I visited the exhibition Matisse in the Studio at the Royal Academy of Arts in Burlington House (Piccadilly). It’s an interesting show, covering not only on the art works by Henri Matisse but also various items he had collected and kept in his studio, some of which appear in his paintings in various forms. Anyway, do go to the exhibition if you can – it’s there until November 12th.

Anyway, all that reminded me of this famous image by Matisse, called Icarus, which seems to fit the theme of this blog. It appears in a small booklet called Jazz which consists of collages and other images as well as text written by the artist himself.

School of Statistics for Astrophysics, Autrans, France, October 9-13

Posted in Uncategorized on September 8, 2017 by telescoper

I just found out about this interesting conference in France coming up next month. Apparently there are still some places left!

Julyan Arbel's avatarStatisfaction

Didier Fraix-Burnet (IPAG), Stéphane Girard (Inria) and myself are organising a School of Statistics for Astrophysics, Stat4Astro, to be held in October in France. The primary goal of the School is to train astronomers to the use of modern statistical techniques. It also aims at bridging the gap between the two communities by emphasising on the practice during works in common, to give firm grounds to the theoretical lessons, and to initiate works on problems brought by the participants. There have been two previous sessions of this school, one on regression and one on clustering. The speakers of this edition, including Christian Robert, Roberto Trotta and David van Dyk, will focus on the Bayesian methodology, with the moral support of the Bayesian Society, ISBA. The interest of this statistical approach in astrophysics probably comes from its necessity and its success in determining the cosmological parameters…

View original post 218 more words

Science and Innovation after Brexit

Posted in Politics, Science Politics with tags , , on September 7, 2017 by telescoper

I’ve been busy most of today so I only have a little time for a short post pointing out that the long-awaited `position paper’ about collaboration on science and innovation between the UK and EU after Brexit has now been published. Those of you intending to remain in the United Kingdom if and when it leaves the European Union might be interested in reading it. I say `might be’ rather than `will be’ as it doesn’t really say anything concrete about anything.

Here’s the overall summary:

In preparing to leave the EU, one of the UK’s core objectives is to “seek agreement to continue to collaborate with European partners on major science, research, and technology initiatives”. It is the UK’s ambition to build on its unique relationship with the EU to ensure that together we remain at the forefront of collective endeavours to improve the world in which we live. The UK believes this is in the joint interest of the UK and EU, and would welcome discussion on how best to shape our future partnership in this area.

The answer to the last bit is, of course, easy. The best way to shape our future partnership in this area is unquestionably for the United Kingdom to remain in the European Union. This document says as much itself. As with most of these papers it consists primarily of a long list of the benefits in this area that the United Kingdom has enjoyed as a direct result of our membership of the European together with a desire to keep most of them after our departure. It offers no real ideas as to how to square the many circles that would involve. In particular, many EU schemes, including those funded by the European Research Council, depend on the freedom of movement the European Union guarantees. Given the leaked Home Office document outlining how it intends to deter EU citizens from coming here I don’t see how we can possibly remain an attractive destination for scientists, or anyone else for that matter.

Meanwhile, today, Parliament is debating the European Union Withdrawal Bill which, if passed, would give the Government sweeping powers – the so-called `Henry VIII’ powers – to bypass Parliament and directly repeal or amend any law it doesn’t like the look of without debate. This is exactly the right-wing power grab that many of who voted Remain feared would happen. If this Bill passes without significant amendment then we can say goodbye to our parliamentary democracy. The parallel with the Enabling Act of 1933 that gave absolute power to Adolf Hitler is frightening.

More Worthless University Rankings

Posted in Bad Statistics, Education with tags , , , on September 6, 2017 by telescoper

The Times Higher World University Rankings were released this week. The main table can be found here and the methodology used to concoct them here.

Here I wish to reiterate the objection I made last year and the year before that to the way these tables are manipulated year on year to create an artificial “churn” that renders them unreliable and impossible to interpret in any objective way. In other words, they’re worthless. This year the narrative text includes:

This year’s list of the best universities in the world is led by two UK universities for the first time. The University of Oxford has held on to the number one spot for the second year in a row, while the University of Cambridge has jumped from fourth to second place.

Overall, European institutions occupy half of the top 200 places, with the Netherlands and Germany joining the UK as the most-represented countries. Italy, Spain and the Netherlands each have new number ones.

Another notable trend is the continued rise of China. The Asian giant is now home to two universities in the top 30: Peking and Tsinghua. The Beijing duo now outrank several prestigious institutions in Europe and the US. Meanwhile, almost all Chinese universities have improved, signalling that the country’s commitments to investment has bolstered results year-on-year.

In contrast, two-fifths of the US institutions in the top 200 (29 out of 62) have dropped places. In total, 77 countries feature in the table.

These comments are all predicated on the assumption that any changes since the last tables represent changes in data (which in turn are assumed to be relevant to how good a university is) rather than changes in the methodology used to analyse that data. Unfortunately, every single year the Times Higher changes its methodology. This time we are told:

This year, we have made a slight improvement to how we handle our papers per academic staff calculation, and expanded the number of broad subject areas that we use.

What has been the effect of these changes? We are not told. The question that must be asked is how can we be sure that any change in league table position for an institution from year to year represents a change in “performance”,rather than a change in the way metrics are constructed and/or combined? Would you trust the outcome of a medical trial in which the response of two groups of patients (e.g. one given medication and the other placebo) were assessed with two different measurement techniques?

There is an obvious and easy way to test for the size of this effect, which is to construct a parallel set of league tables, with this year’s input data but last year’s methodology, which would make it easy to isolate changes in methodology from changes in the performance indicators. The Times Higher – along with other purveyors of similar statistical twaddle – refuses to do this. No scientifically literate person would accept the result of this kind of study unless the systematic effects can be shown to be under control. There is a very easy way for the Times Higher to address this question: all they need to do is publish a set of league tables using, say, the 2016/17 methodology and the 2017/18 data, for comparison with those constructed using this year’s methodology on the 2017/18 data. Any differences between these two tables will give a clear indication of the reliability (or otherwise) of the rankings.

I challenged the Times Higher to do this last year, and they refused. You can draw your own conclusions about why.

P.S. For the record, Cardiff University is 162nd in this year’s table, a rise of 20 places on last year. My former institution, the University of Sussex, is up two places to joint 147th. Whether these changes are anything other than artifacts of the data analysis I very much doubt.

Antiques Roadshow

Posted in Cardiff, Television on September 5, 2017 by telescoper

I passed by this in front of Cardiff Castle on my way home yesterday. I’m thinking of taking a few of my colleagues along to see if they’re worth anything…

😉