Archive for the Science Politics Category

Newer Entries »

The Physics Overview

Posted in Science Politics with tags , , , , , , , , on January 17, 2009 by telescoper

I found out by accident the other day that the Panels conducting the 2008 Research Assessment Exercise have now published their subject overviews, in which they comment trends within each discipline.

Heading straight for the overview produced by the panel for Physics (which is available together with two other panels here),I found some interesting points, some of which relate to comments posted on my previous items about the RAE results (here and here) until I terminated the discussion.

One issue that concerns many physicists is how the research profiles produced by the RAE panel will translate into funding. I’ve taken the liberty of extracting a couple of paragraphs from the report to show what they think. (For those of you not up with the jargon, UoA19 is the Unit of Assessment 19, which is Physics).

The sub-panel is pleased with how much of the research fell into the 4* category and that this excellence is widely spread so that many smaller departments have their share of work assessed at the highest grade. Every submitted department to UoA19 had at least 70% of their overall quality profile at 2* or above, i.e. internationally recognised or above.

Sub-panel 19 takes the view that the research agenda of any group, or of any individual for that matter, is interspersed with fallow periods during which the next phase of the research is planned and during which outputs may be relatively incremental, even if of high scientific quality. In the normal course of events successful departments with a long term view will have a number of outputs at the 3* and 2* level indicating that the groundwork is being laid for the next set of 4* work. This is most obviously true for those teams involved with very major experiments in the big sciences, but also applies to some degree in small science. Thus the quality profile is a dynamic entity and even among groups of very high international standing there is likely to be cyclic variation in the relative amounts of 3* and 4* work according to the rhythm of their research programmes. Most departments have what we would consider a healthy balance between the perceived quality levels. The subpanel strongly believes that the entire overall profile should be considered when measuring the quality of a department, rather than focussing on the 4* component only.

I think this is very sensible, but for more reasons than are stated. For a start the judgement of what is 4* or 3* must be to some extent subjective and it would be crazy to allocate funding entirely according to the fraction of 4* work. I’ve heard informally that the error in any of the percentages for any assessment is plus or minus 10%, which also argues for a conservative formula. However one might argue about the outcome, the panels clearly spent a lot of time and effort determining the profiles so it would seem to make sense to use all the information they provide rather than just a part.

Curiously, though, the panel made no comment about why it is that physics came out so much worse than chemistry in the 2008 exercise (about one-third of the chemistry departments in the country had a profile-weighted quality mark higher than or equal to the highest-rated physics department). Perhaps they just think UK chemistry is a lot better than UK physics.

Anyway, as I said, the issue most of us are worrying about is how this will translate into cash. I suspect HEFCE hasn’t worked this out at all yet either. The panel clearly thinks that money shouldn’t just follow the 4* research, but the HEFCE managers might differ. If they do wish to follow a drastically selective policy they’ve got a very big problem: most physics departments are rated very close together in score. Any attempt to separate them using the entire profile would be hard to achieve and even harder to justify.

The panel also made a specific comment about Wales and Scotland, which is particularly interesting for me (being here in Cardiff):

Sub-panel 19 regards the Scottish Universities Physics Alliance collaboration between Scottish departments as a highly positive development enhancing the quality of research in Scotland. South of the border other collaborations have also been formed with similar objectives. On the other hand we note with concern the performance of three Welsh departments where strategic management did not seem to have been as effective as elsewhere.

I’m not sure whether the dig about Welsh physics departments is aimed at the Welsh funding agency HEFCW or the individual university groups; SUPA was set up with the strong involvement of SFC and various other physics groupings in England (such as the Midlands Physics Alliance) were actively encouraged by HEFCE. It is true, though, that the 3 active physics departments in Wales (Cardiff, Swansea and Aberystwyth) all did quite poorly in the RAE. In the last RAE, HEFCW did not apply as selective a funding formula as its English counterpart HEFCE with the result that Cardiff didn’t get as much research funding as it would if it had been in England. One might argue that this affected the performance this time around, but I’m not sure about this as it’s not clear how any extra funding coming into Cardiff would have been spent. I doubt if HEFCW will do any different this time either. Welsh politics has a strong North-South issue going on, so HEFCW will probably feel it has to maintain a department in the North. It therefore can’t penalise Aberystwyth too badly for its poor RAE showing. The other two departments are larger and had very similar profiles (Swansea better than Cardiff, in fact) so there’s very little justification for being too selective there either.

The panel remarked on the success of SUPA which received a substantial injection of cash from the Scottish Funding Council (SFC) and which has led to new appointments in strategic areas in several Scottish universities. I’m a little bit skeptical about the long-term benefits of this because the universities themselves will have to pick up the tab for these positions when the initial funding dries up. Although it will have bought them extra points on the RAE score the continuing financial viability of physics departments is far from guaranteed because nobody yet knows whether they will gain as much cash from the outcome as they spent to achieve it. The same goes for other universities, particularly Nottingham, who have massively increased their research activity with cash from various sources and consequently done very well in the RAE. But will they get back as much as they have put in? It remains to be seen.

What I would say about SUPA is that it has definitely given Scottish physics a higher profile, largely from the appointment of Ian Halliday to front it. He is an astute political strategist and respected scientist who performed impressively as Chief Executive of the now-defunct Particle Physics and Astronomy Research Council and is also President of the European Science Foundation. Having such a prominent figurehead gives the alliance more muscle than a group of departmental heads would ever hope to have.

So should there be a Welsh version of SUPA? Perhaps WUPA?

Well, Swansea and Cardiff certainly share some research interests in the area of condensed-matter physics but their largest activities (Astronomy in Cardiff, Particle Physics in Swansea) are pretty independent. It seems to me to be to be well worth thinking of some sort of initiative to pool resources and try to make Welsh physics a bit less parochial, but the question is how to do it. At coffee the other day, I suggested an initiative in the area of astroparticle physics could bring in genuinely high quality researchers as well as establishing synergy between Swansea and Cardiff, which are only an hour apart by train. The idea went down like a lead balloon, but I still think it’s a good one. Whether HEFCW has either the resources or the inclination to do something like it is another matter, even if the departments themselves were to come round.

Anyway, I’m sure there will be quite a lot more discussion about our post-RAE strategy if and when we learn more about the funding implications. I personally think we could do with a radical re-think of the way physics in Wales is organized and could do with a champion who has the clout of Scotland’s SUPA-man.

The mystery as far as I am concerned remains why Cardiff did so badly in the ratings. I think the first quote may offer part of the explanation because we have large groups in Astronomical Instrumentation and Gravitational Physics, both of which have very long lead periods. However, I am surprised and saddened by the fact that the fraction rated at 4* is so very low. We need to find out why. Urgently.

7 Comments »

Silver Linings

Posted in Science Politics, The Universe and Stuff with tags on December 19, 2008 by telescoper

They say that bad news sells newspapers, so I shouldn’t be surprised with the large number of hits my previous post and the one before that about the Research Assessment Exercise has generated.

However, I heard some news today which has at least provided a bit of a silver lining and put me in a better mood for the Christmas break. My recent application for a grant to the Science and Technology Facilities Council to fund research over the next three years into departures from the concordance cosmological model has actually been selected.

Owing to a budgetary crisis, STFC grants rounds have been very competitive in recent years so I’m quite relieved to have been successful in the present dire financial context. Obviously, somebody out there seems to like what I do. Being a theorist I’m also quite cheap, which probably helped. Or maybe it was just an administrative error…

Anyway, thanks to this grant I will be able to employ a postdoctoral research assistant and spend a bit more of my time on research. It also helps fund a bit of infrastructure within the department. Overall it amounts to about £350K which sounds a lot, but is actually quite small by the standards of particle physics and astronomy grants. STFC isn’t actually Tesco but every little helps.

All I have to do now is convince a potential postdoc to come and work with me in the 35th 22nd best Physics department in the country. What could be simpler?

8 Comments »

The Authorized Version

Posted in Science Politics with tags , on December 18, 2008 by telescoper

Following on from my previous post about the 2008 Research Assessment Exercise, I’ve been told that Cardiff University’s preferred measure of research activity is not the simple grade point average that I computed there, but an index of research power which is the average multiplied by the number of staff submitted.

Partly out of interest and partly so as not to incur the wrath of the University Thought Police I recalculated the list sorted by the official measure. So here is the authorized version, as sanctioned by the powers that be:

1. University of Cambridge 402.6
2. University of Oxford 371.3
3. Imperial College London 348.7
4. University College London 277.8
5. University of Manchester 215.3
6. University of Durham 191.1
7. University of Edinburgh 169.4
8. University of Warwick 132.6
9. University of Nottingham 126.7
10. University of Glasgow 125.8
11. Queen’s University Belfast 125.0
12. University of Bristol 121.9
13. University of Southampton 120.0
14. University of Birmingham 117.7
15. University of Leicester 114.8
16. University of St Andrews 91.8
17. University of Liverpool 91.7
18. University of Leeds 90.5
19. Queen Mary, University of London 87.5
20. University of Sheffield 86.6
21. Lancaster University 76.6
22. Cardiff University 75.9
23. University of Exeter 75.6
24. University of Strathclyde 74.4
25. University of Hertfordshire 72.8
26. Royal Holloway, University of London 71.3
27. University of Surrey 69.4
28. University of York 67.6
29. University of Bath 57.6
30. University of Sussex 54.0
31. Swansea University 52.9
32. Heriot-Watt University 51.7
33. University of Central Lancashire 51.1
34. Loughborough University 41.9
35. King’s College London 41.8
36. Liverpool John Moores University 39.6
37. Aberystwyth University 35.7
38. Keele University 22.5
39. Armagh Observatory 16.9
40. University of the West of Scotland 6.7
41. University of Kent 6.6
42. University of Brighton 2.3

Well, it’s actually quite surprising how much things change. I don’t think it means very much, but 22nd certainly sounds much better than 35th.

But, being a Newcastle United supporter, I’ve never been a great fan of league tables.

30 Comments »

Res Judicata

Posted in Science Politics with tags , , , , on December 18, 2008 by telescoper

Today is the day people working in British Universities have waited for in a mixture of hope and apprehension for several years. The results of the 2008 Research Assessment Exercise (RAE) were published at 0.01am GMT today (18th December).

I had a look just after midnight and the webserver crashed, but only for a few minutes and I soon got back in and found the bad news. The relevant one for me as an astrophysicist is the table for Unit of Assessment 19 which is Physics & Astronomy. Results are given as a list of numbers, consisting of the number of staff entered (not necessarily an integer, for accounting reasons) followed by the percentage of work judged by the panel to be in each of four categories explained in the following excerpt from the RAE website

The quality profiles displayed on this website are the results of the 2008 Research Assessment Exercise (RAE2008), the sixth assessment in this current format of the quality of research conducted in UK Higher Education Institutions (HEIs). The UK funding bodies for England, Northern Ireland, Scotland and Wales will use the RAE2008 results to distribute funding for research from 2009-10.

The results follow an expert review process conducted by assessment panels throughout 2008. Research in all subjects was assessed against agreed quality standards within a common framework that recognised appropriate variations between subjects in terms of both the research submitted and the assessment criteria.

Submissions were made in a standard form that included both quantitative and descriptive elements. Full details of the contents of, and arrangements for making, submissions were published in ‘Guidance on submissions‘ (RAE 03/2005).

The RAE quality profiles present in blocks of 5% the proportion of each submission judged by the panels to have met each of the quality levels defined below. Work that fell below national quality or was not recognised as research was unclassified.

4* Quality that is world-leading in terms of originality, significance and rigour.
3* Quality that is internationally excellent in terms of originality, significance and rigour but which nonetheless falls short of the highest standards of excellence.
2* Quality that is recognised internationally in terms of originality, significance and rigour.
1* Quality that is recognised nationally in terms of originality, significance and rigour.
Unclassified Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment.

The ‘international’ criterion equates to a level of excellence that it was reasonable to expect for the UOA, even though there may be no current examples of such a level in the UK or elsewhere. It should be noted that ‘national’ and ‘international’ refer to standards, not to the nature or geographical scope of particular subjects.

For my own department, the School of Physics & Astronomy, at Cardiff University, I found the following

Cardiff University (32.30) 5 45 30 20

which means that we entered 32.30 people, but only 5% of the work was judged to be at the top level (4*), 45% at 3*, 30% at 2* and 20% at 1*. On their own these figures don’t mean very much but one can do a quick comparison with the rest of the table to see that for us this is an enormous disappointment. We have a much lower fraction of 4* than the majority of departments, and also a significantly higher fraction of 1*. These findings are very worrying.

If I were working an English University with these results I would be very concerned about their financial implications, but it’s a bit more complicated with us being here in Wales. The numbers given in the table are translated into money by the funding councils and Wales has its own one of these (HEFCW, different from the English HEFCE). There are many fewer physics departments in Wales and we’re not competing with the bigger English ones for funding. We don’t yet know how much our research funds will be cut. It might not be as bad as if we were in England, but it’s clearly not good. We won’t know how much dosh will be involved until March 2009. t’s not just a matter of funding, it’s also the national and international perception of the department in the physics community.

I can see there will be a post mortem to find out what went wrong, as most of us were confident of a much better outcome. Perhaps the format of the RAE (focussing on research papers as the measure of output) is not favourable to a department with so many instrument builders in it?

But with the economy in deep recession making further cuts in research funding likely in the future, and our major external funder (STFC) already struggling to make ends meet, this poor showing in the RAE this has cast a gloomy shadow over Christmas.

Of course many places did much better, including my old department at Nottingham which has

University of Nottingham (44.45) 25 40 30 5

which can be interestingly compared with Cambridge, who have

University of Cambridge (141.25) 25 40 30 5

You can see that apart from the different numbers of staff the profile is exactly the same. I’m sure their publicity machine will pick up on this so I won’t be the last to mention it! Well done, Nottingham!

It will be interesting to see what the newspapers make of the new RAE results. They are significantly more complicated than previous versions which just gave a single number. The scope for flexibility in generating league tables is clearly greatly enhanced by this complexity so we can bet the hacks will have a field day. I thought I’d get a headstart by doing a straightforward ranking using a simple weighted average using 4=4*, 3=3*, etc and then sorting them by the average thus obtained:

1. Lancaster University 2.9
2. University of Bath 2.85
3. University of Cambridge 2.85
4. University of Nottingham 2.85
5. University of St Andrews 2.85
6. University of Edinburgh 2.8
7. University of Durham 2.75
8. Imperial College London 2.75
9. University of Sheffield 2.75
10. University College London 2.75
11. University of Glasgow 2.75
12. University of Birmingham 2.7
13. University of Exeter 2.7
14. University of Sussex 2.7
15. University of Bristol 2.65
16. University of Liverpool 2.65
17. University of Oxford 2.65
18. University of Southampton 2.65
19. Heriot-Watt University 2.65
20. University of Hertfordshire 2.6
21. University of Manchester 2.6
22. University of Warwick 2.6
23. University of York 2.6
24. King’s College London 2.55
25. University of Leeds 2.55
26. University of Leicester 2.55
27. Royal Holloway, University of London 2.55
28. University of Surrey 2.55
29. Swansea University 2.55
30. Queen Mary, University of London 2.5
31. Queen’s University Belfast 2.5
32. Loughborough University 2.45
33. Liverpool John Moores University 2.4
34. University of Strathclyde 2.35
35. Cardiff University 2.35
36. University of Brighton 2.3
37. University of Central Lancashire 2.3
38. Keele University 2.25
39. Armagh Observatory 2.25
40. University of Kent 2.2
41. Aberystwyth University 1.95
42. University of the West of Scotland 1.8

So you can see we are languishing at 35th place out of 42.

This is supposed to be the last RAE and we don’t know what is going to replace it. I don’t at all object to the principle that research funding should be peer-assessed but this particular exercise was enormously expensive in the effort spent at Universities preparing for it, not to mention the ridiculous burden placed on the panel of having to read all those papers.

10 Comments »

Popularisation or Propaganda?

Posted in Science Politics, The Universe and Stuff with tags , , , on November 25, 2008 by telescoper

I was just reading a piece by Jim Al-Khalili in today’s Guardian online science section. Jim is Professor of Physics and of Public Engagement in Science at the University of Surrey. His piece seems to have been inspired by the new appointment of Marcus du Sautoy to a similar position at Oxford University recently vacated by Richard Dawkins. His message is essentially that scientists should not only be more active in popularising science but also do more to “defend our rational, secular society against the rising tide of irrationalism”.

The legitimate interface between science and society has many levels to it. One aspect is the simple need to explain what science tells us about the world in order that people can play an informed part in our increasingly technological society. Another is that there needs to be encouragement for (especially young) people to study science seriously and to make it their career in order to maintain the supply of scientists for the future. And then there is the issue of the wider cultural implications of science, its impact on other belief-systems (such as religions) other forms of endeavour (such as art and literature) and even for government.

I think virtually all scientists would agree with the need for engagement in at least the first two of these. In fact, I’m sure most scientists would love to have the chance to explain their work to a lay audience, but not all subjects are as accessible or inspirational as, say, astronomy. Unfortunately also, not all scientists are very good at this sort of thing. Some might even be counterproductive if inflicted on the public in this way. So it seems relatively natural that some people have had more success than others, and have thus become identified as “science communicators”. Although some scientists are a bit snobby about those who write popular books and give popular talks, most of us agree that this kind of work is vital.

Vital, yes, but there are dangers. The number of scientists involved in this sort of work is probably more limited than it should be owing to the laziness of the popular media, who generally can’t be bothered to look outside London and the South-East for friendly scientists. The broadsheet newspapers employ very few qualified specialists among their staff even on the science pages so it’s a battle to get meaningful scientific content into print in the mass media. Much that does appear is slavishly regurgitated from one of the press agencies who are kept well fed by the public relations experts employed by research laboratories and other science institutes.

These factors mean that what comes out in the media can be a distorted representation of the real scientific process. Head of research groups and laboratories are engaged in the increasingly difficult business of securing enough money to continue their work in these uncertain financial times. Producing lots of glossy press releases seems to be one way of raising the profile and gaining the attention of funding bodies. Most scientists do this with care, but sometimes the results are ludicrously exaggerated or simply wrong. Some of the claims circulating around the time the Large Hadron Collider was switched on definitely fell into one or more of those categories. I realise that there’s a difficult balance to be struck between simplicity and accuracy, and that errors can result from overenthusiasm rather than anything more sinister, but even so we should tread carefully if we want the public to engage with what science really is.

Most worryingly is the perceived need to demonstrate black-and-white certainty over issues which are considerably more complicated than that. This is another situation where science popularisation becomes science propaganda. I’m not sure whether the public actually wants its scientists to make pronouncements as if they were infallible oracles, but the media definitely do. Scientists sometimes become cast in the role of priests, which is dangerous, especially when a result is later shown to be false. Then the public don’t just lose faith with one particular scientist, but with the whole of science.

Science is not about certainty. What it is a method for dealing rationally with uncertainty. It is a pragmatic system primarily intended for making testable inferences about the world using measurable, quantitative data. Scientists look their most arrogant and dogmatic when they try to push science beyond the (relatively limited) boundaries of its applicability and to ride roughshod over alternative ways of dealing with wider issues including, yes, religion.

I don’t have any religious beliefs that anyone other than me would recognize as such. I am also a scientist. But I don’t see any reason why being a scientist or not being a scientist should have any implications for my (lack of) religious faith. God (whatever that means) is, by construction, orthogonal to science. I’m not at all opposed to scientists talking about their religion or their atheism in the public domain, but I don’t see why their opinions are of any more interest than anyone else’s in these matters.

This brings us to the third of Jim’s suggestions: that more scientists should follow Richard Dawkins’ lead and be champions of atheism in the public domain. As a matter of fact, I agree with some of Dawkins’ agenda, such as his argument for the separation of church and state, although I don’t feel his heavy-handed use of the vitriol in The God Delusion achieved anything particularly positive (except for his bank balance, perhaps). But I don’t think it’s right to assume that all scientists should follow his example. Their beliefs are their business. I don’t think we will be much better off if we simply replace one set of priests with another.

So there you have my plea for scientists to accept that science will never have all the answers. There will always be “aspects of human experience that, even in an age of astonishing scientific advance, remain beyond the reach of scientific explanation”.

Can I have the Templeton Prize now please?

7 Comments »

Cerebral Asymmetry: is it all in the Mind?

Posted in Bad Statistics, Science Politics with tags , , on November 12, 2008 by telescoper

After blogging a few days ago about the possibility that our entire Universe might be asymmetric, I found out today that a short comment of mine about a completely different form of asymmetry has been published in the Proceedings of the National Academy of Sciences of New York.

Earlier this summer a paper by Ivanka Savic & Per Lindstrom concerning gender and sexuality differences in brain structure received widespread press coverage and the odd blog comment. They had analysed a group of 90 volunteers divided into four classes based on gender and sexual orientation: male heterosexual, male homosexual, female heterosexual and female homosexual.

They studied the brain structure of these volunteers using Magnetic Resonance Imaging and used their data to look for differences between the different classes. In particular they measured the asymmetry between left and right hemispheres for their samples. The right side of the brain for heterosexual men was found to be typically about 2% larger than the left; homosexual women also had an asymmetry, but slightly smaller than this at about 1%. Gay men and heterosexual women showed no discernible cerebral asymmetry. These claims are obviously very interesting and potentially important if they turn out to be true. It is in the nature of the scientific method that such results should be subjected to rigorous scrutiny in order to check their credibility.

As someone who knows nothing about neurobiology but one or two things about statistics, I dug out the research paper by Savic & Lindstrom and looked at the analysis it presents. I very quickly began to suspect there might be a problem. For each volunteer, the authors obtain measurements of the left and right cerebral volumes (call these L and R respectively). Each pair of measurements is then combined to form an asymmetry index (AI) as (L-R)/(L+R). There is then a set of values for AI, one for each volunteer. The claim is that these are systematically different for the different gender and orientation groups, based on a battery of tests including Analysis of Variance (ANOVA) and t-tests based on sample means.

Of course, it would be better to do this using a consistent, Bayesian, approach because this would make explicit the dependence of the results on an underlying model of the data. Sadly, the statistical methodology available off-the-shelf is of inferior frequentist type and this is what researchers tend to do when they don’t really know what they’re doing. They also don’t bother to read the health warnings that state the assumptions behind the results.

The problem in this case is that the tests done by Savic & Lindstrom all depend on the quantity being analysed (AI) having a normal (Gaussian) distribution. This is very often a reasonable hypothesis for biometric data, but unfortunately in this case the construction of the asymmetry index is such that it is expected to have a very non-Gaussian shape as is commonly the case for distributions of variables formed as ratios. In fact, the ratio of two normal variates has a peculiar distribution with very long tails. Many statistical analyses appeal to the Central Limit Theorem to justify the assumption of normality, but distributions with very long tails (such as the Cauchy distribution) violate the conditions of this Theorem, namely that the distribution must have finite variance. The asymmetry index is probably therefore an inappropriate choice of variable for the tests that Savic & Lindstrom perform. In particular the significance levels (or p-values) quoted in their paper are very low (of order 0.0008, for example, in the ANOVA test) which is surprising for such small samples. These probabilities are obtained by assuming the observations have Gaussian statistics, and they would be much lower for a distribution with longer tails.

Being a friendly chap I emailed Dr Savic drawing this problem to her attention and asking if she knew about this problem and the possible implications it might have for the analysis she had presented. If not, I offered to do an independent (private) check on the data to see how reliable the claimed statistical results actually were. I never received a reply.

Worried that the world might be jumping to all kinds of far-reaching conclusions about gay genes based on these questionable statistics, I wrote instead to the editor of the Journal Proceedings of the National Academy of Sciences of New York, Randy Schekman, who suggested I submit a written comment to the Journal. I did, it was accepted by the editorial committee, and it came out in the 11th November Issue. What I didn’t realise was that Savic & Lindstrom had actually prepared a reply and that this was published alongside my comment. I find it strange that I wasn’t told about this before publication but that aside, it is in principle quite reasonable to let the authors respond to criticisms like mine. Their response reveals that they completely missed the point of the danger of long-tailed distributions I mentioned above. They state that “when the sample size n is big the sampling distribution of the mean becomes approximately normal regardless of the distribution of the original variable“. Not if the distribution of the original variable has such a long tail it doesn’t! In fact, if the observations have a Cauchy distribution then so does the sampling distribution of the mean, whatever the size of sample. You can find this caveat spelled out in many places, including here. Savic & Lindstrom seem oblivous to this pitfall, even after I specifically pointed it out to them.

They also claim that a group size of n=30 is sufficient to be confident that the central limit theorem holds. A pity, then, that none of their groups is of that size. The overall sample is 90, but it is broken down into two groups of 20 and two of 25.

cerebral-asymmetry

(c) 2008 Academy of Sciences of New York

They also say that the measured AI distribution is actually normal anyway and give a plot (above). This shows all the AI values binned into one histogram. Since they don’t give any quantitative measures of goodness of fit, it’s hard to tell whether this has a normal distribution or not. One can, however, easily identify a group of five or six individuals that seem to form a separate group with larger AI values (the small peak to the right of the large peak). Since they don’t give histograms broken down by group it is impossible to be sure, but I would hazard a guess that these few individuals might be responsible for the entire result; remember that the entire sample has n only of 90.

More alarmingly, Savic & Lindstrom state in their reply that “one outlier” is omitted from this graph. Really? On what basis was the outlier rejected? The existence of outliers could be evidence of exactly the sort of problem I am worried about! Unless there was a known mistake in the measurement, this outlier should never have been omitted. They claim that the “recalculation of the data excluding this outlier does not change the results”. It find it difficult to believe that the removal of an outlier from such a small sample could not change the p-values!

In my note I made a few constructive suggestions as to how the difficulty might be circumvented, by Savic & Bergstrom have not followed any of them. Instead they report (without details of the p-values) having done some alternative, non-parametric, tests. These are all very well, but they don’t add very much if their p-values also assume Gaussian statistics. A better way to do this sort of thing robustly would be using Monte Carlo simulations.

The bottom line is that after this exchange of comments we haven’t really got anywhere and I still don’t know if the result is significant. I don’t really think it’s useful to go backwards and forwards through the journal, so I’ve emailed Dr Savic again asking for access to the numbers so I can check the statistics privately. In astronomy it is quite normal for people to make their data sets publically available, but that doesn’t seem to be the case in neurobiology. I’m not hopeful that they will reply, especially since they branded my comments “harsh” and “inappropriate”. Scientists should know how to take constructive criticism.

Their conclusion may eventually turn out to be right, but the analysis done so far is certainly not robust and it needs further checking. In the meantime I don’t just have doubts about the claimed significance of this specific result, which merely serves to illustrate the extremely poor level of statistical understanding displayed by large numbers of professional researchers. This was one of the things I wrote about in my book From Cosmos to Chaos. I’m very confident that a large fraction of claimed results in biosciences are based on bogus analyses.

I’ve long thought that scientific journals that deal with subjects like this should employ panels of statisticians to do the analysis independently of the authors and also that publication of the paper should require publication of the raw data. Science advances when results are subject to open criticism and independent analysis. I sincerely hope that Savic & Lindstrom will release their data in order for their conclusions to be checked in this way.

It’s no wonder that there is so much public distrust of science, when such important claims are rushed into the public domain without proper scrutiny.

13 Comments »

The New Inflationary Universe

Posted in Finance, Science Politics on October 14, 2008 by telescoper

Among the bits of economic information released by the Office of National Statistics today is one item that academics in all disciplines wanted to hear about: the value of the Retail Prices Index (RPI) in the UK for September 2008, which turned out to be 5.0%.

The reason for the fascination with this number is that, in an unusual spasm of farsightedness, the University and College Union stipulated that the final stage of the pay deal it negotiated in 2006 would be applied in October 2008 and this would amount to 2.5% or the RPI whichever is the greater. Two years ago it seemed a very different world, and 2.5 % seemed to be much the likelier eventuality, but energy and commodity prices surged last year and the RPI now stands at double that figure. So we’re all set for a 5% pay rise this month, although probably we won’t actually get any more money until the November pay packet arrives.

It would have been even better had UCU chosen the Consumer Prices Index (CPI), which has now overtaken the RPI and stands at 5.2%. This is the governments preferred measure of inflation, which is based only on the price of consumer goods and household utilities, while the RPI includes other items such as mortgage costs and transport costs.

At least in the short-term, this seems good news for all academics in UK universities.

But even in paradise there was a serpent, and there is a significant danger that some departments’ balance sheets will suffer very badly from these extra salary costs. Many already operate on very tight margins. In the longer term there may be mergers and closures followed by redundancies. Also since the research councils’ cash allocations for the next few years are already fixed, an increase in salaries over that already accounted for will mean a corresponding reduction in the number of positions that can be funded, which is bad news for younger people looking for PDRA positions. Given that the Science and Technology Facilities Council‘s budget wasn’t very generous in the first place, causing a crisis in funding for astronomy and particle physics research the extra wage demands are likely to cause further strain.

Still, a 5% pay rise just before Xmas will be good while it lasts.

Leave a comment »

Wakeham Review

Posted in Science Politics, The Universe and Stuff with tags , , on October 1, 2008 by telescoper

Today is the day of publication of the Wakeham Review of the state of Physics in the United Kingdom. This report was commissioned by the Secretary of State for Innovation, Universities and Skills (DIUS) against the backdrop of the funding crisis that threatened to engulf the
Science and Technology Facilities Council (STFC) in December 2007 and which has led to drastic cuts in research grant funding in particle physics and astronomy throughout the country.

I started blogging a bit too late to join in the chorus of anger surrounding the handling of this crisis by STFC and especially by the behaviour of its Chief Executive, Keith Mason. An investigation of this by a parliamentary Select Committee stated that

“Substantial and urgent changes are now needed in the way in which the Council is run in order to restore confidence and to give it the leadership it desperately needs and has so far failed properly to receive”

If anyone was ever given a clear message that he should resign, this was it. But Keith Mason remains Chief Executive of STFC.

I hoped, therefore, to find some comment about this state of affairs in the Wakeham Review. I haven’t had time to read all of it, but most of it seems bland and rather self-congratulatory. It does, however, describe the strengths of astronomy and space science research in the UK, which is one of the areas placed in jeopardy by STFC’s cack-handed management and woeful lack of political nous. On the other hand, the UK has less impressive impact in other areas. Condensed matter physics was the research area in which most University-based physicists in the UK worked in 2001but their impact, at least in bibliometric terms, was and is unspectacular compared with other countries. Perhaps this is the reason why the number of condensed matter physicists submitted to the Research Assessment Exercise in 2008 has declined, while astronomy and astrophysics have increased.

The Wakeham review does not come to any clear conclusions on why some areas of physics are more popular than others, citing as possibilities laboratory costs and difficulties of attracting people into cross-disciplinary areas like biophysics or nanoscience. Since I’m not a member I don’t have to mince words like the panel did. I think some fields are popular because they are more interesting. And if people wanted to do chemistry or biology they wouldn’t have become physicists in the first place.

There are two paragraphs specifically about STFC, and they make very specific proposals although falling short of asking the current leadership to step down:

6. There is a need to ensure that there is coherence of planning of physics facilities and the allocation of physics research grants, so that research needs are closely aligned with facility provision. For that reason it is not desirable to separate former PPARC-like physics from the funding of its facilities. For this reason the Panel recommends that the current division of physics funding between Research Councils remains. Whilst recognising recent difficulties, the Panel believes that it is important that facilities provided for particle physics and astronomy researchers be directly tensioned with the budget for the research that will utilise those facilities. The current structure provides this tension in part of its remit. However, the panel believes that adding to this tension a further dimension of national facilities and a government Science and Innovation Campuses is just too much.

This is true but I think it’s only a small part of the problem.

The Panel recommends that:
a) the STFC be required at each CSR to bid for and allocate specific funds to former PPARC facilities and grant funding together.This would avoid the undesired tensioning of these grants and facilities support against national facilities and the project for the development of science and innovation campuses.

Good! But will this happen?

b) the existing structure should be allowed time to develop, given it was founded on the basis of extensive positive consultation. However, at an appropriate point following the review of STFC management currently being conducted by Dr David Grant, DIUS should commission a review to examine STFC operations.

*Sigh* Another review. Great.

The next one is a bit stronger:


7. The STFC’s governance structure must be representative of the community it serves in order to gain stakeholders’ confidence going forward.

“..stakeholders’ confidence going forward”? Ugh! Who wrote that bollocks?

The Panel believes that significant damage has been done to the UK’s international reputation in some areas of the discipline of physics following the furore that was generated by the manner, timescale of changes and announcement of recent STFC funding decisions.

You can say that again.

The Panel were very concerned at the make-up of the STFC Council, both in terms of the over representation of the executive and the lack of representation of the community it serves in comparison with other Research Councils. It is understood that this structure was deliberately adopted to deal with the distinct features of STFC that arose because of its multiple missions. However, this has not best served the scientific community in some branches of science whose input was at one level below the Council. This is in sharp distinction to the practice of other Research Councils.


The Panel recommends to DIUS that the membership of STFC’s Council be broadened to include more of the stakeholders in the science activity at the highest level, and to redress the balance between executive presence and non-executive oversight.

Somebody must have deleted the sentence about having to get a new Chief Executive.

I’m sure there’ll be a lot more on physics blogs when there’s been time to digest the whole report, and if I think I’ve missed anything at a first reading I may post some more myself.

1 Comment »

The MacGuffin Factor

Posted in Science Politics, The Universe and Stuff with tags , , , on September 22, 2008 by telescoper

Unpick the plot of any thriller or suspense movie and the chances are that somewhere within it you will find lurking at least one MacGuffin. This might be a tangible thing, such the eponymous sculpture of a Falcon in the archetypal noir classic The Maltese Falcon or it may be rather nebulous, like the “top secret plans” in Hitchcock’s The Thirty Nine Steps. Its true character may be never fully revealed, such as in the case of the glowing contents of the briefcase in Pulp Fiction , which is a classic example of the “undisclosed object” type of MacGuffin. Or it may be scarily obvious, like a doomsday machine or some other “Big Dumb Object” you might find in a science fiction thriller. It may even not be a real thing at all. It could be an event or an idea or even something that doesn’t exist in any real sense at all, such the fictitious decoy character George Kaplan in North by Northwest.

Whatever it is or is not, the MacGuffin is responsible for kick-starting the plot. It makes the characters embark upon the course of action they take as the tale begins to unfold. This plot device was particularly beloved by Alfred Hitchcock (who was responsible for introducing the word to the film industry). Hitchcock was however always at pains to ensure that the MacGuffin never played as an important a role in the mind of the audience as it did for the protagonists. As the plot twists and turns – as it usually does in such films – and its own momentum carries the story forward, the importance of the MacGuffin tends to fade, and by the end we have often forgotten all about it. Hitchcock’s movies rarely bother to explain their MacGuffin(s) in much detail and they often confuse the issue even further by mixing genuine MacGuffins with mere red herrings.

North by North West is a fine example of a multi-MacGuffin movie. The centre of its convoluted plot involves espionage and the smuggling of what is only cursorily described as “government secrets”. But although this is behind the whole story, it is the emerging romance, accidental betrayal and frantic rescue involving the lead characters played by Cary Grant and Eve Marie Saint that really engages the characters and the audience as the film gathers pace. The MacGuffin is a trigger, but it soon fades into the background as other factors take over.

There’s nothing particular new about the idea of a MacGuffin. I suppose the ultimate example is the Holy Grail in the tales of King Arthur and the Knights of the Round Table and, much more recently, the Da Vinci Code. The original Grail itself is basically a peg on which to hang a series of otherwise disconnected stories. It is barely mentioned once each individual story has started and, of course, is never found.

Physicists are fond of describing things as “The Holy Grail” of their subject, such as the Higgs Boson or gravitational waves. This always seemed to me to be an unfortunate description, as the Grail quest consumed a huge amount of resources in a predictably fruitless hunt for something whose significance could be seen to be dubious at the outset.The MacGuffin Effect nevertheless continues to reveal itself in science, although in different forms to those found in Hollywood.

The Large Hadron Collider (LHC), switched on to the accompaniment of great fanfares last week, provides a nice example of how the MacGuffin actually works pretty much backwards in the world of Big Science. To the public, the LHC was built to detect the Higgs Boson, a hypothetical beastie introduced to account for the masses of other particles. If it exists the high-energy collisions engineered by LHC should reveal its presence. The Higgs Boson is thus the LHC’s own MacGuffin. Or at least it would be if it were really the reason why LHC has been built. In fact there are dozens of experiments at CERN and many of them have very different motivations from the quest for the Higgs.

Particle physicists are not daft, however, and they have realised that the public and, perhaps more importantly government funding agencies, need to have a really big hook to hang such a big bag of money on. Hence the emergence of the Higgs as a sort of master MacGuffin, concocted specifically for public consumption, which is much more effective politically than the plethora of mini-MacGuffins which, to be honest, would be a fairer description of the real state of affairs.

Even this MacGuffin has its problems, though. The Higgs mechanism is notoriously difficult to explain to the public, so some have resorted to a less specific but more misleading version: “The Big Bang”. As I’ve already griped, the LHC will never generate energies anything like the Big Bang did, so I don’t have any time for the language of the “Big Bang Machine”, even as a MacGuffin.

While particle physicists might pretend to be doing cosmology, we astrophysicists have to contend with MacGuffins of our own. One of the most important discoveries we have made about the Universe in the last decade is that its expansion seems to be accelerating. Since gravity usually tugs on things and makes them slow down, the only explanation that we’ve thought of for this perverse situation is that there is something out there in empty space that pushes rather than pulls. This has various possible names, but Dark Energy is probably the most popular, adding an appropriately noirish edge to this particular MacGuffin. It has even taken over in prominence from its much older relative, Dark Matter, although that one is still very much around.

We have very little idea what Dark Energy is, where it comes from, or how it relates to other forms of energy we are more familiar with, so observational astronomers have jumped in with various grandiose strategies to find out more about it. This has spawned a booming industry in survey of the distant Universe (such as the Dark Energy Survey) all aimed ostensibly at unravelling the mystery of the Dark Energy. It seems that to get any funding at all for cosmology these days you have to sprinkle the phrase “Dark Energy” liberally throughout your grant applications.

The old-fashioned “observational” way of doing astronomy – by looking at things hard enough until something exciting appears (which it does with surprising regularity) – has been replaced by a more “experimental” approach, more like that of the LHC. We can no longer do deep surveys of galaxies to find out what’s out there. We have to do it “to constrain models of Dark Energy”. This is just one example of the not necessarily positive influence that particle physics has had on astronomy in recent times and it has been criticised very forcefully by Simon White.

Whatever the motivation for doing these projects now, they will undoubtedly lead to new discoveries. But my own view is that there will never be a solution of the Dark Energy problem until it is understood much better at a conceptual level, and that will probably mean major revisions of our theories of both gravity and matter. I venture to speculate that in twenty years or so people will look back on the obsession with Dark Energy with some amusement, as our theoretical language will have moved on sufficiently to make it seem irrelevant.

But that’s how it goes with MacGuffins. Even the Maltese Falcon turned out to be a fake in the end.

p.s. I heard on Saturday that the LHC is having some problems with its magnets and will actually be off-line for a few months. Last week I heard a particle physicist describing the great switch-on as like “Christmas”. This turns out to have been truer than he can have imagined. Only a week has passed and his most expensive toy is already broken…

8 Comments »

The Last Experiment

Posted in Science Politics, The Universe and Stuff with tags , , , on September 16, 2008 by telescoper

I’ve launched myself into the blogosphere just a bit too late for the feeding frenzy surrounding the switching on of the Large Hadron Collider at CERN last week. Obviously the event itself was a bit of a non-event as it will take years for anything interesting to come out the other end of its multi-billion-dollar tunnel. There are a couple of things worth saying in retrospect, though, now that the dust has settled.

The first is about all this nonsense concerning the creation of black holes that could destroy the Earth. If it were possible to create black holes in the LHC then they would be very puny ones, not capable of destroying anything very much at all. The phrase “black hole” conjures up Hollywood-style images of dead stars rampaging through the Galaxy devouring planets and costing a fortune in special effects. But not all black holes are massive enough to be stars in movies. If the LHC could make black holes it would only make very titchy ones. Since the gravitational effect of a black hole depends on its mass – and these little ones have very little of that – any that did pop out of an event in the LHC would be more of a pin-prick than a hole…

Moreover, energetic particles in the form of cosmic rays are constantly raining down on the Earth’s atmosphere, colliding with hadrons as they do so. The most extreme cosmic rays have energies far in excess of the limit that can be reached by the LHC. If an energetic hadron collision were going to produce a black hole that could destroy the planet, it would have happened a very long time ago and we wouldn’t be around to discuss the possibility.

So how did this daft idea come to dominate the news coverage surrounding the switch-on of the LHC? The press are never reluctant to peddle the doomsday scenario whenever they can as it appears to sell newspapers. But there is probably a bit more to it than that. I think part of it is a side-effect of the exaggerated language used by particle physicists in their attempt to use the LHC to capture the public imagination. “The Big Bang Machine” is just one example. If the experiment were really attempting to recreate the Big Bang, then there would indeed be much to be scared about. But the fact of the matter is that the LHC doesn’t reach energies anything like those reached in the Big Bang (nor even in the many smaller bangs that our Universe indulges in from time to time, such as supernova explosions).

The maximum energy reached by the LHC is going to be about 7 TeV (roughly equivalent to the energy of a bumble bee in flight). Although the very earliest stages of the Big Bang itself are not well understood, we are pretty sure that the primordial fireball started off with energies at least a million million (i.e. 1,000,000,000,000) times larger than this. It is doubtful (to say the least) that we’ll ever be able to build a device capable of reaching such energies, so the only “Big Bang Machine” there will ever be is the one we happen to be living in.

This is perhaps the reason why particle physicists are so desperate to glean maximum publicity for the LHC. It’s cost – though not extreme when compared to, for example, military spending – far exceeds that of any other scientific experiment. When it is over, will it be possible to build an even bigger experiment to probe even deeper into the subatomic world? Funding of such experiments generally comes from the public purse and it seems more than likely that the taxpayer will draw the line very soon. Although it won’t destroy the world, perhaps the LHC is nevertheless the end of the line for experimental physics of that kind.

So by all means let’s celebrate the LHC. It’s a wonderful demonstration of what international cooperation can achieve. It is also a response to the need all humans have to ask questions about our Universe. But let us not forget that our ability to probe the inner space of particles with experiments will always be limited, while the outer space beyond the stars offers much wider horizons.

PS. I can’t resist adding this link, as the best example of the worst of the hysteria about the LHC.

PPS. And this one, which explains why the LHC really is safe.

13 Comments »

Newer Entries »