Science Politics | In the Dark

Archive for the Science Politics Category

(Guest Post) Physics and Binary Creep

Posted in Education, Finance, Science Politics with tags chemistry, HEFCE, Paul Crowther, Physics, research, Research Assessment Exercise, university funding on April 15, 2011 by telescoper

His Excel-lence (geddit?) Paul Crowther has been at it again, using his favourite packages sophisticated graph-plotting facilities to produce the interesting figures that go with another guest post….

–0–

Last week’s Times Higher Ed included a news item headlined ‘binary creep’, in which HEFCE were considering restricting support for PhD research students to universities of the highest research quality. Concerns were expressed in the article about a two stream future for universities – research intensives in the fast lane and ‘the rest’ in the slow lane. This reminded me of a recent Times Higher Ed interview with the former Commons’ Science and Technology Committee chairman, Lord (Phil) Willis. Lord Willis argued that the UK could probably sustain “no more than 30” universities with the capacity to attract the best global researchers and carry out world-class research, a view no doubt shared by ministers and civil servants within BIS. I should qualify the following line of thought by emphasising that this is not Government policy, although both stories reflect moves by funding agencies to further concentrate increasingly scarce resources on the highest ranked research universities. For example, in England HEFCE is expected to withdraw all quality-related (QR) support from 2* RAE research from 2012 onwards.

Mindful of the fact that in such a vision for the future, there would be a comparatively few, research intensive universities (`winners’) where would that leave the remainder (‘losers’), especially for physics? Research quality can be quantified in all manner of ways, but for simplicity I have adopted the Quality Index (QI) from Research Fortnight which provides a single mark out of 100 based on RAE quality profiles (4*:3*:2*:1* weighted 8:4:2:1). The chart below shows the QI-ranked list of more-or-less all 120 UK universities who were rated in RAE 2008. It will come as no surprise to anyone that Oxbridge, LSE and Imperial top the rankings, closely followed by UCL and a few other high flyers, but beyond the top 10 perhaps more surprising there are no natural breaks in quality from Durham and QMUL in joint 11th place, to Bolton at 107th.

Thinking out loud about Willis’ assertion that the UK should not be spreading the jam more thinly than, say, the leading 30 universities, there would obviously be individual physics departments currently outside the top 30 which are ranked significantly higher than those within the top 30. To illustrate this, the chart also includes (in blue) physics QI scores for all teaching institutions that were assessed under the UOA 19 in RAE 2008. To blindly follow Lord Willis’ suggestion, 16 out of 42 institutions involved with physics research – comprising 37 per cent of all academic staff – would be clear losers. These would include one physics department raked within the top 10 (scoring 49) because its host institution is ranked 34th overall, while winners would include a department scoring 31, i.e. ranked 40th (out of 42) for physics, as a result of its university squeezing into the top 30. Chemistry – within the same RAE sub-panel as physics – reveals a broadly similar distribution, although there is perhaps a greater concentration of the highest research quality in the overall top 20, as the chart below illustrates.

Alternatively, if there is to be further concentration, one could argue that research funding should focus on, say, the top 20 physics departments regardless of the performance of their host institution. Indeed, already 80 percent of STFC spending goes to only 16 universities. Still, as RAE grades indicate, a strength of UK physics is the breadth of high quality research, with no natural break points until beyond 30th place in the rankings, as the final chart shows. Of course, RAE scores aren’t the sole criterion being discussed, with “critical mass” the other main driver. Due in large part to the big four, 70 per cent of physics academic staff submitted for RAE 2008 are in departments that are currently ranked in the top 20. Chemistry has a similar story to tell in the chart, albeit displaying a somewhat steeper QI gradient.

What might be the long-term consequences of a divergence between a small number of “research-facing” universities and the rest? It is apparent that if the number of physics departments involved in research were reduced by a third, some high quality research groups would be lost, regardless of precisely where the cleaver ultimately fell. Let’s too not forget that astrophysics represents the largest sub-field of physics from the last IOP survey, as measured in numbers of academics.

If policy makers don’t see anything fundamentally wrong with A-level physics being taught by teachers qualified, say, in biology, then they might too wonder whether physics degrees could be taught by academics lacking a physics research background? This might work for first year undergraduate courses, but thereafter isn’t more specialist knowledge needed that a research background most readily provides? How would the third of physics academics outside the top 30 universities react to the prospects of a teaching-only future? Many surely would consider jumping ship either to one of the chosen few or overseas, further decreasing the pool of those with research experience in the remaining physics departments. This is further complicated by the expected political desire that physics departments should be appropriately distributed geographically across England, Scotland, Wales and Northern Ireland.

As a final thought experiment, the fate of physics departments facing the prospect of a teaching-only future might also be binary in nature, either (a) whither and die, decreasing the range of institutions offering degrees in physics (or physical sciences, natural sciences etc.); perversely at a time when the Government are anxious to maintain the number of students studying Science, Technology, Engineering and Mathematics (STEM) subjects, or (b) thriving – free from the distractions of chasing dwinding research grants – by adapting to offer shorter duration physics degrees, described as “cheap and cheerful” by Dr David Starkey during the discussion on student fees on last Thursday’s Newsnight. To reiterate, it is not explicit Government policy to actively reduce the number of physics departments that receive research allocations, but this seems to be the general “direction of travel” in policy-makers speak, so I fear a rocky path ahead..

4 Comments »

Gravity waves goodbye to LISA?

Posted in Science Politics, The Universe and Stuff with tags ESA, gravitational waves, LISA, NASA on April 8, 2011 by telescoper

It seems that we’re not allowed to have any good news these days without a bit of bad to go with it. This week it has emerged here and there that the US National Aeronautics and Space Administration (better known as NASA) is pulling the plug on one of the most exciting space missions on its drawing board. Feeling the pressure of budget constraints and a ballooning overspend on the James Webb Space Telescope (JWST), NASA has decided not to participate further in the Laser Interferometric Space Antenna, a.k.a. LISA. The project teams working on LISA have been disbanded, and the shutters have been pulled down on a project which would have revolutionised astrophysics by opening up new possibilities of observing astronomical objects using gravitational waves, rather than electromagnetic radiation.

This does not mean that LISA is necessarily completely dead. For one thing, it was always planned to be a partnership between NASA and its European counterpart ESA (the European Space Agency); you can find ESA’s LISA page here. In fact a technological demonstrating mission LISA-Pathfinder, operated by ESA, is scheduled for launch in 2013.
It remains possible that ESA will proceed on its own with some version of LISA, although given its own financial constraints it is unlikely that it will be able to fund the full original mission concept. The best we can hope for, therefore, is probably some slimmed-down low-budget version and perhaps an even later launch date.

I still hold out some hope that LISA might come out of mothballs when gravitational waves are actually detected. This may well be accomplished by Advanced LIGO, a ground-based interferometric system based in the states, although it has to be said that gravitational waves have been “on the brink of detection” for at least 30 years and still haven’t actually been found. When detection does become a reality it might galvanise NASA into finding room in its budget again.

This news will be a particularly concern for the sizeable Gravitational Physics group here in the School of Physics & Astronomy at Cardiff University. However, LISA was very much in the planning and development stages so it won’t impact their current work. I haven’t had the chance to discuss the news about LISA with members of this group, so I’d be interested to receive comments from them, or indeed anyone else who knows more about what NASA’s decision may or not mean for the future of gravitational wave physics.

3 Comments »

On My Radio (Telescope) …

Posted in Science Politics, The Universe and Stuff with tags Jodrell Bank, On My Radio, Pauline Black, Square Kilometre Array, STFC, The Selecter, University of Manchester on April 7, 2011 by telescoper

A piece of news I should have passed on sooner than this is the announcement that the Headquarters for the Square Kilometre Array will be based at the Jodrell Bank Observatory which, as you all know, is situated in the English Midlands.

The Square Kilometre Array (known to the astronomical community as SKA) will be, when it’s built, the largest radio telescope, and in fact the largest telescope of any kind, ever constructed. Building it will be a huge technical challenge, and it involves teams from all around the world. Although it hasn’t yet been decided where the actual kit will be sited – Australia and South Africa are two strong contenders – it’s definitely a coup for the UK to be hosting the Project Office. So congratulations to Jodrell Bank and to John Womersley, Director of Science Programmes at the Science and Technology Facilities Council who will be heading up the operation.

I think that the SKA is by far the most exciting project in ground-based astronomy on the STFC books: it has a significantly stronger science case than its competitor in the optical part of the spectrum, the European Extremely Large Telescope (E-ELT), although it is admittedly more of a challenge to build it from a technological point of view. Over the last few years I’ve feared on many occasions that STFC would have to pull out of one of these two very expensive projects and that E-ELT would be the one that survived because it is within the remit of the European Southern Observatory (ESO) to which we pay a hefty subscription. Fortunately the clouds seem to have lifted a bit and it looks like we’re going to remain in both, which is excellent news for UK astronomy.

I was thinking of putting up a bit of music to celebrate the good news. Hmmm….Ska….radio. No brainer really. I wonder who was The Selecter for the location of the SKA Project Office?

P.S. I just looked at the date when On My Radio was in the charts. October 1979, when I was 16. I have to confess that in those days I had a massive crush on lead singer Pauline Black …

12 Comments »

Brush up your CVs!

Posted in Science Politics with tags STFC Chief Executive on April 6, 2011 by telescoper

A job vacancy caught my eye this week, so I thought I’d pass it on (at no extra charge).

This is the long-awaited announcement of a much-needed vacancy as Chief ~~Executioner~~ Executive of the Science and Technology Facilities Council (STFC). The incumbent, Professor Keith Mason, will be leaving the job (although not until next year) and I’m sure his successor will be grateful for the fact that STFC has much less activity to manage now than it did four years ago. Indeed, if you download the further information you’ll find there’s so much talk of “management of change” and “new structures” that you wouldn’t have thought STFC had already existed for four years…

A notable requirement is that the successful candidate must have

…strong emotional intelligence, excellent listening skills; good relationship and influencing skills and the ability to reach out and build consensus and trust;

which will certainly make a change. The ability to “manage budgets” is apparently also necessary.

I gather special training is provided so the successful candidate can learn to read and write TreasurySpeak and that there is a substantial budget for travel. A luxurious office is provided in Swindon, but the Chief Executive is not required to visit it except on special occasions, such as when there is a celebration of the closure of a major national facility. The salary is “competitive”, although it doesn’t say with what.

But, seriously, it’s going to be a tough but vitally important job for UK Science so I hope someone of sufficient stature to take it on does emerge, poisoned chalice though it undoubtedly is. A rumour mill has already started, and I might open a book on the race if there are enough nominations through the comments box…

4 Comments »

What’s your mixing angle?

Posted in Science Politics, The Universe and Stuff with tags astronomy, bilbiometrics, Cabibbo process, Particle Physics, quark mixing on April 5, 2011 by telescoper

Today I’ve been preparing tomorrow’s particle physics lecture on the Cabibbo mechanism for quark mixing, which inspired me to go back to Paul Crowther’s guest post of a couple of days ago to present the data in a slightly different way.

The centrepiece of Paul’s post was the following graph which shows the distribution of two different bibliometric measures for the UK astronomical community. There is the h-index (which is the number h such that the author has h papers cited at least h times) and a normalised version of h in which each paper’s citations are divided by the number of authors of that paper before the index is formed; I call this index h_norm. The results are shown below:

Generally speaking the two indices track each other fairly well, but there are clearly some individuals for whom they diverge. These correspond to researchers whose main mode of productivity is through large consortia and for whom h is correspondingly much larger than h_norm.

The “outliers” are more easily identified by forming the ratio

$l= \frac{h-h_{\rm norm}}{h+h_{norm}}$

which is plotted in the graph below kindly provided by Paul Crowther.

Notice that the “lurker index” $l$ is constructed to normalise out any general trend with h and the data do seem consistent with a constant mean across the ranked list. There is, however, a huge spread even among the top performers.

If this were particle physics rather than astronomy the results wouldn’t be presented in terms of a ratio like $l$ but as a mixing angle like the Weinberg angle or the Cabibbo angle. In this scheme we envisage each researcher’s output publication list as involving a mixture of “solo” and “collaborator” basis states, i.e.

|output>=cos(θ) |solo>+sin(θ) |collaborator>

The angle θ gives a quantitative indication of an author’s inclination to lurk in other people’s publication lists. If θ=0 then the individual’s papers are going to be all single-author affairs with no question marks over attribution of impact. If θ=90° then the individual does primarily collaborative research – perhaps he/she is a good mixer? Most researchers lie somewhere between these two extremes.

I therefore suggest that we should measure bibliometric productivity and impact not just through one “amplitude”, say h, but by the addition of a mixing angle, i.e. the whole output should be summarised as (h,θ). One could estimate the relevant angle fairly straightforwardly as

$\sin\theta = l= \frac{h-h_{\rm norm}}{h+h_{norm}}$ ,

but alternative definitions are possible and a more complete understanding of the underlying process is needed to make this more rigorous.

Stephen Hawking has a particularly small mixing angle (~5.7°); many members of the astronomical Premiership have much larger values of this parameter. The value of θ corresponding to the average value of $l$ is about 23.5° and my own angle is about 8.6°.

And here, courtesy of the ever-reliable Paul Crowther, is a graph of mixing angle versus raw h-index for the whole crowd shown in the above diagram.

P.S. If you thinking this application of mixing angle is daft, then you should read this post.

7 Comments »

(Guest Post) The Astronomical Premiership

Posted in Science Politics with tags astronomy, bibliometric, citations, Paul Crowther on April 2, 2011 by telescoper

Here’s a contribution to the discussion of citation rates in Astronomy (see this blog passim) by the estimable Paul Crowther who in addition to being an astronomer also maintains an important page about issues relating to STFC funding.

–0–

At last week’s Sheffield astrophysics group journal club I gave a talk on astronomical bibliometrics, motivated in part by Stuart Lowe’s H-R diagram of astronomers blog entry from last year, and the subsequent Seattle AAS 217 poster with Alberto Conti. These combined various versions of google search results with numbers of ADS publications. The original one was by far the most fun.

The poster also included Hirsch’s h-index for Americal Astronomical Society members, which is defined as the number of papers cited at least h times. Conti and Lowe presented the top ten of AAS members, with Donald Schneider in pole position, courtesy of SDSS. Kevin Pimblett has recently compiled the h-index for (domestic) members of the Astronomical Society of Australia, topped by Ken Freeman and Jeremy Mould.

Even though many rightly treat bibliometrics with distain, these studies naturally got me curious about comparable UK statistics. The last attempt to look into this was by Alex Blustin for Astronomy and Geophysics in 2007, but he (perhaps wisely) kept his results anonymous. For the talk I put together my attempt at an equivalent UK top ten, including those working overseas. Mindful of the fact that scientists could achieve a high h-index through heavily cited papers with many coauthors, I also looked into using normalised citations from ADS for an alternative, so-called hl,norm-index. I gather there are a myriad of such indices but stuck with just these two.

Still, I worried that my UK top ten would only be objective if I were to put together a ranked list of the h-index for every UK-based astronomy academic. In fact, given the various pros and cons of the raw and hl,norm-indexes, I thought it best to use an average of these scores when ranking individual astronomers.

For my sample I looked through the astrophysics group web pages for each UK institution represented at the Astronomy Forum, including academics and senior fellows, but excluding emeritus staff where apparent. I also tried to add cosmology, solar physics, planetary science and gravitational wave groups, producing a little over 500 in total. Refereed ADS citations were used to calculate the h-index and hl,norm-index for each academic, taking care to avoid citations to academics with the same surname and initial wherever possible. The results are presented in the chart.

Andy Fabian, George Efstathiou and Carlos Frenk occupy the top three spots for UK astronomy. Beyond these, and although no great football fan, I’d like to use a footballing analogy to rate other academics, with the top ten worthy of a hypothetical Champions League. Others within this illustrious group include John Peacock, Rob Kennicutt and Stephen Hawking.

If these few are the creme de la creme, I figured that others within the top 40 could be likened to Premier League teams, including our current RAS president Roger Davies, plus senior members of STFC committees and panels, including Andy Lawrence, Ian Smail and Andrew Liddle.

For the 60 or so others within the top 20 percent, I decided to continue the footballing analogy with reference to the Championship. At present these include Nial Tanvir, Matthew Bate, Steve Rawlings and Tom Marsh, although some will no doubt challenge for promotion to the Premier League in due course. The remainder of the top 40 per cent or so, forming the next two tiers, each again numbering about 60 academics, would then represent Leagues 1 and 2 – Divisons 3 and 4 from my youth – with Stephen Serjeant and Peter Coles, respectively, amongst their membership.

The majority of astronomers, starting close to the half-way point, represent my fantasy non-league teams, with many big names in the final third, in part due to a lower citation rate within certain sub-fields, notably solar and planetary studies. This week’s Times Higher Ed noted that molecular biology citation rates are 7 times higher than for mathematics, so comparisons across disciplines or sub-disciplines should be taken with a large pinch of salt.

It’s only the final 10 percent that could be thought of as Sunday League players. Still, many of these have a low h-index since they’re relatively young and so will rapidly progress through the leagues in due course, with some of the current star names dropping away once they retire. Others include those who have dedicated much of their careers to building high-impact instruments and so fall outside the mainstream criteria for jobbing astronomers.

This exercise isn’t intended to be taken too seriously by anyone, but finally to give a little international context i’ve carried out the same exercise for a few astronomers based outside the UK. Champions League players include Richard Ellis, Simon White, Jerry Ostriker, Michel Mayor and Reinhard Genzel, with Mike Dopita, Pierro Madau, Simon Lilly, Mario Livio and Rolf Kudritzki in the Premier League, so my ball-park league divisions seem to work out reasonably well beyond these shores.

Oh, I did include myself but am too modest to say which league I currently reside in…

37 Comments »

The Ernest Rutherford Fellowships Scheme

Posted in Finance, Science Politics with tags Ernest Rutherford, Ernest Rutherford Fellowships, Physics, STFC on April 1, 2011 by telescoper

It seems timely to use the medium of this blog to pass on some important news from the Science and Technology Facilities Council (STFC) to those who might find it useful.

This week saw the unveiling of a brand new STFC scheme to be called the Ernest Rutherford Fellowships. These will be in some respects similar to the previous Advanced Fellowships in that each Fellowship will last for five years with 12 being offered by STFC each year, and will cover the salary costs of the holder for that period. An important new element, however, is that holders of these Fellowships will be able to bid for “significant additional funds to support their research”.

The announcement of this new programme is sure to be warmly welcomed by the scientific community because the previous Advanced Fellowships have been a stepping stone to an academic career for many a budding scientist (including myself, in fact). There will however be some restrictions on eligibility that did not apply to previous schemes.

The first new restriction is to bring the scheme into line with the attitudes of Ernest Rutherford, in whose honour the new fellowships are to be named. One of the most frequently-quoted remarks by Rutherford is the following:

Don’t let me catch anyone talking about the Universe in my department

Obviously therefore it has proved necessary to close the scheme to astronomers and cosmologists. This shouldn’t prove too much of a problem, however, as the STFC press statement by John Le Mesurier makes it clear that the only notable recipients of Advanced Fellowships in the past are actually particle physicists:

Previous recipients of Advanced Fellowships include Professor Brian Cox who has done much to popularise/demystify physics through his recent TV series, Professor Ruth Gregory who was awarded the IoP Maxwell Medal for outstanding contributions to theoretical, mathematical or computational physics in 2006; and Professor Brian Foster who was awarded the IoP Born medal (for outstanding contributions to physics) in 2003.

The second new rule is intended to control the number of applications in order to make the selection of the recipients of these Elite Fellowships more manageable. The criteria applied to the previous Advanced Fellowship programme were very flexible, with the result that each round typically generated well over a hundred applications. This made the relevant Panel’s task extremely difficult. STFC has therefore decided to impose a restriction on the ~~age~~ seniority of the candidates in order to streamline the process.

To be eligible for an Ernest Rutherford Fellowship, candidates must have completed their PhD between 5 years 11 months and 30 days and 6 years of the date of application. This is in addition to the usual requirement of being a white heterosexual male. According to rigorous investigations by STFC staff, this reduces the pool of potential applicants substantially. To one, actually.

The successful candidate (Dr Jamie B’Stard of Oxbridge University) will be eligible to bid for, and be given on the nod, additional ring-fenced funding to support those things that an Elite Fellow needs, both to carry out their research and to feel generally superior to everyone else (e.g. private jet, fleet of Rolls-Royce motor cars, and gold-plated taps in their private lavatory). Never in the history of British science will a physicist have been so generously endowed. The new scheme will allow science to compete in prestige and public acclaim with other forms of employment, such as in the banking sector.

To liberate the funds needed for this initiative it has inevitably proved necessary to make savings elsewhere in the STFC programme. After minutes of arduous deliberation it was decided, as usual, to pay for it by top-slicing the budget for research grants (this time by 95%). Unfortunately this means that no grants will be available for any other research within the STFC remit. However, as a gesture of goodwill, the Chief Executive of STFC has given the instruction that the remaining 5% of the now defunct grants line will be distributed to universities to help cover the cost of making all existing PDRAs redundant.

I hope this clarifies the situation.

21 Comments »

Beware the “Efficiency Factor” ..

Posted in Finance, Science Politics with tags RCUK, Science funding on March 22, 2011 by telescoper

That sigh of relief we all breathed when the flat-cash settlement for UK science funding was announced last October is now looking decidely premature. For one thing the rate of inflation has climbed to 5.5%, its highest level for 20 years. That’s going to be eating away at the money available for doing science at a much higher rate than we thought it would 6 months ago.

If that weren’t bad enough we now learn that the Dark Lords of the Treasury have been beavering away in the background to come up with a way of squeezing science still further, via so-called “efficiency savings”. Now they have announced their plans under the suitably Orwellian title Ensuring Excellence with Impact.

The full document is (probably deliberately) written in almost unreadable Treasury-speak; after all, you don’t want the lambs to know too much about their impending slaughter. Hidden amid the jargon, however, is a grim message. That grant money you thought you had might not be yours after all.

Some of what is written in the RCUK document was expected. For example, there will be no indexation of grants for the next two years as the public sector pay freeze bites. However, another part of the plan is to tackle the so-called “estates” and other “indirect costs”, the contribution Research Councils pay universities to support basic infrastructure. At the moment, universities cost this themselves. In fact, whenever I’ve applied for grants I have to leave this to other people to fill in as I have no idea how it is calculated. However, different Higher Education Institutes (HEIs) charge at vastly different rates. RCUK has noticed this and will henceforth place HEIs into efficiency groups, with the more expensive being the least efficient. Depending on which efficiency group your HEI is in, the indirect costs will be subject to a squeeze. In other words an “efficiency factor” will be applied.

But this won’t just apply to new grants. Cash you though you had already will be clawed back. Here is a quote from the summary:

To ensure that these changes to indirect cost rates do not present an administrative burden to research organisations, and reflecting the time it takes to prepare an application, existing grants will for this purpose be classed as those submitted via Je-S1 before 30th June 2011. Rather than apply reductions to each individual awarded grant, a top slice will be applied by the Research Councils to research organisations’ portfolio of funding after the 1st July 2011. The percentage of this indirect cost efficiency top slice will be dependent on the efficiency group that a research organisation is in.

Reduced rates of indexation will be used both as part of the efficiency factor for indirect costs and for other elements of grants that are indexed in line with current policies. Reduced rates of indexation for other elements of grants, other than the indirect costs element, will be introduced on 1st April 2011 in line with usual Research Council policies. The indexation changes will be greatest during the first two years to coincide with the period of Public Sector pay restraints, but will be gradually relaxed as the effect of savings being applied to new grants contributes greater efficiencies. The indexation savings will be applied to both new and existing grants. For new grants, new indexation rates will be used for grants awarded from 1st April. For existing grants that have been awarded with different indexation arrangement, i.e. those awarded on or by 31st March, the changes will become part of the “top-slice” by institution.

This is scary. It means money already in departmental and university budgets and used for future planning is going to disappear pretty quickly. How this is going help “Ensuring Excellence” I have no idea, but I have to admit it’s going to have some “Impact”.

Be afraid. Be very afraid.

21 Comments »

What Counts as Productivity?

Posted in Bad Statistics, Science Politics, The Universe and Stuff with tags ADS, astronomy, bibliometric, UKIDSS, UKIRT on March 18, 2011 by telescoper

Apparently last year the United Kingdom Infra-Red Telescope (UKIRT) beat its own personal best for scientific productivity. In fact here’s a graphic showing the number of publications resulting from UKIRT to make the point:

The plot also demonstrates that a large part of recent burst of productivity has been associated with UKIDSS (the UKIRT Infrared Deep Sky Survey) which a number of my colleagues are involved in. Excellent chaps. Great project. Lots of hard work done very well. Take a bow, the UKIDSS team!

Now I hope I’ve made it clear that I don’t in any way want to pour cold water on the achievements of UKIRT, and particularly not UKIDSS, but this does provide an example of how difficult it is to use bibliometric information in a meaningful way.

Take the UKIDSS papers used in the plot above. There are 226 of these listed by Steve Warren at Imperial College. But what is a “UKIDSS paper”? Steve states the criteria he adopted:

A paper is listed as a UKIDSS paper if it is already published in a journal (with one exception) and satisfies one of the following criteria:

1. It is one of the core papers describing the survey (e.g. calibration, archive, data releases). The DR2 paper is included, and is the only paper listed not published in a journal.
2. It includes science results that are derived in whole or in part from UKIDSS data directly accessed from the archive (analysis of data published in another paper does not count).
3. It contains science results from primary follow-up observations in a programme that is identifiable as a UKIDSS programme (e.g. The physical properties of four ~600K T dwarfs, presenting Spitzer spectra of cool brown dwarfs discovered with UKIDSS).
4. It includes a feasibility study of science that could be achieved using UKIDSS data (e.g. The possiblity of detection of ultracool dwarfs with the UKIRT Infrared Deep Sky Survey by Deacon and Hambly).

Papers are identified by a full-text search for the string ‘UKIDSS’, and then compared against the above criteria.

That all seems to me to by quite reasonable, and it’s certainly one way of defining what a UKIDSS paper is. According to that measure, UKIDSS scores 226.

The Warren measure does, however, include a number of papers that don’t directly use UKIDSS data, and many written by people who aren’t members of the UKIDSS consortium. Being picky you might say that such papers aren’t really original UKIDSS papers, but are more like second-generation spin-offs. So how could you count UKIDSS papers differently?

I just tried one alternative way, which is to use ADS to identify all refereed papers with “UKIDSS” in the title, assuming – possibly incorrectly – that all papers written by the UKIDSS consortium would have UKIDSS in the title. The number returned by this search was 38.

Now I’m not saying that this is more reasonable than the Warren measure. It’s just different, that’s all. According to my criterion however UKIDSS measures 38 rather than 226. It sounds less impressive (if only because 38 is a smaller number than 226), but what does it mean about UKIDSS productivity in absolute terms?

Not very much, I think is the answer.

Yet another way you might try to judge UKIDSS using bibliometric means is to look at its citation impact. After all, any fool can churn out dozens of papers that no-one ever reads. I know that for a fact. I am that fool.

But citation data also provide another way of doing what Steve Warren was trying to measure. Presumably the authors of any paper that uses UKIDSS data in any significant way would cite the main UKIDSS survey paper led by Andy Lawrence (Lawrence et al. 2007). According to ADS, the number of times this has been cited since publication is 359. That’s higher than the Warren measure (226), and much higher than the UKIDSS-in-the-title measure (38).

So there we are, three different measures, all in my opinion perfectly reasonable measures of, er, something or other, but each giving a very different numerical value. I am not saying any is misleading or that any is necessarily better than the others. My point is simply that it’s not easy to assign a numerical value to something that’s intrinsically difficult to define.

Unfortunately, it’s a point few people in government seem to be prepared to acknowledge.

Andy Lawrence is 57.

14 Comments »

Batting for Astronomy

Posted in Science Politics with tags astronomy, cuts, funding, Jocelyn Bell Burnell, Keith Mason, Roger Davies, STFC on March 9, 2011 by telescoper

I was too busy teaching this morning to watch streaming video of the meeting of the House of Commons Science & Technology Committee I referred to in a previous post, but then, being a confirmed Luddite, I rarely manage to get such things to work properly anyway. Or is it just that Parliament TV isn’t very good? Anyway, I did get the chance to do a fast-forward skim through the coverage, and also saw a few comments on Twitter.

By all accounts the two big hitters for astronomy, Professor Roger Davies and Dame Jocelyn Bell Burnell both played good innings, watchful in defence, parrying the odd tricky delivery, but also scoring impressively when the opportunity arose. Dame Jocelyn, for example, got in a nice comment to the effect that the shortfall in observatory funding was equivalent to one banker’s bonus.

Any other reactions are welcomed through the comments box.

The e-astronomer (whose pseudonym is Andy Lawrence) has already blogged about the event, including a delightfully pithy summary of the written evidence submitted beforehand . But then Andy’s never reluctant to take the pith when the opportunity arises…

The thing that depresses me most is the contrast between the forthright and well-considered performances of leading figures from the astronomy establishment with the bumbling efforts of the Chief Executive of STFC, Keith Mason. As Andy Lawrence points out, some of the latter’s responses to questions at the last session of the inquiry were downright misleading, giving the impression that he didn’t know what he was talking about. And that’s the more generous interpretation. Combine the poor grasp of detail with his generally unenthusiastic demeanour, and it becomes easy to see that one of the main reasons for the ongoing crisis at STFC is its Chief Executive.

I’ve been told off repeatedly in private for posting items on here that are severely critical of Professor Mason, sometimes on the grounds that my comments are ad hominem, a phrase so frequently misused on the net that it is in danger of losing its proper meaning. It’s not an “ad hominem” attack to state that a person is demonstrably useless at their job. I stand my ground. He should have gone years ago.

Unfortunately we still have to wait another year or so before a replacement Chief Executive will be installed at STFC. Good people elsewhere – both inside and outside science – have lost or are losing their jobs, because of the recession and cutbacks, through no fault of their own. Reality is much less harsh if you’re at the top.

3 Comments »

In the Dark