Negative Impact
After spending the best part of the last couple of days being prodded and poked and subjected to all manner of indignity in the name of medical science, I think it’s appropriate to return to the blogosphere with another rant. Before I start, however, I’d seriously like to thank everyone at the University Hospital of Wales at Heath Park for making my visit there as brief and painless as possible. Everyone was very kind and very efficient. I’m not going to blog about the details, as Columbo doesn’t like reading about other peoples’ ailments.
Over the past few weeks there has been a lot of discussion about the UK government’s agenda for research, particularly science research, that includes something called “impact”. The Research Excellence Framework (REF; successor to the Research Assessment Exercise, RAE) will include such a thing:
Significant additional recognition will be given where researchers build on excellent research to deliver demonstrable benefits to the economy, society, public policy, culture and quality of life
Apparently, however, they don’t really know how to do this so they have set up a number of pilot studies to try to find out. I’d feel a little more comfortable if the bureaucrats had thought about what they were going to do before announcing that our future research funds were going to depend on it. Meanwhile, applicants for grants from any of the research councils must include a statement of the “economic or social” impact their research will have.
Understandably, those of us working in “blue skies” research are very nervous about this new regime. There is more than a suspicion that the new emphasis on impact is intended to divert funds away from “pure” curiosity driven research and into areas where it can have an immediately identifiable short-term economic benefit. This has led to a petition, with over 13000 signatures, by the University and College Union calling for the impact statements to be abandoned.
I don’t know who is going to assess these impact statements, but unless they have a flawless ability to predict future technology I don’t think fundamental physics is going to score very well at all. To see my point, consider the case of J. J. Thomson, who is generally credited with having discovered the electron and who was awarded the Nobel Prize for Physics in 1906. Thomson made extensive use of cathode ray tubes in his studies; these later found their way into sitting rooms across the world as essential components of the classic television set. But that took decades. I doubt if an impact panel looking at Thomson’s work – even if they were physicists rather than grey-suited bureaucrats – would have found any of it likely to lead to immediate economic benefit. The point is that when he discovered the electron it wasn’t because he was actually trying to invent the television set.
I think there are basically two possible interpretations of this impact business. One is that it is a deliberate plan to wind down fundamental research and use the money saved to subsidise UK industry. The other is that it’s another exercise in pointless box-ticking. I am in two minds. On the one hand, it is clear that the recent behaviour of the Science and Technology Facilities Council shows strong evidence of the former. Fundamental research is being slashed, yet projects involving space technology have been funded on the nod without scientific peer review. On the other hand, the RCUK Impact “Champion”, a person by the name of David Delpy, has written in the Times Higher to defend the new agenda. Consider the following paragraph
Recently I have read that some believe it is impossible to predict the economic impact of blue-skies research. To be clear, we are not asking for accurate predictions – simply a consideration of potential. Basic research underpins all disciplines and builds pathways to new technologies with economic and social applications. It may build on an existing body of knowledge, connect to other research around the world or attract new industries to the UK. There are many routes to impact. I believe that I could write a statement indicating potential impact for any proposal I have seen, and to hear that bright academics say they can’t do it sounds a little disingenuous.
Champion Delpy thus suggests he could write a statement for any proposal he has seen, which sounds to me like an admission that what is called for is just a load of flannel. In fact, if he’s paid to be the Impact Champion perhaps he should write all the bullshit and save us scientists the need to jump through these silly hoops? Or perhaps we could get one of those little Microsoft Office Assistant things:
Hello. Looks like you’re writing an Impact Assessment. Would you like me to pad it out with meaningless but impressive-looking socio-economic buzzwords for you?
If it’s just another exercise in vacuous bureaucracy then it’s bad enough, but if it is the other possibility then of course it’s even worse. It could be the end for disciplines like astronomy and particle physics as well as the end of Britain’s history of excellence in those areas. I’ve already blogged about my view of short-termism in research funding. Essentially, my point is that government money should be used to fund precisely those things that don’t have immediate economic benefit. Those that do should be funded by the beneficiaries, i.e. commercial companies.
Politicians probably think that all this complaining about impact means that scientists are arrogantly assuming that the taxpayer should fund them regardless of the cost or the benefit. I can only speak for myself, but I think that’s very unfair. I’m very conscious that my research is funded by Joe Public; that’s one of the reasons I think I should spend time giving public talks and doing other outreach activities. But I think the public funds me and others like me to do “useless” things because, in the end, useless things are more important than money.
The government is probably right to say that the UK economy doesn’t benefit as much from our scientific expertise as is the case with other countries. The reason for that, however, lies not with our universities and research laboratories but with our private industrial and commercial sectors which are, for the most part, managed with a very low level of competence. British universities are demonstrably excellent; our industry is demonstrably feeble. The persistent failure of the private sector to invest in research and development shows that it is in drastic need of a good shake up. British companies, not the taxpayer, should be paying for research that leads to profit for them and for that to happen they will have to learn to engage better with the University sector rather than expecting inventions to be served up on a plate funded by the taxpayer. Universities and research labs should continue do what they’re good at, maintaining a culture within which curiosity and learning are promoted for their own sake not just as part of the dreary materialistic cycle of production and consumption that is all we seem to be able to think about these days.
So at the end I’ve come to the conclusion that, perhaps, insofar as it can be demonstrated, economic impact should be included in the forthcoming Research Excellence Framework. Research which leads directly to the economic gain of the private sector is precisely the type of research that the taxpayer should not be paying for. If it can be proven that a given department has engaged in such activity, its state funding should therefore be cut and it should be told to recover the funds it has misused from the company that has benefitted from it. Economic impact should be included with a negative weight.
And if you think that’s a silly point of view, consider what happens with the other major part of a university’s activity, teaching. Students, we are told, are the primary beneficiaries of their education so they should have to pay fees. In the current regime, however, they only do so when their earnings reach a certain level. If commercial companies are to be the primary beneficiaries of state-funded research, why should they not likewise be asked to pay for it?
In the Dark 
December 2, 2009 at 9:44 pm
Peter,
Glad to see you back in the saddle. All the best,
Adrian
December 2, 2009 at 9:59 pm
Very interesting discussion of the principals, but I think we can be both robust and positive about the impact our work has.
Economic impact is just one of a much longer list of different types of impact that are being considered. When you go out and do a talk about your research, you are having a social and educational impact, perhaps also a cultural impact. We don’t have to rely on buzzwords or verbage (though I do worry that other universities might get away with a flimsy approach), we can measure indicators of success against these other types of impact and demonstrate our case with statistics if we are prepared to go and get them.
There is no denying that blue-sky research can have an unforseen impact, and that can’t be accurately predicted in advance, but surely you’ve never asked for funding for a piece of research saying ‘Its no use to man nor beast but I thought it was entertaining. Any chance of a grant?’. Surely you set out hoping to achieve something?
December 2, 2009 at 10:14 pm
It’s true that forms of impact other than economic are mentioned, but I suspect economic benefit is the only one the powers that be are really interested in. In any case how do you measure cultural impact, for example? How can you compare the cultural impact of cosmology versus that of mediaeval literature or art history? What statistics do we collect? And against what criteria do we measure our performance? Who decides whether an impact statement is good or not? How do they know?
When we write grant applications we do have to put our proposal in the context of ongoing research and show how it can have an impact inside its own sphere. Such proposals are judged by ones peers who should have some chance of deciding whether what you want to do is likely to have impact within your field. But that’s not what we’re talking about. We’re talking about the need to demonstrate impact beyond our field in the immediate short term.
p
December 2, 2009 at 11:16 pm
Peter, good to see you’re back.
It was interesting watching the “Bue Skies Ahead?” debate hosted by THE on Monday (still available at http://www.timeshighereducation.co.uk/sciblue.asp and summarised by Zoe Corbyn at http://www.timeshighereducation.co.uk/story.asp?sectioncode=26&storycode=409385&c=1). There was quite a lot of discussion about impact/funding/STFC, with a little mention of the other subjects on the list (e.g. science “education”).
At one point in the discussion on impact, one of the panel pointed out that the LHC did not have high “impact” on economic grounds. However, he said it was invaluable as an inspirational tool for students. Now, Lord Drayson agreed that this gave it impact, acknowledging that economic impace was not the *only* factor. He asked whether it was only fair to give the LHC an opportunity to promote their “inspirational” impact on future grant applications. The panel responded with “yes, but they shouldn’t be *forced* to”.
Now, the way I see it, the LHC is “cool” for two main reasons: 1) the potential effect it could have on our knowledge of fundamental physics and cosmology, and 2) the fact that it’s really big, very cold, and quite a feat of engineering. (Oh, and it’s quite expensive, too, which tends to grab the public’s attention for either good of bad reasons). There is little “economic” impact obviously available from it on the timescale of decades (I may be paying a little unfair there) apart from giving jobs to contractors and developing a few detector technologies (which won’t be available to the public for a number of years, perhaps decades). The same situation exists largely for other experiments (e.g. Herschel).
The potential impact on physics knowledge should be very apparent from the scientific part of any proposal, so there’s no need for further considerations. The impressive nature of the actual experiment is an obviously very useful way of inspiring future scientists, and should be expressed in the “outreach” section of the proposal. Now, I’m sure Lord Drayson would agree that we shouldn’t be building multi-billion-pound experiments because the look cool, and that if a competing experiment were able to achieve the same of better science with a smaller, cheaper, simpler experiment then that should be funded.
If these are the sort of things that will be allowed in this new category on the funding forms, then “impact” is much more wishy-washy than I first thought. That’s good for anyone doing “cool” or “big” science, but presumably defeat’s the intention of having it.
I also completely agree that if the beneficiaries of University research are commercial companies, then they should contribute towards the funding. Having said that, I’d much rather take part in ground-breaking research for its scientific merits, not because I’m getting a cut from “ACME Cosmologies” or similar.
[Sorry, this comment ended up longer than I intended when I started typing…]
December 3, 2009 at 7:28 am
Hi Peter,
Whilst there is an issue around how impact of all sorts is measured, I cannot see any argument against the principle that all research teams should be doing their best to ensure that their work has an impact on the world beyond academia. This is possibly even more important for blue skies research as the more people know about it, the more likely it is for the serendipitous uses to be identified.
December 3, 2009 at 7:36 am
No, but should they have their funding cut if they can’t provide measurements of the impact they have had? Should they change what they’re doing to something less speculative where they can tick the right boxes? Or should they do less research and spend more time trying to find applications of what they’ve done? I think all of those are wrong.
December 3, 2009 at 10:35 am
Perhaps you and colleagues should submit several proposals for pure research to Delpy that have been knocked back in recent years, and ask him to write impact reviews of them as models to assist scientists with their future proposals. Since he has claimed that he could do this, it would be a reasonable request.
Anton
December 3, 2009 at 8:01 pm
Yes, I agree strongly with most of what Peter has said.
The science minister, Lord Drayson, has argued that it is reasonable for researchers to consider the broader value of their work to society, and it is not entirely unreasonable for the research councils to insist that some impact statement is included in grant applications. What would be wholly unacceptable would be for impact statements to be used in selecting which grant applications are funded. Forcing academics to spend 15 minutes considering the broader consequences of their proposed research is fine, but actually determining funding on the basis of “impact” would not be. In practice, I cannot imagine grant panels in astronomy being significantly influenced by “impact statements”: they are likely to judge applications on the basis of scientific merit regardless of what they are told to do.
The big potential problem here is over the Research Excellence Framework. Allocating 20% or 25% of the research funding of university departments from the higher education funding councils on the basis of “impact” could be very damaging, as Peter rightly pointed out. But this would inevitably raise issues of the relative importance of industrial impact against social impact. This would certainly affect the allocation of funding within units of assessment. For example, within physics (plus astronomy), how would the impact of a mediocre researcher within industrially applied solid state physics be compared with that of an outstanding particle physicist? Could economic impact and cultural impact be compared? Would economic impact be considered more significant than cultural impact? If “impact” is handled (like research “environment”) as a department-wide criterion, would physics departments with research close to industry fare better than those pursuing mostly basic physics?
I’m sure that the research community would not object too strongly if “impact” contributed 5% or 10% of the total funding in the R.E.F., as an initial test of the system. However, the idea of allocating 25% of departments’ funding on the basis of such a woolly concept seems like madness. How could such a subjective concept as impact be measured?
December 3, 2009 at 8:48 pm
I realise that I missed an even better example than J.J. Thomson. A latter in todays Times Higher (from John Dainton) asks whether Sir Ernest Rutherford would have been funded for his work on the atomic nucleus at Manchester and Cambridge with an application adorned with the following impact statement: ” Anyone who expects a source of power from the transformation of the atom is talking moonshine.” After all, that’s what he said at the time.
December 3, 2009 at 9:42 pm
There is a similar story involving Faraday that goes, from memory, something like this.
Michael Faraday met William Gladstone while Gladstone was Chancellor of the Exchequer. Gladstone commented on Faraday’s research on electricity and magnetism and asked, “Pray, Mr. Faraday, what use is it?”
Faraday replied,
“I know not, sir, but I venture to suggest that some day you will tax it.”
(The story may be apocryphal. I did once search for 19th century occurrences of the story but couldn’t find anything earlier than about 1920.)
December 4, 2009 at 12:29 am
Hi, Peter.
Great post. Leslie Ann Goldberg at the University of Liverpool has put together a very helpful web page, The Dangers of Assessing Research by Economic Impact , which comprises a series of links to articles pointing out the flaws in the HEFCE and RCUK impact proposals/procedures.
I guess that I don’t need to tell readers of your blog about Paul Crowther’s excellent STFC Crisis collection of links (which actually covers a much wider “remit” than STFC). Paul deserves some sort of award for maintaining that site.
Your point re. the failure of the private sector to invest in research is well made. Interesting to look at this from the point of view of economists such as Richard R Nelson and Paul David (see, for example, Nelson’s The Market Economy and the Scientific Commons and The Economic Benefits of Publicly Funded Basic Research: A Critical Review by Salter and Martin). It’s perhaps also worth mentioning that a Prime Minister’s Strategy Unit report from 2006 stating that the economic return on basic research was effectively “incalculable”.
Best wishes,
Philip
December 6, 2009 at 1:23 pm
[…] amongst all the doom and gloom about job cuts and the oncoming onslaught that goes by the name of impact, I found in this week’s Times Higher a thought-provoking article about the demise of poetry. […]
December 6, 2009 at 5:31 pm
Peter –
(i) you asked whether the impact plan was about running down fundamental science, or about box ticking. I fear the answer is both. The Research Councils need to be seen to be addressing Government concerns, and depressingly, this is probably more important than actually addressing Government concerns.
(ii) I never met a politician who didn’t understand perfectly well that apparently pure research can have huge unforeseen economic impact decades hence, and so any sane government should continue to fund pure research. But this doesn’t tell you HOW MUCH to spend on pure research.
(iii) Astronomy and particle physics expanded enormously after WWII. This is because of Radar and The Bomb. They decided we were useful and clever, and we figured out how Government worked. What is the reason we shouldn’t shrink back to 1935 levels ?
December 6, 2009 at 9:49 pm
Andy,
Perhaps you’re right. After all we seem to be heading back to the 1930s in many other ways: economy on the skids, fascist parties on the rise, nothing on the telly….
Peter
December 6, 2009 at 11:16 pm
Actually there is far too much on the telly. Or, as The Boss once sang, “fifty seven channels and there’s nothing on”
December 6, 2009 at 11:37 pm
I’m not sure who your boss is but mine doesn’t sing. Apart from the tache he does a fine impression of Swiss Tony though.
December 7, 2009 at 10:44 am
It must be said that if there is a deliberate government policy to switch funding from basic research, this is not the way to go about it. Were that the policy, the change ought to be stated clearly and justified, so that the research community could plan (including some academics taking early retirement and others emmigrating). There would also need to be a large-scale switch in the number of PhD studentships from basic to applied subjects, to prepare the shape of the future research community.
Simply inventing some fluffy concept of “impact”, defining it vaguely to include economic and social factors, and then basing a lot of funding on it, is not the way to go about a switch from basic to applied research. The systems lacks clarity and the research community cannot plan for the consequences.
If the U.K. government intends to scale down astronomy, why did it agree to fund U.K. participation in the Aurora programme a few years ago? Signals from government are confused.
December 10, 2009 at 10:28 pm
[…] as judged by peers. Note that, in the context of the REF, this is a completely different thing to impact which counts a smaller fraction of the assessment and which is supposed measure the influence of […]
December 11, 2009 at 10:43 am
… and here’s another thing. According to this paper, papers with larger numbers of authors get cited more – possibly because they get more auto-citations. Should one normalise the number of citations by the mean number for a paper with a given length of author list? Or is it all getting very silly?
December 23, 2009 at 12:24 pm
[…] Impact again. I’ve explained what that means already […]
January 3, 2010 at 3:25 pm
[…] of the latter organisations heavy-handed top-down management style and gung ho enthusiasm for the impact agenda which may be appropriate for applied sciences and engineering but surely doesn’t make any […]