Archive for the Open Access Category

Particle physics volunteers to be fleeced….

Posted in Open Access with tags , , , , on September 26, 2012 by telescoper

I heard the news yesterday that a body called the Sponsoring Consortium for Open Access Publishing in Particle Physics (SCOAP3) has arranged a deal whereby virtually all articles in particle physics will be available for free on journal websites. The deal will mean that authors will not have to pay thousands of dollars up-front in “article processing charges” in order to have their work available via Open Access media.

So far so good, you’re probably thinking. But read a little bit more about it and it becomes absolutely clear that SCOAP3 has walked straight into a trap laid by the academic publishers with whom it brokered the agreement. The principal deterrent to authors publishing via the “Gold” Open Access model has been that they would have to pay up-front fees, potentially around $2000 for each paper. Any sensible researcher would rather spend $2000 supporting their research than lining the profits of greedy publishers, so would probably opt for a “green” mode instead. Indeed many particle physicists already do this, putting their work on the arxiv where it is available for free anyway.

The publishing industry realises that most authors would simply bypass it and go for self-publication if they could, so it is naturally very keen on deals like this. What actually happens in the SCOAP3 agreement is that an author’s institution pays fees directly to the publisher. According to Nature News:

The consortium will pay the contracts from an annual budget of €10 million, which is funded not by authors or research grants, but by pledges from more than a thousand libraries, funding agencies and research consortia across the world. In effect, existing journal subscription fees are being repurposed to provide the open-access funds.

And there’s the rub. “Existing journal subscription fees” are already extortionately high, and out of all proportion to the actual cost of disseminating scientific knowledge. Authors may think that they’re not paying for Open Access under the new agreement, but in fact they are. It’s just a bit less direct. Their grants will continue to be top-sliced to pay for the SCOAP3 arrangement and, since science budgets are unlikely to rise for the foreseeable future, that means the cash available for actually doing research will fall. This agreement is very good for the publishers, but very bad for science.

The average cost for Open Access publication in Physics Review D. under the new scheme will be $1900 per paper. Ouch! And how does the publisher justify this cost? “To maintain revenue levels…”. I rest my case.

More of the  is going to happen in the UK, where £10M is being set aside from existing Research Council budgets, nominally to “pay for the transition to Open Access” but actually in order to maintain profit levels at the big academic publishing houses. Much of that £10M will no doubt disappear in deals like the one brokered by SCOAP3.  And that means continuing high profits for the publishers at the expense of falling levels of research funding. The whole thing stinks.

And if as an author you decide that you have a moral objection to being scammed in this way, under the SCOAP3 agreement you now have no way out. Even if you bypass the arrangement and just publish on the arXiv, the publishers will get their money directly anyway. You have to admit it’s a clever sting, but I’m still surprised the particle physics community has fallen for it.

This development convinced me even more that the research community has to take matters into its own hands, and organize its own publication strategy. Traditional journals are already virtually redundant and I confidently predict they will die a natural death in just a few years, but while they linger on their publishers will continue to fleece the academic community as long as they can. The sooner we put a stop to it the better.

Nothing Gold Can Stay

Posted in Open Access, Poetry with tags , , , on September 13, 2012 by telescoper

Nature’s first green is gold,
Her hardest hue to hold.
Her early leaf’s a flower;
But only so an hour.
Then leaf subsides to leaf.
So Eden sank to grief,
So dawn goes down to day.
Nothing gold can stay.

by Robert Frost (1874-1963)

Time to go it alone on Open Access

Posted in Open Access, Science Politics with tags , , , , on September 10, 2012 by telescoper

Not at all surprisingly, the government has announced  that existing research council budgets are to be raided to provide funds (to the tune of £10M) to pay for “Gold” Open Access to scientific research. This is the model of open access in which most authors will have to pay publishers a whopping fee up front in order to disseminate their work. The figures being talked about are in the region of £2000 per paper by way of an “article processing fee”.

I put “article processing fee” in quotes there because a fee of that size bears no relation to the actual cost to the publishers of processing an article: articles in most physics journals are typeset by the author, and refereed for free by other academics suggested by the editor (another academic).  What it really represents is the amount of money researchers will have to pay to maintain the humongous profit margins currently enjoyed by the academic publishing industry. Currently they rake in the cash through subscription charges after papers have been published in their journals . In future they will get the dosh in advance, which will probably make their business even more lucrative. And who will pay for maintaining their profitability? Researchers, of course. It’s clear who is going to benefit from the provisions of the Finch Report, and it’s not us.

Not surprisingly the publishing racketeers want to try to make us think they provide a worthwhile service for all the money they sting us for. For example, in this month’s Physics World, there’s a response from Steven Hall (Managing Director of IOP Publishing) to a letter from a certain Dr Garrrett. The original letter pointed the facts of the current state of affairs that I have bemoaned on many occasion on this blog:

Currently, researchers have to typeset their own work, sign away the copyright to publishers and referee the work of their peers – all for no remuneration. They then pay large sums in publication fees or library subscriptions to buy that work back in refereed and collated form.

Steven Hall’s response includes the following paragraph:

Researchers do not perform peer review alone: publishers organize and manage it, invest in people and systems to facilitate it, appoint and support editorial boards to oversee it and develop journals to meet the needs of scientific communities.

This is very far from being an accurate or fair representation of the way things work, at least not in physics. Researchers do carry out peer review alone. And unpaid. The main system that facilitates it is email (which, to my knowledge, was not developed by the academic publishing industry). And the journals that IOP develops are less to do with the “needs” of scientific communities than they are with the desires of a profit-making company to exploit said communities for even greater commercial gain.

Don’t you think it’s very strange that in a time of shrinking library budgets the number of journals seems to be growing all the time? Do we really need new ones? Do we even need the old ones? I think not.

And for those of you who think that IOP Publishing, as a part of the Institute of Physics, must be acting in the best interests of physics research, that’s simply not the case. It’s run as a private publishing company that behaves in exactly the same unscrupulous profiteering manner as, e.g. Elsevier. The IOP’s Open Access journals already charge £1700 per paper in article processing fees. They’re also in the habit of peddling meaningless “impact factor” statistics when trying to market their journals, many of which have lamentably poor citation rates despite their extortionate costs. Hence the IOP’s practice of bundling journal subscriptions in order to force institutions who want the good stuff to pay for the dross as well.

Having looked carefully into the costs of on-line digital publishing I have come to the conclusion that a properly-run, not-for-profit journal, created for and run by researchers purely for the open dissemination of the fruits of their research can be made sustainable with an article processing charge of less than £50 per paper, probably a lot less.

There’s only one response possible to those who’ve hijacked the Finch committee to serve their own ends, and that is to cut them out of the process. I think we can do it better (and cheaper)  ourselves. And very soon I hope to prove it.

Open Journal of Astrophysics: Update

Posted in Open Access with tags , , , , , on August 27, 2012 by telescoper

Regular readers of this blog (Sid and Doris Bonkers) may recall that a few weeks ago I posted an item in which I suggested setting up The Open Journal of Astrophysics. The motivation behind this was to demonstrate that it is possible to run an academic journal which is freely available to anyone who wants to read it, as well as at minimal cost to authors. Basically, I want to show that it is possible to “cut out the middle man” in the process of publishing scientific research and that by doing it ourselves we can actually do it better.

I have been unwell for much of the summer, so haven’t been able to carry this project on as much as I would have liked, and  I also received many messages offering help and advice that I have been unable to reply to individually. But I can assure you that I haven’t forgotten about the idea, nor have I quietly withdrawn the financial backing I suggested in my earlier post. Indeed, my interest in, and excitement, about this project has grown significantly over the summer as new possibilities have been suggested and my resentment about how the academic publishing industry hijacked the Finch Report has deepened.

In fact, quite a lot of effort has already been put in by people elsewhere thinking about how to set this journal up in the best way to make maximal use of digital technology to produce something radically different from the stale formats offered by existing journals.  I hope to be able to report back soon with more details of how it will work, when we propose to launch the site, and even what its name will be, Open Journal of Astrophysics being just a working title. I think it’s far better to wait until we have a full prototype going before going further.

In the meantime, however, I have a request to make. The Open Journal of Astrophysics will need an Editorial Board with expertise across all astrophysics, so they can select referees and deal with the associated correspondence.  The success of this venture will largely depend on establishing trust with the research community and one way of doing that will be by having eminent individuals on the Editorial Board. I will be contacting privately various scientists who have already offered their assistance in this, but if any senior astronomers and/or astrophysicists out there are interested in playing a part please contact me. I can’t offer much in the way of remuneration, but I think this is an opportunity to get involved in a venture that in the long run will benefit the astronomical community immensely.

Oh, and please feel free pass this on to folks you think might be interested even if you yourself are not!

The Impact X-Factor

Posted in Bad Statistics, Open Access with tags , , on August 14, 2012 by telescoper

Just time for a quick (yet still rather tardy) post to direct your attention to an excellent polemical piece by Stephen Curry pointing out the pointlessness of Journal Impact Factors. For those of you in blissful ignorance about the statistical aberration that is the JIF, it’s basically a measure of the average number of citations attracted by a paper published in a given journal. The idea is that if you publish a paper in a journal with a large JIF then it’s in among a number of papers that are highly cited and therefore presumably high quality. Using a form of Proof by Association, your paper must therefore be excellent too, hanging around with tall people being a tried-and-tested way of becoming tall.

I won’t repeat all Stephen Curry’s arguments as to why this is bollocks – read the piece for yourself – but one of the most important is that the distribution of citations per paper is extremely skewed, so the average is dragged upwards by a few papers with huge numbers of citations. As a consequence most papers published in a journal with a large JIF attract many fewer citations than the average. Moreover, modern bibliometric databases make it quite easy to extract citation information for individual papers, which is what is relevant if you’re trying to judge the quality impact of a particular piece of work, so why bother with the JIF at all?

I will however copy the summary, which is to the point:

So consider all that we know of impact factors and think on this: if you use impact factors you are statistically illiterate.

  • If you include journal impact factors in the list of publications in your cv, you are statistically illiterate.
  • If you are judging grant or promotion applications and find yourself scanning the applicant’s publications, checking off the impact factors, you are statistically illiterate.
  • If you publish a journal that trumpets its impact factor in adverts or emails, you are statistically illiterate. (If you trumpet that impact factor to three decimal places, there is little hope for you.)
  • If you see someone else using impact factors and make no attempt at correction, you connive at statistical illiteracy.

Statistical illiteracy is by no means as rare among scientists as we’d like to think, but at least I can say that I pay no attention whatsoever to Journal Impact Factors. In fact I don’t think many people in in astronomy or astrophysics use them at all. I’d be interested to hear from anyone who does.

I’d like to add a little coda to Stephen Curry’s argument. I’d say that if you publish a paper in a journal with a large JIF (e.g. Nature) but the paper turns out to attract very few citations then the paper should be penalised in a bibliometric analysis, rather like the handicap system used in horse racing or golf. If, despite the press hype and other tedious trumpetings associated with the publication of a Nature paper, the work still attracts negligible interest then it must really be a stinker and should be rated as such by grant panels, etc. Likewise if you publish a paper in a less impactful journal which nevertheless becomes a citation hit then it should be given extra kudos because it has gained recognition by quality alone.

Of course citation numbers don’t necessarily mean quality. Many excellent papers are slow burners from a bibliometric point of view. However, if a journal markets itself as being a vehicle for papers that are intended to attract large citation counts and a paper published there flops then I think it should attract a black mark. Hoist it on its own petard, as it were.

So I suggest papers be awarded an Impact X-Factor, based on the difference between its citation count and the JIF for the journal. For most papers this will of course be negative, which would serve their authors right for mentioning the Impact Factor in the first place.

PS. I chose the name “X-factor” as in the TV show precisely for its negative connotations.

The Low-down on the LHC Boson

Posted in Open Access, The Universe and Stuff with tags , , , , , , on August 2, 2012 by telescoper

Although it’s a little late I thought I’d just put up a brief post to draw your attention to the news that a couple of technical papers have appeared on the arXiv giving updated details of the recent discovery at the Large Hadron of a new scalar particle that could be the Higgs boson. I don’t think it’s yet absolutely proven that this is what the new particle is, which is why I’ve called it the “LHC boson” in the title.

The ATLAS paper reports the detection of a Higgs-like particle with a 5.9 sigma confidence level, up from the 5.0 sigma reported on July 4. Here’s the abstract:

A search for the Standard Model Higgs boson in proton-proton collisions with the ATLAS detector at the LHC is presented. The datasets used correspond to integrated luminosities of approximately 4.8 fb^-1 collected at sqrt(s) = 7 TeV in 2011 and 5.8 fb^-1 at sqrt(s) = 8 TeV in 2012. Individual searches in the channels H->ZZ^(*)->llll, H->gamma gamma and H->WW->e nu mu nu in the 8 TeV data are combined with previously published results of searches for H->ZZ^(*), WW^(*), bbbar and tau^+tau^- in the 7 TeV data and results from improved analyses of the H->ZZ^(*)->llll and H->gamma gamma channels in the 7 TeV data. Clear evidence for the production of a neutral boson with a measured mass of 126.0 +/- 0.4(stat) +/- 0.4(sys) GeV is presented. This observation, which has a significance of 5.9 standard deviations, corresponding to a background fluctuation probability of 1.7×10^-9, is compatible with the production and decay of the Standard Model Higgs boson.

The paper from CMS reinforces the discovery of a Higgs-like particle with a mass of 125 GeV at a 5-sigma level of confidence:

Results are presented from searches for the standard model Higgs boson in proton-proton collisions at sqrt(s)=7 and 8 TeV in the CMS experiment at the LHC, using data samples corresponding to integrated luminosities of up to 5.1 inverse femtobarns at 7 TeV and 5.3 inverse femtobarns at 8 TeV. The search is performed in five decay modes: gamma gamma, ZZ, WW, tau tau, and b b-bar. An excess of events is observed above the expected background, a local significance of 5.0 standard deviations, at a mass near 125 GeV, signalling the production of a new particle. The expected significance for a standard model Higgs boson of that mass is 5.8 standard deviations. The excess is most significant in the two decay modes with the best mass resolution, gamma gamma and ZZ; a fit to these signals gives a mass of 125.3 +/- 0.4 (stat.) +/- 0.5 (syst.) GeV. The decay to two photons indicates that the new particle is a boson with spin different from one.

I’ll refrain from commenting on the use of frequentist language in both these papers, but instead just comment that these extremely important papers are available for free on the arXiv. Open access, we call it.

PS. There’s an interesting blog post related to these papers, about citations in particle physics here.

A Modest Proposal – The Open Journal of Astrophysics

Posted in Open Access with tags , , on July 17, 2012 by telescoper

Following on from yesterday’s piece, I thought I’d make a quick suggestion.

Astrophysics has taken the lead for many years in opening up access to scientific publications – all publications of any merit are available for free on the internet via the arXiv and, in my opinion, the traditional journals are already more-or-less redundant even without considering their “astronomical” cost. The one thing that seems a consistent objection to dispensing with journals altogether is the element of peer review.

However, yesterday’s news that many of us are probably going to have to pay up front fees to publish papers (at thousands of pounds  a pop) will no doubt have convinced many that the government’s decision is potentially ruinous for science. I think it’s time for astrophysics to lead the way again.

My suggestion is that we set up a quick-and-easy trial system to circumvent the traditional publishing route. The basic is that authors who submit papers to the arXiv can have their papers refereed by the community, outside the usual system of traditional journals. I’m thinking of a website on which authors would simply have to post their arXiv ID and a request for peer review. Once accepted, the author would be allowed to mark the arXiv posting as “refereed” and an electronic version would be made available for free on the website.

Although there are many issues to be sorted out,  a limited trial of The Open Journal of Astrophysics could I think happen very soon, perhaps even before the REF.

This idea would require community support setting it up (and doing the refereeing/administration)  and it would probably only get off the ground if sufficient senior astronomers get on board. It will no doubt also take money to get it going. However, I feel sufficiently strongly about this that I’m prepared to stump up £10k from my own pocket just to get it started.

If anyone has ideas about how to take this project forward, is willing to help with technical expertise, or if any prominent astronomers would simply like to add their name in support of the idea please do so through the comments box below.  Any further pledges of financial support would of course also be welcome.

And please pass this message on via Twitter/email/whatever, so I can gauge the level of support.

Open Access, of the Closed Kind

Posted in Open Access with tags , , , on July 16, 2012 by telescoper

Last night a story began circulating that the government, through RCUK, was intending to move quickly on the matter of open access to research outputs. This morning there’s a press statement from RCUK, the text of which is here:

Research Councils UK (RCUK) has today, 16th July 2012, unveiled its new Open Access policy. Informed by the work of the National Working Group on Expanding Access to Published Research Findings, chaired by Professor Dame Janet Finch, the policy at once harmonises and makes significant changes to existing Research Councils’ Open Access policies.

Professor Douglas Kell, RCUK Champion for Research and Information Management commented: “Widening access to the outputs of research currently published in journals has the potential to contribute substantially to furthering the progress of scientific and other research, ensuring that the UK continues to be a world leader in these fields. I am delighted that, together, the Research Councils have been able to been able both to harmonise and to make significant changes to their policies, ensuring that more people have access to cutting edge research that can contribute to both economic growth in our knowledge economy and the wider wellbeing of the UK.”

Drs. Astrid Wissenburg, Chair of RCUK Impact Group and RCUK representative on the National Working Group on Open Access commented: “As the bodies charged with investing public money in research, the Research Councils take very seriously their responsibilities in making the outputs from this research publicly available – not just to other researchers, but also to potential users in business; charitable and public sectors; and to the general public. Working with other funders such as HEFCE, DFID and the Wellcome Trust, this new policy signifies a move to a sustainable, affordable and transparent model of making outputs from the research that they fund more openly accessible.”

The new policy, which will apply to all qualifying publications being submitted for publication from 1 April 2013, states that peer reviewed research papers which result from research that is wholly or partially funded by the Research Councils:

  • must be published in journals which are compliant with Research Council policy on Open Access, and;
  • must include details of the funding that supported the research, and a statement on how the underlying research materials such as data, samples or models can be accessed.

Criteria which journals must fulfill to be compliant with the Research Councils’ Open Access policy are detailed within the policy, but include offering a “pay to publish” option or allowing deposit in a subject or institutional repository after a mandated maximum embargo period. In addition, the policy mandates use of ‘CC-BY’, the Creative Commons ‘Attribution’ license, when an APC is levied. The CC_BY licence allows others to modify, build upon and/or distribute the licensed work (including for commercial purposes) as long as the original author is credited.

The Research Councils will provide block grants to eligible UK Higher Education Institutions, approved independent research organisations and Research Council Institutes to support payment of the Article Processing Charges (APCs) associated with ‘pay-to-publish’. In parallel, eligible organisations will be expected to set-up and manage their own publication funds. The Research Councils will work with eligible organisations to discuss the detail of the new approach to funding APCs and to ensure that appropriate and auditable mechanisms are put in place to manage the funds.

Along with HEFCE and other relevant Funding Bodies, we shall monitor these policies actively, both to review their effects and to ensure that our joint objectives on Open Access are being met.

The RCUK policy on Access to Research Outputs is available here .

Although this seems like a victory for open access, it isn’t really. If it’s a victory for anyone it’s a victory for the  cartel of  ruthlessly exploitative profiteers that is the Academic Publishing Industry. For what the RCUK proposal involves is shifting the “cost” of scientific publishing from journal subscriptions to “Article Processing Charges”, which means authors will have to pay upfront to have their work  considered for publication. And when I say “pay”, I mean pay. It’s anticipated that the average APC for a paper will be around £2000. That’s why they call it “Gold” Open Access, I suppose.

An APC of this size  is indefensible. Scientific papers are nowadays typeset by the author and refereed by other academics. The cost to the publisher is tiny. That they need such an extortionate amount to maintain their profit levels just demonstrates the extent to which they’ve  been ripping us of all these years. Worse, having to pay up front  excludes scientists who don’t have access to the funds needed to pay these charges. This isn’t open access, it’s just a slightly different form of the old racket.

Moreover, I understand that no new money is coming to pay these charges. RCUK is finding the funds quoted above from its existing budget. That means that research somewhere will be cut to pay the additional cost of running the new system alongside the old. Better in my view to cut out the publishers altogether, and let universities and researchers do everything themselves. In astrophysics, we’re most of the way there already, in fact.

I for one have no intention of ever paying an Article Processing Charge. If the journals I publish in insist on levying one, I’ll just forget about the journals altogether and put my papers on the arXiv. I urge my colleagues to do the same.

Science 2.0 and all that

Posted in Open Access, Science Politics with tags , , , on July 9, 2012 by telescoper

I cam across this on Twitter today and thought I’d share it. Although I have written at various times about open access and the virtues of sharing scientific data, I hadn’t realised that such things came under the umbrella of “Science 2.0“, a term which is quite new to me. This post contains some very interesting ideas and information on the subject.

katarzynasz's avatarScience 2.0 study

We’re approaching the final stage of our study. So far, we have  opened up our bibliography on our Mendeley group here; our notes through this very blog; our model for open science; and our draft policy recommendations for EU. And we’ve benefited from your comments and insight.

Now, we need your help to improve the evidence about the importance of Science 2.0, if we want policy-makers to take it seriously.

Therefore, we share the final presentation that we have presented to the European Commission, DG RTD here.

Help us improving it, by gathering more data and evidence, showing that Science 2.0 is important and disruptive, and that it’s happening already. In particular, we ask to share evidence and data on the take-up of Science 2.0: how many scientist are adopting it? With what benefits?

We ask all people interested in Science 2.0 to share the evidence at hand, by adding

View original post 15 more words

Clusters, Splines and Peer Review

Posted in Bad Statistics, Open Access, The Universe and Stuff with tags , , , , , on June 26, 2012 by telescoper

Time for a grumpy early morning post while I drink my tea.

There’s an interesting post on the New Scientist blog site by that young chap Andrew Pontzen who works at Oxford University (in the Midlands). It’s on a topic that’s very pertinent to the ongoing debate about Open Access. One of the points the academic publishing lobby always makes is that Peer Review is essential to assure the quality of research. The publishers also often try to claim that they actually do Peer Review, which they don’t. That’s usual done, for free, by academics.

But the point Andrew makes is that we should also think about whether the form of Peer Review that journals undertake is any good anyway.  Currently we submit our paper to a journal, the editors of which select one (or perhaps two or three) referees to decide whether it merits publication. We then wait – often many months – for a report and a decision by the Editorial Board.

But there’s also a free online repository called the arXiv which all astrophysics papers eventually appear on. Some researchers like to wait for the paper to be refereed and accepted before putting it on the arXiv, while others, myself included, just put it on the arXiv straight away when we submit it to the journal. In most cases one gets prompter and more helpful comments by email from people who read the paper on arXiv than from the referee(s).

Andrew questions why we trust the reviewing of a paper to one or two individuals chosen by the journal when the whole community could do the job quicker and better. I made essentially the same point in a post a few years ago:

I’m not saying the arXiv is perfect but, unlike traditional journals, it is, in my field anyway, indispensable. A little more investment, adding a comment facilities or a rating system along the lines of, e.g. reddit, and it would be better than anything we get academic publishers at a fraction of the cost. Reddit, in case you don’t know the site, allows readers to vote articles up or down according to their reaction to it. Restrict voting to registered users only and you have the core of a peer review system that involves en entire community rather than relying on the whim of one or two referees. Citations provide another measure in the longer term. Nowadays astronomical papers attract citations on the arXiv even before they appear in journals, but it still takes time for new research to incorporate older ideas.

In any case I don’t think the current system of Peer Review provides the Gold Standard that publishers claim it does. It’s probably a bit harsh to single out one example, but then I said I was feeling grumpy, so here’s something from a paper that we’ve been discussing recently in the cosmology group at Cardiff. The paper is by Gonzalez et al. and is called IDCS J1426.5+3508: Cosmological implications of a massive, strong lensing cluster at Z = 1.75. The abstract reads

The galaxy cluster IDCS J1426.5+3508 at z = 1.75 is the most massive galaxy cluster yet discovered at z > 1.4 and the first cluster at this epoch for which the Sunyaev-Zel’Dovich effect has been observed. In this paper we report on the discovery with HST imaging of a giant arc associated with this cluster. The curvature of the arc suggests that the lensing mass is nearly coincident with the brightest cluster galaxy, and the color is consistent with the arc being a star-forming galaxy. We compare the constraint on M200 based upon strong lensing with Sunyaev-Zel’Dovich results, finding that the two are consistent if the redshift of the arc is  z > 3. Finally, we explore the cosmological implications of this system, considering the likelihood of the existence of a strongly lensing galaxy cluster at this epoch in an LCDM universe. While the existence of the cluster itself can potentially be accomodated if one considers the entire volume covered at this redshift by all current high-redshift cluster surveys, the existence of this strongly lensed galaxy greatly exacerbates the long-standing giant arc problem. For standard LCDM structure formation and observed background field galaxy counts this lens system should not exist. Specifically, there should be no giant arcs in the entire sky as bright in F814W as the observed arc for clusters at  z \geq 1.75, and only \sim 0.3 as bright in F160W as the observed arc. If we relax the redshift constraint to consider all clusters at z \geq 1.5, the expected number of giant arcs rises to \sim 15 in F160W, but the number of giant arcs of this brightness in F814W remains zero. These arc statistic results are independent of the mass of IDCS J1426.5+3508. We consider possible explanations for this discrepancy.

Interesting stuff indeed. The paper has been accepted for publication by the Astrophysical Journal too.

Now look at the key result, Figure 3:

I’ll leave aside the fact that there aren’t any error bars on the points, and instead draw your attention to the phrase “The curves are spline interpolations between the data points”. For the red curve only two “data points” are shown; actually the points are from simulations, so aren’t strictly data, but that’s not the point. I would have expected an alert referee to ask for all the points needed to form the curve to be shown, and it takes more than two points to make a spline.  Without the other point(s) – hopefully there is at least one more! – the reader can’t reproduce the analysis, which is what the scientific method requires, especially when a paper makes such a strong claim as this.

I’m guessing that the third point is at zero (which is at – ∞ on the log scale shown in the graph), but surely that must have an error bar on it, deriving from the limited simulation size?

If this paper had been put on a system like the one I discussed above, I think this would have been raised…