In the Dark

There’s a short article on arXiv by astronomer Luigi Rolly Bedin with the title The Preprint Evolution: The Rise of Unreviewed Drafts on arXiv and its Implications for Astronomy and abstract:

The arXiv preprint server originally democratized astronomical research by digitizing the circulation of early manuscripts and removing the financial barriers of journal subscriptions. However, a system designed for the lower publication volumes of the 1990s has fractured under modern research output. Today, the sheer volume of unreviewed manuscripts—often tagged as “submitted”, “under review”, or “comments welcome” imposes a severe cognitiv “tax” on the community. Researchers are forced to track multiple, unvetted versions of the same work, creating a massive redundant workload. This practice also bypasses the blinded peer-review process by establishing premature precedence, and it takes a heavy toll on early-career researchers. The pressure to rapidly post unvetted results stifles independent research, pushing young scientists toward massive mainstream collaborations where individual creativity gets lost. In this Short Communication, I analyze a decade of astro-ph metadata to quantify this trend. To safeguard both researchers’ time and research freedom, I propose bifurcating submissions into two separate streams. Level-1 (peer-reviewed science) would host peer-reviewed science (accompanied by a journal acceptance or final DOI), while Level-2 (pre-review drafts) would be reserved for discussion drafts. This distinction would remove the unfair advantage of establishing false precedence through drafts and drastically cut the daily reading burden by allowing researchers to opt out of tracking unvetted versions.

arXiv:2609.35787

I’ve mentioned this paper on this blog because a couple of readers commented about it on a post earlier this week (here and here) I’ll leave it to readers to decide for themselves what they think about the suggestion of having a two-tier arXiv as described in the abstract. A rebuttal has been posted on arXiv here.

For myself I’ll just say that I think that, for such a short paper, it contains a very large number of unsubstantiated claims. I don’t doubt that the number of pre-review submissions has increased over the years, but are researchers really “forced to track multiple, unvetted versions of the same work, creating a massive redundant workload”? Is it really the case that the existence of unrefereed articles on arXiv “stifles independent research, pushing young scientists toward massive mainstream collaborations where individual creativity gets lost”? Where is the evidence for this claim? The claim of a `severe cognitive “tax” on the community’ seems a huge stretch to me too. If you really don’t want to read unrefereed articles on arXiv then you can just ignore those that don’t have a journal reference in their metadata.

I wrote a piece about preprints some time ago here. In it I tried to explain how preprints were initially used to circulate refereed work in advance of the final printed version being available, which often took many months after acceptance. Back in the old days these were usually produced by institutions with a distinctive design, logo, etc which gave them a professional look, which made it easier to distinguish `serious’ papers from crank material (which was also in circulation). This also suggested that some internal refereeing inside an institution had taken place before an “official” preprint was produced, lending it an air of trustworthiness. Smaller institutions couldn’t afford all this, so were somewhat excluded from the preprint business.

With the arrival of the arXiv the practice of circulating hard copies of preprints in astrophysics gradually died out, to be replaced by ever-increasing numbers of electronic articles. The arXiv does have some gatekeeping – in the sense there are some controls on who can deposit a preprint there – but it is far easier to circulate a preprint now than it was.

(I am, incidentally, very surprised that this article – with only reference (to itself) and no evidence to support its claims – passed the arXiv moderation process for astro-ph, but there you go…

Anyway, it is still the case that big institutions and collaborations insist on quite strict internal refereeing before publishing a preprint – and some even insist on waiting for a paper to be accepted by a journal before adding it to the arXiv – but there’s no denying that among the wheat there is quite a lot of chaff, some of which attracts media coverage that it does not deserve. It is also the case that some articles that have passed peer review are flawed in crucial ways not spotted by referees. No system that is operated by human beings will ever be flawless, and peer review is no different. Not everything in “Tier 1” is perfect.

The most important thing is for science to advance. Circulating a paper in advance of review can help to achieve that. I have very often found that comments from other researchers reading the arXiv version supply comments and suggestions far more useful than one gets from a journal reviewer. Sometimes they can spot a fundamental flaw missed by a referee. If you put your paper on arXiv ahead of official peer review you will subject it to an additional level of scrutiny, increasing the chance that the final version will turn out to be a useful contribution to the literature.

Nowadays, in astrophysics, the single most important point of access to scientific literature is through the arXiv, which is why the Open Journal of Astrophysics was set up as an overlay journal to provide a level of rigorous peer review for preprints, not only to provide a sort of quality mark but also to improve the paper through the editorial process. There’s your Tier 1 for you.

P.S. I didn’t think of this at the time but one of the reasons that the fraction of unreviewed preprints is increasing is simply that people are not very good at updating their arXiv entry when a paper is actually published. I noticed this with Euclid papers, for example, which are generally posted on arXiv as preprints and subsequently published in A&A. It was noticeable, however, that some authors do not update the arXiv metadata after acceptance.