Archive for MNRAS

Do “high-quality journals” always publish “high-quality papers”?

Posted in Uncategorized with tags , , , , on May 23, 2023 by telescoper

After a busy morning correcting examination scripts, I have now reached the lunch interval and thought I’d use the opportunity to share a paper I found via Stephen Curry on Twitter with the title In which fields do higher impact journals publish higher quality articles?. It’s quite telling that anyone should ask the question. It’s also telling that the paper, in a Springer journal called Scientometrics is behind a paywall. I can at least share the abstract:

The Journal Impact Factor and other indicators that assess the average citation rate of articles in a journal are consulted by many academics and research evaluators, despite initiatives against overreliance on them. Undermining both practices, there is limited evidence about the extent to which journal impact indicators in any field relate to human judgements about the quality of the articles published in the field’s journals. In response, we compared average citation rates of journals against expert judgements of their articles in all fields of science. We used preliminary quality scores for 96,031 articles published 2014–18 from the UK Research Excellence Framework 2021. Unexpectedly, there was a positive correlation between expert judgements of article quality and average journal citation impact in all fields of science, although very weak in many fields and never strong. The strength of the correlation varied from 0.11 to 0.43 for the 27 broad fields of Scopus. The highest correlation for the 94 Scopus narrow fields with at least 750 articles was only 0.54, for Infectious Diseases, and there was only one negative correlation, for the mixed category Computer Science (all), probably due to the mixing. The average citation impact of a Scopus-indexed journal is therefore never completely irrelevant to the quality of an article but is also never a strong indicator of article quality. Since journal citation impact can at best moderately suggest article quality it should never be relied on for this, supporting the San Francisco Declaration on Research Assessment.

There is some follow-up discussion on this paper and its conclusions here.

The big problem of course is how you define “high-quality papers” and “high-quality journals”. As in the above discussion this usually resolves itself into something to do with citation impact, which is problematic to start with but if that’s the route you want to go down then there is sufficient readily available article-level information for each paper nowadays that you don’t need any journal metrics at all. The academic journal industry won’t agree of course, as it’s in their interest to perpetuate the falsehood that such rankings matter. The fact that correlation between article “quality” measures and journal “quality” measures is weak does not surprise me. I think there are many weak papers that have passed peer review and appeared in high-profile journals. This is another reason for disregarding the journal entirely. Don’t judge the quality of an item by the wrapping, but by what’s inside it!

There is quite a lot of discussion in my own field of astrophysics about what the “leading journals” are. Different ranking methods produce different lists, not surprisingly given the arbitrariness of the methods used. According to this site, The Open Journal of Astrophysics ranks 4th out of 48 journals., but it doesn’t appear on some other lists because the academic publication industry, which acts as gate-keeper via Clarivate, does not seem not to like its unconventional approach. According to Exaly, Monthly Notices of the Royal Astronomical Society (MNRAS) is ranked in 13th place, while according to this list, it is 14th. No disrespect to MNRAS, but I don’t see any objective justification for calling it “the leading journal in the field”.

The top ranked journals in astronomy and astrophysics are generally review journals, which have always attract lots of citations through references like “see Bloggs 2015 and references therein”. Many of these review articles are really excellent and contribute a great deal to their discipline, but it’s not obvious they can be compared with actual research papers. At OJAp we decided to allow review articles of sufficiently high quality because we see the journal primarily as a service to the community rather than a service to the bean-counters who make the rankings.

Now, back to the exams…

4 Comments »

Article Processing Charges at the Royal Astronomical Society

Posted in Open Access with tags , , , on March 2, 2023 by telescoper

As it was foretold, the Royal Astronomical Society has now officially announced that all its journals will be moving to Gold Open Access. The only thing that surprised me about this is the speed that it will be done – from January 1st 2024. The announcement confirms that the “rumour” I reported in 2020 was true (as I knew it was, given the reliability of the source). I did, however, think the timescale would be “within a few years” and it turns out to be much shorter than that.

For the journal of most relevance to myself, Monthly Notices of the Royal Astronomical Society (MNRAS) this decision means that authors will have to pay an Article Processing Charge (APC) at the (suitably astronomical) level of £2310 for each paper (although there will be exemptions in certain situations). Does anyone genuinely believe that it costs that much to publish an article online? Really?

I did actually laugh out loud when I saw the spin the RAS are trying to put on this decision:

The RAS is excited to be a key contributor to the open science movement, helping to drive discoverability and change.

Au contraire. Gold Open Access a serious hindrance to the open science movement, as it involves hugely inflated costs to the authors in attempt to protect revenue in the face of declining subscription income. Switching from a ‘fleece-the-libraries’ model to a ‘fleece-the-authors’ alternative can in no way be regarded as a progressive move.

Other notable astronomy-related journals, such as the Astrophysical Journal (ApJ) and Astronomy & Astrophysics (A&A), have levied “page charges” (effectively APCs by another name) for as long as I can remember, though in the latter case there is a waiver for researchers in “member” countries. ApJ and other journals also have a waiver scheme for those who cannot afford to pay. For those who have to pay, the fee is usually about $100 per page. For a long time MNRAS was the exception and indeed the only feasible choice for people who don’t have access to funding to cover page charges, including many in the developing world. More recently, however, MNRAS introduced a charge for longer papers: £50 per page over 20 pages, so a paper of 21 pages costs £50 and one of 30 pages costs £500, etc. Now there will be a flat fee of £2310 per paper.

It is true that some institutions will pay the APC on behalf of their authors, but that is hardly the point. If institutions have cash to pay for astronomy publications to be open access then they would do far more good to the research community by giving it to the arXiv rather than to the publishing industry. When authors themselves see how much they have to pay to publish their work, many will realize that it is simply not worth the money. (I refuse to pay any APC on principle.)

The Twitter feed for the Open Journal of Astrophysics (OJAp) was buzzing all day yesterday with negative reactions to the RAS announcement. Obviously I am biased in this matter, but I do encourage those thinking of switching to give it a try. The RAS has played into the hands of OJAp, which publishes papers (online only) in all the areas of Astrophysics covered by MNRAS, and more, but is entirely free both for authors and readers. The annual running costs of OJAp are substantially less than one APC at the level proposed by MNRAS.

The comments I have seen brought this image to my mind:

(The allusion to sharks is not accidental.)

The question for the Royal Astronomical Society, and indeed the other learned societies that fund their activities in a similar way, is whether they can find a sustainable funding model that takes proper account of the digital publishing revolution. If their revenue from publishing does fall, can they replace it? And, if not, in what form can they survive? I’d like to think that future operating models for such organizations would involve serving their respective communities, rather than fleecing them.

18 Comments »

Monthly Notices goes Online-only

Posted in Open Access with tags , , on June 14, 2020 by telescoper

I just heard that the journal Monthly Notices of the Royal Astronomical Society which has been publishing astronomy research since 1859, is no longer producing a print edition and instead will be publishing online.

The decision is in response to falling demand for the printed version which has made it no longer economically viable profitable to continue producing it. I choose the ‘profitable’ because the prime purpose of MNRAS is no longer the dissemination of scientific results but the generation of income to fund other activities of the Royal Astronomical Society. Despite the move to the much cheaper digital-only publishing mode, the annual cost of an institutional subscription to this journal is over $10,000. Most of that is goes as profit to Oxford University Press (the actual publisher) and to the Royal Astronomical Society.

Much of what the RAS does with this income is laudible of course, but I don’t think it is fair to inflate institutional subscription costs in order to fund it. University libraries are meant to provide access to research, not to act as cash cows to be milked by learned societies. The Royal Astronomical Society society isn’t the only learned society to use its journals this way, nor is it the most exploitative of those that do, but I believe the approach is indefensible.

My very first research paper was published in MNRAS way back in 1986 and I’ve published many others there over the years, so it’s with a certain amount of nostalgia that I look back on the old style journal. As. Fellow of the Royal Astronomical Society I used to get my own copy in the post at a discounted rate but had to stop and dispose of the old ones when I moved to Nottingham as they took up too much room.

My own belief is that it’s not only the print edition that has had its day but the whole idea of a traditional academic journal.

I’ll just take this opportunity to remind you that The Open Journal of Astrophysics publishes papers (online only) in all the areas of Astrophysics covered by MNRAS, and more, but is entirely free both for authors and readers.

10 Comments »