Archive for llm

On the Use of Generative AI

Posted in Education, Maynooth, Artificial Intelligence with tags , , , , , , on May 18, 2026 by telescoper

We’ve reached the time of year at Maynooth when academic staff are busy grading projects of various kinds. This year we have to be much mindful of the use of Large Language Models (such as ChatGPT) in written reports as these are much more commonplace now. We anticipated this at the start of the academic year, but now we have to see whether are policies work in practice. In the case of the Computational Physics projects that I have to mark, this also extends to the use of Generative AI in writing code. The approach I take there is that I don’t place an absolute ban, but I require students to declare the use and, crucially, describe what steps they used to test and validate the output. By the time they’ve done that they might as well have written the code themselves!

As well as its effect on teaching, GenAI is having a huge impact on research. In my role as Managing Editor of the Open Journal of Astrophysics I have seen a large increase in submissions of papers in which AI plays some role. These vary from pure “slop” – nonsense papers not worthy of serious consideration – to articles that use AI tools in a perfectly reasonable way to speed up certain aspects of the analysis. I think this is the case for most scientific journals.

The approach we have adopted is similar to the policy on teaching outlined above. It is described by the following section we have added to our “For Authors” page:

Use of Generative AI. We do not operate a blanket ban on the use of Large Language Models (LLMs) or other forms of Generative AI. If you do use such tools, however, you must declare it in the acknowledgments section of your paper. Furthermore, if GenAI methods are used for any form of calculation, analysis, or data visualization you must include an account of what steps you have taken to test and validate these methods. Articles containing direct evidence of the use of GenAI, such as hallucinated references or prompts embedded in the text, will not be accepted.

Since the Open Journal of Astrophysics is an arXiv-overlay journal I should also pass on the information that arXiv is itself developing a policy on the use of LLMs. Although it has yet to appear on the arXiv website, a recent communication on social media states:

If there is incontrovertible evidence of LLM slop in a paper, this means the authors did not take the time to read the LLM output and we can’t trust anything else in the paper. Penalty is 1 year ban from arXiv followed by a requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue.

This will be tantamount to a one-year ban from publishing in OJAp, so urge authors should be be very careful in their use of such methods.

It is likely that these policies will have to be extended as the use of GenAI spreads.

1 Comment »

On Papers Written Using Large Language Models

Posted in Uncategorized with tags , , , , , , , on March 26, 2024 by telescoper

There’s an interesting preprint on arXiv by Andrew Gray entitled ChatGPT “contamination”: estimating the prevalence of LLMs in the scholarly literature that tries to estimate how many research articles there are out there that have been written with the help of Large Language Models (LLMs) such as ChatGPT. The abstract of the paper is:

The use of ChatGPT and similar Large Language Model (LLM) tools in scholarly communication and academic publishing has been widely discussed since they became easily accessible to a general audience in late 2022. This study uses keywords known to be disproportionately present in LLM-generated text to provide an overall estimate for the prevalence of LLM-assisted writing in the scholarly literature. For the publishing year 2023, it is found that several of those keywords show a distinctive and disproportionate increase in their prevalence, individually and in combination. It is estimated that at least 60,000 papers (slightly over 1% of all articles) were LLM-assisted, though this number could be extended and refined by analysis of other characteristics of the papers or by identification of further indicative keywords.

Andrew Gray, arXiv:2403.16887

The method employed to make the estimate involves identifying certain words that LLMs seem to love, of which usage has increased substantially since last year. For example, twice as many papers call something “intricate” nowadays compared to the past; there are also increases in the use of the words “commendable” and “meticulous”.

I found this a commendable paper, which is both meticulous and intricate. I encourage you to read it.

P.S. I did not use ChatGPT to write this blog post.

Leave a comment »