My inestimable PhD student Kay Lehnert has been having a look at the capabilities of the Artificial Intelligence platform ChatGPT at writing about string theoretical ideas, specifically the swampland conjectures. It’s remarkable what this does well but also notable what it doesn’t do well at all. What he found was so interesting he wrote it up as a little paper, which you can find on the arXiv here. The abstract is:
In this case study, we explore the capabilities and limitations of ChatGPT, a natural language processing model developed by OpenAI, in the field of string theoretical swampland conjectures. We find that it is effective at paraphrasing and explaining concepts in a variety of styles, but not at genuinely connecting concepts. It will provide false information with full confidence and make up statements when necessary. However, its ingenious use of language can be fruitful for identifying analogies and describing visual representations of abstract concepts.
It took arXiv a while to decide what to do with this paper as it doesn’t fit in any of the usual categories. The arXiv sections that usually cover string theory are General Relativity and Quantum Cosmology (gr-qc) and/or high-energy physics theory (hep-th), which was where it was originally submitted, but this isn’t really a string theory paper per se. After being held by the moderators for a while it eventually it appeared in Popular Physics (physics.pop-ph), cross-listed in Artificial Intelligence (cs.AI) & Computation and Language (cs.CL); the latter two are computer science categories, obviously.
The reclassification of this paper was perfectly reasonable. In fact with this, as with any other arXiv paper, the thing that matters most is that it it is freely available to anyone who wants to read it and is discoverable, i.e. can easily be found via search engines. In the era of Open Access, things will generate interest if they are interesting (and accessible).
We posted the following on the Maynooth University Theoretical Physics Department Twitter account, something we do whenever a new paper by someone in the Department comes out:
Judging by the number of views (101K) by this morning, this one certainly seems to be attracting interest! Hopefully this blog post will generate even more..
Finally, there might be people reading this blog who can suggest a journal that might consider publishing an article on this sort of subject? Whatever you think about ChatGPT I think it’s generating a lot of discussion right now, so the topic is… er… topical.
Please use the box below for any suggestions.
In the Dark 