In the Dark

There have been more news stories about ChatGPT and assessment in universities going around. There’s one here from The Journal and another here from The Conversation to give just two examples.

I wrote about this myself a couple of months ago in a post that included this:

I have to admit that I’ve never really understood the obsession in some parts of academia with “the student Essay” as a form of assessment. I agree that writing skills are extremely important but they’re not the only skills it is important for students to acquire during the course of a degree. Of course I’m biased because I work in Theoretical Physics, an area in which student essays play a negligible role in assessment. Our students do have to write project reports, etc, but writing about something you yourself have done seems to me to be different from writing about what other people have done. While forms of assessment in science subjects have evolved considerably over the last 50 years, other domains still seem to concentrate almost exclusively on “The Essay”.

Whatever you think about the intrinsic value of The Essay (or lack thereof) it is clear that if it is not done in isolation (and under supervision) it is extremely vulnerable to cheating.

A few people have retorted that communication skills are very important in higher education. I agree with that wholeheartedly, but it seems to me that (a) there are other ways of communicating than via formal essays and (b) there are, should be, more to academic study than  writing about things.

That said, I do think we could be doing more in some disciplines, including my own, to cultivate communication skills in general and writing skills in particular. In Theoretical Physics we certainly don’t do this as much as we should. I do have a project report in my 3rd Year computational physics module, but that is a relatively short document and the report itself counts only one-third of the marks (and the project is only 40% of the module mark).

These thoughts somehow reminded me of this. You can click on it to make it bigger if it’s difficult to read. It was the first paper (called colloquially Paper Zero) of my finals examination at the University of Cambridge way back in 1985, getting on for 40 years ago:

As you can probably infer from the little circle around number 4, I decided to write an Essay about topic 4. I’ve always been interested in detective stories so this was an easy choice for me, but I have absolutely no idea what I wrote about for three hours. Nor do I recall actually ever getting a mark for the essay, so I never really knew whether it really counted for anything. I do remember, however, that I had another 3-hour examination in the afternoon of the same day, two three-hour examinations the following day, and would have had two the day after that had I not elected to do a theory project which let me off one paper at the end and for which I got a good mark.

Anyway, to get back to the essay paper, we certainly don’t set essay examinations like that here in the the Department of Theoretical Physics at Maynooth University and I suspect they no longer do so in the Department of Physics at Cambridge either. At the time I didn’t really see the point of making students write such things under examination conditions but then we didn’t have ChatGPT way back then. No doubt it could generate a reasonable essay on any of the topics given.

I am skeptical about whether any of my 3rd year computational physicists would use ChatGPT to write their reports, but they might. But ChatGPT can write Python code too. Am I worried about that? Not greatly. I’ve asked it to write scripts for the various class exercises I’ve set so far and the code it has produced has usually failed. It will get better though….