In the Dark

The arrival of AI bots such as ChatGPT continues to cast a shadow over student assessment in the third-level institutions, as academics are realizing that these algorithms are getting better and better at the tasks asked of students, especially straightforward writing tasks (perhaps including simple calculations) as well as the traditional student essay.

Before going further I have to admit that I’ve never really understood the obsession in some parts of academia with “the Essay” as a form of assessment. I agree that writing skills are extremely important but they’re not the only skills it is important for students to acquire during the course of a degree. Learning how to do things seems to me to be more important than writing about things other people have done. While forms of assessment in science subjects have evolved considerably over the last 50 years, some other domains still seem to concentrate almost exclusively on “The Essay”.

Systems such as ChatGPT can produce text on demand (with a variable degree of success) using sources on the internet. This is not great at dealing with technically complex specialist topics but can produce plausible if somewhat superficial offerings in many circumstances where something less demanding is required. I know that staff in some science departments find that these systems can score essentially 100% on their first-year coursework assignments. Urgent meetings are being called and working groups being set up about this. Panic is in the air.

My immediate response to the situation is very twofold:

1. Don’t panic!
2. If an assessment can be aced by a bot then it should not contribute towards credit unless the students do it in a supervised environment, e.g. as an in-class test rather than a take-home assignment.
3. More importantly, if a student with only a superficial knowledge can score a high mark on an assessment, what is the value of the assessment anyway?

It seems to me that the intervention of ChatGPT should cause academics to reflect much more deeply on what it is that they are trying to assess, and that should lead to new forms of assessment that can’t be performed by AI bots as well as the scrapping of many existing assessment activities, many of which (in my opinion) are pointless. There is so much inertia in academia, however, that such a radical rethink will be forthcoming on the timescale required.

All of which waffly nonsense reminded me of a joke I heard many years ago.

Q: How many academics does it take to change a lightbulb?

A: What do you mean, change?

This entry was posted on May 5, 2023 at 3:40 pm and is filed under Education, Maynooth with tags AI, assessment, Assignment, ChatGPT, The Essay. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.