Worse than AI writing is AI reading. What can we do?

While we’re all worried that assigning home-written essays stopped making sense because students are outsourcing the task to AI, and we’re all scrambling to invent alternative ways of assessment, this particular blogger is even more concerned about the effects of students relying on brief (or not so brief) AI-generated summaries of the readings that they should do before class. In my short post-LLM teaching experience, worse than AI writing is AI “reading”. And, I want to stress, that’s not merely because students aren’t doing the readings. Rather, it’s because they seem to think that what they get from, say, ChatGPT, is enough for them to understand the view of the author in question, and have justified opinions about it. This surely doesn’t work, at least not for readings in philosophy, which is what I teach. Students may know, in a nutshell, what conclusion the author wants to support and a couple of reasons in favour of it. But because they don’t struggle to extract these from an article, or a book chapter, with their own natural intelligence, they fail to appreciate the complexity of the issue we discuss, the difficulty of defending well a particular position on it, the temptation of thinking about that matter very differently and the extraordinary challenge which, sometimes, is to even formulate the right questions. The result is boredom.

In the classroom, boredom is the kiss of death; eyes would like to roll, hands yearn for the phone but remain still, because my students are mostly polite and because I ban mobile phones in class. Everybody seems to be having a mental cramp. Of course we do: since they have not been through the discovery adventure, but instead skipped to the outcome, students’ comments are flat, their questions – which they should prepare in advance of class and be ready to talk about with their colleagues – are pro forma, and most often vague, so as not to betray the lack of familiarity with the text. Boredom is contagious. People appear unable to imagine how one could think differently about the questions we discuss – something that a well-written paper would have made vivid to them. Even incendiary topics (say, culture wars material) are met with apathy.

For many years, Jo Wolff had a wise and funny series of editorials in the Guardian; one of the earliest was praising academic prose for being boring. It’s for fiction writers to create mystery and suspense; philosophers (for instance) should start with the punch line and then deliver the argument for it. I agree with sparing readers the suspense, but after a series of academic conversations with ChatGPT I discovered that, if pushed to the extreme – the formulation of a thesis and the bare bones argument for it – this kind of writing is the worst. It kills curiosity.

What should we do? Perhaps turn some of our classes into reading-together-in-silence events? Back to monastic education! I talked to colleagues, who told me about several things they’re trying to make students read again (without AI.) An obvious possibility is to ban all use of LLMs by students and explain the reasons: Our job is not primarily to populate their minds with theories, but to help them understand arguments, teach them how to pull them apart, and maybe to occasionally build them. I’m not sure about this solution either. For one thing, a well-prompted LLM is better at reconstructing a slightly unclearly and imprecisely presented argument than the average reader and many students; often, AI often produces much better abstracts of academic work than academics themselves, and well-written abstracts are really useful. Another problem is that policies which can’t be enforced are for that reason deficient, and, I suspect, the very attempt to directly police students on their use of AI would be just as anti-pedagogical as the use of AI itself. (Reader, do you learn from those you resent?)

Alternative suggestions are to change how we teach. Quite a few colleagues have started to read out excerpts in class, then discuss them on the spot. One of them goes as far as asking students to memorise them, in an attempt to revive proven methods of Greek and Roman antiquity. This sounds good, time consuming as it is; better do a little, and do it well, than do a lot for naught, though I’d stop short of requiring memorisation. Others ask students to annotate their readings before class, and check, or use Perusall and similar platforms to read the assignments collectively, in preparation for class. I did Perusall to great success in the Covid era, but when I tried it again recently it was a disaster of cheating and complaints. Some teachers are printing out readers, or organising hard copies of books for the students, in the hope that this dissuades them from uploading digital files to LLMs. One colleague introduced 5-10 minutes flash exams at the beginning of each class, to check that students have read. And another one picks two students in each class, randomly, and asks them to co-chair the discussion about the reading of that day.

In the medium term, perhaps universities should double – or triple – the length of time that students spend together, with an instructor, for each class, and earmark the extra time as “study group”, when students read and write. There’s something dystopian about this model and it would massively increase work loads for instructors, so in practice it should mean more jobs, perhaps with lesser compensation. But is this really worse than giving up on the goal of teaching students how to read and write essays? Everybody would resist, no doubt but by the time the value of degrees, including their market value, will be next to nothing, universities might face a choice between closing down and reforming in ways that we find hard to imagine now.

As for the next academic year, I wonder whether I should assign readings that I won’t cover at all in my lecturing, but which will be of great help to students in the discussion section. Those who come to class having read only the LLM-created abstract will be the poorer for it. But, since I won’t ask them to discuss the papers, we might – most of us – escape the boredom mill.

Any thoughts?