AI watermarks are an irrelevant distraction as we engage with assessment in modern universities
The recent notice that Anthropic are introducing digital watermarks into their Claude AI models in compliance with the EU AI law has no doubt led some to believe that we have a way now of detecting AI unambiguously, and therefore the perfect technological quick fix in service of the goal many have of preventing AI use by students. Nothing could be further from the truth.
It suffers from exactly the same problem that many of these tools do - probability and uncertainty, and an inability to deliver unless you operate within a panopticon that will destroy the integrity of the university even faster.
The first problem is that nothing about the implementation is able to be verified by anyone else. We are meant to simply accept their assertion that a piece of text "probably" was manipulated in some way by their models. Even if we trusted them (big ask) the same technology would have to be implemented by every model. While some will do so in order to comply with EU law, not all will, and unless all AI tools are withdrawn from end user modification they never will be.
But assuming they did, this leads to the second problem - any verification of AI non-use would have to involve separate checks with every vendor as they are not using a common solution and shared keys.
The third problem is that using such an approach would involve disclosing student intellectual property and personal information without their consent to multiple third parties, many of whom will then use this information to improve the AIs without the permission or agreement of the students.
Yes, no doubt the universities could enter into agreements with these companies but that then creates a financial burden that will be exploited to the maximum. I'd also be astonished if the use of the tools in this way wasn't legally challenged by student groups and I'd expect them to be successful, setting aside the moral argument that using their power in this way over students further erodes the trust and ethical high ground that are essential to the survival of the university.
The fourth problem is that its unclear what the watermark could usefully do unless the policy of the universities was a complete ban of AI use. The watermark can't tell you if the AI wrote the whole thing, or just fixed grammatical and spelling errors? Using the AI to brainstorm and research an answer that the student then writes is invisible. Using the AI to write rubrics that the student uses to get feedback on and improve their answers, also invisible.
The fifth problem is that technically the watermark will be subverted. Their own paper admits there are many domains where it will be unable to work. There are ways that the text could be reviewed and changed that will remove it from all but the longest texts. All it would take to remove the watermarks from Claude text is to run in through another AI tool that translates into another language and then back again - by the time the user then reviews and cleans up any anomalies the watermark is gone. I doubt it would survive a little shuffling of sentences and paragraphs in all but the longest text. This is merely the latest step in an arms race without end.
The sixth problem is possibly the most important. Pretending that AI doesn't exist is a great way to speed up the irrelevance of our courses and programmes. It exists and will grow in use and legitimacy. We need to teach people how to engage with the world as it is for the vast majority of our courses and programmes or they will go elsewhere and we will fail. Most universities in countries like New Zealand, Australia and the UK cannot survive more than a year or two with even a small decline in student numbers. Students will find the universities that can educate them to use AI well to succeed personally and professionally and will abandon those who do not (and yes if you're Oxford you can rest safely on your reputation for now - everyone else, not so much).