‘An Arms Race’: Australian University Develops New AI Detection Tool to Combat ChatGPT

A smartphone with a displayed ChatGPT logo is placed on a computer motherboard in this illustration taken on Feb. 23, 2023. (Dado Ruvic/Reuters)

As the education sector grapples with the impacts of ChatGPT—an Artificial Intelligence (AI) chatbox—one university in Australia’s Group of Eight has introduced a new detection tool that can help teachers identify any unauthorised use of AI in student submissions.

In a student notice published on April 11, the University of New South Wales (UNSW) said the AI detection software would provide teachers with the percentage of AI-generated text and highlight the relevant passages.

The university noted that the unauthorised use of AI in assignments is “a form of cheating and is considered to be student misconduct.”

While the new tool—developed by a commercial provider—would not be entirely accurate, it will be used as one piece of information to indicate unauthorised use of AI in assignments and trigger further investigation.

“When unauthorised use of AI in assessments is determined, penalties may include a fail and mark of zero for the course, through to suspension or permanent exclusion,” the notice said.

“While the technology changes, our values around academic integrity do not. Your work must be your own and where the use of AI tools like ChatGPT have been permitted by your course convener, they must be properly credited, and your submissions must be substantially your own work.”

UNSW Deputy Vice-Chancellor Merlin Crossley described the new tool as an “anti-cheating software,” saying that educators don’t know how well the new software detects cheating, but “the students don’t know how well it works either.”

He noted that while the software will eventually be bypassed and people will find ways around it, “the next iteration will come out.”

“At the end of the day, part of it is an arms race, but part of it is universities are about learning, most of the students want to learn, and we only have to have a few hurdles that they really can’t get over. And that will keep everyone honest,” Crossley told Sydney’s 2GB Radio on April 14.

He added that Australian universities are “all working together” on figuring out solutions to AI, with most of the universities redesigning tests to combat the use of AI.

Danger Of AI

In a commentary published on The Australian, Crossley also warned educators against embracing ChatGPT entirely, arguing, “the positives are blinding us to the risks.”

“I believe that human lives will continue to unfold in two arenas – periods where ChatGPT is available and periods where it is not. We must train our students for both eventualities. It is not a question of embracing it or not. It is a question of how to train students for moments when it is not available.”

“We need medics who can assist a patient on a bushwalk or after a cyclone. We need leaders who can think on their feet, make decisions, and answer questions. We need teachers who can pass on their knowledge and skills.”

“So, in every course, we now must make sure we have assessments where students can use ChatGPT and some in which they cannot. Because in their lives students will face situations where they can use artificial intelligence and situations where they cannot.”

He suggested that teachers can instruct students on how to use ChatGPT but also continue to teach them how to manage without it.

ChatGPT Can Be Biased, University of Sydney Warns

Another leading Australian university, the University of Sydney, also issued guidelines advising students against using ChatGPT for assessments unless explicitly allowed; and failures to acknowledge the use of AI tools could leave them vulnerable to accusations of cheating.

The university told students that it’s important to remember that the answers ChatGPT provide “reflects biases, opinions and views of the humans that informed the text it has been trained on, which may not align with your own values.”

“While the content ChatGPT produces seems impressive on the surface, the reality is not all responses are worded well or are even correct. This becomes apparent if you ask ChatGPT highly conceptual questions or pose difficult or complex calculations,” the university said on Feb. 24.

“On ChatGPT’s FAQ page, OpenAI confirms that the tool’s ‘outputs may be inaccurate, untruthful, and otherwise misleading at times.’”

Academics Seek To Embrace AI

Meanwhile, some educators are looking to incorporate AI in the classroom. Vaughan Connolly, a researcher at the University of Cambridge argued that ChatGPT can help students present ideas in a clear and organised manner and in the right form, allowing teachers to focus on the ideas themselves.

“In the process, it could shift the focus of education towards critical thinking and big questions. Because it can also summarise an argument, knowledge and concepts, there is real potential to help with formative assessment, especially in situations where teachers have limited time,” he said.

But Steve Watson, co-convener of the Faculty of Education’s Knowledge, Power and Politics at the University of Cambridge said AI is a “double-edged sword” that people can’t just treat as “heralding the arrival of a new utopia.”

“In many ways it takes us back to the old Douglas Adams idea of Deep Thought: we’re very attracted by the prospect of something acting as a source of ultimate intelligence and informational knowledge, but this is not a trustworthy, independent source of information.”