ChatGPT launches tool to distinguish between AI and human input
OpenAI, the creator of the popular new program ChatGPT, has introduced a new tool that can detect whether a piece of writing has been created by artificial intelligence or a human being.
ChatGPT’s ability to instantly write on just about any subject has caused controversy, particularly in the education sector, where students could potentially use it to submit work which is not theirs.
The new tool, however, could soon put an end to any underhand tactics by students, although OpenAI says it is not totally successful in identifying AI text.
“We’ve trained a classifier to distinguish between text written by a human and text written by AIs from a variety of providers,” OpenAI said in a blog post.
“While it is impossible to reliably detect all AI-written text, we believe good classifiers can inform mitigations for false claims that AI-generated text was written by a human: for example, running automated misinformation campaigns, using AI tools for academic dishonesty and positioning an AI chatbot as a human.
“Our classifier is not fully reliable.”
In an evaluation of English texts, the classifier correctly identified 26 per cent of AI-written text as “likely to be AI-written”, but incorrectly labelled human-written text as AI-written 9 per cent of the time.
The classifier’s reliability typically improves as the length of the input text increases.
OpenAI said it has made the classifier publicly available to get feedback on whether imperfect tools like this one are useful.
The classifier is recommended for use only on English text, because it performs “significantly worse” in other languages and is unreliable on code.
ChatGPT was banned in New York City after its launch on November 30 due to concerns about negative effects on student learning, and concerns regarding the safety and accuracy of content, the city’s department of education said.
The Mack Institute for Innovation Management put the program through a Wharton Business School MBA final exam earlier this month.
The end result was that it would have received a B to B- grade on the exam.
“We are engaging with educators in the US to learn what they are seeing in their classrooms and to discuss ChatGPT’s capabilities and limitations, and we will continue to broaden our outreach as we learn,” Open AI said.
