Picture yourself in the role of a judge at a school essay contest. Your aim is to ensure that every participant has penned their own work, excluding any submissions generated by artificial intelligence (AI) tools like ChatGPT. But how do you discern if a piece was crafted by an AI? A recent study reveals an effective technique to test authorship authenticity: just task a bot with rewriting it for you.
“When an AI rephrases content it has authored, the alterations are minimal,” notes Chengzhi Mao. In contrast, when an AI rewrites a human’s creation, it typically introduces significant changes.
Under the leadership of Yang Junfeng, Mao and their team developed a tool named Raidar. This detector employs AI rewriting to detect bot-generated text. Mao, a researcher at Columbia University’s Software Systems Lab in New York, collaborated closely with lab head Yang in this endeavor.
According to Yang, it is imperative to distinguish between bot-generated and human conversations. The prevalence of AI-generated content on social media, product reviews, fake news outlets, and spam publications is alarming. Some students resort to AI for academic dishonesty. Tools like Raidar could play a pivotal role in identifying AI cheaters and deceivers.
The developers of Raidar presented the tool at the International Conference on Learning Representations held in Vienna, Austria, on May 7.
Exclusion of AI
Mao frequently leverages ChatGPT for enhancing his compositions. For instance, he might request the bot to refine and enhance his emails. On the initial rewrite attempt by the bot, Mao acknowledges its impressive execution. Yet, on subsequent edits aimed at correcting the bot’s initial rewrite, minimal alterations are observed.
“This realization prompted us,” Mao elucidates, highlighting that the frequency of edits made by a bot to a textual piece could signify the manner in which the original content was authored.
Amrita Bhattacharjee, a doctoral candidate at Arizona State University in Tempe, applauds the idea as innovative since it addresses AI-generated text detection, although not directly contributing to Raidar’s development.
Raidar serves as a tool to distinguish between human and AI-generated texts. The research team accumulated written samples from humans and diverse chatbots to test Raidar across various text categories, including news articles, Yelp reviews, student essays, and programming code. Subsequently, multiple AI models were employed to rewrite all the human and bot-written content.
By implementing a basic computer program to determine the number of modifications between the original and rewritten versions of each sample – a step devoid of AI involvement – Raidar could accurately classify the text samples as human-generated or AI-generated. This functionality proved effective even when the AI used for rewriting differed from the AI responsible for the initial content creation.
While Raidar’s classification may not be flawless, occasionally misidentifying human text as AI and vice versa, the researchers affirmed its superior performance compared to other AI-written text detection tools.
Unlike other tools relying on AI models and statistical algorithms to learn the distinctive patterns in bot-generated text, Raidar stands out as suitable for short texts, such as social media posts and assignment descriptions, with a minimum word count of 10.
Indicators of AI Content
With the aim of transforming Raidar into an accessible online tool, Yang, Mao, and their team aspire to offer a platform for individuals to submit text and ascertain its AI or human origin.
Promoting Raidar’s user-friendliness, Yang emphasizes that anyone can leverage the tool without possessing a computer or data science background. For instance, a vigilant teacher could task the chatbot with rewriting a student’s assignment. Scant alterations by the bot could serve as a red flag signaling potential AI utilization by the student.
Bhattacharjee advises against solely relying on Raidar’s analysis as the sole determinant for action. He advocates for a comprehensive assessment, recognizing that Raidar may not always yield accurate results. Some students may have legitimate reasons for resorting to AI assistance, like grammar correction.
Furthermore, Yang contemplates extending Raider’s functionality to identify other forms of AI-generated content. Currently, his research delves into the impact of AI model edits on images, videos, and audio clips, hypothesizing that extensive changes indicate human-originated work.
Have a science question? We’re here to help!
Submit your questions here and we may feature them in the upcoming issue of Science News Explores.
Source: www.snexplores.org
