can turnitin detect ai writing

can turnitin detect ai writing

Can AI writing be detected by Turnitin?

1. Introduction

The optimal strategy for AI-created content identification is to employ both AI and intricate writing and examination of various aspects of the content. In real-world cases, the combination of diverse methods remains the better detection resulted from consulting not only staff of internal content experts, e-mail reporting, and manual investigations into excessive similarity reports, but also other internal resources. Given the dynamic nature of AI-created content developments, constant updating and cooperative efforts on content identification can help escape high-profile cheating. The findings indicate that Turnitin readily recognized a significant portion of the AI-created content. With respect to addressing the pigeonhole principle of the algorithm associated with a static dictionary, it is suggested that the rule panel of people should establish non-trivial indicators and leverage to help streamline the full order checking process over the incidents through pattern abnormalities.

Nowadays, artificial intelligence (AI) has impressed this society in a different aspect, especially in the field of education. In the process of plagiarism detection, some within the academic community are seeking ways to enhance the ability of current anti-plagiarism detection policies to escape from the rising of AI-created content. As a renowned originality-detection tool, Turnitin is confronted with an evolving challenge in order to enhance its ability to identify AI-created content. Although Turnitin offers some details through the application of implementation methods and initial outcomes, it is hard to get a comprehensive view of how well it functions in practice.

2. How Turnitin works

The workload of similar documents of billions of evidence from repositories all over the world rises steadily and is hoped to increase rapidly with acquiring its access to data from participating academic publishers if the next generation of an artificial intelligence researcher concentrates on this model or it builds rivals with access to data. Whether Turnitin is able to withstand European GDPR, which mandates users’ right to know how algorithms decide and prohibits them from stopping using “intelligent” algorithms, misses a well-defined model. Also, Turnitin’s relentless efforts to increase the index by including data from commercial publishers have been discussed. Such attempts must deal with two fundamental problems: legitimate document publishers and illegal data publishers can both increase their own benefits and losses by continuously improving their techniques in the context of a central competitor.

Turnitin is based on a particular algorithm and architecture that has achieved significant success in capturing the behavior of “students”. However, the black box underlying Turnitin has not been open to the public. There is no well-defined model, nor are there specific details on how indices are generated and how similarities are calculated between pieces of evidence. User, Ng, and Kumar (2019) proposed to lift the black box and provide details of how Turnitin works by treating it as a Markov decision process problem. The generation of indices and the calculation of technical features are modeled as transitions at different states of the process of making a submission to Turnitin, with the help of an agent who attempts to optimize the performance of the system by manipulating the words in the file.

3. Limitations of Turnitin in detecting AI writing

In one example where Turnitin didn’t detect the AI, the generated output from Eth was “This is a paragraph. This is a sequence of quantum sleeper processes. No one and nothing can stop the forward motion toward progress.” In terms of the GPT-3 and other AIs, it becomes nontrivial to differentiate writing by them to real writing occurring in practice. I have confirmed that with only minimal modification of the GPT-3 output, it wasn’t detected by Turnitin as being an AI-composed paragraph. Not being significantly different from human output, the GPT system is being used for the real world today, and so it is over the 50% solution that AI should be used for writing purposes.

The company Turnitin provides a service for assessing the originality of written work. This is used by universities to detect potential plagiarism by students. Turnitin also provides the same service to businesses, so I suspect that it is used frequently. In an earlier post, I reported that I had tested to see if the AI models produced by OpenAI and EleutherAI were detected as potentially being written by AI. With the current language AI technology, I concluded that an AI-produced text can be made not to be detected by Turnitin as being by AI. I will explain why that is in this post.

4. Strategies to evade detection

The new and poorly understood technology driving the best AI writing tools is OpenAI’s GPT-3 “engine.” By using something called “AI language models,” a piece of software is able to generate entire documents using written prompts or suggestions. What sets GPT-3 apart is the unfettered access to the internet and the vast collections of paid, copyrighted materials within. Consider the advantage a student has at school when papers are assigned and can be written using the questions or suggestions online. Now consider a very good student, a demographically average gifted and talented high school student writing a fifth or sixth-grade paper for a teacher. However, in this case, it takes a million or billion of these “students” and a long time to train, and for the teacher, that older student, to make sure that the student’s chances are improved to at least “best in class.” Most people would not be able to say how or why the tool gives a “best in class paper” each time it watches something new, but their performance would be quite impressive when that same task is assigned to them, especially if the expectations were low.

There is no doubt that writers will come to know that their work is likely to be checked by such tools, and thanks to a variety of writing strategies, that discovery can be delayed longer than most might anticipate. To understand these strategies, one must also understand how the current state of technology in the best AI writers is very different from what most people understand. We’ll first discuss the basic technology of the OpenAI GPT-3 engine, and then the most common strategies already being used to avoid detection by products such as Turnitin.

5. The importance of academic integrity

Most Turnitin users are students. They respect academic integrity because they know the courage it takes to write their own words, as difficult as that is. Many teachers are already aware that an algorithm can only verify whether an essay contains cognitive development that is appropriate to the effort taken to write it. Turnitin is used because faculty don’t want to waste so much of their limited time examining text. But plagiarism detection services like DeleteRequest mean academics no longer have to waste time studying the tiny portion of the class that would try such a thing. Why would students – who know that both they and their professors already have a tool to easily ‘show their work’ – waste their money and time by cheating in ways that are plainly visible to both the teacher and their peers?

The only way to reliably detect AI-generated text is for the AI model itself to reveal that it was behind the generation of the text. In such a case, a signature created by the model – a unique “fingerprint” of the model – is appended to the generated output. Researchers can then use the third-party tool developed at the University of Chicago, which essentially acts as an independent witness to confirm that another tool was indeed used by the authors to produce the output. Our text classifier still can’t do anything about paripassu problems with an existing text generation model being used as a base to fashion a new, undetected model. But our own AI engine will be discharging the Good Samaritan role by alerting the public to the existence of the malfeasance by providing the signature evidence of the AI-generated digital artifact. Taking the Digital Ghostbusting further backwards, the aid of the third-party tool then becomes even more crucial in auditing the use of ‘human aides’ to deceive our AI-detecting system a generation earlier.