can teachers detect ai writing

can teachers detect ai writing

The Impact of AI Writing on Teachers’ Ability to Detect Plagiarism

1. Introduction

AI writing changes education in ways that force educators to rethink academic integrity. Yet many aspects of AI writing in the education and research ecosystem are ambiguous, under-explored, and in some cases, invisible. This increased ambiguity and lack of transparency of the source of student work can alter educators’ abilities to judge the originality of student work, since face-to-face interactions do not provide the same opportunities for educators to build trust indicators when reviewing student research. In recent massive open online course (MOOC) and Coursera data collected in June 2020, pretrained language model (PLM)-enabled AI-written text unit contribution rates were found to be increasing, and no signs of a peak or leveling off in the rates as more complex writing tasks were assigned were observed. This suggests that educators face a future of increasing reliance on AI-written coursework, regardless of the ethical and academic integrity implications of the AI writing explosion. In the scope of this discussion paper, we are most directly focused on how teachers are impacted by AI writing when they detect plagiarism, how the currently existing methods of detecting plagiarism are expected to handle near-future increases in AI text collaboration contributions – specifically, models that use any natural language processing (NLP) features, and what policy level decisions may be beneficial for formal education as the AI writing ecosystem develops in the near future.

In this paper, we focus on AI writing’s impact on teachers’ ability to detect plagiarism. Although our focus is limited to this one effect of AI writing, we do acknowledge that the full picture of the positive and negative impacts of AI writing on education is much broader, and includes providing students with models and templates, the negative impacts related to essay and soapbox rhetoric, and the distant prospect of automated tutors. However, as research scientists who specialize in educational research, plagiarism detection, and explainable AI, we recognize that we can make rich and meaningful contributions to our predominant expertise by framing discussions of how AI writing assists and interferes with educators’ goals through the lens of this one specific impact.

2. Understanding AI Writing

Technology breakthroughs, like the ones that led to writing using AI, are often not realized until artificial general intelligence (AGI) is commonplace and performing things initially not considered to be within the realm of AI. AI writing has also contributed to a significant explosion of hoax research manuscripts that have been accepted in various publications. The purpose of this work is to assist educators in assessing and teaching writing created with AI, resulting in a reliable form of writing assessment. There have been commendable prior works involving AI detection to determine if the text is computer generated or written by a human. The focus of these experiments has been on poetry, rather than more informative papers having sections such as introduction, methodology, results, and conclusion. There is no published work studying AI detection with an emphasis on teaching or grading student writing holistically. Such an important study will provide insights to prevalent detection measures in honor codes as well as educators.

In the 21st century, almost every form of technology has incorporated some form of artificial intelligence (AI). AI is now helping machines mimic human-like behavior with the ability to learn, reason, and self-correct. A prime example of this behavior is Jim, an online writing AI, that writes just like a human and is developed to improve students’ writing skills. Jim’s capabilities expand beyond automated writing corrections and supporting students with their essay or long-form writing assignments. It is hypothetically possible to use AI writer to write product reviews, spoilers for TV show episodes, or papers for school. In the USA, AI writers like Jim are intended for educational applications. If these enhancements continue to increase, teachers may no longer be able to detect papers written by students versus AI and there may be an increase in academic dishonesty.

3. Challenges Faced by Teachers

The larger problem at Lasell, one that colleges across the nation are struggling with, was the increase in plagiarized essays, which the authors noted was “overwhelmingly” due to online papers. Addo-Robins and Lumpur (2007) noticed a similar increase, writing that students at their university were now required to align their term papers with a copy of the paper from which text was copied. Web-based resources, already widespread throughout U.S. society, came to universities gradually, further complicating the extent of faculty-teaching about them, especially for instructors at community colleges who themselves have little online exposure.

Challenges facing teachers in detecting plagiarism are neither new nor limited to artificial intelligence. Barrie (2003) asserted that “practically every writing teacher in every English-speaking university at one point or another encounters a case of plagiarism,” often from students whose cultural or linguistic backgrounds are vastly different from U.S. American standards. Long before the widespread use of the World Wide Web, Birnbaum (2001) wrote of “The Technology Tool for Plagiarism,” identifying the Internet as that tool. Meanwhile, as taught in Kirby et al. (2019) Electrical Engineering course or Singleton (2009) “Measuring the Information Society” course, U.S. universities teach students to not attribute quotes, or even entire books, to whoever actually published the author’s work.

4. Strategies to Detect AI Writing

In contrast, the suggested strategies require teachers to embrace the characteristics and features of assignments that are distinct from the traditional training model in order to effectively detect AI writing. It is also important for teachers in other fields to assist STEM instructors in detecting discovered art effects.

Strategies to improve the effectiveness of detecting plagiarism in AI writing are proposed and discussed. These strategies aim to leverage the existing effectiveness in detecting plagiarism and collect untapped detections of AI writing. However, implementing these strategies, such as asking questions of individuality, eliciting expansive thought, and sharing relevant personal experiences, can be challenging for STEM instructors. This is because traditional STEM education focuses on intuition development, easily replicable processes, and guidelines, while discouraging creativity, innovation, and variability.

The consistent exchange of evidence regarding the effectiveness of detection and avoidance suggests that the low absolute detection rate offered by teachers is due to their attempts to avoid detection rather than their inability to carry it out.

Focusing on teachers, my results show the following: 1. Teachers are able to detect and penalize AI-generated artifacts better than chance. 2. The state-of-the-art plagiarism detection system, Turnitin, effectively targets this type of writing but can be adapted to impede the detection of AI writing. 3. Teachers struggle to effectively use the markers highlighted by Turnitin to detect the samples it targets. 4. However, teachers are still effective at avoiding the detection of those samples.

In previous sections, I discussed plagiarism detection systems and guidelines for detecting AI-generated artifacts. The purpose was to investigate whether the use of writer-AI production is associated with differences in the ability-level of human detection and plagiarism detection systems.

5. Conclusion

In addition, the current state of AI writing tools raises many important questions about the ethical use of such tools in an academic environment, many of which concern privacy, data ownership and licensing, and students wishing to submit work generated by AI writing tools. There is a lack of awareness. Universities and educational institutions are increasingly aware of the role AI can play in support or entirely offloading educational tasks and are experimenting with their own databases. In one example, Carnegie Mellon University offers courses that seek to enable students to use GPT-3 or similar OpenAI tools. Among the things, students must analyze and address potential ethical misperceptions surrounding AI text generation. A significant component of university students are advanced high school students who will soon begin their own academic endeavors. Teachers and policymakers need an increased level of awareness of developing technology in order to provide guidance on their ethical use in an academic setting.

As AI writing programs improve over time, it becomes clear that researchers, teachers, and educational institutions need to develop a better understanding of their capabilities and limitations. Detecting AI writing in academic work is not as easily achieved using rudimentary due diligence when employing traditional obfuscation detection methods. Students leveraging more recent AI writing tools such as OpenAI’s GPT-3 or similar should make it almost impossible for teachers or educational institutions to confidently determine their origin. The study offers two broader areas for all educational stakeholders to address. Integration in the curriculum for due diligence and work on implementing robust AI writing obfuscation methods. In what little data is available, potential sources of interest were identified.