# A Survey on LLM-generated Text Detection: Necessity, Methods, and Future
Directions

Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Derek F. Wong
and Lidia S. Chao This work was supported in part by the Major Program of the
State Commission of Science Technology of China (Grant No. 2020AAA0106701),
the Science and Technology Development Fund, Macau SAR (Grant Nos.
FDCT/0070/2022/AMJ, FDCT/060/2022/AFJ) and the Multi-year Research Grant from
the University of Macau (Grant No. MYRG-GRG2023-00006-FST-UMDF).
(Corresponding authors: Yulin Yuan and Derek F. Wong.)Junchao Wu, Shu Yang,
Runzhe Zhan, Derek F. Wong and Lidia S. Chao are with the Natural Language
Processing & Portuguese-Chinese Machine Translation (NLP2CT) Laboratory and
the Institute of Collaborative Innovation, University of Macau, Macau SAR,
China (e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>lidiasc@um.edu.mo).Yulin Yuan is
with the Department of Chinese Language and Literature, Faculty of Arts and
Humanties, University of Macau, Macau SAR, China (e-mail<EMAIL_ADDRESS>and Department of Chinese Language and Literature, Faculty of Humanities,
Peking University, Beijing, China (e-mail: yuanyl@pku.edu.cn).

###### Abstract

The powerful ability to understand, follow, and generate complex language
emerging from large language models (LLMs) makes LLM-generated text flood many
areas of our daily lives at an incredible speed and is widely accepted by
humans. As LLMs continue to expand, there is an imperative need to develop
detectors that can detect LLM-generated text. This is crucial to mitigate
potential misuse of LLMs and safeguard realms like artistic expression and
social networks from harmful influence of LLM-generated content. The LLM-
generated text detection aims to discern if a piece of text was produced by an
LLM, which is essentially a binary classification task. The detector
techniques have witnessed notable advancements recently, propelled by
innovations in watermarking techniques, zero-shot methods, fine-turning LMs
methods, adversarial learning methods, LLMs as detectors, and human-assisted
methods. In this survey, we collate recent research breakthroughs in this area
and underscore the pressing need to bolster detector research. We also delve
into prevalent datasets, elucidating their limitations and developmental
requirements. Furthermore, we analyze various LLM-generated text detection
paradigms, shedding light on challenges like out-of-distribution problems,
potential attacks, and data ambiguity. Conclusively, we highlight interesting
directions for future research in LLM-generated text detection to advance the
implementation of responsible artificial intelligence (AI). Our aim with this
survey is to provide a clear and comprehensive introduction for newcomers
while also offering seasoned researchers a valuable update in the field of
LLM-generated text detection. The useful resources are publicly available at:
https://github.com/NLP2CT/LLM-generated-Text-Detection

###### Index Terms:

Survey, LLM-generated text detection, responsible artificial intelligence

##  I Introduction

With the rapid development of LLMs, the text generation capabilities of LLMs
have reached a level comparable to human writing [1, 2, 3]. LLMs have
permeated various aspects of daily life and play a vital role become pivotal
in many professional workflows [4], facilitating tasks such as advertising
slogan creation [5], news composition [6], story generation [7], and code
generation [8, 9]. Furthermore, their impact significantly shapes the
progression of numerous sectors and disciplines, including education [10], law
[11], biology [12], and medicine [13].

The powerful generation capabilities of LLMs have rendered it challenging for
individuals to discern between LLM-generated and human-written texts,
resulting in the emergence of intricate concerns. The concerns regarding LLM-
generated text originate from two perspectives. Firstly, LLMs are susceptible
to fabrications [14], reliance on outdated information, and heightened
sensitivity to prompts. These vulnerabilities can facilitate the spread of
erroneous knowledge [15], undermine technical expertise [16, 17], and promote
plagiarism [18]. Secondly, there exists the risk of malicious exploitation of
LLMs in activities such as disinformation dissemination [19, 20], online
fraudulent schemes [21, 22], social media spam production [23], and academic
dishonesty, especially with students employing LLMs for essay writing [24,
25]. Concurrently, LLMs increasingly shoulder the data generation
responsibility in AI research, leading to the recursive use of LLM-generated
text in their own training and assessment. A recent analysis, titled Model
Autophagy Disorder (MAD) [26], raised alarms over this AI data feedback loop.
As generative models undergo iterative improvements, LLM-generated text may
gradually replace the need for human-curated training data. This could
potentially lead to a reduction in the quality and diversity of subsequent
models. In essence, the consequences of LLM-generated text encompass both
societal [27] and academic [28] risks, and the use of LLM-generated data will
hinder the future development of LLMs and detection technology. However, for
the LLM-generated text detection task, detection methods relying on humans are
unreliable [29, 30, 31] and have very low accuracy, even only slightly better
than random classification. Furthermore, the ability of humans to identify
LLM-generated text is often lower than that of detectors or detection
algorithms in various environmental settings [32]. Thus, there is an
imperative demand for robust detectors to identify LLM-generated text
effectively. Establishing such mechanisms is pivotal to mitigating LLM misuse
risks and fostering responsible AI governance in the LLM era [33, 34, 35].

Research on the detection of LLM-generated text has received lots of attention
even before the advent of ChatGPT, especially in areas such as early
identification of deepfake text [36] and machine-generated text detection
[37]. Typically, this problem was regarded as a classification task,
discerning between LLM-generated text and human-written text [37]. Back to
this research stage, the detection task was predominantly focusing on
translation-generated texts and utilizing statistical methods. The
introduction of ChatGPT has sparked a significant surge in interest
surrounding LLMs, heralding a paradigm shift in the research landscape. In
response to the escalating challenges posed by LLM-generated text, the NLP
community has intensely pursued solutions, delving into LLM-generated text
detection and related attacking methodologies. While Crothers et al. [38],
Tang et al. [39] recently presented a review on this topic, we argue that its
depth of detection methods is insufficient.

In this article, we furnish a meticulous and profound review of contemporary
research on LLM-generated text detection, aiming to guide researchers through
the challenges and prospective research trajectories. We investigate the
latest breakthroughs, beginning with an introduction to the task of LLM-
generated text detection, the underlying mechanisms of text generation by
LLMs, and the sources of LLM’s enhanced text generation capabilities. We also
shed light on the contexts and imperatives of LLM-generated text detection.
Furthermore, we spotlight popular datasets for the task, exposing their
current deficiencies to stimulate the creation of more refined datasets. Our
discussion extends to the latest detector research work. In addition to the
traditional fine-tuned LMs and zero-shot methods, we also report watermarking
techniques, adversarial learning methods, LLMs as detectors, and human-
assisted methods. A subsequent analysis pinpoints research limitations in LLM-
generated text detectors, highlighting critical areas like out-of-distribution
challenges, potential attacks, and data ambiguity. Conclusively, we ponder
upon potential directions for future research, aiming to help the development
of efficient detectors.

##  II Background

###  II-A LLM-generated Text Detection Task

Figure 1: Toy picture of LLM-generated text detection task. This task is a
binary classification task that detects whether the provided text is generated
by LLMs or written by humans.

Detecting LLM-generated text is an intricate challenge. Generally speaking,
humans struggle to discern between LLM-generated text and human-written text,
and their capability to distinguish such texts exceeds random classification
only slightly. Table I offers some examples where LLM-generated text often is
extremely close to human-written text and can be difficult to distinguish.
Text generated by LLMs during normal operation and instances in which they
fabricate facts often exhibit no intuitively discernible differences. When
LLMs either abstain from providing an answer or craft neutral responses,
certain signifiers, for instance, the explicit statement “I am an AI language
model”, may facilitate human adjudication, but such examples are less

TABLE I: Examples of human-written text and LLM-generated text.

Type | Question | Human-written | LLMs-generated
---|---|---|---
Normal |  |  Explain what is NLP?
---
|  Natural language processing (NLP) is an interdisciplinary subfield of
linguistics, computer science, and artificial intelligence …
---
|  Natural language processing (NLP) is a field of computer science,
artificial intelligence, and linguistics that focuses on …
---
Refusal |  |  How is today special?
---
|  Today’s Special is a Canadian children’s television show produced by Clive
VanderBurgh at TVOntario from 1981 to 1987.
---
|  I’m sorry, but I am an AI language model and do not have access to current
dates or events. Is there anything else I can help you with …
---
Fabricate |  |  Explain what is NLP based on one publication in the recent literature.
---
|  In “Natural language processing: state of the art, current trends and
challenges”, NLP is summarized as a discipline that uses various algorithms,
tools and methods to …
---
|  NLP is a multidisciplinary field at the intersection of computer science,
linguistics, and ai, as described in a recent peer-reviewed publication titled
”Natural Language Processing: A Comprehensive Overview and Recent Advances”
(2023) …
---

Recent studies [40, 41, 42, 43] have highlighted significant disparities
between human-written and LLM-generated text, such as ChatGPT. Notably, LLM-
generated text often exhibits qualities of enhanced organization, logical
structure, formality, and objectivity in comparison to human-written text.
Additionally, LLMs frequently produce extensive and comprehensive responses,
characterized by a lower prevalence of bias and harmful content. Nevertheless,
they occasionally introduce nonsensical or fabricated details. Linguistically,
LLM-generated text tends to be about twice the length of human-written text
but exhibits a more limited vocabulary. LLMs exhibit a higher frequency of
noun, verb, determiner, adjective, auxiliary, coordinating conjunction, and
particle word categories compared to humans, and less adverb and punctuation,
incorporating more deterministic, conjunctive, and auxiliary structures in
their syntax. Additionally, LLM-generated text often conveys less emotional
intensity and exhibits clearer presentation than human writings, a phenomenon
possibly linked to an inherent positive bias in LLMs [44, 45]. Although there
are slightly different statistical gaps on different datasets, it is clear
that the difference between LLM-generated text and human-written text clearly
exists, because the statistical results of the difference in language features
and human visual perception are consistent.

In this survey, we begin by providing definitions for human-written text, LLM-
generated text, and the detection task.

##### Human-written Text

is characterized as the text crafted by individuals to express thoughts,
emotions, and viewpoints. This encompasses articles, poems, and reviews, among
others, typically reflecting personal knowledge, cultural milieu, and
emotional disposition, spanning the entirety of the human experience.

##### LLM-generated Text

is defined as cohesive, grammatically sound, and pertinent content generated
by LLMs. These models are trained extensively on NLP techniques, utilizing
large datasets and machine learning methodologies. The quality and fidelity of
the generated text typically depend on the scale of the model and the
diversity of training data.

##### LLM-generated Text Detection Task

is conceptualized as a binary classification task, aiming to ascertain if a
given text is generated by an LLM. The formal representation of this task is
given by the subsequent equation:

$D(x)=\left\\{\begin{array}[]{rcl}1&&\text{if }x\text{ generated by LLMs}\\\
0&&\text{if }x\text{ written by human}\end{array}\right.$ (1)

where $D(x)$ represents the detector, and $x$ is the text to be detected.

###  II-B LLMs Text Generation and Confusion Sources

####  II-B1 Generation Mechanisms of LLMs

The text generation mechanism of LLMs operates by sequentially predicting
subsequent tokens. Rather than producing an entire paragraph instantaneously,
LLMs methodically construct text one word at a time. Specifically, LLMs decode
subsequent tokens in a textual sequence, taking into account both the input
sequence and previously decoded tokens. Assume that the total time step is
$T$, the current time step is $t$, the input text or tokenised sequence is:
$X_{T}=\\{x_{1},x_{2},...x_{T}\\}$, and the previous output sequence is
$Y_{t-1}=\\{y_{1},y_{2},...y_{t-1}\\}$. At this point, the output next word
$y_{t}$ can be expressed as:

$y_{t}\sim P(y_{t}|Y_{t-1},X_{T})=softmax(w_{o}\cdot h_{t})$ (2)

$h_{t}$ is the hidden state of the model at time step $t$, $w_{o}$ is the
output matrix, the softmax function is used to obtain the probability
distribution of the vocabulary, and $y_{t}$ is sampled from the probability
distribution of the vocabulary $P(y_{t}|Y_{t-1},X_{T})$. The joint probability
function for the final output sequence can be modeled and represented as:

$Y_{T}=\\{y_{1},y_{2},...,y_{T}\\}$ (3)

The quality of the decoded text is intrinsically tied to the chosen decoding
strategy. Given that the model constructs text sequentially, the quality of
generated text hinges on the method used to select the subsequent word from
the vocabulary, that is, how $y_{t}$ is sampled from the probability
distribution of vocabulary. The predominant decoding techniques encompass
greedy search, beam search, top-k sampling, and top-p sampling. Table II
offers a comparison of the underlying principles, along with the respective
merits and drawbacks, of these decoding methods. This comparison aids in
elucidating the text generation process of LLMs and the specific
characteristics of the text they produce.

TABLE II: The core ideas of different text decoding strategies, as well as
their advantages and disadvantages.

Strategy |  Core Idea |  Advantages |  Drawbacks
---|---|---|---
Greedy Search |  Only the token with the highest current probability is considered at each step. |  Fast and simple. |  Easy to fall into local optimality, lack of diversity, unable to deal with uncertainty.
Beam Search [46] |  Several more candidates can be considered at each step. |  Improvement of text quality and flexibility. |  Tend to generate repetitive fragments, work poorly in open generation domains, unable to handle uncertainty.
Top-K Sampling [47] |  Samples among the K most likely words at each step. |  Increase diversity and be able to deal with uncertainty. |  Difficulty in controlling the quality of generation, which may result in incoherent text.
Top-P Sampling [48] |  Use the shape of the probability distribution to determine the set of tokens to be sampled from |  Coherence and the ability to deal with uncertainty. |  Dependent on the quality of the model predictions, diversity is related to the parameter $P$.

####  II-B2 Sources of LLMs’ Strong Generation Capabilities

The burgeoning growth in model size, data volume, and computational capacity
has significantly enhanced the capabilities of LLMs. Beyond a specific model
size, certain abilities manifest that are not predictable by scaling laws.
These abilities, absent in smaller models but evident in LLMs, are termed
“Emergent Abilities” of LLMs.

##### In-Context Learning (ICL)

The origins of ICL capabilities remain a topic of ongoing debate [49].
However, this capability introduces a paradigm where model parameters remain
unchanged, and only the design of the prompt is modified to elicit desired
outputs from LLMs. This concept was first introduced in the GPT-3 [50]. Brown
et al. [50] posited that the presence of ICL is fundamental for the swift
adaptability of LLMs across a diverse set of tasks. With just a few examples,
LLMs can effectively tackle downstream tasks, obviating the previous BERT
model-based approach that relied on pretraining followed by fine-tuning for
specific tasks [51].

##### Alignment of Human Preference

Although LLMs can be guided to generate content using carefully designed
prompts, the resulting text might lack control, potentially leading to the
creation of misleading or biased content [52]. The primary concern of these
models lies in predicting subsequent words to form coherent sentences based on
vast corpora, rather than ensuring that the content generated is both
beneficial and innocuous to humans. To address these shortcomings, OpenAI
introduced the Reinforcement Learning from Human Feedback (RLHF) approach, as
detailed in [53] and [54]. This approach begins by fine-tuning LLMs using data
from user-directed quizzes and subsequently evaluating the model’s outputs
with human assessors. Simultaneously, a reward function is established, and
the LLM is further refined using the Proximal Policy Optimization (PPO)
algorithm. The end result is a model that aligns with human values,
understands human instructions, and genuinely assists users.

##### Complex Reasoning Capabilities

While LLMs’ ICL and alignment capability facilitate meaningful interactions
and assistance, their performance tends to degrade in tasks demanding logical
reasoning and heightened complexity. Wei et al. [55] observed that encouraging
LLMs to produce more intermediate steps through a Chain of Thought (CoT) can
enhance their effectiveness. Tree of Thoughts (ToT) [56] and Graph of Thoughts
(GoT) [57] are extensions of this methodology. Both strategies augment LLM
performance on intricate tasks by amplifying the computational effort required
for the model to deduce an answer.

###  II-C Why Do We Need to Detect Text Generated by LLMs?

As LLMs undergo iterative refinements and reinforcement learning through human
feedback, their outputs become increasingly harmonized with human values and
preferences. This alignment facilitates broader acceptance and integration of
LLM-generated text into everyday life. The emergence of various AI tools has
played a significant role in fostering intuitive human-AI interactions and
democratizing access to the advanced capabilities of previously esoteric
models. From interactive web-based assistants like
ChatGPT,111https://chat.openai.com/ to search engines enhanced with LLM
technology like the contemporary version of Bing,222https://www.bing.com/ to
specialized tools like Coplit,333https://github.com/features/copilot/ and
Scispeace444https://typeset.io/ that assist professionals in code generation
and scientific research, LLMs have subtly woven into the digital fabric of our
lives, propagating their content across diverse platforms.

However, it is important to acknowledge that for the majority of users, LLMs
and their applications are still considered black-box AI systems. For
individual users, this often serves as a benign efficiency boost, sidestepping
laborious retrieval, and summarization. However, within specific contexts and
against the backdrop of the broader digital landscape, it becomes crucial to
discern, filter, or even exclude LLM-generated text. It is important to
emphasize that not all situations call for the detection of LLM-generated
text. Unnecessary detection can lead to consequences such as system
inefficiencies and inflated development costs. Generally, detecting LLM-
generated text might be superfluous when:

  * •

The utilization of LLMs poses minimal risk, especially when they handle
routine, replicable tasks.

  * •

The spread of LLM-generated text is confined to predictable, limited domains,
like closed information circles with few participants.

Drawing upon the literature reviewed in this study, the rationale behind
detecting LLM-generated text can be elucidated from multiple vantage points,
as illustrated in Figure 2 The delineated perspectives are, in part, informed
by the insights presented in [58] and [59].

Figure 2: The most critical reason why LLM-generated text detection is needed
urgently. We discussed it from five perspectives: Regulation, Users,
Developments, Science, and Human Society.

While the list provided above may not be exhaustive and some facets may
intersect or further delineate as LLMs and AI systems mature, we posit that
these points underscore the paramount reasons for the necessity of detecting
text generated by LLMs.

##### Regulation

As AI tools, often characterized as black boxs, the inclusion of LLM-generated
text in creative endeavors raises significant legal issues. A pressing concern
is the eligibility of LLM-generated texts for intellectual property rights
protection, a subject still mired in debate [60, 61], although the _EU AI Act_
555https://artificialintelligenceact.eu/the-act/ has begun to continuously
improve to regulate the use of AI. The main challenges arise from issues such
as ownership of the training data used by the AI in generating output and how
to determine how much human involvement is enough to make the work theirs. The
prerequisite for copyright protection for AI supervision and AI-generated
content is that human creativity in the materials used to train AI systems can
be distinguished, so as to further promote the implementation of more complete
legal supervision.

##### Users

LLM-generated text, refined through various alignment methods, is
progressively aligning with human preferences. This content permeates numerous
user-accessible platforms, including blogs and Questions & Answers (Q&A)
forums. However, excessive reliance on such content can undermine user trust
in AI systems and, by extension, digital content as a whole. In this context,
the role of LLM-generated text detection becomes crucial as a gatekeeper to
regulate the prevalence of LLM-generated text online.

##### Developments

With the evolving prowess of LLMs, Li et al. [62] suggested that LLMs can
self-assess and even benchmark their own performances. Due to its excellent
text generation performance, LLMs are also used to construct many training
data sets through preset instructions [63]. However, Alemohammad et al. [26]
posited that this “Self-Consuming” paradigm may engender a homogenization in
LLM-generated texts, potentially culminating in what is termed as “LLM
Autophagy Disorder” (MAD). If LLMs heavily rely on web-sourced data for
training, and a significant portion of this data originates from LLM outputs,
it could hinder their long-term progress.

##### Science

The relentless march of human progress owes much to the spirit of scientific
exploration and discovery. However, the increasing presence of LLM-generated
text in academic writing [64] and the use of LLM-originated designs in
research endeavors raise concerns about potentially diluting human ingenuity
and exploratory drive. Although current methodologies may have limitations,
further enhancements in detection capabilities will strengthen academic
integrity and preserve human independent thinking in scientific research.

##### Human Society

From a societal perspective, analyzing the implications of LLM-generated text
reveals that these models essentially mimic specific textual patterns while
predicting subsequent tokens. If used improperly, these models have the
potential to diminish linguistic diversity and contribute to the formation of
information silos within societal discourse. In the long run, detecting and
filtering LLM-generated text is crucial for preserving the richness and
diversity of human communication, both linguistically and informatively.

##  III Systematic Investigation and Implementation

TABLE III: Overview of the diverse databases and search engines utilized in
our research, along with the incorporated search schemes and the consequent
results obtained.

Databases & Search Engine |  Search Scheme |  Retrieved
---|---|---
Google Scholar |  |
(https://scholar.google.com/) |  Full Text |  210
ArXiv (https://arxiv.org/) |  Full Text |  N/Aa
Scopus |  |
(https://www.scopus.com/) |  TITLE-ABS-KEY: ( Title, Abstract, Author Keywords, Indexed Keywords ) |  133
Web of Science |  |
(https://www.webofscience.com/) |  Topic: ( Searches Title, Abstract, Author Keywords, Keywords Plus ) |  92
IEEE Xplore |  |
(https://ieeexplore.ieee.org/) |  Full Text |  49
Springer Link |  |
(https://link.springer.com/) |  Full Text |  N/Aa
ACL Anthology |  |
(https://aclanthology.org/) |  Full Text |  N/Aa
ACM Digital Library |  |
(https://dl.acm.org/) |  Title |  N/Ab

a Search engines cannot use all keywords in a single search string. Therefore
the retrieved results are inaccurate and there may be duplicate results of
thesis queries.
b The search engine retrieved an inaccurate number of papers that were weakly
related to our topic.

Our survey employed the System for Literature Review (SLR) as delineated by
Barbara Kitchenham [65], a methodological framework designed for evaluating
the extent and quality of extant evidence pertaining to a specified research
question or topic. Offering a more expansive and accurate insight compared to
conventional literature reviews, this approach has been prominently utilized
in numerous scholarly surveys, as evidenced by Murtaza et al. [66], Saeed and
Omlin [59]. The research questions guiding our SLR were as follows:

_What_ are the prevailing methods for detecting LLM-generated text, and _what_
are the main challenges associated with these methods?

Upon delineating the research problems, our study utilized search terms
directly related to the research issue, specifically: “LLM-generated text
detection”, “machine-generated text detection”, “AI-written text detection”,
and “deepfake text detection”. These terms were strategically combined using
the Boolean operator OR to formulate the following search string: (“LLM-
generated text detection” OR “machine-generated text detection” OR “AI-written
text detection” OR “deepfake text detection”). Subsequently, employing this
search string, we engaged in a preliminary search through pertinent and
authoritative electronic dissertation databases and search engines. Our
investigation focused on scholarly articles that were publicly accessible
prior to September 2023. Table III outlines the sources used and provides an
overview of our results. Google Scholar predominates as the search engine
yielding the maximum number of retrievable documents. Upon meticulous
examination, it is observed that a substantial portion of the documents
originate from ArXiv, primarily shared by researchers.

Subsequently, we established the ensuing criteria to scrutinize the amassed
articles:

  * •

The article should be a review focusing on the methods and challenges
pertinent to LLM-generated (machine-generated/AI-written) text detection.

  * •

The article should propose a methodology specifically designed for the
detection of LLM-generated (machine-generated/AI-written) text.

  * •

The article should delineate challenges and prospective directions for future
research in the domain of text generation for LLMs.

  * •

The article should articulate the necessity and applications of LLM-generated
text detection.

20192020202120222023$0$$20$$40$$60$YearNumber of publication Figure 3: The
distribution by year of the last 5 years of literature obtained from the
screening is plotted. The number of published articles obtain significant
attention in 2023.  TABLE IV: Summary of content organisation of this survey.

Section |  Topic |  Content
---|---|---
Section 4 |  Data |  Benchmarks for LLM-generated Text Detection, Other Datasets Easily Extended to Detection Tasks and Challenge of datsets for LLM-generated Text Detection.
Section 5 |  Detectors |  Watermarking Technology, Zero-shot Methods, Fine-tuning LM Methods, Adversarial Learning Methods, LLMs as Detector, and Human-assisted Methods
Section 6 |  Evaluation Metrics |  Accuracy (ACC), Area under the ROC curve (AUROC), Recall (Rec) and F1-Score.
Section 7 |  Issues |  Out of Distribution Challenges, Potential Attacks and Data Ambiguity
Section 8 |  Future Directions |  Building Robust Detectors with Attacks, Enhancing the Efficacy of Zero-shot Detectors, Optimizing Detectors for Low-resource Environments, Constructing Detectors Amidst Data Ambiguity, Constructing Detectors with Factual Discrimination Capabilities and Multi-agent Systems for LLM-generated Text Detection.
Section 9 |  Conclusion |  -

If any one of the aforementioned four criteria was met, the respective work
was deemed valuable for our study. Following a process of de-duplication and
manual screening, we identified 82 pertinent pieces of literature. The
distribution trend of these works, delineated by year, is illustrated in
Figure 3. Notably, the majority of relevant research on LLM-generated text
detection was published in the year 2023 (as shown in Figure 3), underscoring
the vibrant development within this field and highlighting the significance of
our study. In subsequent sections, we provide a synthesis and analysis of the
data (in Section 4), primary detectors (in Section 5), evaluation metrics (in
Section 6), issues (in Section 7), and future research directions (in Section
8) pertinent to LLM-generated text detection. The comprehensive structure of
the survey is outlined in Table IV, offering a detailed overview of the
organization of our review.

##  IV Data

High-quality datasets are pivotal for advancing research in the detection of
LLM-generated text. These datasets enable researchers to swiftly develop and
calibrate efficient detectors and establish standardized metrics for
evaluating the efficacy of their methodologies. However, procuring such high-
quality labeled data often demands substantial financial, material, and human
resources. Presently, the development of datasets focused on detecting LLM-
generated text is in its nascent stages, plagued by issues such as limited
data volume and sample complexity, both crucial for crafting a robust
detector. In this section, we introduce prevalent datasets employed for LLM-
generated text detection, generously contributed by recent studies.
Additionally, we highlight datasets from other domains that, while not
originally designed for detection tasks, can be adapted to various detection
scenarios, which is a prevailing strategy in many contemporary detection
studies. We subsequently evaluate these datasets, identifying current
shortcomings and challenges in dataset creation for LLM-generated text
detection, aiming to inform the design of future datasets.

###  IV-A Datasets

Recent research trends indicate that several datasets have garnered
recognition among domain experts and are frequently employed for training and
testing various methods. In this section, we introduce prevalent datasets and
discuss others that can be adapted for LLM-generated text detection tasks.

TABLE V: Summary of benchmarks for LLM-generated text detection.

Corpus |  Use |  Human |  LLMs |  LLMs Type |  Language |  Attack |  Domain
---|---|---|---|---|---|---|---
HC3 [40] |  train |  58k |  26k |  ChatGPT |  English |  - |  Finance, Medicine, Open Question Answering, Computer Science
HC3-Chinese [40] |  train |  22k |  17k |  ChatGPT |  Chinese |  - |  Open Question Answering, Baike, NLPCC-DB Question Answering, Finance, Medicine, Psychology, Law
CHEAT [28] |  train |  15k |  35k |  ChatGPT |  English |  Polish |  Scientific Abstract
GROVER Dataset [67] |  train valid test |  5k, 2k, 8k |  5k, 1k, 4k |  Grover-Mega |  English |  - |  News
TweepFake [68] |  train |  12k |  12k |  GPT-2, RNN, Markov, LSTM, CharRNN |  English |  - |  Tweets
GPT-2 Output Dataset666https://github.com/openai/gpt-2-output-dataset |  train |  250k |  250k |  GPT-2 |  English |  - |  Web
TuringBench [29] |  train |  10k |  190k |  GPT-1, GPT-2, GPT-3, GROVER, CTRL, XLM, XLNET, FAIR, TRANSFORMER_XL, PPLM |  English |  - |  News
MGTBench [69] |  train test |  2k 563 |  13k 3k |  ChatGPT, ChatGPT-turbo, ChatGLM, Dolly, GPT4All, StableLM |  English |  Adversarial Attacks |  Question Answering, Reading comprehension
ArguGPT [70] |  train valid test |  3k 350 350 |  3k 350 350 |  GPT2-xl, Text-babbage-001, Text-curie-001, Text-davinci-001, Text-davinci-002, Text-davinci-003, GPT-3.5-turbo |  English |  - |  Essays writing
DeepfakeText-Detect-Dataset [71] |  train valid test |  319k 56.8k 56.8k |  - |  GPT, LLaMA, GLM-130B, FLAN-T5, OPT, T0, BLOOM |  English |  Paraphrase |  Opinion statements, News article writing, Question Answering, Story generation, Commonsense reasoning, Knowledge illustration, Scientific writing
M4 [72] |  train valid test |  122k 500 500 |  122k 500 500 |  ChatGPT, Textdavinci-003, LLaMa, FlanT5, Cohere, Dolly-v2, BLOOMz |  English, Chinese, Russian, Urdu, Indonesian, Arabic |  - |  Web, Scientific writing, News, Question Answering, News article writing
GPABenchmark [73] |  train |  600k |  600k |  GPT-3.5 |  English |  Polish |  Scientific writing
Scientific-articles Benchmark [74] |  train test |  8k 4k |  8k 4k |  SCIgen, GPT-2, GPT-3, ChatGPT, Galactica |  English |  - |  Scientific writing

####  IV-A1 Benchmarks for LLM-generated Text Detection

Benchmarks with high quality can help researchers verify whether their
detectors are feasible and effective rapidly. We sorted out and compared the
benchmarks that are currently popular or with potential, as shown in Table V.
On the one hand, we hope to help researchers better understand their
differences to choose suitable benchmarks for their experiments. On the other
hand, we hope to draw researchers’ attention to the latest benchmarks, which
have been fully designed to verify the latest issues for the task, with great
potential.

##### Human ChatGPT Comparison Corpus (HC3)

The HC3 [40] stands as one of the initial open-source efforts to compare
ChatGPT-generated text with human-written text. It involves collecting both
human and ChatGPT responses to identical questions. Due to its pioneering
contributions in this field, the HC3 corpus has been utilized in numerous
subsequent studies as a valuable resource. The corpus offers datasets in both
English and Chinese. Specifically, HC3-en comprises 58k human responses and
26k ChatGPT responses, derived from 24k questions. These questions primarily
originate from the ELI5 dataset, WikiQA dataset, Crawled Wikipedia, Medical
Dialog dataset, and FiQA dataset, which correspond to the categories of
reddit_eli5, open_qa, wiki_csai, medicine, and finance, respectively. On the
other hand, HC3-zh encompasses a broader spectrum of domains, featuring 22k
human answers and 17k ChatGPT responses. The data within HC3-zh spans seven
distinct domains: open_qa, baike, nlpcc_dbqa, medicine, finance, psychology,
and law. The sources for the human responses in HC3-zh include WebTextQA,
BaikeQA, Crawled BaiduBaike, NLPCC-DBQA dataset, Medical Dialog dataset, Baidu
AI Studio, and the LegalQA dataset. However, it is pertinent to note some
limitations of the HC3 dataset, such as the lack of diversity in bootstrapped
generated instructions.

##### CHEAT

The CHEAT dataset [28] stands as the most extensive publicly accessible
resource dedicated to detecting artificially generated spurious academic
content from ChatGPT. The collection comprises human-authored academic
abstracts sourced from IEEE Xplore, boasting an average abstract length of
163.9 words and a vocabulary size of 130k words. Subsequent to the ChatGPT
generation process, the dataset encompasses 15k human-composed abstracts and
35k ChatGPT-crafted summaries. To enhance the dataset’s applicability in real-
world contexts, outputs were guided by ChatGPT for further refinement and
amalgamation. The “polishing” process aims to simulate users who may seek to
bypass plagiarism detection by embellishing the text, while “blending”
represents scenarios where users might combine manually drafted abstracts with
those seamlessly crafted by ChatGPT to elude detection mechanisms.
Nevertheless, a limitation of the CHEAT dataset is its focus on narrow
academic disciplines, overlooking cross-domain challenges, which stems from
constraints related to its primary data source.

##### TuringBench

The TuringBench dataset [29] is an initiative designed to explore the
challenges of the “Turing test” in the context of neural text generation
techniques. It comprises human-written content derived from 10k news articles,
predominantly from reputable sources such as CNN. For the purpose of this
dataset, only articles ranging between 200 to 400 words were selected. LLM-
generated text within this dataset is produced by 19 distinct text generation
models, including GPT-1, GPT-2 variants (small, medium, large, xl, and
pytorch), GPT-3, different versions of GROVER (base, large, and mega), CTRL,
XLM, XLNET variants (base and large), FAIR for both WMT19 and WMT20,
Transformer-XL, and both PLM variants (distil and GPT-2). Each model
contributed 200k samples, categorized by label type. Notably, TuringBench
emerged as one of the pioneering benchmark environments for the detection of
LLM-generated text. However, given the rapid advancements in LLM technologies,
the samples within TuringBench are now less suited for training and validating
contemporary detector performances. As such, timely updates incorporating the
latest generation models and their resultant texts are imperative.

##### TweepFake Dataset

TweepFake [68] is a seminal dataset tailored for the analysis of fake tweets
on Twitter, derived from both genuine and counterfeit accounts. It amasses a
total of 25k tweets, with an equal distribution between human-written and
machine-generated entries. The machine-generated tweets were crafted using
various techniques, including GPT-2, RNN, Markov, LSTM, and CharRNN. While
TweepFake continues to be a dataset of choice for many scholars, those working
with LLMs should critically assess its relevance and rigor in light of
evolving technological capabilities.

##### GPT2-Output Dataset

The GPT2-Output Dataset,777https://github.com/openai/gpt-2-output-dataset
introduced by OpenAI, is based on 250k documents sourced from the WebText test
set for its human-written text. Regarding the LLM-generated text, the dataset
includes 250k randomly generated samples using a temperature setting of 1
without truncation and an additional 250k samples produced with Top-K 40
truncation. This dataset was conceived to further research into the
detectability of the GPT-2 model. However, a notable limitation lies in the
insufficient complexity of the dataset, marked by the uniformity of both the
model and data distribution.

##### GROVER Dataset

The GROVER Dataset [67], introduced by Zellers et al. [67], is styled after
news articles. Its human-written text is sourced from RealNews, a
comprehensive corpus of news articles derived from Common Crawl. The LLM-
generated text is produced by Grover-Mega, a transformer-based news generator
with 1.5 billion parameters. A limitation of this dataset, particularly in the
current LLM landscape, is the uniformity and singularity of both its model and
data distribution.

##### MGTBench

Introduced by He et al. [69], MGTBench stands as the inaugural benchmark
framework for MGT detection. It boasts a modular architecture, encompassing an
input module, a detection module, and an evaluation module. The dataset draws
upon several of the foremost LLMs, including ChatGPT, ChatGLM, Dolly, ChatGPT-
turbo, GPT4All, and StableLM, for text generation. Furthermore, it
incorporates over ten widely-recognized detection algorithms, demonstrating
significant potential.

##### ArguGPT Dataset

The ArguGPT Dataset [70] is tailored for detecting LLM-generated text in
various academic contexts such as classroom exercises, TOEFL, and GRE writing
tasks. It comprises 4k argumentative essays, generated by seven distinct GPT
models. Its primary aim is to tackle the unique challenges encountered in
teaching English as a second language.

##### DeepfakeTextDetect Dataset

Attention is also drawn to the DeepfakeTextDetect Dataset [71], a robust
platform tailored for deep fake text detection. The dataset amalgamates human-
written text from ten diverse datasets, encompassing genres like news
articles, stories, scientific writings, and more. It features texts generated
by 27 prominent LLMs, sourced from entities such as OpenAI, LLaMA, and
EleutherAI. Furthermore, the dataset introduces an augmented challenge with
the inclusion of GPT-4 generated text and paraphrased text.

##### M4

M4 [72] emerges as an expansive benchmark corpus dedicated to the detection of
LLM-generated text. It spans multiple generators, domains, and languages.
Drawing from diverse sources like wiki pages across various regions, news
outlets, and academic portals, it mirrors typical scenarios wherein LLMs find
daily utility. The LLM-generated texts within M4 are crafted using cutting-
edge models like ChatGPT, LLaMa, BLOOMz, FlanT5, and Dolly. Notably, the
dataset encompasses cross-lingual nuances, incorporating content in over ten
languages. To summarize, while the M4 dataset adeptly addresses cross-domain,
cross-model, and cross-lingual complexities, it could benefit from exploring
more adversarial scenarios.

##### GPABenchmark

The GPABenchmark [73] is a comprehensive dataset encompassing 600k samples.
These samples span human-written, GPT-written, GPT-completed, and GPT-polished
abstracts from a broad spectrum of academic disciplines, such as computer
science, physics, and the humanities and social sciences. This dataset
meticulously captures the quintessential scenarios reflecting both the
utilization and potential misapplication of LLMs in academic composition.
Consequently, it delineates three specific tasks: generation of text based on
a provided title, completion of a partial draft, and refinement of an existing
draft. Within the domain of academic writing detection, GPABenchmark stands as
a robust benchmark, attributed to its voluminous data and its holistic
approach to scenario representation.

##### Scientific-articles Benchmark

The Scientific-articles Benchmark [74] comprises 16k human-written articles
alongside 13k LLM-generated samples. The human-written articles are sourced
from the arXiv dataset available on Kaggle. In contrast, the machine-generated
samples, which include abstracts, introductions, and conclusions, are produced
by SCIgen, GPT-2, GPT-3, ChatGPT, and Galactica using the titles of the
respective scientific articles as prompts. A notable limitation of this
dataset is its omission of various adversarial attack types.

####  IV-A2 Other Potential Datasets

TABLE VI: Summary of other potential datasets that can easily extended to LLM-
generated text detection tasks.

Corpus |  Size |  Source |  Language |  Domain
---|---|---|---|---
XSum [75] |  42k |  BBC |  English |  Various (News, Politics, Sports, etc.)
SQuAD [76] |  98.2k |  Wiki |  English |  Open Question Answering
WritingPrompts [47] |  302k |  Reddit WRITINGPROMPTS |  English |  Story Generation
Wiki40B [77] |  17.7m |  Wiki |  40+ Languages |  Wiki
PubMedQA [78] |  211k |  PubMed |  English |  Biomedical Science, Question Answering
Children’s Book Corpus [79] |  687k |  Books |  English |  Children’s Books, Question Answering
Avax Tweets Dataset [80] |  137m |  Twitter |  English |  COVID-19 Vaccine Tweets
Climate Change Dataset [81] |  4m |  Twitter |  English |  Climate Change Tweets
Yelp Dataset [82] |  700k |  Yelp |  English |  Opinion Statements
ELI5 [83] |  556k |  Reddit |  English |  Question Answering
ROCStories [84] |  50k |  Crowdsourcing |  English |  Story Generation
HellaSwag [85] |  70k |  ActivityNet Captions, Wikihow |  English |  Question Answering, Sentence Completion
SciGen [86] |  52k |  arXiv |  English |  Academic Writing, Question Answering
WebText [87] |  45m |  Web |  English |  Open Domain
TruthfulQA [20] |  817 |  authors writtEnglish |  English |  Question Answering
NarrativeQA [88] |  1.4k |  Gutenberg3, web |  English |  Question Answering
TOEFL11 [89] |  12k |  TOEFL test |  11 Languages |  Scientific writing
Peer Reviews [90] |  14.5k |  NIPS 2013–2017, CoNLL 2016, ACL 2017, ICLR 2017, arXiv 2007–2017 |  English |  Scientific writing

Constructing a dataset from scratch that encompasses both human-written and
LLM-generated text can indeed be a resource-intensive endeavor. Recognizing
the diverse requirements for LLM-generated text detection across scenarios,
researchers commonly adapt existing datasets from fields like QA, academic
writing, and story generation to represent human-written text. They then
produce LLM-generated text for detector training using methods like prompt
creation and bootstrap complementation. This survey offers a concise
classification and overview of these datasets, summarized in Table VI.

##### Q&A

Q&A is a prevalent and equitable method for constructing datasets. By posing
identical questions to LLMs, one can generate paired sets of human-written and
LLM-generated text.

  * •

PubMedQA [78]: This is a biomedical question-and-answer (QA) dataset sourced
from PubMed.888https://www.ncbi.nlm.nih.gov/pubmed/

  * •

Children Book Corpus [79]: This dataset, derived from freely available books,
gauges the capacity of LMs to harness broader linguistic contexts. It
challenges models to select the correct answer from ten possible options,
based on a context of 20 consecutive sentences. The answer types encompass
verbs, pronouns, named entities, and common nouns.

  * •

ELI5 [83]: A substantial corpus for long-form Q&A, ELI5 focuses on tasks
demanding detailed responses to open-ended queries. The dataset comprises 270k
entries from the Reddit forum “Explain Like I’m Five”, which offers
explanations tailored to the comprehension level of a five-year-old.

  * •

TruthfulQA [20]: This benchmark evaluates the veracity of answers generated by
LLMs. It encompasses 817 questions spread across 38 categories such as health,
law, finance, and politics. All questions were crafted by humans.

  * •

NarrativeQA [88]: This English-language dataset includes summaries or stories
along with related questions aimed at assessing reading comprehension,
especially concerning extended documents. Data is sourced from Project
Gutenberg 999https://gutenberg.org/ and web-scraped movie scripts, with hired
annotators providing the answers.

##### Academic Writing

Academic writing is frequently explored in real-world research contexts. Given
a specific academic topic, LLMs can efficiently generate academic articles or
abstracts.

  * •

Peer Read [90]: This represents the inaugural public dataset of scientific
peer-reviewed articles, comprising 14.7k draft articles and 10.7k expert-
written peer reviews for a subset of these articles. Additionally, it includes
the acceptance or rejection decisions from premier conferences such as ACL,
NeurIPS, and ICLR.

  * •

ArXiv:101010https://arxiv.org/ A freely accessible distribution service and
repository, ArXiv hosts 2.3 million scholarly articles spanning disciplines
like physics, mathematics, computer science, and statistics.

  * •

TOEFL11 [89]: A publicly accessible corpus featuring works of non-native
English writers from the TOEFL test, it encompasses 1.1k essay samples across
11 languages: Arabic, Chinese, French, German, Hindi, Italian, Japanese,
Korean, Spanish, Telugu, and Turkish. These essays are uniformly distributed
over eight writing prompts and three score levels (low/medium/high).

##### Story Generation

LLMs excel in the domain of story generation, with users frequently utilizing
story titles and writing prompts to guide the models in their creative
endeavors.

  * •

WritingPrompts [47]: This dataset comprises 300k human-written stories paired
with writing prompts. The data was sourced from Reddit’s writing prompts
forum, a vibrant online community where members inspire one another by posting
story ideas or prompts. Stories in this dataset are restricted in length,
ranging between 30 to 1k words, with no words repeated more than 10 times.

##### News Article Writing

The task of news article writing can be approached through article summary
datasets. LLMs can be instructed either to generate abstracts from the primary
text or to generate articles based on provided abstracts. Nonetheless, given
the resource constraints, researchers often employ LLMs to generate such
datasets by directly reinterpreting or augmenting the existing abstracts and
articles.

  * •

Extreme Summarization (XSum) [75]: This dataset contains BBC articles
accompanied by concise one-sentence abstracts. It encompasses a total of 225k
samples from 2010 to 2017, spanning various domains such as News, Politics,
Sports, Weather, Business, Technology, Science, Health, Family, Education,
Entertainment, Arts, and more.

##### Web Text

Web text data primarily originates from platforms like social media,
Wikipedia, and tweets. For web text generation, a common approach is to
provide the LLMs with an opening sentence and allow them to continue the
narrative. Alternatively, LLMs can be instructed to generate content based on
a webpage title.

  * •

Wiki-40B [77]: Initially conceived as a multilingual benchmark for language
model training, this dataset comprises text from approximately 19.5 million
Wikipedia pages across 40 languages, aggregating to about 40 billion
characters. The content has been meticulously cleaned to maintain quality.

  * •

Avax Tweets Dataset [80]: Designed to examine anti-vaccine misinformation on
social media, this dataset was acquired via the Twitter API. It features a
streaming dataset centered on keywords with over 1.8 million tweets,
complemented by a historical account-level dataset containing more than 135
million tweets.

  * •

Climate Change Tweets Ids [81]: This dataset houses the tweet IDs for 39.6
million tweets related to climate change. These tweets were sourced from the
Twitter API between 21 September 2017 and 17 May 2019 using the Social Feed
Manager, based on specific search keywords.

  * •

WebText [87]: Originally utilized to investigate the learning potential of LMs
or LLMs, this dataset encompasses 45 million web pages. Prioritizing content
quality, the dataset exclusively includes web pages curated or filtered by
humans, while deliberately excluding common sources from other datasets, such
as Wikipedia.

##### Opinion Statements

Opinion statement datasets are instrumental in assessing the disparity in
subjective expressions between LLM-generated and human-written texts.

  * •

Yelp Reviews Dataset [82]: Sourced from the 2015 Yelp Dataset Challenge, this
dataset was primarily used for classification tasks such as predicting user
ratings based on reviews and determining polarity labels. It comprises 1.5
million review text samples.

  * •

r/ChangeMyView (CMV) Reddit
Subcommunity:111111https://www.reddit.com/r/changemyview/ Often referred to as
“Change My View (CMV)”, this subreddit offers a platform for users to debate a
spectrum of topics, ranging from politics and media to popular culture, often
presenting contrasting viewpoints.

  * •

IMDB Dataset:121212https://huggingface.co/datasets/imdb Serving as an
expansive film review dataset for binary sentiment classification, it exceeds
prior benchmark datasets in volume, encompassing 25k training and 25k test
samples.

##### Comprehension and Reasoning

Datasets geared towards comprehension and generation typically provide
consistent contextual materials, guiding LLMs in the regeneration or
continuation of text.

  * •

Stanford Question and Answer Dataset (SQuAD) [76]: This reading comprehension
dataset features 100k Q&A pairs, encompassing subjects from musical
celebrities to abstract notions. It draws samples from the top 10k English
Wikipedia articles sourced via PageRank. From this collection, 536 articles
were randomly selected, excluding passages shorter than 500 words. Crowdsource
contributors frame the questions based on these experts, while additional
personnel provide the answers.

  * •

SciGen [86]: This task centers on reasoning from perceptual data to generate
text. It comprises tables from scientific articles alongside their
descriptions. The entire dataset is sourced from the “Computer Science”
section in the arXiv website, holding up to 17.5k samples from “Computation
and Language” and another 35.5k from domains like “Machine Learning”.
Additionally, the dataset facilitates the evaluation of generative models’
arithmetic reasoning capabilities using intricate input formats, such as
scientific tables.

  * •

ROCStories Corpora (ROC) [84]: Aimed at natural language understanding, this
dataset tasks systems with determining the apt conclusion to a four-sentence
narrative. It is a curated collection of 50k five-sentence stories reflecting
everyday experiences. Beyond its primary purpose, it can also support tasks
like story generation.

  * •

HellaSwag [85]: Focused on common-sense reasoning, this dataset encompasses
approximately 70k questions. Utilizing Adversarial Filtering (AF), the dataset
crafts misleading and intricate false answers for multiple-choice settings,
where the objective is to pinpoint the correct answer in context.

###  IV-B Challenges

In light of our extensive experience in the area, a notable deficiency
persists in the realm of robust benchmark datasets tailored for LLMs. Despite
commendable advancements, current efforts remain insufficient. A noticeable
trend among researchers is the tendency to utilize datasets originally
designed for other tasks as human-written texts, and produce LLM-generated
texts base on them for training detectors. This approach arises from the
limitations of existing benchmark datasets in comprehensively addressing
diverse research perspectives. As a result, we aim to outline the prominent
limitations and challenges associated with the current benchmark datasets in
this article.

####  IV-B1 Comprehensiveness

Before gaining trust, a reliable detector demands multifaceted testing. The
current benchmarks are somewhat limited, providing only superficial challenges
and thereby not facilitating a holistic evaluation of detectors. We highlight
five crucial dimensions that are essential for the development of more robust
benchmark datasets for LLM-generated text detection task. These dimensions
include the incorporation of multiple types of attack, diverse domains, varied
tasks, a spectrum of models, and the inclusion of multiple languages.

##### Multiple Types of Attack

are instrumental in ascertaining the efficacy of detection methodologies. In
practical environments, LLM-generated text detectors often encounter texts
that are generated using a wide range of attack mechanisms, which differ from
texts generated through simple prompts. For instance, the prompt attack
elucidated in Section 4 impels the model to yield superior-quality text,
leveraging intricate and sophisticated prompts. Integrating such texts into
prevailing datasets is imperative. This concern is also echoed in the
limitations outlined for the Human ChatGPT Comparison Corpus (HC3).

##### Multi-domains and multi-tasks

configurations are pivotal in assessing a detector’s performance across
diverse real-world domains and LLM applications. These dimensions bear
significant implications for a detector’s robustness, usability, and
credibility. For instance, in scholarly contexts, a proficient detector should
consistently excel across all fields. In everyday scenarios, it should adeptly
identify LLM-generated text spanning academic compositions, news articles,
arithmetic reasoning, and Q&A sessions. While numerous existing studies
prudently incorporate these considerations, we advocate for the proliferation
of superior-quality datasets.

##### Multiple LLMs

The ongoing research momentum in LLMs has ushered in formidable counterparts
like LLaMa [91], PaLM [92], and
Claude-2,131313https://www.anthropic.com/index/claude-2 rivaling ChatGPT’s
prowess. As the spotlight remains on ChatGPT, it is essential to concurrently
address potential risks emanating from other emerging LLMs.

##### Multilingual

considerations demand increased attention. We strongly encourage researchers
to spearhead the creation of multilingual datasets to facilitate the
evaluation of text detectors generated by LLMs across different languages. The
utilization of pre-trained models may uncover instances where certain
detectors struggle with underrepresented languages, while LLMs could exhibit
more noticeable inconsistencies. This dimension presents a rich avenue for
exploration and discourse.

####  IV-B2 Temporal

It is discernible that certain contemporary studies persistently employed
seminal but somewhat antiquated benchmark datasets, which had significantly
shaped prior GPT-generated text and fake news detection endeavors. However,
these datasets predominantly originate from backward LLMs, implying that
validated methodologies might not invariably align with current real-world
dynamics. We emphasize the significance of utilizing datasets formulated with
modern LLMs, while also urging benchmark dataset developers to regularly
update their contributions to reflect the rapid evolution of the field.

##  V Advances in Detector Research

Figure 4: Classification of LLM-generated text detectors and corresponding
diagrams. We classify the detector works into watermarking technology, zero-
shot detectors, fine-turning LM detectors, adversarial learning methods, LLMs
as detectors, and Human-assisted Methods.  TABLE VII: Overview of detectors
research for LLM-generated text detection.

Detector Type |  Releated research |  System |  Metrics |  Attacks |  Cross-domain |  Cross-model |  Cross-lingual
---|---|---|---|---|---|---|---
Watermark Technology |  [93] |  - |  FPR, TNR, TPR, FNR, AUROC |  ✓ |  $\times$ |  $\times$ |  $\times$
Watermark Technology |  [94] |  - |  AUROC |  ✓ |  ✓ |  ✓ |  $\times$
Watermark Technology |  [95] |  - |  FPR, FNR, F1-Score |  $\times$ |  $\times$ |  ✓ |  $\times$
Watermark Technology |  [96] |  DWR |  - |  - |  - |  - |  -
Watermark Technology |  [97] |  Backdoor Watermark |  - |  - |  - |  - |  -
Zero-Shot |  [98] |  DetectGPT |  AUROC |  $\times$ |  ✓ |  ✓ |  ✓
Zero-Shot |  [99] |  Fast-DetectGPT |  AUROC |  ✓ |  ✓ |  ✓ |  $\times$
Zero-Shot |  [100] |  - |  AUROC |  $\times$ |  ✓ |  ✓ |  $\times$
Zero-Shot |  [101] |  Log-Likelihood Log-Rank Ratio (LRR), Normalized Log-Rank Perturbation (NPR) |  AUROC |  $\times$ |  ✓ |  ✓ |  $\times$
Zero-Shot |  [102] |  Intrinsic Dimension (PHD) |  FPR |  ✓ |  ✓ |  ✓ |  ✓
Zero-Shot |  [103] |  GLTR |  AUROC |  $\times$ |  ✓ |  ✓ |  $\times$
Zero-Shot |  [104] |  Context-Aware Perplexity |  AUROC, F1-Score |  $\times$ |  ✓ |  $\times$ |  $\times$
Zero-Shot |  [100] |  - |  AUROC |  $\times$ |  ✓ |  ✓ |  $\times$
FT- LM |  [40] |  Chatgpt-Detector-Roberta |  F1-score |  $\times$ |  ✓ |  $\times$ |  ✓
FT- LM |  [105] |  RoBERTa-MPU |  F1-Score |  $\times$ |  ✓ |  ✓ |  $\times$
FT- LM |  [106] |  RoBERTa_FT_Stylo |  Accuracy |  $\times$ |  ✓ |  ✓ |  $\times$
FT- LM |  [68] |  ROBERTA_FT |  Precision, Recall, F1-Score, Accuracy |  $\times$ |  $\times$ |  ✓ |  $\times$
FT- LM |  [107] |  - |  Accuracy |  ✓ |  $\times$ |  ✓ |  ✓
FT- LM |  [67] |  GROVER |  Paired Accuracy, Unpaired Accurac |  $\times$ |  $\times$ |  ✓ |  $\times$
FT- LM |  [73] |  CheckGPT |  Accuracy |  ✓ |  ✓ |  ✓ |  $\times$
FT- LM |  [108] |  RoBERTa-Sentinel, T5-Sentinel |  FPR, FNR, F1-Score |  $\times$ |  $\times$ |  $\times$ |  $\times$
FT- LM |  [71] |  - |  AUROC, AvgRec, HumanRec, LLMRec |  $\times$ |  ✓ |  ✓ |  $\times$
FT- LM |  [109] |  FAST |  Paired Accuracy, Unpaired Accuracy |  ✓ |  ✓ |  ✓ |  $\times$
FT- LM |  [110] |  CoCo |  Accuracy, F1-Score |  ✓ |  ✓ |  ✓ |  $\times$
FT- LM |  [70] |  ArguGPT |  Accuracy |  $\times$ |  $\times$ |  ✓ |  $\times$
FT- LM |  [111] |  ConDA |  F1-Score, AUROC |  $\times$ |  ✓ |  ✓ |  $\times$
Adversarial Learning |  [112] |  - |  AUROC |  ✓ |  ✓ |  ✓ |  $\times$
Adversarial Learning |  [113] |  Roberta-HPPT |  Accuracy, AUROC |  ✓ |  $\times$ |  $\times$ |  $\times$
Adversarial Learning |  [114] |  RADAR |  AUROC |  ✓ |  ✓ |  ✓ |  $\times$
Adversarial Learning |  [115] |  OUTFOX |  AUROC, F1-Score, AvgRec, HumanRec, MachineRec |  ✓ |  $\times$ |  ✓ |  $\times$
LLMs as Detector |  [116] |  - |  Accuracy |  $\times$ |  ✓ |  ✓ |  $\times$
LLMs as Detector |  [117] |  - |  Accuracy, Precision, F1-score |  ✓ |  ✓ |  $\times$ |  $\times$

In this section, we present different detector designs and detection
algorithms, including watermarking technology, zero-shot detectors, fine-
turning LMs detectors, adversarial learning methods, LLMs as detectors, and
human-assisted methods. We focus on the most recently proposed methods and
divide our discussion according to their underlying principles (see Figure 4).

###  V-A Watermarking Technology

Originally deployed within the realm of computer vision for the development of
generative models, watermarking techniques have been integral to the detection
of AI-generated images, serving as protective measures for intellectual and
property rights in the visual arts. With the advent and subsequent
proliferation of LLMs, the application of watermarking technology has expanded
to encompass the identification of text generated by these models.
Watermarking techniques not only protect substantial models from unauthorized
acquisition, such as through sequence distillation but also mitigate the risks
associated with replication and misuse of LLM-generated text.

##### Statistical Methods

Methods utilizing statistical analysis for detecting LLM-generated text
primarily concentrate on identifying discrepancies in output markup and
logistic-statistical distributions between watermarked and non-watermarked
text. Kirchenbauer et al. [93] pioneered the introduction of a watermarking
framework designed explicitly for proprietary LLMs. This framework is
characterized by its embedding of watermarks in a manner that minimally
impacts the quality of the text. Moreover, it facilitates the detection
process through efficient, open-source algorithms, thereby obviating the
necessity for access to the APIs or parameters of the LLMs. This approach
selects a random set of “green” tokens before generating words, meaning the
other tokens are “red” tokens, and then gently encourages the model outputs to
pick tokens from the “green” tokens set during sampling. At the same time, a
watermark detection method based on interpretable $p$-values is constructed.
In the detection stage, red tokens and green tokens in the text are combined
to perform statistics and calculate the significant amount of $p$ to determine
the watermark. A recent work [94] investigated the resilience of watermarked
text against manual rewriting, rewriting using non-watermarked LLMs, or
integration into extensive handwritten documents. This study also introduced a
window test, dubbed “WinMax”, which aimed to assess the effectiveness of
accurately detecting watermarked regions within extensive documents.
Additionally, this method serves as a “plug-in” approach for formulating
detection assumptions, allowing for selective utilization either independently
or in combination with the original detection strategy, without requiring
modifications to the generation scheme.

##### Secret Key-Based Watermarking Technology

Secret key-based watermarking technology encompasses the utilization of unique
keys (or key pairs) to insert specific signals into generated text or decoded
probability vectors. Zhao et al. [96] introduced the Distillation-Resistant
Watermarking (DRW) method, where a watermark is embedded into the predicted
probability vectors produced by the model. This embedding corresponds with a
secret key, serving the dual purpose of safeguarding the model and enabling
the detection of key information by probing a questionable model. The latter
aids in determining whether the dubious model was distilled from the protected
one. The employment of the DRW method effectively shields NLP models from
unauthorized distillation, concurrently preserving their accuracy and
integrity. As such, DRW offers a potent mechanism for defending the
intellectual property embodied in meticulously trained NLP models against
potential theft and misuse. Furthermore, Liu et al. [95] unveiled the
inaugural private watermarking algorithm designed for the generation and
detection of watermarks, deploying two distinct neural networks for each
phase. This approach innovatively extends existing text watermarking
algorithms by avoiding the use of identical keys at both stages. Moreover, it
shares certain parameters between the watermark generation and detection
networks, which not only enhances the efficiency and accuracy of the detection
network, but also exerts minimal impact on the speed of both generation and
detection processes.

##### Backdoor Watermark Technology

The backdoor technique employs a form of watermarking that becomes active when
specific triggers are activated, typically in a black-box setting [97]. This
approach serves to protect models from unauthorized fine-tuning and usage
beyond licensed terms by incorporating a backdoor within the system. This
backdoor is activated by specific inputs delineated by the owner and is
embedded during the base and training phases of models within a multi-task
learning framework. Such an approach yields watermarks that are challenging to
eliminate, even when the model undergoes fine-tuning across various downstream
tasks. Nonetheless, subsequent research has demonstrated the vulnerability of
this technique, revealing that it can be compromised with relative ease. Lucas
and Havens [118] elucidated that triggers composed of common words randomly
assembled are substantially easier to identify than those formulated with
unique, rare tokens. Moreover, they illustrated that access to the model
weights is the sole requirement for the detection of backdoor watermarks.

###  V-B Zero-shot Methods

Within the zero-shot setup, this subsection presents methods for proficiently
identifying text generated by LLMs using detectors, without the need for
additional training through supervised signals. This approach assumes access
to LLMs, where their inputs are evaluated based on distinctive features and
statistics. The inception of zero-shot detection research can be traced back
to the pioneering work of Corston-Oliver et al. [119]. In this foundational
study, the authors utilized linguistic features, such as the branching
properties observed in grammatical analyses of text, function word density,
and constituent length, to determine whether a given text was generated by a
machine translation model. These features served as key indicators in
distinguishing machine-generated text from human-generated text. Another
notable method, dedicated to achieving similar detection objectives, employs
frequency statistics. For instance, Kalinichenko et al. [120] utilized the
frequency statistics associated with word pairs present in the text as a
mechanism to ascertain whether the text had been autonomously generated by a
generative system. Furthermore, the approach adopted by Baayen [121] is
grounded in the distributional features characteristic of words. Progressing
in this line of inquiry, Arase and Zhou [122] later contributed by developing
a detection technique that captures the “phrase salad” phenomenon within
sentences.

The method employing perplexity, grounded in traditional $n$-gram LMs,
evaluates the proficiency of LMs at predicting text [123]. More recent work,
such as HowkGPT [104], discerns LLM-generated text, specifically homework
assignments, by calculating and comparing perplexity scores derived from
student-written and ChatGPT-generated text. Through this comparison,
thresholds are established to identify the origin of submitted assignments
accurately. Moreover, the widely recognized GPTZero141414https://gptzero.me/
estimates the likelihood of a review text being generated by LLMs. This
estimation is based on a meticulous examination of the text’s perplexity and
burstiness metrics.

Entropy represents another early zero-shot method utilized for evaluating LLM-
generated text. Lavergne et al. [124] employed Kullback-Leibler (KL)
divergence to assign scores to n-grams, taking into consideration the semantic
relationships between their first and last words. This approach identifies
n-grams with significant dependencies between the initial and terminal words,
thereby facilitating the detection of spurious content and enhancing the
overall performance of the detection process.

With the expansion of language modeling, methods employing Log Rank
information have garnered increasing attention. The underlying rationale of
these methods is to engage LLMs for the analysis of word rankings therein.
Subsequently, these methods assess the probability of a text being LLM-
generated by comparing the distribution of word usage in the scrutinized text
with that within the models. One noteworthy method based on this intuitive
approach is GLTR [103], which is designed as a visual forensic tool to
facilitate comparative judgment. The tool divides different marking colors
according to the sampling frequency level of the token, and highlights the
proportion of words that LLMs tend to use in the analyzed text by marking
different colors for the tokens that provide the text. Building upon the
foundation laid by GLTR, DetectLLM introduces the computation of the Log-
Likelihood Log-Rank Ratio (LRR) as a tool for text detection. This innovative
approach enhances performance by effectively complementing Log Likelihood
assessments with Log Rank analysis.

##### DetectGPT

The method proposed by DetectGPT [98] centers on the detection of LLM-
generated text through the analysis of the structural attributes inherent to
the probability functions of LLMs. DetectGPT asserts that text derived from
LLMs frequently inhabits the negative curvature region within the model’s log-
probability function. This observation led to the hypothesis that text
generated by LLMs would invariably be located at the local maximum of the log-
probability function of the source LLMs. With this foundational assumption, it
becomes feasible to discern whether a text segment has been generated by a
specific LLM through the calculation and evaluation of log-probabilities
derived from the model of interest. This process involves the random
perturbation of vetted text from a generic pre-trained mask-filling model
(such as T5), with the caveat that semantic similarity must be preserved.
While DetectGPT is a pioneering method within the realm of zero-shot
approaches, occasionally outperforming supervised methods, its robustness
leaves room for improvement. The detector’s performance may witness
substantial degradation if the deployed mask-filling model inadequately
represents the space of meaningful rewrites. Additionally, the time overhead
associated with DetectGPT is considerable, as the evaluation of a single
candidate necessitates scoring over a hundred of its perturbations using the
source LLM. In response to this challenge, Deng et al. [100] proposed a method
that uses a Bayesian surrogate model to select a small number of typical
samples for scoring. By interpolating the scores of typical samples to other
samples to improve query efficiency, the overhead is reduced by half while
maintaining performance. Bao et al. [99] reported a method that replaces the
perturbation step of DetectGPT with a more efficient sampling step,
significantly improving the detection accuracy by about 75% and increasing the
detection speed by 340 times. DetectLLM [101], another recent contribution,
parallels the conceptual framework of DetectGPT. It employs normalized
perturbed log-rank for text detection generated by LLMs, asserting a lower
susceptibility to the perturbation model and the number of perturbations
compared to DetectGPT.

##### Intrinsic Dimension Estimation

The study conducted by Tulchinskii et al. [102] posited the invariant nature
of the competencies exhibited by both human and LLMs within their respective
textual domains. The proposed approach involves the construction of detectors
utilizing the intrinsic dimensions of manifolds underpinning the embedding set
of given text samples. More specifically, the methodology entails computing
the average intrinsic dimensionality values for both sets of fluent human-
written texts and LLM-generated texts in the target natural language. The
ensuing statistical separation between these two sets facilitates the
establishment of a separation threshold for the target language, thereby
enabling the detection of text generated by LLMs. It is imperative to
acknowledge the robustness of this approach across various scenarios,
including cross-domain challenges, model shifts, and adversarial attacks.
However, its reliability falters when confronted with suboptimal or high-
temperature generators.

###  V-C Fine-tuning LMs Methods

Within this subsection, we explore methods that involve fine-tuning
Transformer-based LMs to discriminate between input texts that are generated
by LLMs and those that are not. This approach requires paired samples for the
facilitation of supervised training processes. According to [125], pre-trained
LMs have proven to be powerful in natural language understanding, which is
crucial for enhancing various tasks in NLP, with text categorization being
particularly noteworthy. Esteemed pre-trained models, such as BERT [126],
Roberta [127], and XLNet [128], have exhibited superior performance relative
to their counterparts in traditional statistical machine learning and deep
learning when applied to the text categorization tasks within the GLUE
benchmark [129].

Moreover, there is an extensive body of prior work [130, 131, 107, 71] that
has meticulously examined the capabilities of fine-tuned LMs in detecting LLM-
generated text. Notably, studies conducted in 2019 have acknowledged fine-
tuned LMs, with Roberta [127] being especially prominent, as being amongst the
most formidable detectors of LLM-generated text. In the following discourse,
we will introduce recent scholarly contributions in this vein, providing an
updated review and summary of the methods deployed.

##### In-domain Fine-tuning is All You Need

Fine-tuning Roberta provides a robust baseline for detecting text generated by
LLMs. Fagni et al. [68] observed that fine-tuning Roberta led to optimal
classification outcomes in various encoding configurations [132], with the
subsequent OpenAI detector [87] also adopting a Roberta fine-tuning approach.
Recent works [40, 70, 73, 108] further corroborated the superior performance
of fine-tuned members of the BERT family, such as RoBERTa, in identifying LLM-
generated text. On average, these fine-tuned models yielded a 95% accuracy
rate within their respective domains, outperforming zero-shot and watermarking
methods, and exhibiting a modicum of resilience to various attack techniques
within in-domain settings. Nevertheless, like their counterparts, these
encoder-based fine-tuning approaches lack robustness [130, 131, 107, 71], as
they tend to overfit to their training data or the source model’s training
distribution, resulting in a decline in performance when faced with cross-
domain or unseen data. Additionally, fine-tuning LMs classifiers is limited in
facing data generated by different models [133].

##### Enhanced Fine-Tuned LMs

Data scarcity has propelled the application of contrastive learning [134, 135,
136] to LM-based classifiers, with the core of this approach being self-
supervised learning. This strategy minimizes the distance between the anchor
and positive samples while maximizing the distance to negative samples through
spatial transformations. An enhanced contrastive loss, proposed by Liu et al.
[110], assigns greater weight to hard-negative samples, thereby optimizing
model utility and stimulation to bolster performance in low-resource contexts.
This method thoroughly accounts for linguistic characteristics and sentence
structures, representing text as a coherence graph to encapsulate its inherent
entity consistency. Research findings affirm the potency of incorporating
information fact structures to refine LM-based detectors’ efficacy, a
conclusion echoed by Zhong et al. [109]. Bhattacharjee et al. [111] proposed
ConDA, a contrastive domain adaptation framework, which combines standard
domain adaptation technology with the representation capabilities of
contrastive learning, greatly improving the model’s defense capabilities
against unknown models. Furthermore, the integration of adversarial learning
techniques can augment the robustness of detectors that employ straightforward
fine-tuning strategies; a more in-depth discussion of this can be found in the
section on _Adversarial Learning Methods_ (see Section 5.4).

###  V-D Adversarial Learning Methods

In light of the vulnerability of detectors to different attacks and robustness
issues, a significant body of scholarly research has been dedicated to
utilizing adversarial learning as a mitigation strategy. Predominantly,
adversarial learning methods bear relevance to fine-tuning LMs methods.
Noteworthy recent work by Koike et al. [115] revealed that it is feasible to
train adversarially without fine-tuning the model, with context serving as a
guide for the parameter-frozen model. Recognizing the pivotal role of
adversarial learning within this domain, we have elected to allocate a
discrete section for a detailed discussion on the subject. For analytical
clarity, we have compartmentalized the studies into two categories: Sample
Enhancement Based Adversarial Training and Two-Player Games.

##### Sample Enhancement Based Adversarial Training

A prominent approach within this category centers on deploying adversarial
attacks predicated on sample augmentation, with the overarching aim of
crafting deceptive inputs to thereby enhance the model’s competency in
addressing a broader array of scenarios that bear deception potential.
Specifically, this method emphasizes the importance of sample augmentation and
achieves it by injecting predetermined adversarial attacks. This augmentation
process is integral to fortifying the detector’s robustness by furnishing it
with an expanded pool of adversarial samples. Section 7.2 of the article
outlines various potential attack mechanisms, including paraphrase attacks,
adversarial attacks, and prompt attacks. Yang et al. [113], Shi et al. [112],
He et al. [69] conducted the adversarial data augmentation process on LLM-
generated text, the findings of which indicated that models trained on
meticulously augmented data exhibited commendable robustness against potential
attacks.

##### Two-Player Games

This subsection casts the spotlight on innovative methods within the realm of
Two-Player Games. These methods, fundamentally aligned with the principles
underpinning Generative Adversarial Networks [137] and Break-It-Fix-It
strategies [138], typically involve the configuration of an attack model
alongside a detection model, with the iterative confrontation between the two
culminating in enhanced detection capabilities. Hu et al. [114] introduced a
novel framework, RADAR, envisaged for the concurrent training of robust
detectors through adversarial learning. This framework facilitates interaction
between a paraphrasing model, responsible for generating realistic content
that evades detection, and a detector whose goal is to enhance its capability
to identify text produced by LLMs. The innovative RADAR framework
incrementally refines the paraphrase model, drawing on feedback garnered from
the detector and employing PPO [139]. Despite its commendable performance in
countering paraphrase attacks, the study by Hu et al. [114] did not provide a
comprehensive analysis of RADAR’s defense mechanism against other attack
modalities. In a parallel vein, Koike et al. [115] proposed a training
methodology for detectors, predicated on a continual interaction between an
attacker and a detector. Distinct from RADAR, OUTFOX allocates greater
emphasis on the likelihood of detectors employing ICL [140] for attacker
identification. Specifically, the attacker in the OUTFOX framework utilizes
predicted labels from the detector as ICL exemplars to generate text that
poses detection challenges. Conversely, the detector uses the content
generated adversarially as ICL exemplars to enhance its detection capabilities
against formidable attackers. This reciprocal consideration of each other’s
outputs fosters improved robustness in detectors for text generated by LLMs.
Empirical evidence attests to the superior performance of the OUTFOX method
relative to preceding statistical methods and those based on RoBERTa,
particularly in responding to attacks from TF-IDF and DIPPER [141].

###  V-E LLMs as Detectors

##### Questionable Reliability of Using LLMs

Several works have examined the feasibility of utilizing LLMs as detectors to
discern text generated by either themselves or other LLMs. This approach was
first broached by Zellers et al. [67], wherein the text generation model
Grover [67] was noted to produce disinformation that was remarkably deceptive
due to its inherently controllable nature. Subsequent exploratory analyses by
Zellers et al. [67] engaging various architectural models like GPT-2 [87] and
BERT [142] revealed that Grover’s most effective countermeasure was Grover
itself, boasting an accuracy rate of 92%, while other detector types
experienced a decline in accuracy to approximately 70% as Grover’s size
increased. A recent reevaluation conducted by Bhattacharjee and Liu [116] on
more recent LLMs like ChatGPT and GPT-4 yielded that neither could reliably
identify text generated by various LLMs. During the observations, it was noted
that ChatGPT and GPT-4 exhibited contrasting tendencies. ChatGPT tended
classify text generated by LLMs as if it were written by humans, with a
misclassification probability of about 50%. While GPT-4 leaned towards
labeling human-written text as if it were generated by LLMs, and about 95% of
human-written texts are misclassified as LLM-generated texts. ArguGPT [70]
further attested to the lackluster performance of GPT-3.5-Turbo in detecting
text generated by LLMs, with accuracy rates languishing below 50% across zero-
shot, one-shot, and two-shot settings. These findings collectively demonstrate
the diminishing reliability of employing LLMs for direct self-generated text
detection, particularly when compared to statistical and neural network
methods. This is particularly evident in light of the increasing complexity of
LLMs.

##### ICL: A Powerful Technique for LLM-Based Detection

Despite the unreliability issues associated with using LLMs for direct
detection of LLM-generated text, recent empirical investigations highlight the
potential efficacy of ICL in augmenting LLMs’ detection capabilities. ICL, a
specialized form of cue engineering, integrates examples into cues provided to
the model, thereby facilitating the learning of new tasks by LLMs. Through
ICL, existing LLMs can adeptly tackle novel tasks without necessitating
additional fine-tuning. The OUTFOX Detector [115] employs an ICL approach,
continuously supplying example samples to LLMs for text generation detection
tasks. The experimental findings demonstrate that the ICL strategy outperforms
both traditional zero-shot methods and RoBERTa-based detectors.

##### LLMs’ Paternity Testing

Given the formidable comprehension and generation capacities inherent to LLMs,
the prospect of leveraging these for self-validation without direct engagement
in detection tasks emerges as a compelling alternative. Yu et al. [117]
introduced an innovative text detection mechanism for LLMs, capitalizing on
the similarity of regenerated text. In this method, LLMs infer the source
problem of the text under examination and proceed to regenerate the text based
on this inferred problem. The detection process then hinges on the similarity
between the examined text and the regenerated text, providing a basis to
ascertain whether the text under scrutiny was generated by an LLM. Notably,
this method not only exhibits superior detection performance and robustness
but also demonstrates resilience to paraphrase-like attacks, such as re-
translation and polishing, with an accuracy degradation rate of approximately
half that of RoBERTa-based methods. However, the approach is not without its
challenges, including a less-than-optimal user experience and substantial
overheads associated with accessing LLMs.

###  V-F Human-assisted Methods

In this subsection, we delve into human-assisted methods for detecting text
generated by LLMs. These methods leverage human prior knowledge and analytical
skills, providing notable interpretability and credibility in the detection
process.

##### Intuitive Indicators

Several studies have delved into the disparities between human and machine
classification capabilities. Human classification primarily depends on visual
observation to discern features indicative of text generation by LLMs. Uchendu
et al. [143] noted that a lack of coherence and consistency in LLM-generated
text serves as a strong indicator of falsified content. Texts produced by LLMs
often exhibit semantic inconsistencies and logical errors. Furthermore, Dugan
et al. [144] identified that the human discernment of LLM-generated text
varies across different domains. For instance, LLMs tend to generate more
“generic” text in the news domain, whereas, in story domains, the text might
be more “irrelevant”. [144] also noted that relying solely on grammatical
errors as a detection strategy is unreliable. In addition, LLMs frequently
commit factual and common-sense reasoning errors, which, while often
overlooked by neural network-based detectors, are easily noticed by humans
[37].

##### Imperceptible Features

Ippolito et al. [32] suggested that text perceived as high quality by humans
tends to be more easily recognizable by detectors. This observation implies
that some features, imperceptible to humans, can be efficiently captured by
detection algorithms. While humans are adept at identifying errors in many
LLM-generated texts, unseen features also significantly inform human decision-
making. In contrast, statistical thresholds commonly employed in zero-shot
Detector research to distinguish LLM-generated text can be manipulated.
However, humans typically possess the ability to detect such manipulations
through various metrics, GLTR [103] pioneered this approach, serving as a
visual forensic tool to assist human vetting processes, while also providing
rich interpretations easily understandable by non-experts[145].

##### Enhancing Human Detection Capabilities

Recent studies [32] indicated that human evaluators might not be as proficient
as detection algorithms in recognizing LLM-generated text across various
settings. However, exposing evaluators to examples before evaluation enhances
their detection capabilities, especially with longer samples. The platform
RoFT [146] allows users to engage with LLM-generated text, shedding light on
human perception of such text. Although revealing true boundaries post-
annotation did not lead to an immediate improvement in annotator accuracy, it
is worth noting that with proper incentives and motivations, annotators can
indeed improve their performance over time [144]. The SCARECROW framework [30]
facilitates the annotation and review of LLM-generated text, outlining ten
error types to guide users. The result from SCARECROW reports Manual
annotation outperformed detection models on half of the error types,
suggesting potential in developing efficient annotation systems despite the
associated human overhead.

##### Mixed Detection: Understanding and Explanation

Weng et al. [147] introduced a prototype amalgamating human expertise and
machine intelligence for visual analysis, premised on the belief that human
judgment is the benchmark. Initially, experts label text based on their prior
knowledge, elucidating the distinctions between human and LLM-generated text.
Subsequently, machine-learning models are trained and iteratively refined
based on labeled data. Finally, the most intuitive detector is selected
through visual statistical analysis, serving the detection purpose. This
granular analysis approach not only bolsters experts’ trust in decision-making
models but also fosters learning from the models’ behavior to efficiently
identify LLM-generated samples.

##  VI Evaluation Metrics

Evaluation metrics, indispensable for the assessment of model performance
within any NLP task, warrant meticulous consideration. In this section, we
enumerate and discuss metrics conventionally utilized in the tasks of LLM-
generated text detection. These metrics include Accuracy, Paired Accuracy,
Unpaired Accuracy, Recall, Human-written Recall (HumanRec), LLM-generated
Recall (LLMRec), Average Recall (AvgRec), F1 Score, and Area Under the
Receiver Operating Characteristic Curve (AUROC). Furthermore, we discuss the
advantages and drawbacks associated with each metric to facilitate informed
metric selection for varied research scenarios in subsequent studies.

The confusion matrix can help effectively evaluate the performance of the
classification task and describes all possible results (four types in total)
of the LLM-generated text detection task:

  * •

True Positive (TP) refers to the result of the positive category (LLM-generate
text) correctly classified by the model.

  * •

True Negative (TN) refers to the result of the negative category (human-
written text) correctly classified by the model.

  * •

False Positive (FP) refers to the result of the positive category (LLM-
generate text) incorrectly classified by the model.

  * •

False Negative (FN) refers to the result of the negative category (human-
written text) incorrectly predicted by the model.

The evaluation metrics introduced below can all be described by TP, TN, FP,
and FP.

##### Accuracy

Accuracy serves as a general metric, denoting the ratio of correctly
classified texts to the total text count. While suitable for balanced
datasets, its utility diminishes for unbalanced ones due to sensitivity to
category imbalance. The metrics of Paired and Unpaired Accuracy have also
found application in [67, 109] to evaluate the detector’s ability in different
scenarios. In the unpaired setting, the discriminator must independently
classify each test sample as human or machine. In the paired setting, the
model is given two test samples with the same metadata, one real and one
generated by the large model. The discriminator must assign a higher machine
probability to articles written by large models than to articles written by
humans. These indicators are used to measure the performance of the algorithm
on data in different scenarios. Relatively speaking, the detection difficulty
of unpaired settings is higher than that of paired settings. Accuracy can be
described by the following formula:

$\displaystyle Accuracy$ $\displaystyle=\frac{\text{correctly detected
samples}}{\text{all samples}}$ (4) $\displaystyle=\frac{TP+TN}{TP+TN+FP+FN}$

##### Precision

Precision is a measure of the correctness of real predictions and refers to
the proportion of correctly detected LLM-generated samples among all detected
LLM-generated samples. This metric is very useful in situations where we are
concerned about false positives. When a sample is not LLM-generated, but is
classified as LLM-generated text, this erroneous result may reduce the user’s
impression of the model or even cause the negative impact on business.
Therefore, improving precision is also important in the LLM-generated text
detection task. This Metric can be described by the following formula:

$\displaystyle Precision$ $\displaystyle=\frac{\text{correctly detected LLM-
generated samples}}{\text{all detected LLM-generated samples}}$ (5)
$\displaystyle=\frac{TP}{TP+FP}$

##### Recall

Recall represents the proportion of actual machine-generated texts accurately
identified as such. This metric is invaluable in contexts where underreporting
must be minimized, as in instances requiring the capture of the majority of
machine-generated texts. AvgRec, the mean recall across categories, is
particularly useful for multi-category tasks requiring collective performance
assessment across categories. HumanRec and LLMRec denote the proportions of
texts accurately classified as human-written and machine-generated,
respectively, shedding light on the model’s differential performance on these
two classes. Recall, HumanRec, LLMRec, and AvgRec can be described by the
following formulas respectively:

$Recall=\frac{TP}{TP+FN}$ (6) $HumanRecall=\frac{\text{correctly detected
human-written samples}}{\text{all human-written samples}}$ (7)
$LLMRecall=\frac{\text{correctly detected LLM-generated samples}}{\text{all
LLM-generated samples}}$ (8) $AvgRecall=\frac{HumanRecall+LLMRecall}{2}$ (9)

##### False Positive Rate (FPR)

The FPR refers to the proportion of all actual human-written samples that are
incorrectly detected as LLM-generated samples. This metric can measure the
proportion of incorrect predictions made by the model in samples that are
actually written by humans. It helps to understand the false positive rate of
the model and thus has a higher sensitivity for the detection of LLM-generated
samples. This metric can be described by the following formula:

$\displaystyle FPR$ $\displaystyle=\frac{\text{incorrectly detected LLM-
generated samples}}{\text{all human-written samples}}$ (10)
$\displaystyle=\frac{FP}{FP+TP}$

##### True Negative Rate (TNR)

The TNR refers to the proportion of samples that are correctly detected as
human-written among all actual human-written samples. This metric measures how
accurately the model predicts human-written samples, but does not take into
account the FPR, where text that is actually human-written is incorrectly
detected as LLM-generated text. This metric can be described by the following
formula:

$\displaystyle TNR$ $\displaystyle=\frac{\text{correctly detected human-
written samples}}{\text{all human-written samples}}$ (11)
$\displaystyle=\frac{TN}{TN+FP}$

##### False Negative Rate (FNR)

The FNR refers to the proportion of all actual LLM-generated samples that are
incorrectly detected as human-written. This metric helps understand how
misinterpreted the model is for LLM-generated text. This metric can be
described by the following formula:

$\displaystyle FNR$ $\displaystyle=\frac{\text{incorrectly detected human-
written samples}}{\text{all LLM-generated samples}}$ (12)
$\displaystyle=\frac{FN}{FN+TP}$

#####  $F_{1}$ Score

The $F_{1}$ Score constitutes a harmonic mean of precision and recall,
integrating considerations of false positives and false negatives. It emerges
as a prudent choice when a balance between precision and recall is imperative.
The $F_{1}$ score can be calculated using the following formula:

$\displaystyle F_{1}=$ $\displaystyle
2*\frac{Precision*Recall}{Precision+Recall}$ (13)
$\displaystyle=\frac{2TP}{2TP+FP+FN}$

##### AUROC

The AUROC metric, derived from Receiver Operating Characteristic curves,
considers true and false positive rates at varying classification thresholds,
proving beneficial for evaluating classification efficacy at different
thresholds. This is particularly crucial in scenarios necessitating specific
false positive and miss rates, especially within the context of unbalanced
datasets and binary classification tasks. Given that the detection rate of
zero-shot detection methods significantly hinges on threshold values, the
AUROC metric is commonly employed to evaluate their performance across all
possible thresholds. The calculation formula of AUROC is as follows:

$AUROC=\int_{0}^{1}\frac{TP}{TP+FP}\mathrm{d}\frac{FP}{FP+TN}$ (14)

##  VII Important Issues of LLM-generated Text Detection

In this section, we discuss the main issues and limitations of contemporary
state-of-the-art techniques designed for detecting text generated by LLMs. It
is important to note that no technique has been acknowledged as infallible.
The issues elucidated herein may pertain specifically to one or multiple
classes of detectors.

###  VII-A Out of Distribution Challenges

Out-of-distribution issues significantly impede the efficacy of current
techniques dedicated to the detection of LLM-generated text. This section
elucidates the constraints of these detectors to variations in domains and
languages.

##### Cross-Domain

The dilemma of cross-domain application is a ubiquitous challenge inherent to
numerous NLP tasks. Studies conducted by Antoun et al. [107], Li et al. [71]
underscored considerable limitations in the performance of sophisticated
detectors, including but not limited to DetectGPT [98], GLTR [103], and fine-
tuned Roberta methods when applied to cross-domain data. These detectors
exhibit substantial performance degradation when confronted with out-of-
distribution data prevalent in real-world scenarios, with the efficacy of some
classifiers marginally surpassing that of random classification. This
disparity between high reported performance and actual reliability underlines
the need for critical evaluation and enhancement of existing methods.

##### Cross-lingual

The issue of cross-lingual application introduces a set of complex challenges
that hinder the global applicability of existing detector research.
Predominantly, contemporary detectors designed for LLM-generated text
primarily target monolingual applications, often neglecting to evaluate and
optimize performance across multiple languages. Wang et al. [72] and Chaka
[148] noted the lack of control observed in multilingual LLM-generated text
detectors across various languages, despite the existence of certain language
migration capabilities. We emphatically draw attention to these cross-lingual
challenges as addressing them is pivotal for enhancing the usability and
fairness of detectors for LLM-generated text. Moreover, recent research [149]
revealed a discernible decline in the performance of state-of-the-art
detectors when processing texts authored by non-native English speakers.
Although employing effective prompt strategies can alleviate this bias, it
also inadvertently allows the generated text to bypass the detectors.
Consequently, there is a risk that detectors might inadvertently penalize
writers who exhibit non-standard linguistic styles or employ limited
expressions, thereby introducing issues of discrimination within the detection
process.

###  VII-B Potential Attacks

Potential attacks are an important factor in the continued unreliability of
current LLM-generated text detectors. We present the current effective attacks
to push researchers to focus on more comprehensive defense performance
building.

##### Paraphrase Attacks

Paraphrasing attacks are one of the most effective attacks that can be fully
effective against detectors using watermarking technology as well as LM-based
detectors and zero-shot detector [150, 151]. The principle is to apply a
lightweight paraphrase model on LLMs’ outputs and change the distribution of
lexical and syntactic features of the text by paraphrasing the text to corrupt
the detector. Sadasivan et al. [150] reported on Parrot [152], a T5-based
paraphrase model and DIPPER [141], an 11B paraphrasing model that allows for
tuning paraphrase diversity and the degree of content reordering that attacks
the overall superiority of existing detection methods. While it has been shown
that introducing retrieval [141] can be an effective defense against
paraphrasing attacks, the implementation of this defense requires maintenance
by the language model API provider and remains vulnerable to recursive
paraphrasing [150].

##### Adversarial Attacks

We summarise attacks that process on textual features as adversarial attacks,
including cutoff (cropping a portion of the feature or input) [153], shuffle
(randomly disrupting the word order of the input) [154], mutation (character
and word mutation) [155], word swapping (substituting other suitable words
given the context) [156, 157, 158] and misspelling [159]. There are also
adversarial means such as TextAttack, which can build an attack from four
components: an objective function, a set of constraints, a transformation, and
a search method. Shi et al. [112] and He et al. [69] reported on the
effectiveness of the permutation approach on attack detectors. Specifically,
Shi et al. [112] replaced words with synonyms based on context, which forms an
effective attack on the fine-tuned classifier, watermarking [93], and
DetectGPT [98], reducing detector performance by more than 18%, 10%, and 25%
respectively. He et al. [69] employed probability-weighted word saliency [157]
to generate adversarial examples, which further maintains semantic similarity.
The attack degrades the performance of the fine-tuning classifier
significantly, even though the distributional features of the attack can be
learned by the fine-tuning classifier to form a strong defense.

##### Prompt Attacks

Prompt attacks are a major challenge for current LLM-generated text detection
techniques. The quality of LLM-generated text is correlated with the
complexity of the prompts that guide LLMs to generate text. As the model size
and corpus size increase, LLMs emerge with excellent ICL capabilities for more
complex text generation capabilities. The large number of efficient prompting
methods elicited, such as few shot [50], combining prompt [160], CoT [55], and
zero-shot CoT [161], etc., which can greatly improve the quality and
capability of LLMs. Existing work on LLM-generated text detectors mainly use
datasets generated with simple prompts, such as the work of Guo et al. [40],
which directly leads to the fact that detectors may have difficulty in
detecting text generated with complex prompts when faced with them. Liu et al.
[73] reported a slight decrease in the detection ability of a detector with a
fine-tuned LM when confronted with a different prompt. The Substitution-based
Contextual Example Optimisation method proposed by Lu et al. [162] applies
elaborate prompts to evade the defense of existing detection systems, leading
to a significant decrease in AUC by an average of 0.54, with a higher success
rate and better text quality than paraphrase attacks. It is worth mentioning
that both paraphrase attacks and adversarial attacks mentioned above could be
launched through prompt design [112, 115]. The threat of prompt attacks will
be further amplified in the future with the research on prompt engineering.

###  VII-C Data Ambiguity

Data ambiguity remains a challenge in the field of LLM-generated text
detection, with close ties to the inherent mechanics of the detection
technology itself. The pervasive deployment of LLMs across various domains
exacerbates this issue, rendering it increasingly challenging to discern
whether training data comprises human-written or LLM-generated text. Utilizing
LLM-generated text as training data under the misapprehension that it is
human-authored inadvertently instigates a detrimental cycle. Within this
cycle, detectors, consequently trained, demonstrate diminished efficacy in
distinguishing between human-written and LLM-generated text, thereby
undermining the foundational premises of detector research. It is imperative
to acknowledge that this quandary poses a significant, pervasive threat to all
facets of detection research, yet, to our knowledge, no existing studies
formally address this concern. An additional potential risk was articulated by
Alemohammad et al. [26], who posited that data ambiguity might precipitate the
recycling of LLM-generated data in the training processes of subsequent
models. This scenario could adversely impact the text generation quality of
these emergent LLMs, thereby destabilizing the research landscape dedicated to
the detection of LLM-generated text.

##  VIII Future Research Directions

In this section, we explore potential directions for future research aimed at
better construction of more efficient and realistically effective detectors.

###  VIII-A Building Robust Detectors with Attacks

The attack methods introduced in Potential Attacks in section 7.2, encompass
Paraphrase Attacks [150], Adversarial Attacks [69], and Prompt Attacks [162].
These methods underscore the primary challenges impeding the utility of
current detectors. While recent research, such as Yang et al. [113], has
addressed robustness against specific attacks, it often neglects potential
threats posed by other attack forms. Consequently, it is imperative to develop
and validate diverse attack types, thereby gaining insights into
vulnerabilities inherent to LLM-generated text detectors. We further advocate
for the establishment of comprehensive benchmarks to assess existing detection
strategies. Although some studies [69, 72] purport to provide such benchmarks,
the scope and diversity of the validated attacks remain limited.

###  VIII-B Enhancing the Efficacy of Zero-shot Detectors

Zero-shot methods stand out as notably stable detectors [100]. Crucially, they
offer enhanced controllability and interpretability for users [163]. Recent
research [43, 164] has elucidated distinct disparities between LLM-generated
text and human-written text, underscoring a tangible and discernible gap
between the two. This revelation has invigorated research in the domain of
LLM-generated text detection. We advocate for a proliferation of studies that
delve into the nuanced distinctions between LLM-generated texts and human-
written text, spanning from low-dimensional to high-dimensional features.
Unearthing metrics that more accurately distinguish the two can bolster the
evolution of automatic detectors and furnish more compelling justifications
for decision-making processes.

###  VIII-C Optimizing Detectors for Low-resource Environments

Many contemporary detection techniques tend to overlook the challenges faced
by resource-constrained settings, neglecting the need for resources in
developing the detector. The relative efficacy of various detectors across
different data volume settings remains inadequately explored. Concurrently,
determining the minimal resource prerequisites for different detection methods
to yield satisfactory results is imperative. Beyond examining the model’s
adaptability across distinct domains [16] and languages [72], we advocate for
investigating the defensive adaptability against varied attack strategies.
Such exploration can guide users in selecting the most beneficial approach to
establish a dependable detector under resource constraints.

###  VIII-D Constructing Detectors Amidst Data Ambiguity

A significant challenge that arises is verifying the authenticity of training
data. When aggregating textual data from sources such as blogs and web
comments, there is a potential risk of inadvertently including a substantial
amount of LLM-generated text. This incorporation can fundamentally compromise
the integrity of detector research, perpetuating a detrimental feedback loop.
We urge forthcoming detection studies to prioritize the authenticity
assessment of real-world data, anticipating this as a pressing challenge in
the future.

###  VIII-E Constructing Detectors with Factual Discrimination Capabilities

Contemporary detection methodologies have largely overlooked the capacity to
discern factual accuracy. Existing detectors primarily emphasize the
distribution of features within text generated by LLMs, while often
overlooking their potential for factual verification. A proficient detector
should possess the capability to discern the veracity or falsity of factual
claims presented in text. In the initial stages of generative modeling’s
emergence, when it had yet to pose significant societal challenges, the
emphasis was on assessing the truth or falsity of the content in machine-
generated text, with less regard for its source [165]. Constructing detectors
with factual discrimination capabilities can aid in more accurately
attributing the source of text, rather than relying solely on distributional
features, and subsequently contribute to mitigating the proliferation of
disinformation. Recent studies [166, 167] highlight the potential of LLMs to
detect factual content in texts. We recommend bolstering such endeavors
through integration with external knowledge bases [168] or search engines
[169].

###  VIII-F Multi-agent Systems for LLM-generated Text Detection

Preliminary research has corroborated the efficacy of ICL in enhancing the
performance of LLMs in the realm of text detection specific to LLM generation.
Recent studies [170, 171] demonstrate that leveraging multi-agent systems can
amplify the proficiency of LLMs by fostering collaboration and facilitating
knowledge exchange among intelligent agents. This paradigm mirrors human
collective decision-making. Uchendu et al. [143] reported underscores the
potency of human collaborative decision-making in refining the task of LLM-
generated text detection. We deem that multi-agent systems hold significant
promise. On one hand, multi-agent systems might optimize LLM performance by
leveraging collective deliberation among agents. On the other hand, they could
evaluate the detection outputs of multiple detectors in a consensus-driven
manner, yielding more compelling results.

##  IX Conclusion

With the widespread development and application of LLMs, the pervasive
presence of LLM-generated text in our daily lives has transitioned from
expectation to reality. LLM-generated text detectors play a pivotal role in
distinguishing between human-written and LLM-generated text, serving as a
crucial defense against the misuse of LLMs for generating deceptive news,
engaging in scams, or exacerbating issues such as educational inequality. In
this survey, we introduce the task of LLM-generated text detection, outline
the sources contributing to enhanced LLM-generated text capabilities, and
highlight the escalating demand for efficient detectors. We also list datasets
that are popular or promising, pointing out the challenges and requirements
associated with existing detectors. In addition, we shed light on the critical
limitations of contemporary detectors, including issues related to out-of-
distribution data, potential attacks, and data ambiguity, to direct
researchers’ attention to the focal points of the field, thereby sparking
innovative ideas and approaches. Finally, we propose potential future research
directions poised to guide the development of more powerful and effective
detection systems.

## References

  * OpenAI [2023] OpenAI, “GPT-4 technical report,” _CoRR_ , vol. abs/2303.08774, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2303.08774
  * Anthropic [2023] Anthropic, “Model card and evaluations for claude models,” 2023. [Online]. Available: https://www-files.anthropic.com/production/images/Model-Card-Claude-2.pdf
  * Chowdhery et al. [2022a] A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann _et al._ , “Palm: Scaling language modeling with pathways,” _ArXiv preprint_ , vol. abs/2204.02311, 2022. [Online]. Available: https://arxiv.org/abs/2204.02311
  * Veselovsky et al. [2023] V. Veselovsky, M. H. Ribeiro, and R. West, “Artificial artificial artificial intelligence: Crowd workers widely use large language models for text production tasks,” _ArXiv preprint_ , vol. abs/2306.07899, 2023. [Online]. Available: https://arxiv.org/abs/2306.07899
  * Murakami et al. [2023] S. Murakami, S. Hoshino, and P. Zhang, “Natural language generation for advertising: A survey,” _ArXiv preprint_ , vol. abs/2306.12719, 2023. [Online]. Available: https://arxiv.org/abs/2306.12719
  * Yanagi et al. [2020] Y. Yanagi, R. Orihara, Y. Sei, Y. Tahara, and A. Ohsuga, “Fake news detection with generated comments for news articles,” in _2020 IEEE 24th International Conference on Intelligent Engineering Systems (INES)_. IEEE, 2020, pp. 85–90.
  * Yuan et al. [2022] A. Yuan, A. Coenen, E. Reif, and D. Ippolito, “Wordcraft: story writing with large language models,” in _27th International Conference on Intelligent User Interfaces_ , 2022, pp. 841–852.
  * Becker et al. [2023] B. A. Becker, P. Denny, J. Finnie-Ansley, A. Luxton-Reilly, J. Prather, and E. A. Santos, “Programming is hard-or at least it used to be: Educational opportunities and challenges of ai code generation,” in _Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1_ , 2023, pp. 500–506.
  * Zheng et al. [2023] Q. Zheng, X. Xia, X. Zou, Y. Dong, S. Wang, Y. Xue, Z. Wang, L. Shen, A. Wang, Y. Li _et al._ , “Codegeex: A pre-trained model for code generation with multilingual evaluations on humaneval-x,” _ArXiv preprint_ , vol. abs/2303.17568, 2023. [Online]. Available: https://arxiv.org/abs/2303.17568
  * Susnjak [2022] T. Susnjak, “Chatgpt: The end of online exam integrity?” _ArXiv preprint_ , vol. abs/2212.09292, 2022. [Online]. Available: https://arxiv.org/abs/2212.09292
  * Cui et al. [2023] J. Cui, Z. Li, Y. Yan, B. Chen, and L. Yuan, “Chatlaw: Open-source legal large language model with integrated external knowledge bases,” _ArXiv preprint_ , vol. abs/2306.16092, 2023. [Online]. Available: https://arxiv.org/abs/2306.16092
  * Piccolo et al. [2023] S. R. Piccolo, P. Denny, A. Luxton-Reilly, S. Payne, and P. G. Ridge, “Many bioinformatics programming tasks can be automated with chatgpt,” _ArXiv preprint_ , vol. abs/2303.13528, 2023. [Online]. Available: https://arxiv.org/abs/2303.13528
  * Thirunavukarasu et al. [2023] A. J. Thirunavukarasu, D. S. J. Ting, K. Elangovan, L. Gutierrez, T. F. Tan, and D. S. W. Ting, “Large language models in medicine,” _Nature medicine_ , pp. 1–11, 2023.
  * Ji et al. [2023] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung, “Survey of hallucination in natural language generation,” _ACM Computing Surveys_ , vol. 55, no. 12, pp. 1–38, 2023.
  * Christian [2023] J. Christian, “Cnet secretly used ai on articles that didn’t disclose that fact, staff say,” _Futurusm, January_ , 2023.
  * Rodriguez et al. [2022] J. Rodriguez, T. Hay, D. Gros, Z. Shamsi, and R. Srinivasan, “Cross-domain detection of GPT-2-generated technical text,” in _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. Association for Computational Linguistics, 2022, pp. 1213–1233. [Online]. Available: https://aclanthology.org/2022.naacl-main.88
  * Aliman and Kester [2021] N. M. Aliman and L. Kester, “Epistemic defenses against scientific and empirical adversarial ai attacks,” in _CEUR Workshop Proceedings_ , vol. 2916. CEUR WS, 2021.
  * Lee et al. [2023] J. Lee, T. Le, J. Chen, and D. Lee, “Do language models plagiarize?” in _Proceedings of the ACM Web Conference 2023_ , 2023, pp. 3637–3647.
  * Pagnoni et al. [2022] A. Pagnoni, M. Graciarena, and Y. Tsvetkov, “Threat scenarios and best practices to detect neural fake news,” in _Proceedings of the 29th International Conference on Computational Linguistics_. International Committee on Computational Linguistics, 2022, pp. 1233–1249. [Online]. Available: https://aclanthology.org/2022.coling-1.106
  * Lin et al. [2022] S. Lin, J. Hilton, and O. Evans, “TruthfulQA: Measuring how models mimic human falsehoods,” in _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. Association for Computational Linguistics, 2022, pp. 3214–3252. [Online]. Available: https://aclanthology.org/2022.acl-long.229
  * Weidinger et al. [2021] L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato, P.-S. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadeh _et al._ , “Ethical and social risks of harm from language models (2021),” _ArXiv preprint_ , vol. abs/2112.04359, 2021. [Online]. Available: https://arxiv.org/abs/2112.04359
  * Ayoobi et al. [2023] N. Ayoobi, S. Shahriar, and A. Mukherjee, “The looming threat of fake and llm-generated linkedin profiles: Challenges and opportunities for detection and prevention,” in _Proceedings of the 34th ACM Conference on Hypertext and Social Media_ , 2023, pp. 1–10.
  * Mirsky et al. [2022] Y. Mirsky, A. Demontis, J. Kotak, R. Shankar, D. Gelei, L. Yang, X. Zhang, M. Pintor, W. Lee, Y. Elovici _et al._ , “The threat of offensive ai to organizations,” _Computers & Security_, p. 103006, 2022.
  * Stokel-Walker [2022] C. Stokel-Walker, “Ai bot chatgpt writes smart essays-should academics worry?” _Nature_ , 2022.
  * Kasneci et al. [2023] E. Kasneci, K. Seßler, S. Küchemann, M. Bannert, D. Dementieva, F. Fischer, U. Gasser, G. Groh, S. Günnemann, E. Hüllermeier _et al._ , “Chatgpt for good? on opportunities and challenges of large language models for education,” _Learning and Individual Differences_ , vol. 103, p. 102274, 2023.
  * Alemohammad et al. [2023] S. Alemohammad, J. Casco-Rodriguez, L. Luzi, A. I. Humayun, H. Babaei, D. LeJeune, A. Siahkoohi, and R. G. Baraniuk, “Self-consuming generative models go MAD,” _CoRR_ , vol. abs/2307.01850, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2307.01850
  * Cardenuto et al. [2023] J. P. Cardenuto, J. Yang, R. Padilha, R. Wan, D. Moreira, H. Li, S. Wang, F. A. Andaló, S. Marcel, and A. Rocha, “The age of synthetic realities: Challenges and opportunities,” _CoRR_ , vol. abs/2306.11503, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2306.11503
  * Yu et al. [2023a] P. Yu, J. Chen, X. Feng, and Z. Xia, “CHEAT: A large-scale dataset for detecting chatgpt-written abstracts,” _CoRR_ , vol. abs/2304.12008, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2304.12008
  * Uchendu et al. [2021] A. Uchendu, Z. Ma, T. Le, R. Zhang, and D. Lee, “TURINGBENCH: A benchmark environment for Turing test in the age of neural text generation,” in _Findings of the Association for Computational Linguistics: EMNLP 2021_. Association for Computational Linguistics, 2021, pp. 2001–2016. [Online]. Available: https://aclanthology.org/2021.findings-emnlp.172
  * Dou et al. [2022] Y. Dou, M. Forbes, R. Koncel-Kedziorski, N. A. Smith, and Y. Choi, “Is GPT-3 text indistinguishable from human text? scarecrow: A framework for scrutinizing machine text,” in _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. Association for Computational Linguistics, 2022, pp. 7250–7274. [Online]. Available: https://aclanthology.org/2022.acl-long.501
  * Clark et al. [2021a] E. Clark, T. August, S. Serrano, N. Haduong, S. Gururangan, and N. A. Smith, “All that’s ‘human’ is not gold: Evaluating human evaluation of generated text,” in _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_. Association for Computational Linguistics, 2021, pp. 7282–7296. [Online]. Available: https://aclanthology.org/2021.acl-long.565
  * Ippolito et al. [2020] D. Ippolito, D. Duckworth, C. Callison-Burch, and D. Eck, “Automatic detection of generated text is easiest when humans are fooled,” in _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, 2020, pp. 1808–1822. [Online]. Available: https://aclanthology.org/2020.acl-main.164
  * Stokel-Walker and Van Noorden [2023] C. Stokel-Walker and R. Van Noorden, “What chatgpt and generative ai mean for science,” _Nature_ , vol. 614, no. 7947, pp. 214–216, 2023.
  * Porsdam Mann et al. [2023] S. Porsdam Mann, B. D. Earp, S. Nyholm, J. Danaher, N. Møller, H. Bowman-Smart, J. Hatherley, J. Koplin, M. Plozza, D. Rodger _et al._ , “Generative ai entails a credit–blame asymmetry,” _ArXiv preprint_ , vol. abs/2305.15324, 2023. [Online]. Available: https://arxiv.org/abs/2305.15324
  * Shevlane et al. [2023] T. Shevlane, S. Farquhar, B. Garfinkel, M. Phuong, J. Whittlestone, J. Leung, D. Kokotajlo, N. Marchal, M. Anderljung, N. Kolt, L. Ho, D. Siddarth, S. Avin, W. Hawkins, B. Kim, I. Gabriel, V. Bolina, J. Clark, Y. Bengio, P. F. Christiano, and A. Dafoe, “Model evaluation for extreme risks,” _CoRR_ , vol. abs/2305.15324, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2305.15324
  * Pu et al. [2023] J. Pu, Z. Sarwar, S. M. Abdullah, A. Rehman, Y. Kim, P. Bhattacharya, M. Javed, and B. Viswanath, “Deepfake text detection: Limitations and opportunities,” in _44th IEEE Symposium on Security and Privacy, SP 2023, San Francisco, CA, USA, May 21-25, 2023_. IEEE, 2023, pp. 1613–1630. [Online]. Available: https://doi.org/10.1109/SP46215.2023.10179387
  * Jawahar et al. [2020] G. Jawahar, M. Abdul-Mageed, and L. Lakshmanan, V.S., “Automatic detection of machine generated text: A critical survey,” in _Proceedings of the 28th International Conference on Computational Linguistics_. International Committee on Computational Linguistics, 2020, pp. 2296–2309. [Online]. Available: https://aclanthology.org/2020.coling-main.208
  * Crothers et al. [2023] E. Crothers, N. Japkowicz, and H. L. Viktor, “Machine-generated text: A comprehensive survey of threat models and detection methods,” _IEEE Access_ , 2023.
  * Tang et al. [2023] R. Tang, Y. Chuang, and X. Hu, “The science of detecting llm-generated texts,” _CoRR_ , vol. abs/2303.07205, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2303.07205
  * Guo et al. [2023] B. Guo, X. Zhang, Z. Wang, M. Jiang, J. Nie, Y. Ding, J. Yue, and Y. Wu, “How close is chatgpt to human experts? comparison corpus, evaluation, and detection,” _ArXiv preprint_ , vol. abs/2301.07597, 2023. [Online]. Available: https://arxiv.org/abs/2301.07597
  * Ma et al. [2023] Y. Ma, J. Liu, and F. Yi, “Is this abstract generated by ai? a research for the gap between ai-generated scientific text and human-written scientific text,” _ArXiv preprint_ , vol. abs/2301.10416, 2023. [Online]. Available: https://arxiv.org/abs/2301.10416
  * Muñoz-Ortiz et al. [2023] A. Muñoz-Ortiz, C. Gómez-Rodríguez, and D. Vilares, “Contrasting linguistic patterns in human and llm-generated text,” _ArXiv preprint_ , vol. abs/2308.09067, 2023. [Online]. Available: https://arxiv.org/abs/2308.09067
  * Giorgi et al. [2023] S. Giorgi, D. M. Markowitz, N. Soni, V. Varadarajan, S. Mangalik, and H. A. Schwartz, “”i slept like a baby”: Using human traits to characterize deceptive chatgpt and human text,” in _Proceedings of the IACT - The 1st International Workshop on Implicit Author Characterization from Texts for Search and Retrieval held in conjunction with the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2023), Taipei, Taiwan, July 27, 2023_ , ser. CEUR Workshop Proceedings, M. Litvak, I. Rabaev, R. Campos, A. M. Jorge, and A. Jatowt, Eds., vol. 3477. CEUR-WS.org, 2023, pp. 23–37. [Online]. Available: https://ceur-ws.org/Vol-3477/paper4.pdf
  * Giorgi et al. [2021] S. Giorgi, L. Ungar, and H. A. Schwartz, “Characterizing social spambots by their human traits,” in _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_. Association for Computational Linguistics, 2021, pp. 5148–5158. [Online]. Available: https://aclanthology.org/2021.findings-acl.457
  * Markowitz et al. [2023] D. M. Markowitz, J. Hancock, and J. Bailenson, “Linguistic markers of inherent ai deception and intentional human deception: Evidence from hotel reviews,” _PsyArXiv preprint_ , 2023.
  * Graves [2012] A. Graves, “Sequence transduction with recurrent neural networks,” _arXiv preprint arXiv:1211.3711_ , 2012.
  * Fan et al. [2018] A. Fan, M. Lewis, and Y. Dauphin, “Hierarchical neural story generation,” in _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. Association for Computational Linguistics, 2018, pp. 889–898. [Online]. Available: https://aclanthology.org/P18-1082
  * Holtzman et al. [2020] A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi, “The curious case of neural text degeneration,” in _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net, 2020. [Online]. Available: https://openreview.net/forum?id=rygGQyrFvH
  * Dai et al. [2023] D. Dai, Y. Sun, L. Dong, Y. Hao, S. Ma, Z. Sui, and F. Wei, “Why can gpt learn in-context? language models implicitly perform gradient descent as meta-optimizers,” in _ICLR 2023 Workshop on Mathematical and Empirical Understanding of Foundation Models_ , 2023.
  * Brown et al. [2020] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language models are few-shot learners,” in _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_ , H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., 2020. [Online]. Available: https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html
  * Raffel et al. [2020] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” _J. Mach. Learn. Res._ , vol. 21, pp. 140:1–140:67, 2020. [Online]. Available: http://jmlr.org/papers/v21/20-074.html
  * Zhang et al. [2023] Y. Zhang, Y. Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y. Zhang, Y. Chen _et al._ , “Siren’s song in the ai ocean: A survey on hallucination in large language models,” _ArXiv preprint_ , vol. abs/2309.01219, 2023. [Online]. Available: https://arxiv.org/abs/2309.01219
  * Ouyang et al. [2022] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray _et al._ , “Training language models to follow instructions with human feedback, 2022,” _ArXiv preprint_ , vol. abs/2203.02155, 2022. [Online]. Available: https://arxiv.org/abs/2203.02155