# On the Amplification of Linguistic Bias through Unintentional Self-
reinforcement Learning by Generative Language Models — A Perspective

Minhyeok Lee
School of Electrical and Electronics Engineering
Chung-Ang University
Seoul 06974, Republic of Korea
<EMAIL_ADDRESS>

###### Abstract

Generative Language Models (GLMs) have the potential to significantly shape
our linguistic landscape due to their expansive use in various digital
applications. However, this widespread adoption might inadvertently trigger a
self-reinforcement learning cycle that can amplify existing linguistic biases.
This paper explores the possibility of such a phenomenon, where the initial
biases in GLMs, reflected in their generated text, can feed into the learning
material of subsequent models, thereby reinforcing and amplifying these
biases. Moreover, the paper highlights how the pervasive nature of GLMs might
influence the linguistic and cognitive development of future generations, as
they may unconsciously learn and reproduce these biases. The implications of
this potential self-reinforcement cycle extend beyond the models themselves,
impacting human language and discourse. The advantages and disadvantages of
this bias amplification are weighed, considering educational benefits and ease
of future GLM learning against threats to linguistic diversity and dependence
on initial GLMs. This paper underscores the need for rigorous research to
understand and address these issues. It advocates for improved model
transparency, bias-aware training techniques, development of methods to
distinguish between human and GLM-generated text, and robust measures for
fairness and bias evaluation in GLMs. The aim is to ensure the effective,
safe, and equitable use of these powerful technologies, while preserving the
richness and diversity of human language.

##  1 Introduction

As we navigate the digitized age, we increasingly witness the penetration of
generative language models (GLMs) into a multitude of sectors (OpenAI, 2023;
Radford et al., 2019, 2018; Brown et al., 2020). These advanced artificial
intelligence tools are being harnessed to craft a broad array of written
content; from concocting engaging novels, developing intriguing scripts, to
composing relevant news pieces from mere topical cues. This expanding
footprint of generative language models in the realm of writing presents an
intriguing shift in how we create and consume text, marking a potential
turning point in the annals of written communication.

In addition to the impact of GLMs in written content creation, it is important
to acknowledge the role of ChatGPT in the widespread adoption of these tools.
ChatGPT (OpenAI, 2023), a powerful language model developed by OpenAI, has
played a significant role in expanding the reach and accessibility of AI-
generated text. With its ability to understand and respond to human prompts,
ChatGPT has become a potential tool for various applications, including
customer service, virtual assistants, and interactive conversations.

The proliferation of written pieces crafted, or at least aided, by these
generative models is inevitable given their growing acceptance and widespread
adoption. The convenience and efficiency offered by these tools have seen them
embraced in numerous applications, bringing about a transformation in content
generation processes. Yet, it is not without concern. In particular, the
possibility that these generative language models might unintentionally
amplify biases present in the data they were trained on, raises critical
questions.

The potential risk of self-reinforcing bias in learning from these models
poses a significant challenge. As more content is generated by these models,
and in turn, used to train future iterations, the opportunity for biases to be
reinforced and perpetuated increases. This phenomenon could lead to a spiral
of bias amplification, undermining the objectivity and fairness of the content
produced, a factor that necessitates thorough scrutiny.

Our exploration here is motivated by the need to better understand these
implications, with a focus on unintended self-reinforcement learning. We posit
that an unchecked adoption and uncritical consumption of content generated by
these models could inadvertently facilitate an echo chamber of biases. In the
following sections, we shall explore this issue, discussing potential
ramifications and suggesting possible countermeasures to mitigate this
unintended consequence.

##  2 Linguistic Bias in Generative Language Models

Before exploring the potential consequences of bias propagation in GLMs, it is
essential to define the type of bias that is the focus of our discussion in
this paper. Bias, in the context of machine learning and more specifically,
GLMs, can manifest in numerous ways. However, in this work, we predominantly
concentrate on the linguistic biases in vocabulary selection, tone, and the
flow of writing.

In the field of machine learning, bias generally refers to a pattern of errors
that the model tends to make based on the training data it was provided
(Mehrabi et al., 2021). While this notion of bias holds in our context as
well, we take a step further to highlight the more subtle, less noticeable,
and yet, profoundly impactful dimensions of bias that emerge in the outputs of
GLMs.

Before exploring the potential consequences of bias propagation in GLMs, it is
essential to define the type of bias that is the focus of our discussion in
this paper. Bias, in the context of machine learning and more specifically,
GLMs, can manifest in numerous ways. However, in this work, we predominantly
concentrate on the linguistic biases in vocabulary selection, tone, and the
flow of writing.

While some biases, such as non-linguistic biases, might seem more egregious or
apparent, we argue that they may not pose as substantial a problem in the
context of GLMs. This is due to the fact that users of these models often act
as primary filters for explicit biases. Non-linguistic biases typically
manifest as overt preferences towards a particular gender, race, or minority,
or they may involve the production of inaccurate information or hallucinations
(Lee, 2023a). When GLMs generate content exhibiting such biases, human users
can usually identify and discard these skewed outputs, preventing this biased
information from proliferating in the public domain.

On the contrary, the more pervasive yet subtle problem lies in the linguistic
bias inherent in the operation of GLMs. These models function by predicting
the next word based on the context provided by preceding tokens or words. In
making this prediction, the model relies heavily on probability distributions
that have been learned from the training data. Given a particular token or
word, the model is programmed to select the next word from those with the
highest associated probabilities. This process leads to a pattern where, given
a similar context or initial token, the model tends to produce similar outputs
over repeated instances.

What this implies is that, over time, GLMs can end up generating text that
displays a consistent bias in terms of word selection, tone, and flow of
writing, as these are the patterns that the model has learned from the
training data. The issue with this form of linguistic bias is that it is far
more difficult for users to detect, let alone filter. Unlike non-linguistic
biases that are often explicit and recognizably deviant, linguistic biases are
nuanced, blending seamlessly into the fabric of the text, making it hard for
users to even realize their presence.

Vocabulary selection bias refers to the model’s preference for certain words,
phrases, or terminologies over others. This bias could be a result of the
frequency of word usage in the training data, or it could reflect the
prominence of certain cultural, social, or professional contexts in the data
pool. Tonal bias involves the tendency of a model to reflect a particular
style, emotion, or sentiment in its output. A GLM trained predominantly on
formal, academic literature may naturally adopt a more formal tone, even when
generating text for informal or casual contexts. Conversely, a GLM trained on
a data pool laden with slang or colloquial language might incorporate such
elements even in contexts requiring formal expression. The flow of writing is
another aspect where bias can emerge. This could involve the structure of
sentences, the sequence of ideas, the use of transition words, and the overall
cohesiveness and coherence of the generated text. Models may develop a default
writing flow based on the predominant patterns in their training data, which
can subsequently influence the way ideas are organized and communicated in the
generated text.

These types of bias, while not overtly harmful or discriminatory, can still
have substantial influence over how language is perceived and utilized. They
are subtle, often going unnoticed, and yet, their impact on language
development, communication, and the representation of knowledge can be
profound.

It is important to note that such biases are not a product of intentional
design decisions. Rather, they emerge as a consequence of the training data’s
characteristics and the model’s learning mechanisms. The biases in vocabulary,
tone, and writing flow are reflections of the patterns the models have learned
from their training data, and they are perpetuated and reinforced when these
models generate text, which then serves as training data for subsequent
models.

##  3 Possibility of Unintended Self-reinforcement Learning of GLMs

GLMs, with their adaptive learning capabilities and extensive application, can
shape the textual landscape of the internet in significant ways. However, the
effectiveness of these models hinges largely on the dataset they learn from.
The more diverse and comprehensive the dataset, the better equipped the model
is in delivering accurate and contextually correct content (Tirumala et al.,
2022).

The issue arises when GLMs, as part of their learning process, use internet-
crawled data to fine-tune their capabilities. As GLMs gain popularity and
their outputs multiply across the digital space, these AI-generated texts
inevitably become a part of the data pool from which future GLMs learn. While
this seems like a benign cycle, it presents a unique challenge, the potential
for self-reinforcement of biases.

GLMs, similar to any machine learning models, are susceptible to biases
present in their training data. These biases could be related to language use,
tone, gender, race, knowledge accuracy, or any other facet of human discourse
and knowledge representation. If an initial GLM has a certain bias, it is
likely that the text it generates will also exhibit the same bias. As this
biased content contributes to the learning material for subsequent models, the
bias can unintentionally be reinforced and amplified, leading to a self-
perpetuating cycle.

The self-reinforcing nature of learning in GLMs can thus create an echo
chamber of sorts, where biases are amplified rather than mitigated with each
iteration of learning. Given the difficulty in distinguishing human-written
text from GLM-generated text, these biases can easily propagate unimpeded.

This inadvertent self-reinforcement is particularly potent when it comes to
linguistic biases. As previously discussed, such biases, rooted in vocabulary
selection, tonal preferences, and writing flow, are considerably subtle and
often blend into the generated text without causing overt disruptions.
Consequently, these biases are not easy for users or automated filters to
detect and correct, which allows them to permeate the generated text and, in
time, the overall linguistic landscape shaped by GLMs.

Furthermore, given the expansive and ubiquitous use of GLMs, such as chatbots,
automated content creators, and translators, these biases can diffuse across a
vast array of contexts. When GLM outputs, with their embedded linguistic
biases, become a part of the learning pool for subsequent models, we encounter
the risk of these biases becoming deeply entrenched and normalised. This
could, in effect, lead to the unintentional shaping of language use and
expression in the digital domain, to mirror the biases of the models, rather
than a balanced representation of human language and thought.

This cycle of learning, generation, and re-learning effectively serves as an
unintentional feedback loop, where the biases in the initial output are fed
back into the learning system, only to emerge stronger in the subsequent
output. Without appropriate measures in place to identify and correct these
biases, they can continue to propagate through this loop, growing more
pronounced with each iteration.

Figure 1: Process of Unintentional Self-reinforcement Learning of Linguistic
Bias. Green lines represent the process of self-reinforcement learning of
linguistic bias

##  4 Possibility of Unintended Bias Learning by Humans

As GLMs increasingly become intertwined with our digital interactions, it is
crucial to contemplate not just the self-reinforcing bias within the models
themselves, but also the inadvertent bias that may be learned by humans. This
is particularly relevant in the context of education, where the internet has
assumed a significant role. It is thus necessary to scrutinize the subtle ways
in which GLM-generated content can impact the linguistic and cognitive
development of future generations.

False information or blatantly biased writing is generally identifiable and
hence, can be avoided or corrected in educational contexts. However, the
challenge lies with the subtler, more pervasive forms of bias, those which are
not glaringly wrong but are nonetheless, influential. These could be biases in
vocabulary selection, tone, or the flow of writing that emerge from the GLMs.
Given the widespread use of the internet in advanced education, children are
likely to be exposed to and subsequently, unconsciously absorb these biases,
impacting their linguistic growth and their interpretation of the world.

This inadvertent learning of bias has implications for the evolution of
language and communication. The vocabulary choices, tonal preferences, and
writing styles of early GLMs could seep into the lexicon of future
generations, potentially shaping and constraining how they express ideas and
emotions. This phenomenon, while not harmful in a traditional sense, still
represents a significant influence on the natural development and diversity of
language.

Furthermore, once these GLM-induced biases are learned and reproduced by
humans, they become embedded in human-generated text. Consequently, even if
future GLMs are trained primarily on human-written text, they are still likely
to encounter and learn from the biases of earlier GLMs. This presents a
paradoxical situation where attempting to reduce the influence of GLMs by
focusing on human-written content might inadvertently sustain the very biases
we seek to eliminate.

Thus, the widespread adoption of GLMs may catalyze a loop of bias
reinforcement and amplification that extends beyond the models themselves,
permeating human language and communication. This underscores the importance
of addressing biases in GLMs, not just to improve the models, but also to
protect the diversity and integrity of human language and discourse. Figure 1
illustrates the process of unintentional self-reinforcement learning of
linguistic bias during human learning and GLM training.

##  5 Pros and Cons of Linguistic Bias Amplification in Widespread Adoption of
GLMs

###  5.1 Advantages

Educational Benefits The first major advantage is tied to the domain of child
education. The statistical nature of GLMs allows them to generate logical and
rational text from a linguistic point of view. This outcome is a byproduct of
their training, where they learn from vast corpora to generate text that
closely mirrors high-quality human language (Lee, 2023b). As such, the output
of GLMs can be beneficial in an educational context. Children learning from
GLM-generated text can potentially acquire enhanced writing skills due to
exposure to rational and well-constructed language. The inherent linguistic
bias of the GLMs, in this case, might result in the propagation of refined
language skills.

Facilitation of Future GLM Learning The second advantage lies in the ease of
learning for future GLMs. Under the premise that the linguistic bias inherent
in the text generated by current GLMs is diffused widely and utilized in
subsequent GLM training, this can essentially expedite the learning process of
future GLMs. This can be attributed to the principle of knowledge distillation
in deep learning (Hinton et al., 2015). Knowledge distillation refers to the
process wherein one deep learning model is trained using the outputs of
another, allowing the model to assimilate key data and knowledge in a
relatively shorter span of time (Gou et al., 2021; Cho and Hariharan, 2019).
In this context, despite the potential linguistic bias, the core information
embedded in the training data is preserved in the generated text.
Consequently, this not only sustains the essence of the original information
but also expedites the training of subsequent GLMs, rendering the learning
process more efficient and swift. Hence, the linguistic bias, while it may be
perceived as a challenge, might conversely serve as an enabler in streamlining
the learning of future GLMs.

###  5.2 Disadvantages

Reduction of Linguistic Diversity Conversely, there are potential drawbacks to
the widespread adoption of GLMs, one of which concerns the preservation of
human linguistic diversity. Language is a fundamental aspect of human culture
and identity, and it is inherently diverse and evolving. If GLM-produced text,
with its inherent linguistic bias, becomes a predominant source of language
learning for humans, there is a risk of these biases being unconsciously
absorbed and propagated by human language users. This could lead to a scenario
where human language begins to mirror the language of GLMs, leading to a
reduction in the diversity and richness of human language. Moreover,
considering the Sapir-Whorf Hypothesis which postulates that the structure of
a language affects its speakers’ world view (O’Neill, 2015; Kay and Kempton,
1984), a reduction in linguistic diversity might also impact the diversity of
human thought.

Dependency on Initial GLMs The second disadvantage pertains to the dependence
on initial GLMs. As we have discussed, the biases inherent in the output of
the currently developed GLMs like ChatGPT could potentially propagate through
subsequent GLMs and human language, creating a self-reinforcing cycle of bias.
This introduces a dependency on the initial GLMs, a situation which might be
disadvantageous due to several reasons. First, it could limit the flexibility
and adaptability of future GLMs and human language, as they are constrained by
the linguistic biases of the initial GLMs. Second, this early GLM-dependency
scenario makes it difficult to predict the future impacts of GLMs without
sufficient research and efforts to mitigate their current linguistic biases.
Until these biases are adequately understood and addressed, the self-
reinforcing cycle might continue, further exacerbating these problems.

##  6 Challenges and Future Directions

The exploration of the benefits and drawbacks of linguistic biases present in
GLMs highlights the complexity of their widespread use. While GLMs promise
numerous advantages, their latent ability to unintentionally reinforce and
amplify inherent biases calls for rigorous scrutiny. To ensure the effective,
safe, and fair application of these technologies, a comprehensive
understanding of these issues, coupled with dedicated efforts to address them,
is of paramount importance.

###  6.1 Challenges

The primary challenge lies in the dependency on initial GLMs. The current
generation of GLMs, such as ChatGPT, inadvertently influence future models and
human language through their intrinsic linguistic biases. This interdependency
poses several difficulties. On the one hand, it restricts the dynamism of
subsequent GLMs and human language, as both are perpetually influenced by the
biases of the initial models. On the other hand, it obscures the
foreseeability of the impacts of GLMs, as the extent and depth of the biases
in current GLMs have not yet been fully explored or understood.

The propagation of linguistic bias is a subtle and complex process, which
makes it difficult to detect and quantify. Unlike explicit misinformation or
overt bias, linguistic bias often manifests in nuanced ways, such as a
preference for certain vocabulary or syntax, or a skew towards a particular
tone or style. These biases, while they might appear trivial, can have
significant consequences when they are unknowingly adopted and perpetuated by
both humans and machines.

###  6.2 Future Directions

Given these challenges, future work should be directed towards a more profound
understanding of the linguistic biases inherent in current GLMs. Thorough
research should be conducted to identify, quantify, and comprehend the extent
of these biases. This knowledge would not only shed light on the possible
impacts of GLMs on human language and thought, but also guide the development
of strategies to mitigate these biases.

Efforts should also be made to improve the transparency and interpretability
of GLMs. The underlying mechanisms that give rise to linguistic bias should be
dissected, and the reasons why certain biases are more likely to be amplified
than others should be investigated. This could be accomplished through
techniques such as feature importance analysis or visualization of hidden
layer activations.

In addition, strategies should be developed to prevent the reinforcement and
amplification of linguistic bias. These could include techniques for bias-
aware model training, such as bias regularization or bias-attentive data
augmentation. Alternatively, post-processing methods could be employed to
adjust the output of GLMs to reduce bias.

Moreover, there should be ongoing work to develop techniques to distinguish
between human-written text and GLM-generated text, in order to prevent the
unintentional ingestion of GLM-generated text into the training data of future
GLMs. The development of such techniques could help to curb the self-
reinforcement of linguistic bias.

It is essential to develop robust and rigorous measures to evaluate the
fairness and bias of GLMs. These measures should not only consider the output
of the models but also the processes that generate these outputs. By holding
GLMs to high standards of fairness and bias, we can ensure that they
contribute positively to our linguistic landscape, rather than distort it with
amplified bias.

##  7 Conclusion

The widespread adoption of GLMs holds profound implications for our digital
interactions and the evolution of human language. As GLMs like ChatGPT are
increasingly utilized across various applications, their influence on shaping
the digital linguistic landscape is undeniable. However, this influence brings
with it the latent potential for the unintentional reinforcement and
amplification of inherent linguistic biases. These biases, rooted in the
initial training data of the GLMs, might not only propagate through subsequent
models but also infiltrate human language and thought, leading to a self-
reinforcing cycle of bias amplification.

While the utility of GLMs in areas such as education and the facilitation of
future GLM learning is commendable, the inadvertent propagation of linguistic
biases poses significant challenges. The consequences of these biases are
multifaceted, ranging from the reduction of linguistic diversity to the
creation of an unintended dependency on initial GLMs. Recognizing these issues
is crucial to foreseeing and preparing for the potential impacts of widespread
GLM usage.

Addressing this complex situation demands a robust research approach aimed at
understanding and mitigating the biases in current GLMs. Future research
should be directed towards uncovering the nature and extent of these biases,
improving model transparency, and devising methods to prevent bias
amplification. It is crucial to develop techniques to distinguish between
human-written and GLM-generated text and establish robust measures for
evaluating GLM fairness and bias.

The journey towards bias-free GLMs is undoubtedly challenging but essential
for the future of digital interactions and the preservation of the richness
and diversity of human language. By investing in rigorous research and taking
conscientious strides towards mitigating these biases, we can unlock the true
potential of GLMs in a manner that is not only technologically beneficial but
also ethically sound. In so doing, we can ensure that the widespread adoption
of GLMs bolsters the diversity of human language and thought, rather than
unintentionally constraining them. Thus, the path towards this vision, while
demanding, promises to be a crucial undertaking for the future of artificial
intelligence and its coexistence with humanity.

## Acknowledgments and Disclosure of Funding

We would like to bring to your attention that the refinement of this paper,
achieved through the utilization of GLMs, may have unintentionally introduced
linguistic bias into the main text.

## References

  * OpenAI [2023] OpenAI.  Gpt-4 technical report.  _OpenAI Technical Report_ , 2023.
  * Radford et al. [2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al.  Language models are unsupervised multitask learners.  _OpenAI Technical Report_ , 2019.
  * Radford et al. [2018] Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al.  Improving language understanding by generative pre-training.  _OpenAI Technical Report_ , 2018.
  * Brown et al. [2020] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901, 2020.
  * Mehrabi et al. [2021] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan.  A survey on bias and fairness in machine learning.  _ACM Computing Surveys (CSUR)_ , 54(6):1–35, 2021\.
  * Lee [2023a] Minhyeok Lee.  A mathematical investigation of hallucination and creativity in gpt models.  _Mathematics_ , 11(10), 2023a.  ISSN 2227-7390.
  * Tirumala et al. [2022] Kushal Tirumala, Aram Markosyan, Luke Zettlemoyer, and Armen Aghajanyan.  Memorization without overfitting: Analyzing the training dynamics of large language models.  _Advances in Neural Information Processing Systems_ , 35:38274–38290, 2022.
  * Lee [2023b] Minhyeok Lee.  A mathematical interpretation of autoregressive generative pre-trained transformer and self-supervised learning.  _Mathematics_ , 11(11):2451, 2023b.
  * Hinton et al. [2015] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean.  Distilling the knowledge in a neural network.  _arXiv preprint arXiv:1503.02531_ , 2015.
  * Gou et al. [2021] Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao.  Knowledge distillation: A survey.  _International Journal of Computer Vision_ , 129:1789–1819, 2021.
  * Cho and Hariharan [2019] Jang Hyun Cho and Bharath Hariharan.  On the efficacy of knowledge distillation.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 4794–4802, 2019.
  * O’Neill [2015] Sean P O’Neill.  Sapir–whorf hypothesis.  _The international encyclopedia of language and social interaction_ , pages 1–10, 2015.
  * Kay and Kempton [1984] Paul Kay and Willett Kempton.  What is the sapir–whorf hypothesis?  _American anthropologist_ , 86(1):65–79, 1984\.


# Playing with Words: Comparing the Vocabulary and Lexical Richness of ChatGPT
and Humans

Pedro Reviriego  Javier Conde  Elena Merino-Gómez  Gonzalo Martínez and José
Alberto Hernández  P. Reviriego is with Universidad Politécnica de Madrid,
28040 Madrid, Spain. Email<EMAIL_ADDRESS>
J. Condeis with Universidad Politécnica de Madrid, 28040 Madrid, Spain. Email:
<EMAIL_ADDRESS>
E. Merino-Gómez is with Universidad de Valladolid, 47011 Valladolid, Spain.
Email<EMAIL_ADDRESS>
G. Martínez is with Universidad Carlos III de Madrid, 28911, Leganés, Spain.
Email<EMAIL_ADDRESS>
J.A. Hernández is with Universidad Carlos III de Madrid, 28911, Leganés,
Spain. Email<EMAIL_ADDRESS>
###### Abstract

The introduction of Artificial Intelligence (AI) generative language models
such as GPT (Generative Pre-trained Transformer) and tools such as ChatGPT has
triggered a revolution that can transform how text is generated. This has many
implications, for example, as AI-generated text becomes a significant fraction
of the text, would this have an effect on the language capabilities of readers
and also on the training of newer AI tools? Would it affect the evolution of
languages? Focusing on one specific aspect of the language: words; will the
use of tools such as ChatGPT increase or reduce the vocabulary used or the
lexical richness? This has implications for words, as those not included in
AI-generated content will tend to be less and less popular and may eventually
be lost. In this work, we perform an initial comparison of the vocabulary and
lexical richness of ChatGPT and humans when performing the same tasks. In more
detail, two datasets containing the answers to different types of questions
answered by ChatGPT and humans, and a third dataset in which ChatGPT
paraphrases sentences and questions are used. The analysis shows that ChatGPT
tends to use fewer distinct words and lower lexical richness than humans.
These results are very preliminary and additional datasets and ChatGPT
configurations have to be evaluated to extract more general conclusions.
Therefore, further research is needed to understand how the use of ChatGPT and
more broadly generative AI tools will affect the vocabulary and lexical
richness in different types of text and languages.

##  I Introduction

The rapid development of tools that can generate text in a wide range of
tasks, such as ChatGPT [1] is poised to change the way text is created. This
has huge implications for many industries and jobs as noted by different
analysts and experts [2],[3]. However, it may also have an impact on the
future of languages. There are indeed concerns that AI tools may benefit some
languages over others which may compromise the future of minority languages
[4],[5]. As AI-generated content populates the Internet, it is likely that it
will be used to train newer versions of generative AI tools creating a
feedback loop in which AI tools depend on the content generated by previous
AI-tools [6]. This can degrade the quality of newer AI tools and even lead to
a collapse of the AI models [7]. Similarly, users will be exposed to AI-
generated text when they are learning languages (including their mother
tongue), and thus it may influence how languages are learned.

It seems reasonable to consider that the potential impact of AI-generated text
will depend on how different it is from human-generated text. Evaluating these
differences is far from trivial. A few metrics such as fidelity, diversity, or
the Fréchet Inception Distance (FID) have been proposed focusing on generative
AI image tools [8]. However, applying them (or similar concepts) to tools such
as ChatGPT is not straightforward. In this work, we consider the differences
between the text generated by ChatGPT and humans focusing on a specific
aspect: the vocabulary used and lexical richness in a given task. Words are a
fundamental element of languages and the number of words known varies with
age, education, and other factors [9]. The lexicon of any language understood
as the comprehensive list of all its words, forms the very foundation of human
language’s constitution. Words serve as the medium through which the meanings
of universal knowledge are conveyed, making them an indispensable heritage.
For instance, words classified as obsolete, often excluded from general
dictionaries [10] of "current language," are nonetheless crucial for
understanding texts from different eras. The definitive loss of a word implies
losing the essential keys needed to understand the world in its wholeness.
Since the 19th century, the proliferation of dictionaries that include
historical lexicography has become common in many languages111For example in
Spanish https://www.rae.es/banco-de-datos/corde. Throughout the 20th century,
the use of corpora that aim to comprehensively collect the existing lexicon in
each language through written and oral language samples has become
widespread222For example for Spanish CREA: https://www.rae.es/banco-de-
datos/crea; for English the British National Corpus (BNC):
http://www.natcorp.ox.ac.uk/corpus. In the 21st century, enriching corpora
with databases has become much easier and faster, thanks to the combination of
digitizing materials gathered in the previous century along with the vast
amount of written and audio resources available on the Internet333For English,
the Oxford English Corpus https://www.sketchengine.eu/oxford-english-corpus/;
for American English, the Corpus of Contemporary American English (COCA):
https://www.english-corpora.org/coca/; for oral French, le corpus de la
parole. In France work continues on the CRFC: corpus de référence du français
contemporain [11]; for Spanish on the CORPES XXI: https://www.rae.es/banco-de-
datos/corpes-xxi. The size of such databases today, in addition to their
different formats (written, sound, and videographic) necessarily requires
electronic support.

Currently, online dictionaries provide us with vast repositories of words that
were once considered obsolete and had disappeared from the “current use”
dictionaries of all languages not too long ago [12]. Indeed, the removal of
words that have fallen into disuse and the inclusion of new lemmas have been
constant processes throughout the history of dictionaries [10]. At present,
thanks, among other factors, to the possibilities of digital storage and the
ease and speed of word searches, the trend is to conserve each and every word,
in an effort to preserve the knowledge associated with them.

Looking beyond individual words, it is of interest to consider how words are
used in a given text [13]. This has been widely studied and is related to the
concept of lexical richness that tries to capture the diversity of the
vocabulary used [14]. A large number of metrics to quantify lexical richness
have been proposed like Type-Token-Ratio (TTR), the Root-Type-Token-Ratio
(RTTR) or the Corrected-Type-Token-Ratio (CTTR) that are computed from the
number of words (tokens) and number of distinct words (types) in the text
[15]. More advanced metrics that use logarithmic relationships have also been
proposed to try to obtain lexical richness values that are independent of the
length of the text, like for example, the one proposed by Mass [16]. These
metrics can be used to compare the lexical richness of different texts.

In this context, it is of interest to compare the lexical richness of the text
generated by humans and ChatGPT for a given task to understand how it may
impact the evolution of the vocabulary. To the best of our knowledge, this has
not been explored, only the distribution of the frequency of words in AI
generated languages has been studied [17] showing that it follows a Zipf [18]
distribution as is the case in human languages.

In this paper, we take a first step in this direction by comparing the
vocabulary used and lexical richness in human and ChatGPT answers for several
types of questions and when ChatGPT is used for paraphrasing. The rest of the
paper is organized as follows, section II describes the datasets used in our
evaluation. The analysis is presented in section III and the results, the
limitations of this initial study as well as a few ideas for future work are
discussed in section IV. The paper ends with the conclusion in section V.

##  II Datasets

To compare the vocabulary used by ChatGPT and humans, we need text generated
by both humans and ChatGPT for the same tasks. This is precisely what is done
in the datasets used to evaluate algorithms to detect if a given text was
generated by ChatGPT [19],[20],[21].

For example, the dataset described in [20] contains 126 TOEFL essays written
by ChatGPT and humans444https://github.com/rexshijaku/chatgpt-generated-text-
detection-corpus/tree/main. The essays are written in response to a question,
for example: "How do movies or television influence people’s behavior? Use
reasons and specific examples to support your answer." The questions cover
different topics and the essays have approximately 300-400 words.

A second dataset was presented in [19] and contains human and ChatGPT answers
to questions on different topics555https://huggingface.co/datasets/Hello-
SimpleAI/HC3. In our analysis, we consider four topics: Computer Science,
Medicine, Finance and open questions, they have 842, 1248, 3933, and 1187
questions respectively. The questions and human answers were taken from
different sources in each category which means that they can be considered
independent datasets. The answers are on average shorter, most in the range of
100-200 words.

In summary, we have five sets of 126, 842, 1248, 3933, and 1187 questions and
answers to compare human and ChatGPT answers. Note that different from other
papers, we are not interested in discriminating if a given answer was
generated by ChatGPT or a human but in comparing the vocabulary used by both
when doing the same tasks. To that end, rather than individual responses to
questions, the entire corpus of responses to each type of question is more
useful. Therefore, we have merged all the answers for a given topic in a
single file for ChatGPT and in another for humans; those files are used for
vocabulary analysis in the next section.

In addition to these two datasets that contain responses to questions, a third
dataset [22] with paraphrases of short questions and sentences has also been
evaluated. In this case, ChatGPT is asked to generate five paraphrases for
each input sentence or question. The inputs are much shorter, most having
fewer than 20 words. There are two categories: sentences and questions with
172059 and 247138 elements respectively. For each category, all human elements
are merged in a single file. The same is done for ChatGPT but including only
one of the paraphrases to have a fair comparison as including several
paraphrases of the same sentence intuitively reduces the number of distinct
words compared to having unrelated sentences. The data used in the evaluation
is summarized in Table I, all the datasets used have been generated using
ChatGPT3.5.

TABLE I: Summary of the data used in the evaluation Category | Type | Elements | Word per element
---|---|---|---
TOEFL | Answers | 126 | 200-400
Medicine | Answers | 842 | 100-200
Computing | Answers | 1248 | 100-200
Finance | Answers | 3933 | 100-200
Open questions | Answers | 1187 | 100-200
Questions | Paraphrase | 172959 | 10-30
Sentences | Paraphrase | 247138 | 10-30

##  III Vocabulary Analysis

Once all the answers for a given category are merged into a single file, a
standard preprocessing through the python’s Natural Language
Toolkit666https://www.nltk.org/ (NLTK) is applied. First, we tokenized the
text, that is, we divided the text into words. Then, we carried out a Part-Of-
Speech (POS) tagging [23] process by assigning the respective grammatical
category to each token taking into account the text context. For the
experiment, we used the Penn Treebank Tagset [24] and the Universal Tagset
[25] versions of NLTK. For example, the sentence ‘‘He met six friends from
Spain and seven from Germany.’’ is tokenized and POS tagged into the following
structure (Word, Penn grammatical category, Universal grammatical category):

⬇

("He", "PRON", "PRP"),

("met", "VERB", "VBD"),

("six", "NUM", "CD"),

("friends", "NOUN", "NNS"),

("from", "ADP", "IN"),

("US", "NOUN", "NNP"),

("and", "CONJ", "CC"),

("seven", "NUM", "CD"),

("from", "ADP", "IN"),

("Germany", "NOUN", "NNP")

(’.’, ’.’, ’.’)

Then, we lemmatized the tagged tokens using the English lexical database
WordNet [26] (included in NLKT) and we removed all punctuation. In the example
shown, the result would be the following:

⬇

("He", "PRON", "PRP"),

("meet", "VERB", "VBD"),

("six", "NUM", "CD"),

("friend", "NOUN", "NNS"),

("from", "ADP", "IN"),

("US", "NOUN", "NNP"),

("and", "CONJ", "CC"),

("seven", "NUM", "CD"),

("from", "ADP", "IN"),

("Germany", "NOUN", "NNP")

After that, the total number of words ($n$) and of distinct words ($t$) and
their frequencies have been computed. To evaluate lexical richness, two
metrics, the Root-Type-Token-Ratio (RTTR) and Mass have been used [15]. The
RTTR is computed as:

$RTTR=\frac{t}{\sqrt{n}}$ (1)

and grows with lexical richness. Instead, the Mass metric given by:

$Mass=\frac{log(n)-log(t)}{(log(n))^{2}}$ (2)

decreases with lexical richness.

The same metrics were applied by removing the most common English stop words
[23]. In the example, the resultant words would be:

⬇

("meet", "VERB", "VBD"),

("six", "NUM", "CD"),

("friend", "NOUN", "NNS"),

("US", "NOUN", "NNP"),

("seven", "NUM", "CD"),

("Germany", "NOUN", "NNP")

Finally, we applied the same metrics by selecting only nouns, verbs, adverbs,
and adjectives.

⬇

("meet", "VERB", "VBD"),

("friend", "NOUN", "NNS"),

("US", "NOUN", "NNP"),

("Germany", "NOUN", "NNP"

The results of the experiment are summarized in Tables II, III, and IV for all
lemmas, after removing punctuation, common stop words, and selecting only
nouns, verbs, adverbs, and adjectives, respectively. It can be observed that
while the number of words or distinct words is higher for humans or for
ChatGPT777Note that the medicine, finance, and open question categories have
several ChatGPT answers for some of the questions, see Table 1 in [19].
depending on the category, the RTTR and Mass metrics indicate that the lexical
richness of human answers is higher for all categories. In more detail, the
RTTR is consistently higher for human answers and the Mass metric is
consistently lower for human answers. This holds both when keeping and
removing commonly used stop words and when selecting only nouns, verbs,
adverbs, and adjectives. Depending on the category, the words include
technical terms or names of places and persons, analyzing the number of
distinct words by those types of words would also be of interest. Figures
1,2,3 show the RTTR and metrics in graphical form. It can be seen that the
largest differences between human and ChatGPT texts occur for questions while
the differences are smaller for paraphrasing. This may be related to the
nature of the tasks, answering questions seems to provide more freedom to use
different vocabulary than paraphrasing. This suggests that the lexical
richness differences will depend on the task (instructions, questions,
summarization, translation, paraphrasing, etc.).

In summary, these initial results seem to suggest that the version of ChatGPT
used to generate those datasets tends to use fewer distinct words than humans
and has smaller lexical richness. However, the results are very preliminary,
and further investigation is needed to understand how the different parameters
influence the vocabulary used by humans and ChatGPT. For example, the task,
the persons, the settings and version of ChatGPT, and the questions.

TABLE II: Number of words, number of distinct words, RTTR, and Mass metrics for each category. In bold the largest (lowest) value for each (the mass) metric Category | Type | Number of words | distinct words | RTTR | Mass
---|---|---|---|---|---
TOEFL | Human | 49768 | 5901 | 26.45 | 0.0182
TOEFL | ChatGPT | 41784 | 4229 | 20.69 | 0.0202
Medicine | Human | 106180 | 13133 | 40.30 | 0.0156
Medicine | ChatGPT | 250230 | 10183 | 20.36 | 0.0207
Computing | Human | 167410 | 18587 | 45.43 | 0.0152
Computing | ChatGPT | 155803 | 11743 | 29.75 | 0.0181
Finance | Human | 707628 | 31114 | 36.99 | 0.0172
Finance | ChatGPT | 944355 | 16299 | 16.77 | 0.0214
Open questions | Human | 35126 | 10585 | 56.48 | 0.0109
Open questions | ChatGPT | 419462 | 31755 | 49.03 | 0.0154
Paraphrase of questions | Human | 2781777 | 71146 | 63.91 | 0.0148
Paraphrase of questions | ChatGPT | 3105856 | 65175 | 52.51 | 0.0157
Paraphrase of sentences | Human | 3771366 | 119569 | 83.83 | 0.0137
Paraphrase of sentences | ChatGPT | 3577265 | 107096 | 74.69 | 0.0142
TABLE III: Results after removing stop words. Number of words, number of distinct words, RTTR, and Mass metrics for each category. In bold the largest (lowest) value for each (the mass) metric Category | Type | Number of words | distinct words | RTTR | Mass
---|---|---|---|---|---
TOEFL | Human | 24020 | 5665 | 36.55 | 0.0141
TOEFL | ChatGPT | 21733 | 4081 | 27.66 | 0.0168
Medicine | Human | 60319 | 12847 | 52.31 | 0.0128
Medicine | ChatGPT | 134542 | 10001 | 27.27 | 0.0186
Computing | Human | 103237 | 18370 | 57.17 | 0.0130
Computing | ChatGPT | 86685 | 11557 | 39.25 | 0.156
Finance | Human | 368636 | 30642 | 50.47 | 0.0151
Finance | ChatGPT | 500566 | 16040 | 22.67 | 0.0200
Open questions | Human | 22273 | 10422 | 69.83 | 0.0076
Open questions | ChatGPT | 233756 | 31527 | 65.21 | 0.0131
Paraphrase of questions | Human | 1423867 | 106703 | 89.42 | 0.0129
Paraphrase of questions | ChatGPT | 1598740 | 92551 | 73.20 | 0.0140
Paraphrase of sentences | Human | 2279774 | 163500 | 108.29 | 0.0123
Paraphrase of sentences | ChatGPT | 2174323 | 141779 | 96.15 | 0.0128
TABLE IV: Results after removing stop words and selecting only nouns, verbs, adverbs, and adjectives. Number of words, number of distinct words, RTTR, and Mass metrics for each category. In bold the largest (lowest) value for each (the mass) metric Category | Type | Number of words | distinct words | RTTR | Mass
---|---|---|---|---|---
TOEFL | Human | 23005 | 5549 | 36.59 | 0.0141
TOEFL | ChatGPT | 21072 | 3997 | 27.53 | 0.0168
Medicine | Human | 57910 | 12542 | 52.12 | 0.0127
Medicine | ChatGPT | 132020 | 9757 | 26.85 | 0.0186
Computing | Human | 97135 | 17633 | 56.58 | 0.0129
Computing | ChatGPT | 84681 | 11218 | 38.55 | 0.0157
Finance | Human | 343135 | 28530 | 48.70 | 0.0153
Finance | ChatGPT | 485529 | 15306 | 21.97 | 0.0202
Open questions | Human | 20362 | 9777 | 68.52 | 0.0075
Open questions | ChatGPT | 221897 | 30044 | 63.78 | 0.0132
Paraphrase of questions | Human | 1349778 | 101495 | 87.36 | 0.0130
Paraphrase of questions | ChatGPT | 1515505 | 88264 | 71.70 | 0.0140
Paraphrase of sentences | Human | 2110285 | 152653 | 105.08 | 0.0124
Paraphrase of sentences | ChatGPT | 2009146 | 133187 | 93.96 | 0.0129
Figure 1: Comparison of RTTR (top) and Mass (bottom) metrics for humans (blue)
and ChatGPT (red). Larger (lower) RTTR (Mass) corresponds to more richness
Figure 2: Comparison of RTTR (top) and Mass (bottom) metrics after removing
stop words for humans (blue) and ChatGPT (red). Larger (lower) RTTR (Mass)
corresponds to more richness Figure 3: Comparison of RTTR (top) and Mass
(bottom) metrics after removing stop words and selecting only nouns, verbs,
adverbs, and adjectives for humans (blue) and ChatGPT (red). Larger (lower)
RTTR (Mass) corresponds to more richness

##  IV Discussion

As ChatGPT generated content populates the Internet, it is of interest to
understand the differences between the vocabulary used by ChatGPT (and similar
tools) and humans as the vocabulary used by AI tools may have implications for
the language learning and use of persons and also for the training of newer AI
tools. This is however challenging as the vocabulary used depends on the
person, the tasks, the AI model version and parameters, etc., leading to a
complex scenario. Our preliminary comparison of the vocabulary used and
lexical richness of ChatGPT and humans when answering different categories of
questions and when paraphrasing sentences and questions shows that ChatGPT
consistently uses a smaller vocabulary and has lower lexical richness.
However, our results are based on a very limited set of data and a single AI
tool and version (ChatGPT3.5) and thus have to be taken with a pinch of salt.

Further research is needed to check if these observations are generally
applicable to other tasks, tools, and languages and to find additional
insights into how ChatGPT and humans use the vocabulary in different
scenarios. To make this possible further datasets containing text generated by
ChatGPT and humans for the same tasks are needed. One possibility is to use
datasets that are created to evaluate algorithms that detect text written by
AI tools. However, this has limitations as the datasets are not designed to
test the vocabulary. Therefore, it would also be of interest to create
datasets to evaluate the vocabulary of AI tools. More generally evaluating the
vocabulary of AI tools seems to be a new interesting area for research in the
intersection of computing and linguistics.

It seems that automating the analysis is needed to perform tests at scale. A
possible avenue to automate the analysis could be to take human written texts,
such as books and ask ChatGPT to rewrite them and then compare the number of
distinct words. The translation from one language to another and then
comparing with a human translation could also be another avenue for
automation. Developing software to process entire books or even larger sets of
texts using ChatGPT API would enable the evaluation of different versions,
parameters (including role) and prompts for ChatGPT to further understand how
those settings impact the vocabulary used by ChatGPT and how it evolves across
versions of the tool.

##  V Conclusions

In this work, we have for the first time compared the vocabulary used and
lexical richness of ChatGPT and humans when performing the same tasks. The
results show that in the datasets considered, ChatGPT uses a smaller number of
distinct words. However, these results are very preliminary and further
research is needed to understand the differences in the vocabulary used by AI
tools such as ChatGPT and humans. Those may depend on the tasks, person,
version, and configuration of the AI tool among other parameters and thus
additional datasets are needed to provide a comprehensive evaluation of the
vocabulary used.

Analyzing the vocabulary used by AI tools is a new topic for research in the
intersection of computing and linguistics that we think deserves further
attention as it may have implications for the evolution of languages in the
future. For example, as persons are exposed to AI-generated text, how will
this influence the learning and use of the language? This will among other
things depend on the vocabulary used by AI tools. This vocabulary will also
influence the frequency of words and probably their future as those not used
by AI tools may become less and less used.

To analyze the vocabulary of ChatGPT, software tools that automate the process
of testing are needed. In more details, tools that take large amounts of human
text and interact with ChatGPT to produce the equivalent text generated by the
AI model. Once those tools are available, AI models such as ChatGPT could be
compared on their vocabulary for different tasks or languages as they are now
compared on other performance metrics.

##  VI Acknowledgements

This work was supported by the FUN4DATE project funded by the Spanish Agencia
Estatal de Investigacion (AEI) 10.13039/501100011033.

## References

  * [1] T. Wu, S. He, J. Liu, S. Sun, K. Liu, Q.-L. Han, and Y. Tang, “A brief overview of chatgpt: The history, status quo and potential future development,” _IEEE/CAA Journal of Automatica Sinica_ , vol. 10, no. 5, pp. 1122–1136, 2023.
  * [2] P. P. Ray, “Chatgpt: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope,” _Internet of Things and Cyber-Physical Systems_ , vol. 3, pp. 121–154, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S266734522300024X
  * [3] J. Keller, “Chat, anyone? [president’s message],” _IEEE Computational Intelligence Magazine_ , vol. 18, no. 3, pp. 4–5, 2023.
  * [4] M. R. Costa-jussà, J. Cross, O. Çelebi, M. Elbayad, K. Heafield, K. Heffernan, E. Kalbassi, J. Lam, D. Licht, J. Maillard _et al._ , “No language left behind: Scaling human-centered machine translation,” _arXiv preprint arXiv:2207.04672_ , 2022.
  * [5] P. Reviriego and E. Merino-Gómez, “Text to image generation: leaving no language behind,” _arXiv preprint arXiv:2208.09333_ , 2022.
  * [6] G. Martínez, L. Watson, P. Reviriego, J. A. Hernández, M. Juarez, and R. Sarkar, “Combining generative artificial intelligence (ai) and the internet: Heading towards evolution or degradation?” _arXiv, 2303.01255_ , 2023.
  * [7] I. Shumailov, Z. Shumaylov, Y. Zhao, Y. Gal, N. Papernot, and R. Anderson, “The curse of recursion: Training on generated data makes models forget,” _arXiv, 2305.17493_ , 2023.
  * [8] M. F. Naeem, S. J. Oh, Y. Uh, Y. Choi, and J. Yoo, “Reliable fidelity and diversity metrics for generative models,” in _International Conference on Machine Learning_. PMLR, 2020, pp. 7176–7185.
  * [9] M. Brysbaert, M. Stevens, P. Mandera, and E. Keuleers, “How many words do we know? practical estimates of vocabulary size dependent on word definition, the degree of language input and the participant’s age,” _Frontiers in Psychology_ , vol. 7, 2016. [Online]. Available: https://www.frontiersin.org/articles/10.3389/fpsyg.2016.01116
  * [10] C. B. de la Mata, “La décima edición del diccionario de la lengua castellana de la real academia española (1852): el aumento y la supresión de voces,” 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:212862457
  * [11] D. Siepmann, C. Bürgel, and D. Sascha, “Le corpus de référence du français contemporain (crfc), un corpus massif du français largement diversifié par genres,” _SHS Web of Conferences_ , vol. 27, p. 11002, 01 2016.
  * [12] L. Pierre and B. Cerquiglini, _Les mots disparus de Pierre Larousse_. Larousse, 2018.
  * [13] N. Schmitt, _Researching vocabulary: A vocabulary research manual_. Springer, 2010.
  * [14] K. Kyle, “Measuring lexical richness,” _The Routledge handbook of vocabulary studies_ , pp. 454–475, 2019.
  * [15] R. Hout and A. Vermeer, “Comparing measures of lexical richness,” _In: H. Daller, J. Milton and J. Treffers-Daller (eds.), Modelling and assessing vocabulary knowledge (93-116). Cambridge: Cambridge University Press._ , 01 2007\.
  * [16] F. J. Tweedie and R. H. Baayen, “How variable may a constant be? measures of lexical richness in perspective,” _Computers and the Humanities_ , vol. 32, pp. 323–352, 1998.
  * [17] J. Diamond, “"genlangs" and zipf’s law: Do languages generated by chatgpt statistically look human?” _arXiv, 2304.12191_ , 2023.
  * [18] G. K. Zipf, _Selected Studies of the Principle of Relative Frequency in Language_. Harvard University Press, 2013\.
  * [19] B. Guo, X. Zhang, Z. Wang, M. Jiang, J. Nie, Y. Ding, J. Yue, and Y. Wu, “How close is chatgpt to human experts? comparison corpus, evaluation, and detection,” _arXiv preprint arxiv:2301.07597_ , 2023.
  * [20] R. Shijaku and E. Canhasi, “Chatgpt generated text detection,” 01 2023.
  * [21] P. Yu, J. Chen, X. Feng, and Z. Xia, “Cheat: A large-scale dataset for detecting chatgpt-written abstracts,” _arXiv,2304.12008_ , 2023.
  * [22] M. K. Vladimir Vorobev, “Chatgpt paraphrases dataset,” 2023.
  * [23] C. Chai, “Comparison of text preprocessing methods,” _Natural Language Engineering_ , vol. 29, pp. 1–45, 06 2022.
  * [24] M. Marcus, B. Santorini, and M. A. Marcinkiewicz, “Building a large annotated corpus of english: The penn treebank,” 1993.
  * [25] S. Petrov, D. Das, and R. McDonald, “A universal part-of-speech tagset,” in _Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12)_. Istanbul, Turkey: European Language Resources Association (ELRA), May 2012, pp. 2089–2096.
  * [26] G. A. Miller, “Wordnet: a lexical database for english,” _Communications of the ACM_ , vol. 38, no. 11, pp. 39–41, 1995.