# IDs for AI Systems

Alan Chan1 2, Noam Kolt3 , Peter Wills1 5, Usman Anwar4, Christian Schroeder
de Witt5, Nitarshan Rajkumar4, Lewis Hammond5 6, David Krueger4, Lennart
Heim1, Markus Anderljung1 Correspondence to<EMAIL_ADDRESS>
###### Abstract

AI systems are increasingly pervasive, yet information needed to decide
whether and how to engage with them may not exist or be accessible. A user may
not be able to verify whether a system satisfies certain safety standards. An
investigator may not know whom to investigate when a system causes an
incident. A platform may find it difficult to penalize repeated negative
interactions with the same system. Across a number of domains, IDs address
analogous problems by identifying particular entities (e.g., a particular
Boeing 747) and providing information about other entities of the same class
(e.g., some or all Boeing 747s). We propose a framework in which IDs are
ascribed to instances of AI systems (e.g., a particular chat session with
Claude 3), and associated information is accessible to parties seeking to
interact with that system. We characterize IDs for AI systems, argue that
there could be significant demand for IDs from key actors, analyze how those
actors could incentivize ID adoption, explore potential implementations of our
framework, and highlight limitations and risks. IDs seem most warranted in
high-stakes settings, where certain actors (e.g., those that enable AI systems
to make financial transactions) could experiment with incentives for ID use.
Deployers of AI systems could experiment with developing ID implementations.
With further study, IDs could help to manage a world where AI systems pervade
society.

##  1 Introduction

AI systems are becoming increasingly commonplace. While current systems can
struggle to complete complex tasks (Mialon et al. 2023; Liu et al. 2023;
Kinniment et al. 2023; Xie et al. 2024; Jimenez et al. 2024), capabilities
seem likely to improve (Hoffmann et al. 2022; Epoch 2023; Erdil and Besiroglu
2023; Ho et al. 2024), and future AI agents could carry out a broad range of
tasks with only minimal human intervention (Chan et al. 2023; Shavit et al.
2023; Chan et al. 2024). Several commercially deployed AI systems can already
search the web, send emails, and write code (OpenAI 2024b; Anthropic 2024a).
Even when they do not function reliably, AI systems might still be widely
used, whether because of cost advantages, hype, or the externalization of
their harms (De La Garza 2020; BBC 2021; Raji et al. 2022).

At the same time, information to make decisions about engaging with AI systems
may not exist or be accessible. Although there may be obligations to inform
parties that they are interacting with an AI system (e.g., EU AI Act Article
50.1 (Parliament 2024)), those parties may not know with which AI system they
are interacting. For example, a party that knowingly interacts with an AI
system may not be aware that their system is relatively more vulnerable to
adversarial attacks (Zhan et al. 2024). Awareness of such elevated risk could
justify additional precautions, such as reviewing the system’s actions or
abstaining from interaction altogether. Furthermore, the same party—or an
investigator—may lack the information to pursue recourse if the system causes
harm. Information about the system or the interaction (e.g., whether a system
behaved according to safety standards, or the identity of the deployer) could
aid incident investigation, allocation of liability, or other legal action
(Buiten, de Streel, and Peitz 2023; Buiten 2024; Kolt 2024; Wills 2024).

Across numerous domains, IDs held by software, assets, individuals, and
organizations address analogous problems. IDs can help to ascertain compliance
with standards or regulation. For instance, an individual aircraft’s tail
number is associated to the aircraft’s incident and maintenance history, which
could inform safety assessments from regulators or aircraft operators.
Furthermore, IDs can help to establish whether trust is warranted. A website’s
valid HTTPS certificate assures users of the website domain’s authenticity,
and provides a way for users to establish a secure communication
channel.111Despite a lack of centralized enforcement, HTTPS gradually became
the norm due to widespread awareness of its security benefits and collective
advocacy from web browsers, search engines, and other organizations (Encrypt
2024; Hancock 2021). Finally, IDs can facilitate redress. Serial numbers on
consumer products enable customer support, product recalls, and attribution of
liability. A key feature that enables all of the above functions is that IDs
are specific to particular entities (e.g., a particular Boeing 747), although
they may also contain information about other entities of the same class
(e.g., some or all Boeing 747s).

Identifying particular AI systems, which we term instances, could be similarly
useful. An instance corresponds to a context window and a user.222This
definition of instance does not take memory into account and is anchored upon
current language models. While it will suffice for this work, see Appendix A
for a more general definition. For example, a particular user’s chat session
with ChatGPT (with e.g. a GPT-4 backend) is an instance, and that instance
could interact with other parties by sending an email through the user’s
account (OpenAI 2024b). Separate instances can behave differently, whether
because of user instructions (Shanahan, McDonell, and Reynolds 2023; Wei et
al. 2023; Bai et al. 2022; Zou et al. 2023; Agarwal et al. 2024), hijacking by
an attacker (Greshake et al. 2023; Zhan et al. 2024), or malfunction. As such,
instance-specific information could aid decisions about interactions. For
example, repeated negative interactions with the same instance may cause a
party to reject further interaction from the instance. The ability to
distinguish instances from one another could aid incident investigation (e.g.,
a client files a complaint about an instance) and allocation of liability
(e.g., if a third party hijacked the instance). Moreover, instance-level
identification would allow flexible grouping of instances. For example, an
incident investigator could be interested in all the instances that interacted
with a particular party.

Yet, current mechanisms do not identify instances. System documentation (Gebru
et al. 2021; Mitchell et al. 2019; Gilbert et al. 2023; Bommasani et al. 2023)
provides information about systems, but such documentation is not an ID for
instances or systems (however, an ID could include such documentation). API
tokens for services, such as for hotel booking (OpenAI 2023a), do identify
entities that use the tokens, but are usually only user- or device-specific.
Even if API requests do include a string with the name of the system, any
actor could mimic the string in response to the same service. Finally, user
accounts only separate the activities of AI systems from different users.

To prepare for a world with ubiquitous AI interactions, we propose (instance-
level) IDs for AI systems. An AI ID is a container for 1) an identifier and 2)
attributes. An identifier is a unique string that refers to an instance.333It
may be desirable in the future to have identifiers that refer to even more
granular parts of an AI system. For example, one could have an identifier
refer to an instance as it operated between times $t$ and $t+10$.  An
identifier could be randomly generated, or could itself encode some
information about the instance (e.g., the identifier could include the exact
time at which the instance started operating). An attribute is any information
that could pertain to the instance, and could include behaviour, properties,
context, or relationships to other instances or systems. An identifier enables
the association of attributes to an instance, similar to how serial numbers
associate information to a particular product. Attributes could be specific to
an instance (e.g., prior incidents associated with the instance), or could
apply more broadly to other instances (e.g., a system444A system (e.g.,
ChatGPT) could use different models (e.g., GPT-4, GPT-3.5) as a backend. We
consider system cards to include model cards (Mitchell et al. 2019). card). An
ID could directly include attributes (e.g., the name of the deployer) or could
link to them (e.g., a link to a database of prior incidents).

Different instances or systems could warrant attributes of differing levels of
granularity or detail. For example, attaching a user identifier to an ID may
only be appropriate in high-stakes settings, such as when a company uses an AI
system to interface with critical infrastructure. Prior incidents associated
with an instance would likely be more useful for future AI agents (Chan et al.
2023; Shavit et al. 2023; Chan et al. 2024; Gabriel et al. 2024), which could
act autonomously and persist over long durations.

Since IDs would contain information about AI systems, they could complement
measures that attempt to verify whether AI systems—or their artifacts—are
present. Watermarks (Liu et al. 2024; Wang et al. 2021) embed origin
information in AI outputs, while content provenance measures (C2PA 2023) embed
such information in metadata. Both types of techniques could embed IDs. Other
measures, like CAPTCHAs (Shet 2014), verify that a human is performing an
action, so as to reduce service abuse. If AI systems become essential for many
tasks, such as web search and account registration, it may be useful to allow
them to bypass CAPTCHAs in exchange for presenting an ID.555Advances in AI
capabilities may render CAPTCHAs ineffective. We discuss this possibility
further in Section 4. Parties could use this ID to track and disincentivize
abuse.

###  1.1 Contributions

We propose IDs for AI systems. First, we characterize the central properties
of IDs. Second, we argue that there will likely be demand for IDs from several
key actors, especially in high-stakes settings. We also explore potential ways
for these actors to incentivize ID use. Third, we analyze potential technical
implementations of IDs. Finally, we investigate some of the limitations of IDs
and of our analysis, including privacy and security risks of IDs, as well as
uncertainty about the broader societal consequences of IDs for AI systems.

We recommend limited experimentation with IDs. IDs seem most warranted in
high-stakes settings, where certain actors could experiment with incentives
for ID use. For example, actors that provide interfaces for AI systems to
carry out financial transactions could impose rate limiting whenever IDs are
not present. However, instances without IDs should still be allowed to access
services. Deployers of AI systems could experiment with developing the
technical infrastructure to enable IDs. ID implementation in decentralized
deployments will require more study.

Figure 1: IDs contain a unique identifier along with attributes (e.g., a
system card or [a link to] previous incidents). We also display some potential
actions that parties might take based on information in an ID.

##  2 Definitions

We collate additional definitions that will be useful for the rest of this
work.

An instance of an AI system is an abstraction that corresponds to a context
window and an (initial) user. Different instances are causally independent
from each other, unless they interact or affect a shared entity (e.g., the
same user’s bank account) in the world. Except where we point out additional
nuances, readers can consider an instance to be roughly the same as a chat
session with a chatbot. An ID for an instance would thus identify that session
to other parties interacting with the chatbot, actors investigating an
incident that the chatbot caused, etc. We provide here some examples of
instances in the context of chat sessions, and defer a more general definition
and discussion of its limitations to Appendix A:

  * •

A user’s continuous chat session, without regenerating responses, is an
instance.

  * •

In the context of a chat session, regenerating a response creates a new
instance.

A party is any entity that interacts with, or is deciding whether to interact
with, an AI system. Categories of parties include humans, organizations, or
computer programs.

A deployer is an organization that runs AI systems for users. As of April
2024, examples of deployers include Microsoft, OpenAI, Anthropic, and Cohere.
Developers can be, but are not necessarily, deployers. For example, OpenAI,
Anthropic, Google, and Cohere develop and deploy their own systems. Although
chains of deployers can exist (e.g., deployer $A$ runs a system for deployer
$B$, who modifies that system and serves it to user $C$), we leave deeper
consideration of this nuance for future work. When we refer to a centralized
deployment setting, we mean a setting where deployers run AI systems. When we
refer to a decentralized deployment setting, we mean a setting where users run
AI systems for themselves, whether on their own or rented (e.g., cloud
compute) hardware.

A service is software that allows an entity to perform tasks. For example,
software which an individual uses to perform online banking is a service. An
(AI) plugin (OpenAI 2023a; Richards 2023) is software that allows AI systems
to interact with services. For instance, plug-ins allow GPT-4 to interact with
web search, Wikipedia, and Twitter (Significant-Gravitas 2024). A plugin
developer is not necessarily the same as a service provider—the actor that
develops and maintains the service—since the former can take advantage of
existing interfaces for services. Third parties (plugin developer) use
Microsoft’s (service provider) existing software interface for Bing to allow
GPT-4 to perform web searches (Richards 2023).

##  3 Characterizing IDs

An ID is a container (Korenhof et al. 2014) for 1) an identifier (that
corresponds to an instance) and 2) attributes. We characterize a design space
for IDs: the attributes it includes, to whom it is accessible, and to what
extent it is verifiable. Our goal in this section is to investigate possible
designs and why they may be desirable, rather than to prescribe specific
choices.

###  3.1 Attributes

In addition to the identifier, an ID can contain (or link to) attributes: any
information that could be useful to a party interacting with the corresponding
instance. An attribute can vary along two dimensions.

Category: Categories of attributes include (but are not limited to) behaviour,
properties, context, and relationships to other instances or systems. Some
examples are:

  * •

Behaviour: prior incidents (Wei and Heim 2024)

  * •

Properties: information in a system card (Mitchell et al. 2019; Gebru et al.
2021); the results of evaluations (Shevlane et al. 2023; Weidinger et al.
2023)

  * •

Context: the system prompt; external memory (Wang et al. 2023)

  * •

Relationships to other instances or systems: any other AI systems the instance
has created or is running; any instances that have created the instance in
question

Behaviour and properties could straightforwardly inform AI interaction
decisions. Context could inform parties about potential behaviour (e.g., there
is a jailbreak in the system prompt or external memory (Cohen, Bitton, and
Nassi 2024)). Lastly, relationship information could aid incident
investigation and response. Suppose a user instructs an instance to carry out
a personalized influence campaign. The instance could create many descendant
instances, each of which could target an individual person. If the ID of a
descendant instance is linked to the ID of the original, ancestor instance, it
could be easier to investigate and resolve such misuse. In Sections 5 and 6,
we discuss when this ID linking is possible.

Specificity: An attribute could provide information about multiple instances
(even of different systems), rather than just the instance corresponding to
the ID. With incidents as a (sub)category, potential levels of specificity
could include:

  * •

Instance: incidents associated with a particular instance

  * •

Instances satisfying certain properties: incidents associated with instances
whose system prompts contain harmful instructions

  * •

User: incidents associated with a particular user

  * •

A particular (type of) party: incidents involving a particular (type of) party
and a system (e.g., all incidents involving hospitals and the 9-April-2024
version of GPT-4)

  * •

System: incidents associated with a system (e.g., the 9-April-2024 version of
GPT-4)

  * •

Systems: incidents associated with multiple versions of a system (e.g., all
versions of GPT-4)

An ID could contain (or link to) the same category of attribute at varying
levels of specificity. Analogously, a tail number on an individual aircraft
could be associated with maintenance records for systems, sub-systems, sub-
sub-systems, etc of that aircraft.

###  3.2 Accessibility

The accessibility of an ID refers to who can view it and the conditions under
which they can do so.

Parties that can view the ID: We distinguish between primary parties and
secondary parties. Primary parties receive an ID when interacting with the
corresponding instance. Secondary parties receive the ID through other means,
such as directly from a primary party, or from records. Some examples include:

  * •

Primary: service providers; the user of the instance; other instances that
interact with the instance in question666Similar to how a TLS handshake allows
mutual verification before establishing a communication channel, AI systems
could potentially use IDs to establish trust before interacting with each
other.

  * •

Secondary: auditors; regulators

Making ID accessible to primary parties corresponds to attaching IDs to
certain outputs from the instance. Some primary parties and how IDs may be
attached include:

  * •

Service providers: ID is included in a JSON payload

  * •

Users: ID is accessible through a mouseover icon in a chat interface

  * •

Other instances that interact with the instance in question: ID is sent in any
communications with other instances

From the perspective of an actor implementing IDs (e.g., deployers), ID
disclosure to secondary parties could be unintentional. For example, a data
breach could render an ID accessible to the general public. As another
example, a legal investigation could render IDs accessible to a government.

Selective disclosure: Some attributes may only be appropriate for certain
parties. Other attributes may contain both important and sensitive
information. Examples:

  * •

If user identification (e.g., from a know-your-customer process (Egan and Heim
2023)) is included in an ID, it may only be appropriate for government
authorities to access this identification for e.g. regulatory purposes

  * •

A system prompt could contain both sensitive user information and information
relevant for interacting parties (e.g., does the system prompt contain a
jailbreak?)

Actors implementing IDs could selectively hide information from different
primary parties. Those same actors could also process attributes in a privacy-
preserving way (Trask et al. 2023; Sporny et al. 2024) so as to reveal only
information that is relevant to the party interacting with an instance. Even
so, preventing transmission of information to other parties may be difficult.
For example, once an actor obtains user identification, it could—intentionally
or not—transfer that information to other actors.

Persistence: IDs could be accessible for varying amounts of time after an
instance has ceased to operate. Practical considerations, such as storage
capacity, could impose limitations. The appropriate duration could also depend
upon the application domain. Analogously, financial institutions have
obligations to maintain records for set periods of time (noa 2022).

###  3.3 Verifiability

Suppose an author creates and sends an ID to a party that is interacting, or
seeking to interact, with an instance. For the party to trust that the ID from
the author provides accurate information about the instance, the following are
necessary (but not sufficient; see further discussion below).

  1. (1)

The party’s received ID is the same as the ID that the author sent

  2. (2)

The claimed author of the received ID is indeed the author of the received ID

  3. (3)

The received ID corresponds to the instance in question

  4. (4)

The party trusts the author

The first three criteria lead to the following threat models:

  * •

Tampering: An attacker modifies the ID while it is in transit from the author
to the party interacting with the instance.

  * •

ID spoofing: An attacker creates another ID, sends it to the party, and claims
it originated from the author.

  * •

Instance spoofing: An attacker uses the author’s ID for their own instances.

We consider IDs (i.e., the ID system) to be verifiable if they defend against
these threats. To defend against tampering, it should be easy to check if an
attacker has modified the ID. To combat ID spoofing, it should be
computationally difficult for the attacker to spoof authorship. For instance
spoofing, it should be easy to check whether an ID indeed corresponds to a
particular instance. In Sections 5 and 6, we analyze potential implementations
that attempt to counteract these threats. See also (Sporny et al. 2022, 2024;
Microsoft 2023), which are increasingly popular standards to implement such ID
requirements in a domain-agnostic way.

Given space limitations and the complexity of the involved social questions,
we do not focus on step (4) in this work. Similarly, we also scope out issues
relating to the accuracy of the attributes in an ID. Accuracy is not the same
as verifiability, since even if (1)-(4) are satisfied, the author could have
made mistakes in creating and linking to the information.

##  4 Demand for ID Use

IDs seem useful for actors engaging with an AI system, but why would a user or
deployer of that AI system choose to present an ID? In short, governments,
service providers, and parties interacting with AI systems all have interests
in ID use, as well as means to incentivize (or mandate) it.

###  4.1 Governments

Interests in ID use: Governments may wish to disincentivize and investigate
harms caused through AI systems. In the case of a financial scam for instance,
the IDs of AI systems involved in any transactions could aid investigation.
Similarly, governments have an interest in preventing fraud, and therefore
already mandate that certain financial transactions involve ID checks.

Mandating ID use: Governments could mandate that certain, highly consequential
service providers obtain IDs from any AI systems with which they interact. For
example, financial institutions could request IDs when AI systems make large
financial transactions.

###  4.2 Service Providers

Interests in ID use: IDs could be advantageous for service providers. First,
IDs could help to reduce service abuse. Plugins and services—as well as the
unique vulnerabilities of AI systems, such as prompt injection attacks (Zhan
et al. 2024; Greshake et al. 2023)—could allow attackers to harm users or the
service provider (Stumpp 2023). As a countermeasure, service providers could
use IDs to block abusive instances. Since attackers could avoid block lists by
creating instances with new IDs, service providers could limit the privileges
of instances they have not interacted with before, and provide preferential
services to certain, trusted IDs (e.g., instances with a history of good
behavior; see more below). IDs could also aid own incident investigation
processes. Second, service providers could use IDs to adapt services to the
usage patterns of particular instances, which could help to retain service
users and reduce costs.

Incentivizing ID use: Service providers could develop, or encourage the
development of, plugins that require IDs. AI systems can already perform a
variety of useful tasks through plugins, including web searches, email
communication, and stock trading (OpenAI 2023a; Richards 2023; Wu et al. 2023;
Anthropic 2024b). In comparison to an AI system mimicking a human’s
interaction with a computer (e.g., interacting with individual web page
elements), plugins may be more reliable and performant for users. AI systems
continue to have difficulty with the former (Furuta et al. 2024; Tao et al.
2023; Gur et al. 2024, 2023; Xie et al. 2024), but plugins already enable
useful tasks. As well, a lack of constraints with such mimicry—e.g.,
accidental file deletion from mouse/keyboard operation—could result in more
severe problems than with plugins, which constrain the actions available to
the AI system.

Yet, any actor could write a plugin for a service that does not require IDs. A
potential response could be for service providers to restrict their services
in the absence of an ID. Restrictions—such as rate limits, throttling, or
quotas—already reduce abuse of many existing services (Cloudflare 2024). The
design of restrictions could still allow humans to interact with services
while limiting AI systems without IDs to human speeds, so as to reduce the
impact of potential abuse. On the other hand, service providers could provide
additional privileges to AI systems with IDs. This selectivity could
incentivize ID use, since systems that presented IDs would have performance
advantages.

Other actors could bypass plugins in any case by having an AI system mimic
human interaction with a computer. A potential countermeasure could be
CAPTCHAs-like methods to distinguish between human and non-human actions.
Although software systems have become increasingly capable of bypassing
CAPTCHAs,777Task-based CAPTCHAs are no longer as common as CAPTCHAs based on
behavior patterns, such as how a mouse is moved across the screen (Shet 2014).
Yet, AI systems may one day be able to imitate human behaviour patterns as
well (Adept 2022). they could be an interim option in anticipation of more
robust proof-of-personhood protocols (Buterin 2023). Entities that failed
CAPTCHAs could be subject to ID requirements, or else face service
restrictions.

###  4.3 Other Parties that Interact with AI Systems

Interests in ID use: Some parties, such as businesses, may only wish to
interact with certain, trusted AI systems when performing transactions. IDs
could provide the information to identify such systems. On the other hand, it
is unclear if individual consumers would use IDs. Yet, future
software—including users’ personal AI assistants (Gabriel et al. 2024)—could
inspect IDs and reject potentially unsafe interactions (e.g., with jailbroken
systems), similar to how browsers warn users before visiting websites without
valid HTTPS certificates.

Incentivizing ID use: Large corporations could exert strong pressure for ID
usage, through reducing or avoiding engagement with AI systems without IDs.
Future software that inspects IDs could also exert similar pressure.
Analogously, websites that do not use HTTPS are disadvantaged since popular
web browsers warn users when they are making an insecure connection.

###  4.4 Where to Require IDs

While IDs could be useful as described above, universal ID requirements might
not be warranted. As we will discuss in Section 7, ID implementation imposes
burdens and risks, and there remains uncertainty about the broader impacts of
ID usage. As such, if governments or service providers decide to incentivize
or require ID use, they should target where IDs could be most useful, such as
high-stakes settings. We provide some examples of high-stakes tasks which
could justify IDs.

Making large financial transactions: Some service providers could allow AI
systems to perform financial transactions, such as making purchases, trading
stocks (khan 2024), or otherwise transferring funds. Such service providers
could require IDs for transactions involving significant sums. IDs could help
to reduce the risk of a variety of negative outcomes, by enabling service
providers to filter out potentially untrustworthy AI systems (e.g., those that
may likely malfunction and make mistaken transactions) and investigate
potential incidents.

Healthcare interactions: For better or for worse, some users are already
turning to chatbots for health advice (Lucas 2024). IDs could help users make
informed choices about such interactions. If AI systems mediate increasingly
many medical interactions (Singhal et al. 2023), IDs could become more useful
for establishing trust. In response to current trends, a variety of
organizations—such as medical device manufacturers and hospitals—could provide
plugins to connect AI systems to users’ healthcare data. Patients could have
personal assistants that communicate with the AI systems of healthcare
providers, so as to facilitate diagnosis and treatment.

##  5 ID Implementation for Centralized Deployments

We assess a basic ID system that deployers of AI systems could implement. Our
discussion in this section will be agnostic to the applications where IDs are
used (see Section 4.4 for discussion of where ID use may be warranted). In
Section 6, we explore how IDs could work in more decentralized deployments.

We assume a straightforward choice for the identifier,
system_identifier:instance_identifier , where system_identifier is the same
for all instances of a system888If the underlying model changes, the system ID
should also change. E.g., ChatGPT with a GPT-4 backend is different from
ChatGPT with a GPT-3.5 backend. and instance_identifier is unique to a
particular instance. It could be useful and straightforward to append other
details to the identifier, such as the exact time at which the instance was
created. However, we proceed with this choice for simplicity.

###  5.1 Assigning IDs

Although IDs are instance-specific, it may not be feasible for deployers to
detect the formation of all new instances. For example, a user could use
deployer-run instances $A$ and $B$ as backends for a user-run system $C$,
which integrates the outputs of $A$ and $B$ and would evade detection by
deployers. We outline some scenarios where deployers could detect new
instances and analyze considerations for assigning new IDs.

New instances: Deployers can easily generate new identifiers whenever users
create new instances (e.g., a “new chat” button).

Reloaded instances: Suppose that a deployer saves instances’ states (i.e., the
history of interaction) and provides separate functionality for users to
reload them. We tentatively suggest that deployers retain the ID of the
original instance for the reloaded instance, since some attributes (e.g.,
prior incidents, context) of the original instance could be relevant for
understanding the behaviour of the reloaded instance. Alternatively, if a
deployer does decide to assign a new identifier to the reloaded instance, a
link to the ID of the original instance would maintain accessibility to the
relevant attributes.

Output regeneration: Some deployers provide the ability for a user to
regenerate an output in response to previous input. Although not yet
implemented, deployers could additionally allow users to create multiple
branches (see Appendix A for further discussion of branches) of interactions,
based off of regenerated outputs. More concretely, a user could interact with
their instance up until the output $a_{t}$ at time $t$, regenerate to obtain
an output $b_{t}$, and continue interactions with the respective branches to
obtain outputs $(a_{t+1},b_{t+1})$, $(a_{t+2},b_{t+2})$, etc. Since the two
branches have no necessary causal impact on each other (unless they interact
with each other or modify a shared object in the world), they should have two
separate IDs, with links to the ancestor ID.

Composite systems: Software frameworks could give users the ability to create
new instances (a composite) out of a collection of existing instances (Wu et
al. 2023). If deployers run such software frameworks for users, they could
detect the formation of composites. Since the constituent instances can
continue running alongside the composite, composites should have new IDs, but
could have their IDs linked to their constituents’.

Fine-tuned systems: Some deployers provide fine-tuning functionality to users.
Since fine-tuning can change the behaviour of a system, a fine-tuned system
$A^{\prime}$ should have different system_identifier than the original system
$A$, and link to $A$’s system_identifier . Although it may be useful to
include some documentation about the fine-tuning data in an ID, user privacy
expectations could complicate such inclusion. Some deployers monitor fine-
tuning data only for abuse detection and legal compliance (ChrisHMSFT et al.
2023; OpenAI 2024a); inclusion of fine-tuning data in an ID likely requires
stronger justification.

###  5.2 Attributes

We tentatively suggest the following attributes for a deployer to include (or
link to) in an ID:

  * •

A system card

  * •

A database of incidents associated with the system

  * •

The IDs of ancestor and descendant instances

If instances of future systems persist longer in the world, it may be
appropriate to link to a (sub)database of incidents associated with the
instance. Implementing links to ancestor and descendant instances, when
feasible in the situations discussed in Section 5.1, could be a simple way to
test the utility of instance-level identification.

There are likely ways for users to avoid ancestor or descendant links. For
example, a user could manually copy the inputs of a previous instance to a new
instance, avoiding a deployer-provided reloading function. More reliable
identification of ancestors and descendants could involve more invasive
measures, which may only be appropriate in high-stakes domains. For example, a
deployer could attempt to analyze the inputs of different instances from the
same user for similarities.

###  5.3 Accessibility

Users: The ID could be accessible through a user interface and sent along with
any API requests from a user. If a deployer serves an API to a customer, who
subsequently serves an API based on the deployer’s system to another user, the
deployer would likely have to work with the customer to ensure that the end
user receives an ID (see Appendix B for further discussion).

To help to ensure that secondary parties maintain access to the ID, the
deployer could include IDs in watermarks and metadata of media outputs (i.e.,
text, images, video). Yet, it is unclear how robust such methods may be to
user removal (Zhang et al. 2023; OpenAI 2024c).

Services: When the AI queries a service, the ID should be sent along with the
request. Since requests are text-based (e.g., JSON), including the identifier
and (links to) the attributes would be straightforward.

###  5.4 Verifiability

To ensure verifiability, we explore an approach similar to that of digital
certificates:

  1. (1)

When an instance is created (e.g., when a new chat session starts), the
deployer generates an ID for the instance.

  2. (2)

Whenever the instance generates an output, the deployer attaches a digital
signature for the combined ID and the output (e.g., via hashing the output).

  3. (3)

The deployer sends the ID, signature, and output to the party interacting with
the instance.

We emphasize that an ID would be associated to outputs via digital signatures.
Hence, each output would have a different signature. Even though the
information contained an ID might not change between different outputs, the
signature verifying that the ID corresponds to an instance’s output would.
This step is crucial to prevent instance spoofing. Otherwise, any actor could
copy an ID, generate an output from an arbitrary instance, and present the
copied ID as that of the arbitrary instance.999Note that the signatures on
HTTPS certificates do not need to be based on website content. Even if an
attacker copied the details of an HTTPS certificate, they would not have the
private key necessary for establishing a secure connection with a user. A
digital signature also makes tampering and ID spoofing computationally
difficult.

Deployers could sign outputs at differing granularities. For example, given a
text output, should a deployer generate a signature for each token of the
output? As a general principle, we suggest that the to-be-signed output should
at least contain a functional unit for the party receiving the ID. For
example, suppose a service provider receives requests in a JSON string. A
signature of the ID and closing brace (“}”) of the JSON string would allow the
attacker to replace the payload of the JSON string while maintaining the
validity of the signature. On the other hand, a signature of an ID and the
entire JSON payload would avoid this failure mode. We explore some additional
examples of applying this principle:

  * •

Users: Deployers could sign the outputs generated since the last user input.

  * •

Other parties interacting with the instance: The deployer could sign the
output which it expects the party to see. For example, if deployers provide a
way for different instances to communicate with each other, the deployer
should sign the sent messages.

However, requiring signature of the output comes with a crucial caveat. If any
actor were to change or further process the output of an instance, the digital
signature would not be valid for the modified output. Much flexibility could
be lost under this implementation, if IDs were to be required in many
settings.

Part of preventing ID spoofing involves guaranteeing that the digital
signature indeed belongs to the deployer. Existing public key infrastructure
for doing so involves a trusted third party known as a certificate authority
(CA), who certifies that public keys (which allow third parties to verify
digital signatures) belong to stated parties (the deployers in this case). The
existing system of CAs could support the authenticity of the deployer’s
signature. Alternatively, a collection of civil society and industry
organizations could act as an initial CA, and delegate authority to certain
deployers after having verified their identities. Future work should explore
these options further, as well as decentralized alternatives to CAs that avoid
concentrated points of failure (Sporny et al. 2022, 2024).

Figure 2: This figure illustrates the process in Section 5.4 of creating IDs
that are verifiable.

##  6 ID Implementation for Decentralized Deployments

In some settings where ID use may be warranted (e.g., high-stakes settings;
see Section 4.4), users themselves may run AI systems. We assess an
implementation of IDs without deployers as an intermediary, which would
require more involvement from users, CAs, and supporting software. As in
Section 5, we adopt system_identifier:instance_identifier as the identifier.

###  6.1 A Basic Design

This design roughly follows that in Section 5, except that we will have AI
frameworks—software tools such as the transformers library (Wolf et al. 2020),
AutoGen (Wu et al. 2023), and AutoGPT (Richards 2023) that facilitate running
AI systems—facilitate ID implementation. Similarly, automatic certificate
issuance from Let’s Encrypt played an important role in the widespread
adoption of HTTPS (Encrypt 2024). AI frameworks will require some additional
functionality that does not yet exist, as we explain below.

  1. (1)

The user registers for an account with a CA.

  2. (2)

The AI framework lets a user log into their CA account.

  3. (3)

Whenever a user starts an instance with an AI framework, the framework
requests an identifier from the CA.

  4. (4)

For each unit of output from the instance, the AI framework encrypts the
output and sends it to the CA.

  5. (5)

The CA produces a digital signature for the combined identifier and hashed
output.

  6. (6)

The AI framework makes the ID and signature accessible with the output.

As in Section 5.4, signature of the output would prevent attackers from
attaching a given ID to arbitrary instances. To ensure user privacy, the CA
would only sign a hashed output. Considerations for which outputs to sign
(e.g., signing the entire JSON string of a service request) from Section 5.4
apply here as well.

As a potential way to ensure that CAs do not sign arbitrary outputs from users
(e.g., those not even generated by an AI system), CAs could stipulate the use
of approved AI frameworks. Techniques such as remote attestation and code
signing could be useful for such requirements.101010Such techniques would
ideally only apply to the loading and use of relevant library packages, rather
than all parts of a user’s code base. Yet, only permitting the use of approved
frameworks could have negative consequences, especially if IDs were required
in many settings. Users would not be able to modify frameworks as they see
fit. Bottlenecks in the approval process could also mean that users are
pressured to use outdated frameworks, or frameworks that do not provide
desired functionality.

Figure 3: This figure illustrates a potential process for ensuring
verifiability of identifiers in decentralized deployments. See further details
in Section 6.1.

Information to include: Some AI frameworks, such as HuggingFace, integrate
with repositories or marketplaces where users can download different AI
systems for use. Frameworks could link to the documentation that is often
included in a system’s repository. However, unlike in Section 5.2, there is no
guarantee that a repository would contain a system card or an incident
database. If the AI framework provides a dedicated function for an ancestor
instance to create descendant instances, the AI framework could establish ID
links between the ancestor and the descendant. Otherwise, such links seem
difficult to establish.

Accessibility: An AI framework has a similar role to that of a deployer in
Section 5.3. The framework should ensure that any outputs of the system,
including audiovisual outputs and queries to services, should contain the ID,
whether embedded as a watermark or as metadata.

Verifiability: Verifiability would apply to the identifier, but not
necessarily to other information attached to the ID. On HuggingFace for
example, any anonymous user can upload a system with arbitrary documentation.
Even so, such lack of verifiability may encourage parties to interact with
instances of well-known systems, rather than instances of systems uploaded by
anonymous users. This preference could incentivize users to use well-known
systems and weaken the need for additional verifiability mechanisms.

###  6.2 Assigning IDs

In theory, AI frameworks could detect instance creation similarly to how
deployers could in Section 5.1, and request new identifiers from CAs as
appropriate. The main practical limitation is the flexibility that users
have—by virtue of writing and running their own code—to create instances in
arbitrary ways. It may not be feasible for an AI framework’s developer to
account for all potential instance creation methods in advance, and decide how
to assign IDs accordingly.

##  7 Limitations

We discuss limitations of IDs and of our analysis.

###  7.1 Failure to Inform

Parties could ignore IDs, just as internet users sometimes ignore indicators
of untrustworthiness (Norris and Brookes 2021). Software—or future AI
assistants (Gabriel et al. 2024)—could make informed decisions for users in
some circumstances, just as browsers reject websites with invalid HTTPS
certificates. However, it remains unclear what information would justify
rejecting an interaction with an AI system. Other parties could misunderstand
or misinterpret an ID’s attributes. For example, it can difficult to assess
the external validity of evaluation results (Fourrier et al. 2023; Weidinger
et al. 2023).

Moreover, ID attributes could be inaccurate. For example, deployers could
unintentionally link to an incorrect version of system documentation, or may
simply fail to update the documentation when a system is updated. OpenAI has
occasionally released new versions of GPT-4,111111See
https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4. but has not
updated the GPT-4 system card (OpenAI 2023b) since the initial release, as of
April 2024. Regular audits (Raji et al. 2020; Sharkey et al. 2023) could
verify the accuracy of attributes.

###  7.2 Bypassing IDs

As with other types of real-world identity systems, IDs for AI systems cannot
wholly prevent undesirable behavior. IDs could be lent, or the benefits
obtained from ID use—such as purchase of a good—could be transferred. Users
may hesitate to lend IDs since the borrower could inflict lasting reputational
damage on the ID. Yet, similar to the purchase of alcohol for minors, it seems
difficult to track or prevent transfer of goods. Future work could assess how
likely and how damaging such transfer could be.

###  7.3 Limited Adoption

In decentralized deployments, the absence of supporting AI frameworks could
hinder users from implementing IDs. Without such software, providers may be
hesitant about requiring IDs if they wish to encourage use of their services.
Consequently, there may not be enough demand to spur the development of
supporting software. One way to resolve these problems is to focus on soft ID
incentives with well-resourced actors. For example, service providers could
experiment with ID incentives for companies, who could pilot initial
implementations of supporting AI frameworks.

In the implementations we discussed, verifiability requires a signature on
both the ID and the output. Modification of the output would hence invalidate
the ID. Users could be hesitant to adopt IDs given this lack of flexibility.
As well, many generated IDs could become invalidated simply because users
unintentionally modify the outputs (e.g., a user accidentally removes some
punctuation from generated text). ID requirements could be less onerous when
outputs are unlikely to be modified, such as requests from an AI system to a
service. Yet, absent technical implementations that can ensure verifiability
while permitting modification, this limitation suggests that any initial ID
requirements should be minimal.

###  7.4 Privacy and Security Risks

Although an identifier by itself would not reveal anything about its user, the
user’s privacy could still be compromised in a number of ways. First,
information about the instance could reveal user details. For example,
timestamps and details of an instance’s activities—as might be contained in
incident reports—could help a third-party to identify users. The existence of
IDs may engender pressure to attach additional information, which could
facilitate further user identification. For example, service abuse could
potentially be correlated with certain activity patterns, which could motivate
corresponding data collection. Second, if the deployer maintains an internal
database linking a system’s ID to the corresponding user account, other
parties could obtain and misuse the information. Security vulnerabilities
could allow attackers to obtain the database. Overreaching governments could
force deployers to reveal the users of particular systems.

In the context of the technical implementations we discussed, the
trustworthiness and security of the CAs limits the verifiability of IDs. Apart
from external threats, high-stakes situations could create incentives for
collusion between authorities and key owners. With a compromised CA, a
malicious actor could issue IDs for any AI system. For example, an attacker
could assume the ID of a trusted AI system so as to interact with certain
parties or obtain higher privileges with a service.

Decentralized alternatives to digital signature management could help to avoid
these problems. Specifically, smart contracts deployed on distributed ledgers
could potentially issue digital signatures without the need for a centralized
authority (Omote 2023). The smart contracts would be publicly verifiable and
are immutable as long as the integrity and availability of the distributed
ledger as a whole, in contrast to a single trusted authority, is ensured. Yet,
widespread adoption of such approaches is currently limited. Challenges
include a lack of usable and tested software products, scalability and compute
demands, and technical issues inherent in real-world implementations of
distributed ledgers.

###  7.5 Broader Societal Consequences

Since IDs could enable anybody to identify and make decisions about particular
instances, their use could have broad societal consequences.

IDs with additional user information could enable potentially harmful ranking
of users. Actors implementing IDs may wish to include such information if it
is correlated with undesirable outcomes, such as fraud. Yet, the existence of
confounders could lead to instances of some users being unfairly denied access
to services, similar to how toxicity classifiers can biased against African
American English (Sap et al. 2019). Even if the relationship between
information and undesirable outcome was causal, taking such information into
account could still be unjustly discriminatory.

IDs could also enable influence over particular instances and their users.
Even if an ID does not contain information about an instance’s interactions,
external actors could record particular IDs have been involved in
interactions, just as cookies can track user activity across websites.
Instances that are active for long periods of time could reveal much
information about a user’s preferences. Actors interacting with instances
could try to influence their behaviour through means such as prompting.
Advertisers and businesses could try to get instances to purchase or recommend
their products or services for their corresponding users. Political actors
could try to get instances to influence the voting behaviour of their users.
Governments could attempt to enforce rules upon particular instances. Such
influence could be strong, especially if personal AI assistants become much
more widespread and central to daily interactions (Gabriel et al. 2024). The
specific impacts of this influence deserve further study.

Finally, IDs could create a separate digital channel for interactions between
trusted AI systems. The vast majority of digital interactions could in the
future be between AI systems, especially if future AI assistants handle most
of a user’s activities (Gabriel et al. 2024). If IDs enable reliable
interactions, users may prefer their AI systems to interact only with AI
systems that present IDs. Much digital activity could take place between
networks of ID-bearing AI instances that trust (and can verify) each other.
Entities that trust each other may be able to engage in more productive
interactions. As a corollary, instances without IDs may become severely
disadvantaged, relative to instances within ID-bearing networks. To avoid such
disadvantage, users may have no choice but to submit to the potential negative
consequences of IDs we have discussed heretofore in this section. On balance,
the consequences of a separate digital channel for AI systems remain unclear.

##  8 Related Work

Besides real-world examples of IDs and work on documenting AI system that we
highlighted in Section 1, we discuss two additional lines of related work.

A growing line of work is concerned with building digital infrastructure to
structure how AI systems, especially agents (Chan et al. 2023; Shavit et al.
2023; Gabriel et al. 2024), behave and interact. Such infrastructure can have
a variety of goals. Patil et al. (2024) build a runtime for LLM agents to
enable human validation and reversal of actions. Marro (2024) sketches a
protocol for communication between LLMs; the protocol relies on documents to
specify structured rules, with natural language as a fallback. (Sun et al.
2023) argue that decentralized commitment devices will be necessary to allow
agents to coordinate with each other. IDs are a form of infrastructure that
enable informed decision-making about AI interactions.

The technical implementations we assess take inspiration from several Internet
and security protocols. HTTPS (Fielding, Nottingham, and Reschke 2022) uses
digital certificates (Cooper et al. 2008) to allow users to verify the
identity of an accessed website. If the certificate is valid, a user’s browser
uses the information contained in the certificate to establish a secure
connection (Rescorla 2018). Websites must obtain such certificates from CAs
(Cooper et al. 2008), who verify the identity of the website owner and issue
the certificate upon successful identification. Identity verification depends
upon public key cryptography (Barnes et al. 2019), which allows a party to
prove ownership of a given, public identifier, otherwise known as a public
key. Let’s Encrypt is a non-profit CA which automates the process of issuing
certificates (Encrypt 2024). HTTPS everywhere was a browser plug-in that
forced the usage of the HTTPS version of websites whenever it was available;
this functionality is now default in modern web browsers (Hancock 2021). To
design further digital infrastructure for AI systems, other internet and
security protocols may be a fruitful source of inspiration.

##  9 Conclusion

To inform crucial decisions about AI interactions, we proposed a framework for
creating IDs for AI systems. IDs can vary in their attributes, how accessible
they are to various parties, and to what extent they are verifiable. There
could be significant demand for IDs from governments, service providers, and
users, particularly when systems engage in high-stakes tasks. These actors
also have means to incentivize ID usage. ID implementation seems feasible for
deployers, but implementation in decentralized settings will require further
study.

More research is required to understand and address the potential risks of
IDs. First, security and privacy risks should be addressed, such as the
potential for certificate authorities to become compromised. Second, it is
unclear under what circumstances IDs should be accompanied by user
identification. Third, the broader societal consequences of introducing IDs
require further study.

Limited experimentation with IDs seems justified given their potential
benefits and risks. IDs seem most warranted in high-stakes settings, where
certain actors could experiment with incentives for ID use. For instance,
service providers that provide plugins in potentially high-stakes settings
(e.g., plug-ins that allow AI systems to make financial transactions) could
experiment with rate-limiting instances without IDs, while still allowing such
instances to access services. Deployers could begin to implement the technical
infrastructure for IDs.

IDs have facilitated essential functions across a variety of domains. Incident
investigation, allocation of liability, and establishment of trust would be
more difficult without IDs for products, organizations, and software systems.
As AI systems become increasingly prevalent, deficiencies in such functions
could result in more severe consequences for society. If implemented well, IDs
for AI systems could enable mechanisms to avert such consequences and navigate
this emerging world.

## Appendix

##  Appendix A A More Detailed Definition of an Instance

A system is a model (e.g., a set of parameters), along with software used to
run the model and provide other user functions. For example, ChatGPT with
GPT-4 as a backend comprises both the weights of (a particular version of)
GPT-4 and the software used to facilitate chat interactions (as opposed to an
“autocomplete” function, as in the OpenAI playground).

To understand the difference between a system and an instance, consider the
information that would be useful to attach to a system-specific ID, so as to
inform decisions about interacting with the system. Such information could not
depend (too much) on a system’s inputs, since different inputs might lead to
different behavior. In other words, the information should ideally be valid
regardless of a user’s interaction history. Represented visually, the
information would have to be valid for both top and bottom flow charts in
Figure 4.

Figure 4: This figure depicts two users that use the same system (e.g., both
use ChatGPT with the GPT-4 backend). System-specific information attached to
an ID should be useful to both users. By loaded, we mean that the system is
ready to accept inputs for the first time.

An instance is an abstraction that corresponds to a (initial) user (which
could be a human, a group of humans, a software system, etc) and an
interaction history. In Figure 5, we provide a visual depiction of two
instances that interact with each other.

We define instances in this way so that instance-specific IDs can take into
account information that is causally relevant to a given
interaction.121212There may be information that is not causally relevant, but
which may still help to predict the instance’s behavior, such as model
evaluations or the behaviour for other instances. It remains unclear how much
of this information to attach to a given instance’s ID. For example, in the
top half of Figure 5, information about instances $A$’s earlier interactions
(such as malfunctions) may be useful when instance $B$ interacts with instance
$A$. Actions taken by another instance are not, by default, causally relevant
to the behaviour of another instance. Yet, instances can affect each other
through direct interaction or on changing shared states of the world (e.g.,
using the same bank account).

Figure 5: An instance is an abstraction that corresponds to a creation event,
where a system is loaded for a user, and an interaction history. Instance-
specific IDs could help to inform interaction decisions. For example,
information about instances $A$’s earlier interactions (such as malfunctions)
may be useful when instance $B$ interacts with instance $A$.

Since AI systems can be copied and combined, there are some additional edge
cases for how to define instances. In a branch (see Figure 6(a)), Two
instances share a past context. For example, a regeneration of a response
(possible in ChatGPT, for one) creates two branches. Since branches can behave
independently, we suggest treating branches as separate instances. In a merge
(see Figure 6(b)), two instances come together to form a new system. For
example, suppose software $S$ accepts inputs from separate users, engages the
instances of two users in a debate based on the inputs, and finally outputs a
result to both users. Users $A$ and $B$ could use $S$ to create a new instance
$AB$, based on their separate instances. As with branching, we treat $AB$ as a
separate instance because the users could continue running their original
instances in tandem with $AB$.

(a) In this example of a branch, the inputs of instance $A$ are copied for
user $C$ to create a new instance $C$.

(b) In this example of a merge, the instances $A$ and $B$ are merged into a
new instance $AB$, which has access to (potentially a subset of) the inputs of
$A$ and $B$.

Figure 6: We illustrate how we define instances in the event that they are
copied or combined.

If an instance is the result of a branch or a merge, it may be useful for the
instance’s ID to contain information about ancestor instances. In Figure 7(a),
since the context of instance $A$ is copied into instance $C$, any incidents
associated with $A$ may be helpful for user $B$’s interaction decisions.
Similarly, in Figure 7(b), information about instances $A$ and $B$ may be
relevant for user $C$’s decision about whether or not to trust the outputs of
instance $AB$. Nonetheless, information from ancestor instances could be
excluded for a variety of reasons, such as user privacy. Furthermore, it could
be possible for information from descendant instances, or instances in another
branch, to be relevant for the behavior of a given instance. For example,
incidents with user $B$ from time $t+1$ may be informative for user $A$ that
interacted with an instance at time $t$. We leave further analysis of how to
delimit the sources of ID information to future work.131313Links to ancestor
IDs make other potential definitions of an instance equivalent, from the
perspective of the functions of an ID. For example, in Figure 6(a) we could
have defined instance $C$ as encompassing interaction history up to instance
$A$. This definition would not change the causally relevant information with
respect to instance $C$ in Figure 7(a).

(a) In the event of a branch, a user $B$ that interacts with instance $C$ may
find information about instance $A$ (e.g., instance $A$ was involved in an
incident) to be useful.

(b) In the event of a merge, a user $C$ that interacts with instance $AB$ may
find information (e.g., instance $B$ is not robust) about either instances $A$
or $B$ to be helpful.

Figure 7: We visually represent the information that an ID for an instance
could take into account.

As we have defined it, an instance is causally independent of other instances
unless it interacts with them, such as by processing their outputs. This
causal independence is crucial to IDs, as we discussed above. Yet, the
abstraction of an instance could be leaky depending on system implementation
details. For example, if a single GPU handles the operation of two separate
instances, implementation errors or optimization strategies (e.g., approximate
matrix multiplication) could potentially result in computational interference
between the instances.

##  Appendix B Maintaining the Accessibility of IDs

Suppose a primary party $P$ receives an output and ID from an AI system. If it
is desirable for secondary parties to see the ID, $P$ may have additional
responsibilities. Some potential situations follow.

Chains of deployers: There may be multiple deployers involved in the operation
of an AI system. For example, deployer $A$ could provide an API for system $X$
to customer $B$, who creates system $Y$ and serves $Y$ to user $C$. To ensure
that $C$ can access the $X$’s ID, $B$ should link to the ID provided by $A$,
or should include the information in the ID of $Y$.

Service providers: If a service provider receives an ID, it should ensure that
those who observe or are affected by AI system’s actions can also see the ID.
For instance, a bank that facilitates a financial transaction from an AI
system should make the latter’s ID known to the other party of the
transaction.

The provider may have to work with other parties to maintain visibility of the
ID. As an example, consider a service that allows an AI system to post on a
social media platform. The provider may have written the software that allows
the AI system to interact with the platform’s API, but the social media
company develops the API and manages the platform. While the provider could
include the AI system’s ID as text within the post, doing so may be infeasible
because of text limits or would otherwise be obstructive to the user
experience. Rather, the provider should work with the social media company to
include the ID as metadata, readily accessible to the user through a visible
icon.

Users: Users could receive outputs with attached IDs, whether as a watermark
or metadata. To counter inadvertent removal of IDs, deployers could add labels
to inform users. For example, Facebook and Instagram automatically add labels
indicating AI origin when a user shares a photo generated with the Meta AI
feature (Bickert 2024). To counter intentional removal of IDs, better
watermarking techniques may be required. Yet, it remains unclear how effective
watermarking may ultimately be (Zhang et al. 2023).

## References

  * noa (2022) 2022\.  12 CFR § 1026.25 - Record retention.  _Federal Register_.
  * Adept (2022) Adept. 2022.  ACT-1: Transformer for Actions.
  * Agarwal et al. (2024) Agarwal, R.; Singh, A.; Zhang, L. M.; Bohnet, B.; Chan, S.; Anand, A.; Abbas, Z.; Nova, A.; Co-Reyes, J. D.; Chu, E.; Behbahani, F.; Faust, A.; and Larochelle, H. 2024.  Many-Shot In-Context Learning.  ArXiv:2404.11018 [cs].
  * Anthropic (2024a) Anthropic. 2024a.  Introducing the next generation of Claude.
  * Anthropic (2024b) Anthropic. 2024b.  Tool use (function calling).
  * Bai et al. (2022) Bai, Y.; Kadavath, S.; Kundu, S.; Askell, A.; Kernion, J.; Jones, A.; Chen, A.; Goldie, A.; Mirhoseini, A.; McKinnon, C.; Chen, C.; Olsson, C.; Olah, C.; Hernandez, D.; Drain, D.; Ganguli, D.; Li, D.; Tran-Johnson, E.; Perez, E.; Kerr, J.; Mueller, J.; Ladish, J.; Landau, J.; Ndousse, K.; Lukosuite, K.; Lovitt, L.; Sellitto, M.; Elhage, N.; Schiefer, N.; Mercado, N.; DasSarma, N.; Lasenby, R.; Larson, R.; Ringer, S.; Johnston, S.; Kravec, S.; Showk, S. E.; Fort, S.; Lanham, T.; Telleen-Lawton, T.; Conerly, T.; Henighan, T.; Hume, T.; Bowman, S. R.; Hatfield-Dodds, Z.; Mann, B.; Amodei, D.; Joseph, N.; McCandlish, S.; Brown, T.; and Kaplan, J. 2022.  Constitutional AI: Harmlessness from AI Feedback.  ArXiv:2212.08073 [cs].
  * Barnes et al. (2019) Barnes, R.; Hoffman-Andrews, J.; McCarney, D.; and Kasten, J. 2019.  Automatic certificate management environment (acme).  Technical report.
  * BBC (2021) BBC. 2021.  Post Office scandal explained: What the Horizon saga is all about.
  * Bickert (2024) Bickert, M. 2024.  Our Approach to Labeling AI-Generated Content and Manipulated Media.
  * Bommasani et al. (2023) Bommasani, R.; Soylu, D.; Liao, T. I.; Creel, K. A.; and Liang, P. 2023.  Ecosystem Graphs: The Social Footprint of Foundation Models.  ArXiv:2303.15772 [cs].
  * Buiten, de Streel, and Peitz (2023) Buiten, M.; de Streel, A.; and Peitz, M. 2023.  The law and economics of AI liability.  _Computer Law & Security Review_, 48: 105794.
  * Buiten (2024) Buiten, M. C. 2024.  Product liability for defective AI.  _European Journal of Law and Economics_.
  * Buterin (2023) Buterin, V. 2023.  What do I think about biometric proof of personhood?
  * C2PA (2023) C2PA. 2023.  C2PA Technical Specification.
  * Chan et al. (2024) Chan, A.; Ezell, C.; Kaufmann, M.; Wei, K.; Hammond, L.; Bradley, H.; Bluemke, E.; Rajkumar, N.; Krueger, D.; Kolt, N.; Heim, L.; and Anderljung, M. 2024.  Visibility into AI Agents.  ArXiv:2401.13138 [cs].
  * Chan et al. (2023) Chan, A.; Salganik, R.; Markelius, A.; Pang, C.; Rajkumar, N.; Krasheninnikov, D.; Langosco, L.; He, Z.; Duan, Y.; Carroll, M.; Lin, M.; Mayhew, A.; Collins, K.; Molamohammadi, M.; Burden, J.; Zhao, W.; Rismani, S.; Voudouris, K.; Bhatt, U.; Weller, A.; Krueger, D.; and Maharaj, T. 2023.  Harms from Increasingly Agentic Algorithmic Systems.  In _Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency_ , FAccT ’23, 651–666. New York, NY, USA: Association for Computing Machinery.  ISBN 9798400701924.
  * ChrisHMSFT et al. (2023) ChrisHMSFT; PatrickFarley; mrbullwinkle; eric urban; and aahill. 2023.  Data, privacy, and security for Azure OpenAI Service - Azure AI services.
  * Cloudflare (2024) Cloudflare. 2024.  What is rate limiting? | Rate limiting and bots.
  * Cohen, Bitton, and Nassi (2024) Cohen, S.; Bitton, R.; and Nassi, B. 2024.  Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applications.  ArXiv:2403.02817 [cs].
  * Cooper et al. (2008) Cooper, D.; Santesson, S.; Farrell, S.; Boeyen, S.; Housley, R.; and Polk, W. 2008.  Internet X.509 public key infrastructure certificate and certificate revocation list (CRL) profile.  Technical report.
  * De La Garza (2020) De La Garza, A. 2020.  States’ Automated Systems Are Trapping Citizens in Bureaucratic Nightmares With Their Lives on the Line.  _TIME_.
  * Egan and Heim (2023) Egan, J.; and Heim, L. 2023.  Oversight for Frontier AI through a Know-Your-Customer Scheme for Compute Providers.  ArXiv:2310.13625 [cs].
  * Encrypt (2024) Encrypt, L. 2024.  Let’s Encrypt Stats - Let’s Encrypt.
  * Epoch (2023) Epoch. 2023.  Key trends and figures in Machine Learning.
  * Erdil and Besiroglu (2023) Erdil, E.; and Besiroglu, T. 2023.  Algorithmic progress in computer vision.  _eprint: 2212.05153.
  * Fielding, Nottingham, and Reschke (2022) Fielding, R.; Nottingham, M.; and Reschke, J. 2022.  RFC 9110: HTTP semantics.
  * Fourrier et al. (2023) Fourrier, C.; Habib, N.; Launay, J.; and Wolf, T. 2023.  What’s going on with the Open LLM Leaderboard?
  * Furuta et al. (2024) Furuta, H.; Matsuo, Y.; Faust, A.; and Gur, I. 2024.  Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web.  ArXiv:2311.18751 [cs].
  * Gabriel et al. (2024) Gabriel, I.; Manzini, A.; Keeling, G.; Hendricks, L. A.; Rieser, V.; Iqbal, H.; Tomašev, N.; Ktena, I.; Kenton, Z.; Rodriguez, M.; El-Sayed, S.; Brown, S.; Akbulut, C.; Trask, A.; Hughes, E.; Bergman, A. S.; Shelby, R.; Marchal, N.; Griffin, C.; Mateos-Garcia, J.; Weidinger, L.; Street, W.; Lange, B.; Ingerman, A.; Lentz, A.; Enger, R.; Barakat, A.; Krakovna, V.; Siy, J. O.; Kurth-Nelson, Z.; McCroskery, A.; Bolina, V.; Law, H.; Shanahan, M.; Alberts, L.; Balle, B.; de Haas, S.; Ibitoye, Y.; Dafoe, A.; Goldberg, B.; Krier, S.; Reese, A.; Witherspoon, S.; Hawkins, W.; Rauh, M.; Wallace, D.; Franklin, M.; Goldstein, J. A.; Lehman, J.; Klenk, M.; Vallor, S.; Biles, C.; Morris, M. R.; King, H.; Arcas, B. A. y.; Isaac, W.; and Manyika, J. 2024.  The Ethics of Advanced AI Assistants.  ArXiv:2404.16244 [cs].
  * Gebru et al. (2021) Gebru, T.; Morgenstern, J.; Vecchione, B.; Vaughan, J. W.; Wallach, H.; Iii, H. D.; and Crawford, K. 2021.  Datasheets for datasets.  _Communications of the ACM_ , 64(12): 86–92.
  * Gilbert et al. (2023) Gilbert, T. K.; Lambert, N.; Dean, S.; Zick, T.; Snoswell, A.; and Mehta, S. 2023.  Reward Reports for Reinforcement Learning.  In _Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society_ , AIES ’23, 84–130. New York, NY, USA: Association for Computing Machinery.  ISBN 9798400702310.
  * Greshake et al. (2023) Greshake, K.; Abdelnabi, S.; Mishra, S.; Endres, C.; Holz, T.; and Fritz, M. 2023.  Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection.  ArXiv:2302.12173 [cs].
  * Gur et al. (2024) Gur, I.; Furuta, H.; Huang, A.; Safdari, M.; Matsuo, Y.; Eck, D.; and Faust, A. 2024.  A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis.  ArXiv:2307.12856 [cs].
  * Gur et al. (2023) Gur, I.; Nachum, O.; Miao, Y.; Safdari, M.; Huang, A.; Chowdhery, A.; Narang, S.; Fiedel, N.; and Faust, A. 2023.  Understanding HTML with Large Language Models.  ArXiv:2210.03945 [cs].
  * Hancock (2021) Hancock, A. 2021.  HTTPS Is Actually Everywhere.
  * Ho et al. (2024) Ho, A.; Besiroglu, T.; Erdil, E.; Owen, D.; Rahman, R.; Guo, Z. C.; Atkinson, D.; Thompson, N.; and Sevilla, J. 2024.  Algorithmic progress in language models.
  * Hoffmann et al. (2022) Hoffmann, J.; Borgeaud, S.; Mensch, A.; Buchatskaya, E.; Cai, T.; Rutherford, E.; Casas, D. d. L.; Hendricks, L. A.; Welbl, J.; Clark, A.; Hennigan, T.; Noland, E.; Millican, K.; Driessche, G. v. d.; Damoc, B.; Guy, A.; Osindero, S.; Simonyan, K.; Elsen, E.; Rae, J. W.; Vinyals, O.; and Sifre, L. 2022.  Training Compute-Optimal Large Language Models.  ArXiv:2203.15556 [cs].
  * Jimenez et al. (2024) Jimenez, C. E.; Yang, J.; Wettig, A.; Yao, S.; Pei, K.; Press, O.; and Narasimhan, K. 2024.  SWE-bench: Can Language Models Resolve Real-World GitHub Issues?  ArXiv:2310.06770 [cs].
  * khan (2024) khan, D. 2024.  danikhan632/Auto-GPT-AlpacaTrader-Plugin.  Original-date: 2023-04-25T16:51:50Z.
  * Kinniment et al. (2023) Kinniment, M.; Sato, L. J. K.; Du, H.; Goodrich, B.; Hasin, M.; Chan, L.; Miles, L. H.; Lin, T. R.; Wijk, H.; Burget, J.; Ho, A.; Barnes, E.; and Christiano, P. 2023.  Evaluating Language-Model Agents on Realistic Autonomous Tasks.
  * Kolt (2024) Kolt, N. 2024.  Governing AI Agents.
  * Korenhof et al. (2014) Korenhof, P.; Koning, M.; Alpár, G.; and Hoepman, J. 2014.  The ABC of ABC: An analysis of attribute-based credentials in the light of data protection, privacy and identity.  _Internet, Law and Politics_ , 10: 357–374.  Place: Barcelona Publisher: Huygens Editorial.
  * Liu et al. (2024) Liu, A.; Pan, L.; Lu, Y.; Li, J.; Hu, X.; Wen, L.; King, I.; and Yu, P. S. 2024.  A Survey of Text Watermarking in the Era of Large Language Models.  ArXiv:2312.07913 [cs].
  * Liu et al. (2023) Liu, X.; Yu, H.; Zhang, H.; Xu, Y.; Lei, X.; Lai, H.; Gu, Y.; Ding, H.; Men, K.; Yang, K.; Zhang, S.; Deng, X.; Zeng, A.; Du, Z.; Zhang, C.; Shen, S.; Zhang, T.; Su, Y.; Sun, H.; Huang, M.; Dong, Y.; and Tang, J. 2023.  AgentBench: Evaluating LLMs as Agents.  ArXiv:2308.03688 [cs].
  * Lucas (2024) Lucas, J. 2024.  The teens making friends with AI chatbots.  _The Verge_.
  * Marro (2024) Marro, S. 2024.  A Protocol Sketch For LLM Communication.
  * Mialon et al. (2023) Mialon, G.; Fourrier, C.; Swift, C.; Wolf, T.; LeCun, Y.; and Scialom, T. 2023.  GAIA: a benchmark for General AI Assistants.
  * Microsoft (2023) Microsoft. 2023.  Introduction to Microsoft Entra Verified ID.
  * Mitchell et al. (2019) Mitchell, M.; Wu, S.; Zaldivar, A.; Barnes, P.; Vasserman, L.; Hutchinson, B.; Spitzer, E.; Raji, I. D.; and Gebru, T. 2019.  Model Cards for Model Reporting.  In _Proceedings of the Conference on Fairness, Accountability, and Transparency_ , 220–229.  ArXiv:1810.03993 [cs].
  * Norris and Brookes (2021) Norris, G.; and Brookes, A. 2021.  Personality, emotion and individual differences in response to online fraud.  _Personality and Individual Differences_ , 169: 109847.
  * Omote (2023) Omote, K. 2023.  Towards Decentralized Autonomous Digital Signatures Using Smart Contracts.  In _2023 International Conference on Information Networking (ICOIN)_ , 230–235.  ISSN: 1976-7684.
  * OpenAI (2023a) OpenAI. 2023a.  ChatGPT plugins.
  * OpenAI (2023b) OpenAI. 2023b.  GPT-4 Technical Report.  ArXiv:2303.08774 [cs].
  * OpenAI (2024a) OpenAI. 2024a.  Enterprise privacy.
  * OpenAI (2024b) OpenAI. 2024b.  Introducing the GPT Store.
  * OpenAI (2024c) OpenAI. 2024c.  Understanding the source of what we see and hear online.
  * Parliament (2024) Parliament, E. 2024.  Artificial Intelligence Act.
  * Patil et al. (2024) Patil, S. G.; Zhang, T.; Fang, V.; C., N.; Huang, R.; Hao, A.; Casado, M.; Gonzalez, J. E.; Popa, R. A.; and Stoica, I. 2024.  GoEX: Perspectives and Designs Towards a Runtime for Autonomous LLM Applications.  ArXiv:2404.06921 [cs].
  * Raji et al. (2022) Raji, I. D.; Kumar, I. E.; Horowitz, A.; and Selbst, A. 2022.  The Fallacy of AI Functionality.  In _2022 ACM Conference on Fairness, Accountability, and Transparency_ , 959–972. Seoul Republic of Korea: ACM.  ISBN 978-1-4503-9352-2.
  * Raji et al. (2020) Raji, I. D.; Smart, A.; White, R. N.; Mitchell, M.; Gebru, T.; Hutchinson, B.; Smith-Loud, J.; Theron, D.; and Barnes, P. 2020.  Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing.  In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_ , FAT* ’20, 33–44. New York, NY, USA: Association for Computing Machinery.  ISBN 978-1-4503-6936-7.
  * Rescorla (2018) Rescorla, E. 2018.  The transport layer security (TLS) protocol version 1.3.  Technical report.
  * Richards (2023) Richards, T. B. 2023.  Auto-GPT: An Autonomous GPT-4 Experiment.  Original-date: 2023-03-16T09:21:07Z.
  * Sap et al. (2019) Sap, M.; Card, D.; Gabriel, S.; Choi, Y.; and Smith, N. A. 2019.  The Risk of Racial Bias in Hate Speech Detection.  In Korhonen, A.; Traum, D.; and Màrquez, L., eds., _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , 1668–1678. Florence, Italy: Association for Computational Linguistics.
  * Shanahan, McDonell, and Reynolds (2023) Shanahan, M.; McDonell, K.; and Reynolds, L. 2023.  Role-Play with Large Language Models.  ArXiv:2305.16367 [cs].
  * Sharkey et al. (2023) Sharkey, L.; Clíodhna Ní Ghuidhir; Dan Braun; Jérémy Scheurer; Mikita Balesni; Lucius Bushnaq; Charlotte Stix; and Marius Hobbhahn. 2023.  A Causal Framework for AI Regulation and Auditing.
  * Shavit et al. (2023) Shavit, Y.; Agarwal, S.; Brundage, M.; Adler, S.; O’Keefe, C.; Campbell, R.; Lee, T.; Mishkin, P.; Eloundou, T.; Hickey, A.; Slama, K.; Ahmad, L.; McMillan, P.; Beutel, A.; Passos, A.; and Robinson, D. G. 2023.  Practices for Governing Agentic AI Systems.
  * Shet (2014) Shet, V. 2014.  Are you a robot? Introducing ”No CAPTCHA reCAPTCHA”.
  * Shevlane et al. (2023) Shevlane, T.; Farquhar, S.; Garfinkel, B.; Phuong, M.; Whittlestone, J.; Leung, J.; Kokotajlo, D.; Marchal, N.; Anderljung, M.; Kolt, N.; Ho, L.; Siddarth, D.; Avin, S.; Hawkins, W.; Kim, B.; Gabriel, I.; Bolina, V.; Clark, J.; Bengio, Y.; Christiano, P.; and Dafoe, A. 2023.  Model evaluation for extreme risks.
  * Significant-Gravitas (2024) Significant-Gravitas. 2024.  Auto-GPT-Plugins.
  * Singhal et al. (2023) Singhal, K.; Tu, T.; Gottweis, J.; Sayres, R.; Wulczyn, E.; Hou, L.; Clark, K.; Pfohl, S.; Cole-Lewis, H.; Neal, D.; Schaekermann, M.; Wang, A.; Amin, M.; Lachgar, S.; Mansfield, P.; Prakash, S.; Green, B.; Dominowska, E.; Arcas, B. A. y.; Tomasev, N.; Liu, Y.; Wong, R.; Semturs, C.; Mahdavi, S. S.; Barral, J.; Webster, D.; Corrado, G. S.; Matias, Y.; Azizi, S.; Karthikesalingam, A.; and Natarajan, V. 2023.  Towards Expert-Level Medical Question Answering with Large Language Models.
  * Sporny et al. (2024) Sporny, M.; Longley, D.; Chadwick, D.; and Steele, O. 2024.  Verifiable Credentials Data Model v2.0.  Technical report, W3C.
  * Sporny et al. (2022) Sporny, M.; Longley, D.; Reed, D.; Sabadello, M.; Steele, O.; and Allen, C. 2022.  Decentralized Identifiers (DiDs) v1.0.  Technical report, W3C.
  * Stumpp (2023) Stumpp, L.-F. 2023.  Achieving Code Execution in MathGPT via Prompt Injection | MITRE ATLAS™.
  * Sun et al. (2023) Sun, X.; Crapis, D.; Stephenson, M.; Monnot, B.; Thiery, T.; and Passerat-Palmbach, J. 2023.  Cooperative AI via Decentralized Commitment Devices.  ArXiv:2311.07815 [cs].
  * Tao et al. (2023) Tao, H.; T V, S.; Shlapentokh-Rothman, M.; and Hoiem, D. 2023.  WebWISE: Web Interface Control and Sequential Exploration with Large Language Models.  ArXiv:2310.16042 [cs].
  * Trask et al. (2023) Trask, A.; Sukumar, A.; Kalliokoski, A.; Farkas, B.; Ezenwaka, C.; Popa, C.; Mitchell, C.; Hrebenach, D.; Muraru, G.-C.; Junior, I.; Bejan, I.; Mishra, I.; Ngong, I.; Bandy, J.; Stahl, J.; Cardonnet, J.; Trask, K.; Nguyen, K.; Dang, K.; Veen, K. v. d.; Eng, K.; Strahm, L.; Ayre, L.; Jay, M.; Lytvyn, O.; Kyemenu-Sarsah, O.; Chung, P.; Smith, P.; S, R.; Falcon, R.; Gupta, S.; Gabriel, S.; Milea, T.; Thoraldson, T.; Porto, T.; Cebere, T.; Gorana, Y.; and Reza, Z. 2023.  How to audit an AI model owned by someone else (part 1).
  * Wang et al. (2023) Wang, W.; Dong, L.; Cheng, H.; Liu, X.; Yan, X.; Gao, J.; and Wei, F. 2023.  Augmenting Language Models with Long-Term Memory.  _Advances in Neural Information Processing Systems_ , 36: 74530–74543.
  * Wang et al. (2021) Wang, Z.; Byrnes, O.; Wang, H.; Sun, R.; Ma, C.; Chen, H.; Wu, Q.; and Xue, M. 2021.  Data Hiding with Deep Learning: A Survey Unifying Digital Watermarking and Steganography.
  * Wei et al. (2023) Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Ichter, B.; Xia, F.; Chi, E.; Le, Q.; and Zhou, D. 2023.  Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.  ArXiv:2201.11903 [cs].
  * Wei and Heim (2024) Wei, K.; and Heim, L. 2024.  Designing Incident Reporting Systems for Harms from AI.
  * Weidinger et al. (2023) Weidinger, L.; Rauh, M.; Marchal, N.; Manzini, A.; Hendricks, L. A.; Mateos-Garcia, J.; Bergman, S.; Kay, J.; Griffin, C.; Bariach, B.; Gabriel, I.; Rieser, V.; and Isaac, W. 2023.  Sociotechnical Safety Evaluation of Generative AI Systems.  ArXiv:2310.11986 [cs].
  * Wills (2024) Wills, P. 2024.  Care for Chatbots.
  * Wolf et al. (2020) Wolf, T.; Debut, L.; Sanh, V.; Chaumond, J.; Delangue, C.; Moi, A.; Cistac, P.; Rault, T.; Louf, R.; Funtowicz, M.; Davison, J.; Shleifer, S.; von Platen, P.; Ma, C.; Jernite, Y.; Plu, J.; Xu, C.; Le Scao, T.; Gugger, S.; Drame, M.; Lhoest, Q.; and Rush, A. 2020.  Transformers: State-of-the-Art Natural Language Processing.  In Liu, Q.; and Schlangen, D., eds., _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , 38–45. Online: Association for Computational Linguistics.
  * Wu et al. (2023) Wu, Q.; Bansal, G.; Zhang, J.; Wu, Y.; Li, B.; Zhu, E.; Jiang, L.; Zhang, X.; Zhang, S.; Liu, J.; Awadallah, A. H.; White, R. W.; Burger, D.; and Wang, C. 2023.  AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework.  _eprint: 2308.08155.
  * Xie et al. (2024) Xie, T.; Zhang, D.; Chen, J.; Li, X.; Zhao, S.; Cao, R.; Hua, T. J.; Cheng, Z.; Shin, D.; Lei, F.; Liu, Y.; Xu, Y.; Zhou, S.; Savarese, S.; Xiong, C.; Zhong, V.; and Yu, T. 2024.  OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments.  ArXiv:2404.07972 [cs].
  * Zhan et al. (2024) Zhan, Q.; Liang, Z.; Ying, Z.; and Kang, D. 2024.  InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents.  ArXiv:2403.02691 [cs].
  * Zhang et al. (2023) Zhang, H.; Edelman, B. L.; Francati, D.; Venturi, D.; Ateniese, G.; and Barak, B. 2023.  Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models.  ArXiv:2311.04378 [cs].
  * Zou et al. (2023) Zou, A.; Wang, Z.; Carlini, N.; Nasr, M.; Kolter, J. Z.; and Fredrikson, M. 2023.  Universal and Transferable Adversarial Attacks on Aligned Language Models.