# Teamwork under extreme uncertainty:
AI for Pokémon ranks 33rd in the world

Nicholas R. Sarantinos
University of Cambridge
<EMAIL_ADDRESS>
###### Abstract

The highest grossing media franchise of all times, with over $90 billion in
total revenue, is Pokémon. The video games belong to the class of Japanese
Role Playing Games (J-RPG). Developing a powerful AI agent for these games is
very hard because they present big challenges to MinMax, Monte Carlo Tree
Search and statistical Machine Learning, as they are vastly different from the
well explored in AI literature games. An AI agent for one of these games means
significant progress in AI agents for the entire class. Further, the key
principles of such work can hopefully inspire approaches to several domains
that require excellent teamwork under conditions of extreme uncertainty,
including managing a team of doctors, robots or employees in an ever changing
environment, like a pandemic stricken region or a war-zone. In this paper we
first explain the mechanics of the game and we perform a game analysis. We
continue by proposing unique AI algorithms based on our understanding that the
two biggest challenges in the game are keeping a balanced team and dealing
with three sources of uncertainty. Later on, we describe why evaluating the
performance of such agents is challenging and we present the results of our
approach. Our AI agent performed significantly better than all previous
attempts and peaked at the 33rd place in the world, in one of the most popular
battle formats, while running on only 4 single socket servers.

##  1 Introduction

Since the early days of Artificial Intelligence, a lot of research was focused
on developing intelligent agents for games. Despite the significant amount of
work done in deterministic and perfect information games, including Chess [1]
and GO [2], there is limited research in games like Pokémon, that combine a
significant random factor with imperfect information, outside of Poker.

Pokémon, the highest grossing media franchise of all times, long ago inspired
fans to develop intelligent agents capable of competing with human players [3]
[4] [5] [6]. The franchise began in 1995 with the introduction of the first
Pokémon games for the Nintendo Game Boy platform. In these games players
explore a vast game world, collect, train, evolve and trade creatures known as
Pokémon as well as battle against other players using a team of Pokémon. Since
1995 over 70 Pokémon games have been released and together with merchandise,
movies, TV shows, cards and more they grossed over $90 billion in total
revenue. The Pokémon games can be classified as either core series [7] or
spin-offs [8]. The differences between the two categories are outside the
context of this work and here we focus only on the former. On 18 November 2022
the first 9th generation core series games, Pokémon Scarlet and Pokémon
Violet, were released. When this work took place there were 8 generations of
games and the latest core series games were Pokémon Sword and Pokémon Shield.
Screenshots of these games are in Figure 1. While every new generation brings
changes, the games always allowed two players to connect their consoles
locally and battle against each other. Since 2006, players can do the same via
Internet. In 2009 and every year afterwards, the game publisher, Nintendo,
organizes the “Video Game Championships (VGC)” where the top players across
the globe are invited to battle against each other for the title of World
Champion. This and other events led to the emergence of professional Pokémon
players. Nintendo designed the games to be easy for young kids to play but
difficult for professional players to master as the battle mechanics are very
complicated. Numerous fan made websites provide extensive analysis of the game
mechanics, in order to aid professional and amateur players [9] [10] [11].

(a) A player is exploring the vast game world.

(b) Two Pokémon battle against each other. This work proposes an AI agent for
battles like this one.

Figure 1: Pokémon Sword and Pokémon Shield, the first 8th generation core
series Pokémon games.

Developing a powerful AI agent for pokémon battles can inspire all sorts of
applications that require team management under conditions of extreme
uncertainty, including managing a team of doctors, robots or employees in an
ever changing environment, like a pandemic stricken region or a war-zone.
Further, such an agent can be used with tweaks in numerous other popular
Japanese Role Playing Games (J-RPG), like Final Fantasy, as many of these
games share similar battle mechanics and victory depends on teamwork under
extreme uncertainty.

There have been several attempts by others to develop AI agents for Pokémon
[12] [13] [14] [3] [4] [15] [16] [17]. These pioneer attempts gave us a better
understanding of the challenges involved. They mostly relied on work done in
Chess or GO or tackled the entire problem with a reinforcement learning
algorithm and did not invest the enormous amount of time and effort required
to invent the new approaches Pokémon requires, as it is a vastly different
problem.

We will now describe how this paper is organized. In section 2 (“Background”),
we explain in adequate depth the mechanics of the game from the perspective of
an AI researcher, we introduce the online battling platform our work relies
on, we present the work by others so far and finally we observe that our work
can inspire approaches in many real world problems. In section 3 (“Game
analysis from an AI perspective”), we perform a game analysis that vastly
expands our understanding of the problem. In section 4 (“Methods of this
paper”) we propose unique AI algorithms based on our understanding that the
two biggest challenges in the game are keeping a balanced team and dealing
with three different sources of uncertainty. In section 5 (“Experiments”), we
describe why evaluating the performance of Pokémon AI agents is challenging
and we present the results of our unique AI algorithms. The results place our
AI agent above all attempts by others so far by quite a margin. It peaked at
the 33rd place in the world, in one of the most popular battle formats. To the
best of our knowledge, there is no other AI agent for Pokémon that has managed
to reach the top 500 players in the world in any battle format.

##  2 Background

###  2.1 The mechanics of Pokémon battles

In this section we describe the mechanics behind Pokémon battles. We first
describe how battles are played, then we proceed to describe the properties of
the creatures known as Pokémon and finally we describe the format of the
battle states. We will not describe how the game world is explored as well as
how Pokémon are collected, trained, evolved and traded because these mechanics
are outside the context of this work. The battles we focus on are known as
competitive because the only goal is to win, unlike other battles in the game
where the goal is to train a Pokémon or capture a wild one. The following
apply to the core series games of generations 6 to 8. The reader can find more
in-depth information at online sites [9] [10] [11].

There are two battling sides and one, two or three active Pokémon at every
side at every moment. Every side has also reserve Pokémon that can be called
to replace active ones at any point. Pokémon battles are categorized as
Single, Double or Triple, depending on the number of active Pokémon per side.
The battle sides usually have the same number of active Pokémon. This work
focuses on Single battles but can be expanded to support the other two
categories too. The total number of Pokémon per team, both active and reserve,
is defined by the rules of the specific battle and can never exceed six. Our
work focuses on battles with the maximum number of Pokémon per team.

The games use a turn based battling system. At the start of every turn the two
sides must select their actions simultaneously, without knowing the selection
of the opponent. Actions are only executed after both sides make their
selections. We describe the order of execution below. These actions, as seen
in Figure 1 (b), can be:

  1. 1.

Make an active Pokémon use a move. Every Pokémon can use at most one move per
turn.

  2. 2.

Switch/replace an active Pokémon with a reserve. Every Pokémon can be switched
with another at most once per turn.

  3. 3.

Use an item from the bag [18]. There are several kinds of items stored there
and their use is to aid the exploration of the game world as well as the
collection and training of Pokémon.

  4. 4.

Flee the battle. This is allowed only if the opponent is a wild Pokémon and
not another player.

In competitive battles, which we are focusing, only the first two are
possible. Players cannot use their bag in these battles but each Pokémon is
allowed to hold an item. The order selections are executed depends on game
mechanics. Barring rare exceptions, Pokémon switching is executed before any
move. The order at which moves are executed depends on many of the Pokémon
properties we describe later on. The three most important are the speed
statistic, ability and move effect. Every player knows the properties of their
Pokémon but not these of the enemy ones. As a result, players have to guess
this hidden information in order to determine which Pokémon will move first.

If a player switches an active Pokémon with another, while their active
Pokémon is still able to battle, this will cause the newly switched in Pokémon
to receive a hit upon entering the battle arena, without having the time to
attack back, if the opponent chooses to use a move. For this reason switching
Pokémon without being forced to do so is risky and should ideally be done only
if the expected opponent move will cause little damage or if the opponent is
also expected to switch Pokémon too.

The winner in the battle is the player that manages to reduce to zero the life
of all opponent Pokémon first. One has to consider team balance, as a team
contains multiple Pokémon and one will have to decide if it is best to have
one fully healthy or many damaged ones. The goal is not just to damage the
enemy as much as possible but to make sure all their Pokémon will faint at the
end. This is complicated by the vast amount of conditions that affect the
battle, as we describe later on. Estimating the value of battle states is
incredibly complicated as, on top of taking into account the team balance, the
battle conditions and the uncertainty hidden information result in, there is
an extraordinary amount of corner cases. Sometimes it is a mistake to reduce
the life points of an opponent Pokémon, unless they reach zero, as this may
activate an ability that doubles its attacking power. In many cases even
professional players have heated arguments as to which battle state is
preferred.

Having described how battles are played, we will now explain in adequate
detail the most important, in the context of this work, properties of Pokémon.

  1. 1.

The species: Every Pokémon belongs to one of the over 800 possible species
[19] [20]. The species is very important because it impacts indirectly the
battle in many ways, as described below.

  2. 2.

The level: Every Pokémon has a level in the range from 1 to 100 [21]. The
level does not impact the battle in any direct way but is significant because
it has major indirect impact since, when a Pokémon level ups:

    1. (a)

Its statistics are increased

    2. (b)

New moves can be learned

    3. (c)

It can evolve to another compatible species

  3. 3.

The type(s): Every Pokémon can have one or two types [22] out of: Normal,
Fire, Water, Grass, Electric, Ice, Fighting, Poison, Ground, Flying, Psychic,
Bug, Rock, Ghost, Dragon, Dark, Steel and Fairy. For example Pikachu is
Electric type while Charizard is Fire and Flying type. Types are important
mainly because they increase or decrease the effectiveness of moves. For
example using a Fire type move against a Water type Pokémon will halve the
damage dealt. Consult a type chart [23] for details on effectiveness.

  4. 4.

The statistics: Every Pokémon has: Hit Points (HP), Attack, Defense, Special
Attack, Special Defense and Speed [24]. These are determined by a formula [25]
that takes into account the Species, Level, Nature [26], Individual Values
(IV) [27] and Effort Values (EV) [28]. The last three parameters are not
explained here because they exist only to impact the statistics and have no
other value in battles. Once the formula calculates the value of a statistic,
this can be temporally modified by items, abilities, move effects and battle
conditions.

  5. 5.

The ability: Every Pokémon can have one ability [29] out of one to three
possible that depend on the species. For example the possible abilities for
Pikachu are Static and Lightning Rod while for Charizard are Blaze and Solar
Power. The ability impacts directly the battle in many ways as it can modify
the statistics, paralyze the opponent, change the effectiveness of the moves
and have many other effects. Over 250 abilities exist in total [30].

  6. 6.

The item: Every Pokémon can optionally hold one of the over 1,500 items
available in the game [31] [32]. Examples of items are Leftovers and Choice
Band. Items can have numerous effects in battles and are usually used to heal
or boost the Pokémon that holds them.

  7. 7.

The move(s): Every Pokémon knows one to four moves [33]. The pool of possible
moves depends on the species and can range from only 1 to over 100. For
example Pikachu can learn Volt Tackle and Thunderbolt but not Fire Blast.
Every move has six properties: type, category, damage, accuracy, power points
(PP) and optionally an effect. A non deterministic formula [34] takes into
account the properties of the move as well as the properties of the attacking
and the defending Pokémon and determines the damage. The move category is
either Physical, Special or Status and determines if the Attack and the
Defense or if the Special Attack and the Special Defense statistics will be
used in the damage calculation. Power points is the maximum number of times
the move can be used in a battle. A few move effects can heal the user. Over
700 moves exist in the games [35].

Having described the key properties of the creatures known as Pokémon, we will
now describe the format of the battle states. As one would expect, due to the
hidden information in the game, only the computer running the battle software
can fully observe the battle state. Every state consists of:

  1. 1.

The player teams: The team of every player is stored in the battle state. As
mentioned above, depending on the battle rules, the properties of the
opponent’s Pokémon, including in many cases the species, are unknown unless
revealed or inferred using the process of elimination. Every player attempts,
to the best of their ability, to infer as much of the hidden information about
their opponent’s team as possible. These inferred information can be
considered to be part of the battle state the player observes.

  2. 2.

The battle conditions [36]: These can be classified into three categories,
depending on their scope, as we do below. There are other ways to classify
them too.

    1. (a)

The battlefield conditions: These impact the entire battlefield and usually
last a few turns. An example is the change of the weather into rain. This
boosts the power of water type moves and makes certain electric type moves to
never miss the target. There are about 10 such conditions.

    2. (b)

The battle-side conditions: These impact only one side of the battlefield.
Some last a few turns and others for the entire battle, unless removed by an
effect. One example is placing toxic spikes at the enemy side, so all enemies
will immediately be poisoned when they are switched in. Another is summoning a
tailwind at the ally side so all allies will have their speed statistic
doubled for a few turns. There are about 20 such conditions.

    3. (c)

The Pokémon conditions: These impact only individual Pokémon. Some last a few
turns, others last until the Pokémon is switched out and others for the entire
battle, unless removed by an effect. One example is changing the status of a
Pokémon into frozen, burned or paralyzed. Another is boosting or decreasing
any of the statistics of a Pokémon. It is even possible to change the form of
a Pokémon mid-battle temporally. There are about 100 such conditions.

###  2.2 A simulator and a platform for Pokémon battles

The core series Pokémon games have been officially available only in Nintendo
platforms, including Nintendo Switch. Until recently, the games made it
difficult to play ranked battles against others online. In these battles
players are ranked in a global leader-board, based on a ranking system, like
professional Chess players. Further, many online features are only available
for a year or two after the release of a game, requiring players to keep
buying the latest version, almost every year, to keep playing online battles.
The passion of Pokémon fans along with these issues led to the creation of a
fan made Pokémon battle simulator, Pokémon Showdown.

Pokémon Showdown was started in 2011 by Guangcong Luo and is currently open
source with over 20,000 commits and over 300 contributors on GitHub [37]. The
project successfully simulates the extremely complicated Pokémon battle system
and is virtually perfect, barring rare bugs. It does not simulate the
exploration of the game world as well as the collection, training, evolution
and trading of Pokémon. An online platform was created around the simulator
that allows players to play ranked battles. It is accessible via a web page (
https://play.pokemonshowdown.com/ ). Players are ranked based on their wins
and loses using the ELO rating system. The platform also displays the Glicko
rating as well as a custom variation of this. Players are matched with an
opponent based on their ELO rating. If either of the players asks for it, the
platform sets a time limit of a few seconds per decision. This is very strict
and an issue for AI agents.

The online platform offers battles in several different formats [38]. Each
battle format has different rules and presents different challenges to
players. The most important differences between the most popular battle
formats are the following:

  1. 1.

Team selection: At the start of every battle, players either select a team of
Pokémon to use or they are provided with a team of randomly generated Pokémon.
Building a great Pokémon team is no easy task. On the other hand, it is
significantly harder for players to master every possible random team than to
master a few carefully selected.

  2. 2.

Opponent team preview: At the start of every battle, the species property of
every opponent Pokémon is either revealed or kept hidden until the opponent
Pokémon in question is used at least once in the battle. In general, if the
opponent Pokémon species property is kept hidden, the battle is considered
significantly harder, as players have to make assumptions about this hidden
information.

  3. 3.

Number of active Pokémon: As we discussed above, Pokémon battles are
categorized as Single, Double or Triple, depending on the number of active
Pokémon per side.

  4. 4.

Number of Pokémon per team: As we discussed above, Pokémon teams can have a
maximum of six Pokémon, including both active and reserve.

  5. 5.

Restrictions: The rules of the battle format can include several restrictions,
including banned Pokémon, moves, items, abilities and more.

  6. 6.

Generation: Every generation of Pokémon games differs in the available
Pokémon, moves, items, abilities as well as battle mechanics. All generations
from 1 to 9 are available on the platform.

The two most popular battle formats in the online platform are named “Random
Battles” and “OverUsed (OU) Battles” [39]. In the former, players are provided
with a random team at the start of the battle and there is no opponent team
preview. In the later, players select their team and there is opponent team
preview. Both of them are Single battles as players have one active Pokémon at
a time. Further, in both of them players have teams of six Pokémon. There are
differences between the restrictions of the two battle formats but they will
not be discussed here due to space constraints. In this work we focus on the
7th generation of core series Pokémon games. We also focus on the format
“Random Battles” which is the most popular and is considered by many more
challenging than the “OverUsed (OU) Battles” format. At this paper we propose
an algorithm for team selection, making it easy for others to extend our work
and develop an AI agent that supports the other very popular format too.

The simulator and the online platform are the two pieces of infrastructure
required to make a Pokémon AI agent. Without the simulator it is not possible
for an AI to employ look-ahead. Nintendo, so far, have not provided an API to
interact with their Pokémon games programmatically. As a result, the only way
an AI agent can play against human players in a Nintendo platform is by
analyzing the video captured from the screen of the console. The online
platform based on Pokémon Showdown solves this problem as it provides a
websocket API that can be used by AI agents to receive the stream of battle
events and respond back with battle commands.

###  2.3 Work by others

As the Pokémon franchise remains as popular as ever, in part thanks to the
very successful spin-off game Pokémon GO [40], there have been a lot of
pioneer attempts to create Pokémon AI agents. In addition to what we will
present below, there are many other attempts that are only available on GitHub
and BitBucket as open source projects [41] [5] [6].

Ho and Ramesh [13] while students at Stanford University developed one of the
first AI agents that relied on MinMax and a handcrafted evaluation function.

Khosla et al. [14], also while students at Stanford University, developed an
agent that relied on Expectiminimax and compared its performance to MinMax.
Instead of using a handcrafted evaluation function, they used one based on TD
learning, a form of reinforcement learning. This work is important because it
made clear that evaluating battle states in Pokémon is very hard and a
handcrafted evaluation function is a poor choice.

Kalose et al. [3], again while students at Stanford University, developed an
AI agent that used a model free approach and relied on Q-Learning. The
conclusion was that this approach requires much more effort than Minmax and so
far achieved lower performance.

Chen and Lin [4], once again while students at Stanford University, used Deep
Reinforcement learning and feature space embeddings. The conclusion was that
approach did not performed as well as others.

Lee and Togelius [12] proposed a competition for Pokémon AI agents. Their work
focused on comparing AI agents by having them battle each other.

Sanchez [15] did an extensive study on the area. This work attempted to use
the approaches Google Deepmind used to master the game of GO [2] but concluded
that it is not appropriate for the game of Pokémon. He also proposed further
work on reinforcement learning approaches.

Norström [16] did another extensive study in the area. This work focused on a
simplified form of the 1st generation core series games. It relied on Monte
Carlo Tree search but was not evaluated in the online platform based on
Pokémon Showdown.

Huang and Lee [17] developed two reinforcement learning agents that were
trained in 3.8 million self play battles over the course of six days. Their
first AI agent achieved decent performance against human players as it
performed on pair with the best work of others. Further, the agent was put to
play against the AI agent [5] but performed worse. Their second AI agent was
trained to play with a few selected teams. This AI agent managed to win most
battles against the AI agent [5] but performed worse than the first against
human players.

Dotson [42] presented an AI agent in a website and a YouTube video. The author
did not describe his approach in detail. The results reported indicate that
the agent is stronger than the others described above. As we will see below,
it is not straightforward to compare the performance of AI agents.

To the best of our knowledge, until now no AI agent for Pokémon has managed to
reach the top 500 players in any battle format.

###  2.4 Applications in other domains

As we describe in section 3 (“Game analysis from an AI perspective”), victory
in the game of Pokémon depends on managing a team and ensuring excellent
teamwork under conditions of extreme uncertainty. This lies at the heart of
many significant real world problems that span several domains. Examples
include managing a team of doctors in a case of a polytrauma patient with an
ever changing condition, managing a team of employees to complete a project in
an very dynamic environment like a pandemic stricken region, managing a team
of autonomous drones in a rescue operation against a team of adversaries and
many more. We hope that the key principles of our work will inspire approaches
that solve problems in these domains. Further, this work can be applied to
several other extremely popular Japanese Role Playing Games (J-RPG), like
Final Fantasy.

##  3 Game analysis from an AI perspective

In this section we describe our key conclusions about the game. These are very
important as they serve the basis of the unique approach to AI agent
development we present in the next section.

###  3.1 How battles can be won

From our description of the Pokémon battle mechanics, it should be clear that
great players have memorized a lot of information: they know the properties of
the over 800 Pokémon species, over 700 moves, over 1,500 items, over 250
abilities as well as the over 100 possible battle conditions of generation 7.
Most importantly, great players understand that victory depends on:

  1. 1.

Keeping the team as balanced as possible during the battle. Sometimes it is
worth to take a short term loss in order to keep the team balanced and capable
to beat the opponent’s team. For example, it is worth to lose a fully healthy
Pokémon in order to make sure the only water type in our team remains
undamaged and is able to beat the two Fire type in the opponent’s team,
otherwise, assuming we have no other way to beat these, we will lose the
battle. Every player wants to maximize the probability that their team will
win the opponent’s team. In other words, at every point, they want to be sure
that they have at least a way to beat all opponent Pokémon with at least one
of their Pokémon still standing. An AI agent cannot achieve this just by
looking ahead with MinMax or Monte Carlo Tree Search.

  2. 2.

Dealing with the three different sources of uncertainty present in battles. A
well designed AI agent will approach every one differently.

    1. (i)

Players decide simultaneously and often intentionally make unexpected moves.
As a result, opponent prediction is critical.

    2. (ii)

There are many kinds of hidden information in the game as many of the opponent
Pokémon properties are unknown unless revealed or inferred.

    3. (iii)

The damage and the effect of the vast majority of Pokémon moves is non
deterministic. Given a state, the very same moves can lead to over 1,000
different states, depending on the output of a RNG.

###  3.2 Considerations when evaluating states using Machine Learning

In our opinion, the following are some of the most important considerations
one should take into account before applying Machine Learning to this problem.

  1. 1.

The game mechanics are very complicated, in contrast to games like GO or
Chess. Furthermore, the number of possible battle states in the game is very
large, significantly more than the $3^{81}$ possible states in the game of GO.
These factors make it hard for Neural Networks to infer patterns in the states
that can be exploited. Based on this, directly asking the Neural Networks to
evaluate the states is something we do not believe is optimal as it depends,
among others, on the Neural Networks learning the very complicated game
mechanics indirectly. This approach worked well in games like Chess or GO
because their mechanics are very simple.

  2. 2.

Every Pokémon battle can be considered set in an almost unique environment
because the teams of the players will almost never be the same, especially in
the battle format we focus. As a result, training Neural Networks only using
past battles, with simple supervised learning, will likely result in having
zero samples of the ongoing battle’s environment in the train set. For this
reason, we believe it is important for Neural Networks to be trained on the
past turns of the ongoing battle too and possibly place more weight on these.

  3. 3.

The multiple sources of uncertainty in the game make it difficult to determine
what role luck played in victory. Given a state, the very same moves can
sometimes lead to over 1,000 different states due to the RNG. As a result we
believe using a traditional Reinforcement Learning approach where the AI agent
is receiving rewards based on the outcome of the battle is not optimal as it
is common for players to win even if they play very poorly compared to their
opponent and the opposite. In a few cases the team of a player is so much more
powerful than the opponent’s that the outcome of the battle is determined
before it even starts, regardless of the skill of the players.

###  3.3 The game trees are pathological

Over 30 years ago, Dana Nau [43] [44] discovered that in some games, looking
ahead too many turns leads to worse decision making. This phenomenon was named
“game tree pathology”. Following excessive game analysis, we concluded that
the game trees are pathological. To the best of our knowledge, we present the
most popular game where this phenomenon is apparent in all game trees. In
Pokémon, every time we look ahead another turn we have to make assumptions
about the decisions of the opponent that turn, guess the hidden information
about the properties of the opponent’s Pokémon and assume the numbers the RNG
will generate. The game tree is pathological because the more we look ahead
the more we base assumptions on top of assumptions making the AI delusional as
it looks into a future far different from what will actually happen.
Unfortunately, not looking ahead is not an option in this game. As a result,
we have to find the right number of turns to look-ahead in order to see into
the future while minimizing the issue we described.

##  4 Methods of this paper

The approaches we describe in this section depend on the conclusions we
described above. We first describe our approach to Pokémon battling and then
we explain in detail how our approach deals with every one of the three
different sources of uncertainty present in the game. Finally, we propose an
algorithm for team building.

###  4.1 A battle algorithm that ensures teamwork

####  4.1.1 The main algorithm

The algorithm we describe below places a lot of weight into keeping the team
of the AI agent balanced while minimizing the issue of game tree pathology,
which we discussed in the subsection 3.3. It uses a distributed transposition
table to greatly reduce the number of states explored. Further, it relies on a
forward pruning algorithm to prune states because alpha beta pruning cannot be
used in games like this where players decide simultaneously.

  1. Step 1:

We start from the current battle state and we look-ahead for 2 turns. For both
players we consider all possible moves during this process and we end up with
several terminal states.

  2. Step 2:

For every of the terminal states of step 1 we want to calculate the
probability the AI agent will win from that state. This is akin to determining
“how strong is our team compared to the opponent’s”. We calculate this in two
steps.

    1. Step 2.1:

We set each Pokémon in our side to battle alone against each Pokémon in our
opponent’s side. In order to determine the outcome of these one-versus-one
battles, we rely on 1 or 2 turn look-ahead. The terminal states of this look-
ahead are scored with a function. Each Pokémon’s value at that point is
determined based on its remaining life points (HP). It also gets a big bonus
if that value is above zero. The scoring function returns the score of our
Pokémon minus the opponent’s Pokémon score.

    2. Step 2.2:

We combine the scores of all one-versus-one battles into a single score by
calculating their average. The goal of this is to determine how high chance
our team has to win the opponent’s. In our current implementation we use a
simple average but it would be more appropriate to weight the scores based on
the probability the corresponding battle has to occur in the future. We
determined that, the more even a battle is, the higher this probability is,
since in general players will avoid using Pokémon that are weak against the
enemy if they have a better choice.

  3. Step 3:

Every time the scores of all children of a state are available, in either of
the above steps, we construct a payoff matrix. We use this matrix to predict
the next move of the opponent at that state, as we describe in the subsection
4.1.2. In that subsection we also describe how the algorithm deals with the
other two sources of uncertainty during the look-ahead, namely hidden
information and excessive RNG use.

A forward pruning approach: For each player, this algorithm iteratively
removes the dominated strategies. To do so it relies on static analysis of the
game states. It checks if one of several conditions are met in order to
determine if a state is worse than another for the AI agent. For example, if
two battle states differ in only one or two parameters, it is not challenging
to write pruning conditions for someone familiar with the game. Unlike chess,
the unique nature of the game of Pokémon makes this pruning approach both very
efficient and low risk as it rarely prunes non dominated strategies. To the
best of our knowledge, this is the first time such an approach is proposed for
this game.

####  4.1.2 Dealing with uncertainty

In this subsection we will describe in detail how the algorithm we presented
deals with every one of the three sources of uncertainty we introduced in the
section 3.1 (“How battles can be won”).

  1. (i)

Players decide simultaneously and often intentionally make unexpected moves.
As a result, opponent prediction is critical.

At every point in the algorithm we described above we need to predict the move
of our opponent we use the following algorithm. Our approach is based on a
regret minimization Machine Learning algorithm.

    1. Step 1:

Our algorithm receives as inputs a payoff matrix as well as the history of
opponent moves with the corresponding payoff matrices. Since the game is zero
sum, our best move is associated with the largest number in the payoff matrix.
The opposite is true for the opponent’s best move.

    2. Step 2:

A Machine Learning algorithm takes the inputs we described above and returns
the probability of the opponent making each of their available moves.

    3. Step 3:

We multiply the probabilities returned by the Machine Learning algorithm with
the payoffs of the possible opponent moves.

    4. Step 4:

From the resulting numbers of the above step, we consider the opponent move
associated with the lowest number.

    5. Step 5:

Our algorithm returns as output our best response to the opponent’s move we
considered in the above step.

  2. (ii)

There are many kinds of hidden information in the game as many of the opponent
Pokémon properties are unknown unless revealed or inferred.

We deal with this source of uncertainty with two approaches, depending on
whether we know the species property of the opponent’s Pokémon.

    1. Case 1 -

We know the species: In this case we determine the most probable value for all
the unknown properties of the opponent’s Pokémon in question by relying on
detailed usage statistics. During this process we consider the known
properties other than species, if any. As the game progresses and more
properties are being revealed, our predictions of the unknown properties are
getting more and more accurate. In general, the same species do not have
extreme variations in the rest of their properties. For this reason this
approach is very successful and great human players think this way too.

    2. Case 2 -

We do not know the species: In this case all the other properties will be
unknown too. We decide with which Pokémon we will represent every of the
entirely unknown opponent Pokémon by solving the following mixed integer
linear programming problem using the IBM CPLEX Optimizer [45].

N is the number of all Pokémon available in the game, for example, in
generation 7 we have N = 802.

M is the number of non fainted Pokémon in our team, where 1 $\leq$ M $\leq$ 6.

K is the number of entirely unknown Pokémon in the opponent’s team, where 0
$\leq$ K $\leq$ 5.

The objective of the optimization is to select K Pokémon such that, the
outcome of the hypothetical battle between the M non fainted Pokémon in our
team and all the N Pokémon available in the game is approximated as accurately
as possible by the outcome of the hypothetical battle between the M non
fainted Pokémon in our team and the K selected Pokémon.

$\displaystyle\min\quad$
$\displaystyle\sum_{j=1}^{M}{|\sum_{i=1}^{N}x_{i}(s_{ji}-{{1}\over{N}}\sum_{t=1}^{N}s_{jt})|}$
s.t. $\displaystyle\sum_{i=1}^{N}x_{i}=K$ $\displaystyle x_{i}\in\\{0,1\\}$
$\displaystyle i=1,\dots,N$

We define N Boolean decision variables, $x_{1},...,x_{N}$ where each one
corresponds to a specific Pokémon from all the available in the game. If
$x_{150}=1$ this indicates that we select the Pokémon with index 150, named
Mewtwo, to represent one of the entirely unknown opponent Pokémon. Since there
are K entirely unknown Pokémon in the opponent’s team, we must select exactly
K Pokémon to represent them and thus we set the appropriate constrain.

The matrix s, given the index of a Pokémon in our team and the index of any of
the available Pokémon in the game, contains a positive or a negative number
that represents the approximate outcome of the battle between these two
Pokémon. We derive all possible such matrices from a matrix of NxN where the
outcome of any Pokémon versus any other is stored. We pre compute offline this
NxN matrix by having every two Pokémon battle each other without teammates for
2 turns.

  3. (iii)

The damage and the effect of the vast majority of Pokémon moves is non
deterministic. Given a state, the very same moves can lead to over 1,000
different states, depending on the output of a RNG.

We use the approach we are describing below to deal this source of uncertainty
during look-ahead. Our approach achieves much higher accuracy and is
significantly more efficient compared to other alternatives. Further, it is
deterministic which aids the evaluation of the AI agent. Before we describe
our approach we will describe why two obvious alternatives are not a good
choice.

#### Determinization

Starting from a state, considering that the same moves can lead to a very high
number of different states, sometimes over 1,000, determinization cannot be
considered because it will lead to an extremely high branching factor.

#### Monte Carlo sampling

If we have not found a better alternative, Monte Carlo sampling would have
been our choice. The primary issue is that it is very inefficient as it
requires many samples, and thus a lot of time, to return an accurate
approximation of the value of a battle state.

#### Our approach

    1. Step 1:

Each random event in the game requests from the RNG a floating point number in
the range [0, 1). We have observed that virtually all events do not nearly use
this entire range. For this reason, in our approach we are restricting the RNG
to only generate a number among N predefined ones. Following experiments we
determined that N should be in range [8, 20]. In the examples below we use N =
8.

    2. Step 2:

The above is not enough on its own because there are usually multiple events
that request a random number per turn. Assuming there are five such events and
each player has only one available move in a turn, even with our approach, we
would need to consider $8^{5}=32,768$ states. We tackle this thanks to two
observations. First, the vast majority of random events in the game impact the
side of a specific player only. Further, at every turn usually one random
event per side has the most impact in the state. The second observation means
it will not be a big compromise in terms of accuracy to use the same random
number for all events of a side in a turn.

Following our observations, we are asking the RNG to generate only two numbers
per turn. Every time a random event asks for a number, we are proving it with
one of the two generated that turn, depending on which side the event will
impact. As a result, if both players have only one available move in a turn,
we only have to consider $8^{2}=64$ states that turn. Usually players have 4
to 9 moves per turn so we have to consider ${8^{2}}\times{9^{2}}=5,184$ states
as they decide simultaneously. In practice this number is significantly lower,
around 500, thanks to the use of a transposition table, as we described in our
main algorithm.

    3. Step 3:

The approach we described in the above steps has a high branching factor. As a
result, we use it in only one turn of look-ahead, the next turn of the battle.
In all turns of look-ahead after the next, we force the RNG to always generate
0.5, effectively seeding all these random events with “average luck”. As the
next turn of the battle is usually the most important in look-ahead, little
accuracy is lost from this.

###  4.2 Team building using a genetic algorithm

Given a powerful AI agent for battles, a simple and effective team building
approach is to rely on a genetic algorithm. In the battles we focus players
are provided with random teams, so there was no need for us to implement team
building. Regardless, we felt that describing this would make a contribution,
as team building was discussed in several papers.

  1. Step 1 -

Initialization: In our case, we consider as an individual an entire Pokémon
team and as a gene a Pokémon property. The teams used by human players in the
online platform can be very useful if available, as they can be used to seed
the initial population of individuals.

  2. Step 2 -

Fitness function: The fitness function of the genetic algorithm in our case
will take as input a Pokémon team and evaluate its performance against as many
other teams as possible. In order for the fitness function to work well, the
teams a candidate team will battle against should represent these used by
humans players in the online platform. The evaluation of teams will rely on AI
agent self play.

  3. Step 3 -

Selection: During the selection phase some of the best performing teams are
selected.

  4. Step 4 -

Crossover: During the crossover phase, we generate new teams by taking
properties of Pokémon from two successful teams.

  5. Step 5 -

Mutation: During this phase, we sightly modify the properties of Pokémon in
the newly generated team.

  6. Step 6 -

Termination: We terminate the algorithm when we find a few teams with a very
high fitness score. We test these teams in the online platform, since the
fitness function will have a margin of error.

##  5 Experiments

###  5.1 Experimental setup

We connect our AI agent to the online battle platform based on Pokémon
Showdown using the WebSocket API provided. We play battles against human
players in the format “[Generation 7] Random Battle”. The platform
automatically matches the AI agent with opponents based on their ELO rating.

Our AI agent is distributed across 4 Amazon WebServices c5a.24xlarge instances
connected to each other using 10 Gbps Ethernet, each featuring a single
48-core AMD EPYC 7R32 processor and 192 GB of main memory. The AI agent must
run on high end hardware because the online platform enforces a strict time
limit of a few seconds per decision. The most notable distributed component is
the transposition table where the servers communicate in an asynchronous and
almost entirely peer to peer manner. This approach minimizes delays in
decision making due to network delays. Our distributed approach is based on
similar approaches developed for Chess [46] [47].

###  5.2 Evaluation method

Evaluating the performance of AI agents in games is critical because it allows
researchers to track the progress made, develop AI agents that can learn and
improve on their own as well as compare them with each other and human
players.

The most popular rating systems by far are the ELO [48] and Glicko [49]. These
measure the relative skill of a number of players that play against each
other. ELO is more popular but Glicko uses “rating reliability” that can be
used to penalize players that have a long time to play the game. AI agents in
Chess and GO are usually compared with the ELO rating but it is incorrect to
compare the rating of humans and AI agents directly, unless they have been
playing against each other.

Games that rely on both luck and skill like Pokémon present challenges to
rating systems. The game of Poker, due to its unique nature, is evaluated with
unique approaches. The majority of other popular such games, including video
games such as League of Legends, are evaluated with the ELO rating. While
there has been research to design rating systems that try to rank players
while minimizing the impact of randomness in these rankings [50], these
systems are not as widely adopted as ELO and Glicko. Game developers,
including Nintendo in official games, rely on ELO to rank players. Due to the
nature of these games, when evaluating the performance of a player, it is
recommended to keep the ELO rating over time, like we do in this work. This
would make it possible to determine if luck temporally impacted the rating. If
this is the case, there will be big rating spikes or drops.

Pokémon Showdown, the platform our AI agent uses to play against humans, ranks
players based on the ELO rating [51]. Glicko and a variation are also
displayed. The ladder never resets and the platform makes two modification to
the ELO rating system: It adds ladder decay that reduces the ranking of
players if they stop playing for some time and it limits the maximum rating
gain or loss per battle as the ELO increases. One improvement we suggest to
the Pokémon Showdown team is to provide, in addition to the existing one, an
alternative, seasonal ladder based on the ELO rating without the two
modifications we mentioned. That ladder will reset every few weeks or months,
exactly like the ladder systems work in many popular games, including the
latest generation of official Pokémon games. This will make it possible to
evaluate AI agents with the pure ELO rating system which is the golden
standard used in most game AI agent papers. Furthermore, this will prune from
the ladder inactive players, instead of just reducing their ELO rating, which
will make calculating the percentiles of players and AI agents more accurate,
as we discuss below.

The popularity of different game formats changes over time in the online
platform. This is dramatic when a new generation is released, as most players
will switch to the newer and the previous will not be as popular. Despite
this, a number of very hardcore players will keep playing the previous. As a
result, reaching the very top of the ladder does not get noticeably easier but
the ELO range becomes much shorter. This in turn means that the best players
in the world in the previous generation will have a much lower ELO rating
compared to the best players in the new generation, despite having very
similar skill level. This observation means that the ELO rating of AI agents
can be compared directly if and only if they were evaluated in the exact same
battle format and exact same time period. If this is not the case, the only
way to compare the AI agents would be via their percentiles. Because it is
very unlikely for two AI agents to be evaluated in the exact same time period,
we suggest the percentile to be the golden standard in evaluating AI agents in
this game. Unfortunately, currently the online platform does not display the
rank of players outside the top 500 and thus it is not possible to find the
percentile of lower ranked AI agents. We suggest this functionality to be
implemented, in addition to the seasonal ladder.

In summary, we believe the percentile is by far the most accurate way to
evaluate the performance of Pokémon AI agents as long as this information is
provided by the platform and not approximated by the researcher. When this is
not available, if the AI agent reaches the top 500 players, the world rank can
be relied upon. In addition to the world rank, in our evaluation below we
provide the ELO rating over time while we clearly specify the battle format
and time period the AI agent was evaluated.

###  5.3 Results

(a) The world rank of the AI agent after every battle played.
The platform provides the rank only for the top 500 players.

(b) The ELO rating of the AI agent after every battle played.
Every player starts with ELO 1,000.

Figure 2: The results of our AI agent after 643 battles against human players
in the Pokémon Showdown platform.

We set our AI agent, named “Athena2020”, to play 643 battles against human
players in the format “[Generation 7] Random Battle” in the Pokémon Showdown
platform. The AI agent was started at approximately 22 Aug 2020 15:03 GMT and
was stopped at approximately 25 Aug 2020 17:21 GMT. The online platform
automatically matches players to battle against each other based on their ELO
rating. Links to the replays of the 643 battles our AI agent played, which are
stored at the Pokémon Showdown platform, are available in the Appendix.

In the Figure 2 (a) the world rank of the AI agent after every battle, is
shown. As we discussed, the platform currently provides the rank only for the
top 500 players. As it can be seen on the figure, the AI, starting from a new
account, first reached the top 500 players after 79 battles. After battle 227
it reached its peak by ranking 33rd in the world. Other notable highs were
ranking 52nd after battle 107, 70th after battle 369, 35th after battle 461
and 50th after battle 602. Following a query to the developers of the online
platform, we were informed that the total number of players that played at
least one battle, in the format the AI was evaluated on, is as of late August
2020 over 15 million. While many of these are inactive and thus we cannot
calculate the percentiles over time by diving the ranks over time with this
number, we felt it was important to mention it, as it gives an idea of how
fierce the competition is on the platform. Since reaching the top 500 players
after battle 79, the AI remained ranked among the top 500 players in the world
exactly the 2/3rds of the time. We believe this indicates that the performance
of the AI agent was not due to mere luck but it truly deserved the rankings it
reached. Furthermore, it is obvious that luck does indeed play a role. This
supports our claims of high uncertainty in the game.

In the Figure 2 (b) the ELO rating of the AI agent after every battle played
is shown. Every player starts with ELO 1,000. Taking a look at the figure
makes apparent that there is a clear correlation between the ELO rating and
the rank, as one would expect. As we discussed above, the ELO rating of AI
agents can be compared directly if and only if they were evaluated in the
exact same battle format and exact same time period. We provide the ELO rating
in order to show the performance of the AI agent when it is not ranked among
of the top 500 players.

To the best of our knowledge, this is the only AI agent for Pokémon that has
managed to reach the top 500 players in the world in any battle format.

This work was ready for publication in late 2020. We planned to spend a few
more months to improve the AI’s ranking further as there is great potential to
do so. However, due to other obligations, we decided to publish this now.

##  6 Conclusions

In this work we explained in depth the mechanics of the Pokémon battle system
from the perspective of an AI researcher and we performed a game analysis that
vastly expanded our understanding of the problem. We continued by proposing
unique AI algorithms based on our understanding that the two biggest
challenges in the game are keeping a balanced team and dealing with three
different sources of uncertainty. Our AI agent performed significantly better
than all previous attempts and peaked at the 33rd place in the world, in one
of the most popular battle formats, while running on only 4 single socket
servers.

##  7 Future work

Our work can be extended to support Double and Triple battles as well as
formats other than Random Battles. This last suggestion would require the
implementation of the approach we described above titled “Team building using
a genetic algorithm”. Further, a more sophisticated pruning algorithm can be
used. One option is to use the work of Saffidine et al. [52] where they
propose an algorithm inspired by alpha beta pruning for games where players
decide simultaneously, since alpha beta pruning itself cannot be used in such
games. Another is to develop a new, more suited for our use case, pruning
algorithm. Preliminary investigation made us conclude that a pruning algorithm
based on the principle of backwards induction could be applicable to the game
of Pokémon as well as other games and problems.

We also suggest to the Pokémon Showdown team to implement the functionality to
determine the percentile and rank of a player who is not among the top 500.
This would make it possible to compare AI agents of all skill levels even if
they were evaluated in different time periods. Furthermore, a seasonal ladder,
as we have described above, will aid the evaluation of AI agents.

## Acknowledgments

The author would like to thank Prof. Ion Androutsopoulos for his suggestions
and fruitful discussions about Machine Learning and Game Theory. Furthermore,
he is very grateful to Prof. Giannis Marias and Ioannis Sazonof for providing
him with access to the computers of the CS Laboratories of Athens University
of Economics and Business that helped with the testing of early ideas.

This work would not have been possible without the amazing work of Guangcong
Luo, Kirk Scheibelhut and the other developers of the Pokémon Showdown online
platform and battle simulator.

Finally, special thanks to Nintendo and Game Freak for creating the Pokémon
franchise.

## References

  * [1] Deep blue, IBM’s supercomputer, defeats chess champion garry kasparov in 1997. https://www.nydailynews.com/news/world/kasparov-deep-blues-losingchess-champ-rooke-article-1.762264
  * [2] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis, Mastering the game of Go with deep neural networks and tree search, _Nature_ , vol. 529, no. 7587, pp. 484–489, Jan 2016. https://doi.org/10.1038/nature16961
  * [3] A. Kalose, K. Kaya, and A. Kim, Optimal battle strategy in Pokémon using reinforcement learning. https://web.stanford.edu/class/aa228/reports/2018/final151.pdf
  * [4] K. Chen and E. Lin, Gotta Train Ém All: Learning to Play Pokémon Showdown with Reinforcement Learning. http://cs230.stanford.edu/projects_fall_2018/reports/12447633.pdf
  * [5] A Pokémon Showdown Battle Bot written in Python. https://github.com/pmariglia/showdown
  * [6] AI project for Pokémon Showdown. https://github.com/taylorhansen/pokemonshowdown-ai
  * [7] Core series Pokémon games. https://bulbapedia.bulbagarden.net/wiki/Core_series
  * [8] Spin-off Pokémon games. https://bulbapedia.bulbagarden.net/wiki/Spin-off_Pok%C3%A9mon_games
  * [9] Bulbapedia - Pokémon encyclopedia. https://bulbapedia.bulbagarden.net/
  * [10] Smogon - Pokémon encyclopedia and battle strategy. https://www.smogon.com/
  * [11] Serebii - Pokémon encyclopedia and news. https://serebii.net/
  * [12] S. Lee and J. Togelius, Showdown AI Competition, in _2017 IEEE Conference on Computational Intelligence and Games (CIG)_. IEEE Press, 2017, p. 191–198. https://doi.org/10.1109/CIG.2017.8080435
  * [13] H. Ho and V. Ramesh, Percymon: A Pokémon Showdown Artifical Intelligence. https://varunramesh.net/content/documents/cs221-final-report.pdf
  * [14] A. Kalose, K. Kaya, and A. Kim, Optimal battle strategy in Pokémon using reinforcement learning. https://web.stanford.edu/class/aa228/reports/2018/final151.pdf
  * [15] M. L. Sanchez, Learning complex games through self play - Pokémon battles. https://upcommons.upc.edu/bitstream/handle/2117/121655/134419.pdf
  * [16] L. Norström, Comparison of Artificial Intelligence Algorithms for Pokémon Battles. https://odr.chalmers.se/bitstream/20.500.12380/300015/1/Linus%20Norstr%C3%B6m.pdf
  * [17] D. Huang and S. Lee, A Self-Play Policy Optimization Approach to Battling PokéMon, in _2019 IEEE Conference on Games (CoG)_. IEEE Press, 2019, p. 1–4. https://doi.org/10.1109/CIG.2019.8848014
  * [18] Pokémon Bag on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Bag
  * [19] Pokémon species on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Pok%C3%A9mon_(species)
  * [20] List of Pokémon species on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/List_of_Pok%C3%A9mon_by_National_Pok%C3%A9dex_number
  * [21] Pokémon level on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Level#In_the_core_series
  * [22] Pokémon type on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Type
  * [23] Pokémon type chart on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Type/Type_chart
  * [24] Pokémon statistics on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Statistic
  * [25] Pokémon statistics formula on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Statistic#Generation_III_onward
  * [26] Pokémon Natures on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Nature
  * [27] Pokémon Individual Values (IV) on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Individual_values
  * [28] Pokémon Effort Values (EV) on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Effort_values
  * [29] Pokémon Abilities on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Ability
  * [30] List of Pokémon Abilities on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Ability#List_of_Abilities
  * [31] Pokémon Items on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Item
  * [32] List of Pokémon Items on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/List_of_items_by_index_number_(Generation_VIII)
  * [33] Pokémon Moves on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Move
  * [34] Pokémon damage calculation formula on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Damage#Damage_calculation
  * [35] List of Pokémon Moves on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/List_of_moves
  * [36] Pokémon status conditions on Bulbapedia. https://bulbapedia.bulbagarden.net/wiki/Status_condition
  * [37] Pokémon Showdown on GitHub. https://github.com/smogon/pokemon-showdown
  * [38] Pokémon Showdown battle formats. https://www.smogon.com/dex/ss/formats/
  * [39] Pokémon Showdown OverUsed battle format. https://www.smogon.com/dex/ss/formats/ou/
  * [40] Official Pokémon GO website. https://pokemongolive.com/en/
  * [41] Technical Machine Pokémon AI. https://bitbucket.org/davidstone/technical-machine/src/master/
  * [42] A. Dotson III, Future Sight AI. https://www.pokemonbattlepredictor.com/FSAI
  * [43] D. S. Nau, M. Luštrek, A. Parker, I. Bratko, and M. Gams, When is it better not to look ahead?, _Artificial Intelligence_ , vol. 174, no. 16, pp. 1323–1338, 2010. https://www.sciencedirect.com/science/article/pii/S0004370210001402
  * [44] D. S. Nau, An investigation of the causes of pathology in games, _Artificial Intelligence_ , vol. 19, no. 3, pp. 257–278, 1982. https://www.sciencedirect.com/science/article/pii/0004370282900029
  * [45] IBM CPLEX Optimizer. https://www.ibm.com/analytics/cplex-optimizer
  * [46] K. Hasan, A Distributed Chess Playing Software System Model Using Dynamic CPU Availability Prediction, 07 2011.
  * [47] C. Frayn and C. Justiniano, _The ChessBrain Project — Massively Distributed Chess Tree Search_. Berlin, Heidelberg: Springer Berlin Heidelberg, 2007, pp. 91–115. https://doi.org/10.1007/978-3-540-72705-7_5
  * [48] A. E. Elo, _The rating of Chessplayers, past and present_. Ishi Press International, 2008.
  * [49] M. E. Glickman, The glicko system, 1995.
  * [50] P. Duersch, M. Lambrecht, and J. Oechssler, Measuring skill and chance in games, _European Economic Review_ , vol. 127, no. C, 2020. https://ideas.repec.org/a/eee/eecrev/v127y2020ics0014292120301045.html
  * [51] Rating Systems on Pokémon Showdown. https://www.smogon.com/smog/issue43/elo-hello
  * [52] A. Saffidine, H. Finnsson, and M. Buro, Alpha-Beta Pruning for Games with Simultaneous Moves, _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 26, no. 1, pp. 556–562, Sep. 2021. https://ojs.aaai.org/index.php/AAAI/article/view/8148

## Appendix

### A. Battle Replays

http://sarantinos.me/pokemon_ai_battle_replays