This AI Can ‘Catch Them All’ in Pokemon Battles As Well as Humans

Gotta catch them all? There’s an AI for that. Introducing POKE´LLMON, a new LLM-based AI agent designed to play Pokémon battles with human-like proficiency.

POKE´LLMON is the brainchild of researchers at the Georgia Institute of Technology, who say it uses in-context reinforcement learning and knowledge-augmented generation to learn from its gameplay experiences and makes decisions with remarkable accuracy.

The model is so good, in fact, it notched notable win rates against real human players in Pokemon battles.

Pokemon AI?

The university AI researchers set out to develop a cutting-edge AI agent—a persona powered by an AI model that both plays the game and learns from it, mirroring human learning and decision-making processes. Unlike the legacy approach, in which a machine-controlled player would follow pre-programmed rules, its developers say their AI model evolves, tries new things, and behaves more like a human player than an algorithm.

It’s also designed to work on other virtual battlefields.

“[It’s] the first LLM embodied agent that achieves human-parity performance in tactical battle games, as demonstrated in Pokemon battles,” the research team wrote. “The architecture of POKE´LLMON is general and can be adapted for the design of LLM-embodied agents in many other games.”

Image: Georgia Institute of Technology

The core of POKE´LLMON’s prowess lies in its advanced in-context reinforcement learning mechanism, which effectively evolves as it wins and loses battles, getting increasingly adept at predicting and countering opponents’ moves.

Complementing its learning ability, POKE´LLMON also applies what its creators call a knowledge-augmented generation technique. This approach allows the AI to integrate external, verified knowledge into its decision-making process, ensuring high accuracy and contextually relevant choices during battles.

This strategy is especially helpful in countering potential hallucinations – a common challenge in AI systems. As implemented, POKE´LLMON gameplay is both creative and grounded in solid, game-specific information.

Developers also made sure that POKE´LLMON is not a shrinking violet. The model applies a consistent action generation technique to ensure that it remains composed and strategically consistent, even when facing formidable opponents. This aspect of the AI implementation prevents the panic-driven decisions that plagues human competitors.

“Action generation conditioned on panic thoughts leads the agent to continuously switch Pokemons instead of attacking,” the researchers note. “In comparison, consistent action generation with SC (self-consistency) decreases the continuous switch ratio by independently generating actions multiple times and voting out the most consistent action.”

The AI’s performance in the arena is nothing short of impressive. With a 49% win rate in “ladder competitions” and an even more remarkable 56% win rate in invited battles, it has proven its mettle against a spectrum of challengers, human and not.

Don’t let the playful vibe of Pokémon fool you—there’s a world of competitive strategy to explore beneath its colorful surface. Research like POKE´LLMON could serve as the stepping stone for new models that power new games.

The nearest comparable game is probably chess, and online chess sites detect cheaters based on their moves and the probability of executing an attack versus what a human can or would do. Computer algorithms are configured to execute the best move every time, which gives them—or people who use them—a distinct advantage.

With adaptable, human-like AI, however, these cheating tools may soon be obsolete, making human-versus-machine battles more fun and challenging.

Source