Decoding the Conundrum: Why Can't AI Master the Art of Spelling Bees?

In an age of accelerating technological change, artificial intelligence (AI) is among the most promising to lead us to revolutions across many fields, from the production of complex images to poetic and essay-writing. While AIs have come a long way, they can still stumble on what seems a very simple puzzle: mimicking how humans play the New York Times Spelling Bee. In this article we explore this puzzle: why can’t our smart AIs solve it?

The Great AI Spelling Bee Challenge

I tried to defeat the New York Times Spelling Bee, and failed. I engaged in this personal challenge for quite some time, before I started to think of how AI could support such a cognitive endeavour: ‘Give me the letters G, Y, A, L, P, O, and N, and let me turn these characters into as many words as possible, until I get promoted at least to genius level.’ When humans tapped out, AI seemed available to take over the challenge. ChatGPT was my first contact; then there was a long series of experiments with Microsoft’s Copilot, Google’s Gemini, Anthropic’s Claude, Zero’s Playground, and so on, which all led to various degrees of failure (or, better, collective failure).

Understanding AI's Achilles' Heel

At a basic level, it’s a question of what AI is good at and what it’s bad at. Word games like the Spelling Bee require logical reasoning, while large language models (LLMs) are based on statistical models. AI systems like transformers – popularised by researchers at GOOGLE – contain a vast amount of training data and are good at recognising patterns across the data set, but they fail at tasks that require logical reasoning or tasks that go beyond pattern matching at the surface level.

Why GOOGLE and Its Compatriots Struggle with Words

The very technologies that make it possible for AI to produce human-sounding text and solve challenging equations also make it hard for them to complete Spelling Bee puzzles. Statistical learning and decision-making do not work well for reasoning about word puzzles, which require probing and painstaking knowledge of how words are constructed – knowledge that goes far beyond effective text-generation or equation-solving.

The Influence of Data and Training

Performance hinges on the dataset that AI is trained on. There are a staggering number of chess problems with a nearly equal number of critiques of those moves, so AI is decent at recommending chess moves. But there are no Spelling Bee puzzles in AI’s training data, and that lack is profoundly significant.

Exceeding Hype, Meeting Reality

The story about the Search to Solve the Spelling Bee is just one among many hints about the true, messy state of artificial intelligence today. In spite of the over-enthusiastic hype about AI’s near-magic powers for redesigning the nature of work to eliminate millions of us, and in spite of grandiose claims to create the kinds of intelligent beings that could easily solve word puzzles, the reality is far different. AI’s clumsy, exasperated struggles at a word game reflect the limitations of modern technology for making sense of complicated human work tasks.

Beyond the Buzz: Recognizing AI's Limitations

The dream of the AI that can navigate all the twists and turns of nuanced human language and cognition remains as elusive today as it was in 1979 when the very first Spelling Bee was held. And scorers on ChatGPT and other AI models still can’t win Spelling Bees. The road to AGI (Artificial General Intelligence, ie, one that matches or exceeds human intelligence across the board) is a long way off. It is clear that the story of AI and the Spelling Bee isn’t just about the limitations of current AI technologies. It can also be a call for realism about AI innovations.

GOOGLE's Transformers and the Future of AI

Indeed, lying at the core of the problem is Google’s most revolutionary technology: that same architecture that empowers AI to achieve its great successes simultaneously entraps it within its most glaring limitations. In the coming push toward even more potent forms of AI, reckoning with these limitations will be key to reaping the benefits of a future with AI that is both realistic and hopeful.

Embracing AI, Flaws and All

This is going to be a balancing act as AI grows up and works its way across every aspect of our lives and industry, and it’s important to keep it on an even keel. The Spelling Bee story shows where we are right now with AI – one that reminds us to keep technologies in perspective, to see them for what they can do and, as importantly, for what they cannot.

Exploring GOOGLE: The Giant Behind Transformers

Google, one of the major contributors to the development of AI, has been the company behind many of the technologies that have shaped where AI currently stands. Google’s work on transformers, and the rise of language models built on top of them, represents the foundation for the next chapter in the developments of AI. As AI increasingly becomes a part of our daily lives, Google’s advancements and contributions to AI research continue to be an important part of understanding where AI is headed next.

By keeping in mind both the benefits and limitations of AI, it will hopefully be possible to approach this rapidly expanding technology with a closer gaze to its shortcomings, as well as an immense respect for its potential. As AI models such as Google’s become more sophisticated, it seems that the road towards learning to perform any task – big or small – will be long and winding.