Unleashing the Future: How Generative AI Paves the Way for GENERAL-PURPOSE Humanoids

The goal of humanoid robotics has always been to create machines capable of performing a broad range of tasks like humans do, but making the change from fantasy into reality is a tall order. Building general-purpose humanoids is just one of many problems we’ll have to tackle along the way. But this endeavour provides one possible plot for humanity’s technological future – generative AI being a part-solution.

The Dawn of a New Era in Robotics

Humanoids have long been a fantasy object, a robot that could act human, and ideas about what that would be like have transfixed average imaginations across the globe. The path to a ‘general purpose humanoid’ – a robot that could adapt to any task as easily as a human – has always been an incredibly difficult one to traverse. Generative AI could fundamentally change the narrative, essentially becoming a key turning point within the plot.

The PRIME Challenge: Training for Versatility

Underlying this challenge is the problem of training: for decades, researchers have sought to churn out robots that can perform a very specific task. Now the ambition is to leap to making flexible, general-use systems that can handle a multitude of environments and needs. This vision requires a fundamental shift in training approaches – and addressing this, above all else, is the reason generative AI is being hailed as a possible solution.

Generative AI: A PRIME Mover in Robotics

At institutes such as MIT, recent insights into generative AI have shown how it can radically improve the way robots learn and adapt. The robots that result from this research are called ‘PolCo’ (for ‘policy composition’) models, meaning that they use generative AI models to capture knowledge from multiple data sources and to leverage that knowledge to be able to perform a larger set of tasks simultaneously.

The Unveiling of PoCo: A Generative Leap Forward

When these ‘policies’ (defined as strategies for accomplishing a given task) are combined, the system can accomplish a plethora of tasks, from hammering or using a spatula to generalising to tasks it has never seen before. The enhanced performance by diffusion models has been a key contributor to this success, and represents a new dawn of dexterous robots.

A Symbiosis of Real-World Dexterity and Digital Generalization

What makes generative AI in robotics so beautiful is that it takes the best of each of these learning environments and combines them. As the researcher Lirui Wang explained it: ‘By combining policies trained on real-world data with those trained in the simulator, a robot can have the optimal level of both dexterity and abilities to adapt to unseen situations.’ The digital and physical worlds coming together to make robots better at their tasks is a good example of synergy.

Toward the Horizon of GENERAL-PURPOSE Humanoids

These implications also go far beyond laboratory settings. Once swapping tools and adapting to new tasks becomes easy, it should be time for true, general-purpose humanoids to enter our world. Our best guess is that they will be heavily involved in construction work, domestic service, factory tasks and house calls to the sick. But that’s speculation.

PRIME Possibilities: Envisioning the Future

Generative AI-powered robotics is poised to offer a goldmine of development and research pathways. As AI-driven robots learn to adapt to new environments and teach themselves from varied datasets, the dream of humanoid general-purpose robots is closer than ever. The potential applications are as diverse as they are transformative, with a future that could see the robots assisting humanity in solving our most complex problems, right where we live.

Understanding PRIME in the Context of Robotics

When in the article we say a word is ‘prime’, we mean it to stand for crucial or essential. Whether it’s the prime challenge of how to train robots, the use of generative AI as a prime mover of robotics research, or if we point to prime examples of technological lighthouses, it’s always important to emphasise that these are the prime movers of robotics. It’s those issues that matter. If humanoid technology is going to make our lives better, those are the issues we have to focus on.

Finally, generative AI represents what I believe is the beginning of a long-anticipated new phase of humanoid robotics, a phase that will overcome many barriers to versatility and generalisation. My argument in this essay is that, on the most promising path forward with generative AI, ‘prime capabilities’ will indeed exist, and in large numbers. This is because AI systems capable of prime capabilities will be necessary to achieve the high goals we’re after. Armed with these prime capabilities, researchers and developers will make progress towards the ultimate target: general-purpose humanoids who become part of our everyday lives. The road to this new, grand phase for robotics will not be easy. Problems are plentiful. But the potential payoff – prime capabilities on a grand scale – is so great that it’s worth pressing on. It’s evocative of a new (not entirely science-fictional) future, where robots and humans can work together in novel ways.