Can AI Teach Robots to Fold Laundry? Scientists Are Making Progress
Artificial intelligence has already demonstrated impressive capabilities, from generating images to writing essays. Now, researchers are working to apply AI to the world of robotics, with ambitious goals like creating robots capable of performing everyday tasks.
Chelsea Finn, an engineer and researcher at Stanford University, believes AI could revolutionize robotics. “In the long term we want to develop software that would allow the robots to operate intelligently in any situation,” she says.
A company she co-founded, Physical Intelligence, has already developed a general-purpose AI robot capable of folding laundry and other tasks. Other research teams are exploring the potential of AI to improve robots’ abilities in areas ranging from package sorting to drone racing. Google recently unveiled an AI-powered robot designed to pack lunches, indicating the growing interest in the field.
However, the research community is divided over the potential for generative AI tools to transform robotics in the same way they have transformed some online work. Robots require real-world data and face challenges that are significantly more complex than those faced by chatbots. “Robots are not going to suddenly become this science fiction dream overnight,” says Ken Goldberg, a professor at UC Berkeley. “It’s really important that people understand that, because we’re not there yet.”
Dreams and Disappointment in Robotics
Robotics has often been subject to inflated expectations, with reality lagging behind the visions. The very term “robot” was coined by Karel Čapek, a Czech writer, who in the 1920s, wrote a play about human-like beings that could perform any task their owner commanded. In reality, robots have struggled to perform even relatively simple tasks.
Robots excel at repetitive movements in controlled environments, such as on an automotive assembly line. But the real world is filled with unexpected obstacles and uncommon objects. In Finn’s Stanford University laboratory, graduate student Moo Jin Kim is demonstrating how AI-powered robots are beginning to address these challenges.
Kim has been developing “OpenVLA,” a program that stands for Vision, Language, Action. “It’s one step in the direction of ChatGPT for robotics, but there’s still a lot of work to do,” he says.
The robot itself, with its mechanical arms and pincers, appears fairly unremarkable. Its distinctiveness lies in what’s inside. Unlike regular robots, which require specific programming, this one is powered by a trainable AI neural network. Scientists believe that these networks function in a way that is similar to the human brain, with mathematical “nodes” connected in a way that resembles the way neurons are connected in the brain.
Kim can train the robot by showing it a task. He uses joysticks to control the arms, effectively “puppeteering” the robot to perform the desired action. This repetition conditions the system, and the robot then reinforces the connections that matter, weakening the ones that don’t. “Basically like whatever task you want it to do you just keep doing it over and over like 50 times or 100 times,” he says. Soon the robot can repeat the task without the help of an operator.
To demonstrate the setup, Kim presents a tray of trail mix. Having already taught the robot how to scoop, he types a command: “Scoop some green ones with the nuts into the bowl.” The robot’s arms respond slowly, and on a video feed, OpenVLA places a marker over the correct bin. Kim remarks, “That’s the part where we hold our breath.” The robot then slowly reaches for the scoop and retrieves the trail mix. “It looks like it’s working!” says Kim excitedly. It’s a small scoop, but the direction is correct.
The Future of AI-Powered Robots
Finn’s company, Physical Intelligence, aims to advance this training approach. She envisions robots that can quickly adapt to performing simple jobs, like restocking shelves in a grocery store. Finn believes that training a single model to perform a wide variety of tasks will be the most successful approach, in contrast to the current strategy of developing systems for a single purpose. “We actually think that trying to develop generalist systems will be more successful than trying to develop a system that does one thing very, very well,” she says.
However, the next step is to create training data to enable the AI system; this process is considerably more difficult than collecting text from the Internet to train a chatbot. “This is really hard,” Finn concedes. “We don’t have an open internet of robot data, and so oftentimes it comes down to collecting the data ourselves on robots.”
Despite optimism, Goldberg is skeptical that the real-world data gap can be readily bridged. AI chatbots have improved substantially due to the enormous amount of data they can learn from. Building an equivalent amount of data for real-world robots will take considerably longer. “At this current rate, we’re going to take 100,000 years to get that much data,” he says.
Pulkit Agrawal, a robotics researcher at MIT, agrees that current training methods need refinement. Agrawal advocates for simulation, a method that involves running the robot’s AI neural network in a virtual world. Researchers in Switzerland recently used this approach to effectively train a drone for racing. However, simulations have limitations, as real-world factors such as wind or sunlight can throw results off-track.
Goldberg says that more complex manual tasks, like grasping and manipulation, are particularly difficult to replicate in a computer. “Basically there is no simulator that can accurately model manipulation,” he says.
Looking Ahead: What Is the Right Approach?
Matthew Johnson-Roberson, a researcher at Carnegie Mellon University, believes that overcoming data limitations alone will not be enough. He suggests that the framing of the problem may be the core challenge. He points out that the task of chatbots, which is to predict the next word in a sentence, is simpler than the tasks robots are expected to perform, such as executing a task in space and time.
Johnson-Roberson believes that more fundamental research into how neural networks process space and time is required. He cautions the field to learn from past experiences, such as the rush to build self-driving cars. “So much capital rushed in so quickly,” he comments. “It incentivized people to make promises on a timeline they couldn’t possibly deliver on.”
Even the skeptics concede that AI will forever change robotics. Goldberg co-founded Ambi Robotics, a company that produces an AI-driven system that identifies the optimal pick-up points for packages and has led to a substantial reduction in dropped packages. Yet, he adds with a laugh, “if you put this thing in front of a pile of clothes, it’s not going to know what to do with that.”
Back at Stanford, Finn reiterates that expectations must be kept in check. “I think there’s still a long way for the technology to go,” she says. She doesn’t expect that AI-powered robots will completely replace human labor, especially for complex jobs, but she believes they hold promise in addressing labor shortages. “I’m envisioning that this is really going to be something that’s augmenting people and helping people,” she says.
