Robots learn to walk and work. While robot dogs aren’t yet man’s best friend, true autonomy and reasoning will make them useful companions in industry, search and rescue, and space exploration. But you have to walk before you can run, and the machines are learning biology lessons for better walking robots.
The opening chords of the 1960s Motown song Do You Love Me, by the Contours ring out over the speakers as the robots begin to dance. Several models, including a bipedal humanoid version and a four-legged dog-like craft, are seen dancing with each other. They hang around, pirouette and sway.
Introduced by American robotics company Boston Dynamics, the viral robot video with dancing legs caused a stir in late 2020. Reactions ranged from people suggesting it was made using CGI, to fearing that robots were taking over the world. Yet for all the impressive engineering, the video also showed the limitations that legged robots face. While for humans dancing is fairly easy, for robots it’s incredibly difficult, and the three-minute video meant that every robot move had to be manually scripted in detail.
“Today, robots are still relatively stupid,” said Marco Hutter, a professor at ETH Zurich and an expert in robotics. “A lot of Boston Dynamics videos are handcrafted movements for specific environments. They need human oversight. In terms of actual autonomy and reasoning, we’re still a long way from humans, animals, or whatever. what we expect from science fiction.
Still, these types of robots could be very useful to mankind. They could help us with disasters, they could improve industrial operations and logistics, and they could even help us explore outer space. But for that to happen, we need to improve legged robots in basic tasks like walking and teach them to do it without supervision.
The ERC project Le Mo is one of the investigations launched by European researchers to make robots more autonomous. Their basic premise is that locomotion by legs is not what it could be and that machine learning techniques could improve it. LeMo specifically focuses on what is called reinforcement learning.
“Reinforcement learning uses simulation to generate big data for neural network control policy training,” explained Hutter, who is also LeMo’s project manager. “The better the robot advances in the simulation, the more reward it gets.” If the robot falls or slips, it is punished.
The robot they use in the project is a 50-kilogram dog-like four-legged robot. Above, several sensors and cameras allow it to detect its environment. This part has become fairly standard for legged robots, but the advancement produced by LeMo lies in the software. Instead of using a model-based approach, where researchers program rules into the system, like “when there’s a rock on the ground, raise your feet higher,” they “train” an AI system in a simulation.
Here, the robot system runs through a virtual terrain simulation again and again, and each time it performs well, it receives a reward. Each time he fails, he receives a punishment. By repeating this process millions of times, the robot learns to walk by trial and error.
“LeMo is one of the first times reinforcement learning was used on legged robots,” Hutter said. “Thanks to this, the robot can now traverse difficult terrain, such as slippery floors and inclined steps. We hardly fall anymore.
Using this technology, the ETH Zurich team recently won a $2 million Defense Advanced Research Projects Agency (DARPA) competition in which teams were challenged to deploy a fleet of robots to explore difficult underground areas by themselves.
“Legged robots are already being used for industrial inspections and other observational tasks,” Hutter said. “But there are also applications such as search and rescue and even space exploration, where we need better locomotion. Using techniques like reinforcement learning, we can achieve this.
Another ERC project, called M-runners, works on how to build legged robots that work in space. Today, when we launch robots to places like the moon or Mars, they are usually wheeled robots. These must land and taxi on relatively flat terrain.
“But the interesting things for geologists are usually not found on the plains,” said Professor Alin Albu-Schäffer, of TU Munich and the German Aerospace Center. “They’re found in places like canyons, where rovers can’t easily go.”
This is why there is strong interest in sending legged robots into space. But before we can do that, more research needs to be done to make them more effective. M-Runner is inspired by nature here.
“Our hypothesis is that biology is more energy efficient,” Albu-Schäffer said. “Our muscles and tendons have a certain elasticity. Animals, like a galloping horse, use this elasticity to store and release energy. Traditional robots, on the other hand, are rigid and don’t do that.
This means that legged robots aren’t as efficient as they could be. But actually understanding these processes and transferring them to robots is quite a challenge. This requires a deep understanding of the biology, but also the math behind how movements are performed and repeated.
The complex system of the limb, with a large number of interdependent parts like muscles, tendons and bones, working together very closely to repeat movements like walking or running. “Modeling this mathematically is a scientifically unresolved question,” Albu-Schäffer said.
This is what the M-Runner project tries to solve, and to transfer to robots, a strongly interdisciplinary quest. “We work on biomechanics and biological systems,” Albu-Schäffer said. “But also neuroscience, mathematics and physics. In turn, we build tools that apply this to real robots.
So far, the project has already built a prototype robot, a dog-sized variant, on which researchers are testing different types of running and gaits. The end goal is to apply this theoretical research to a role such as space exploration. “We also think about low gravity in simulations,” says Albu-Schäffer. “The robot here can make more spectacular jumps and go farther.”
Beyond this research, legged robots are already integrated into our economy and our society today. “These machines are already in use,” Hutter said. “It’s not yet a household item. But in industrial settings, it’s becoming increasingly popular, and in China, even domestic use cases are being explored.
But their mass-market appeal hinges on improving how these robots walk and act in the real world. This is why more research is needed. “Legged robots aren’t just about Boston Dynamics,” Albu-Schäffer said. “In Europe, cutting-edge research is also taking place, and we are seeing real advances in technology.”
The research in this article was funded by the EU. This article was originally publishedin Skylinethe European magazine for research and innovation.