A reinforcement learning framework for improving soccer shooting skills of quadruped robots

0

Credit: Ji et al.

Researchers from the University of California at Berkeley (UC Berkeley), University of Montreal and Mila have recently developed a hierarchical reinforcement learning framework to improve the accuracy of quadrupedal robots in soccer shooting. This framework, presented in a pre-published article on arXiv, was deployed on a Unitree A1, a quadruped robot developed by UnitreeRobotics.

“Human legs are not only for locomotion, but can also be used for manipulations like playing football, and we want to enable quadrupedal robots to achieve this ability as well,” said Zhongyu Li, one of the researchers. who conducted the study, to TechXplore. “There is a notable league in the robotics community called ‘RoboCup’ (Robot World Cup), which has been inviting researchers to train their robots to play soccer games for decades.”

Recent advances have enabled the creation of more reliable hardware and advanced control algorithms for robots. As a result, robots are now more agile and could potentially tackle more complex tasks, including playing football alongside humans. The framework developed by Li and his colleagues could help improve the ability of quadrupedal robots to throw the ball during football matches.

The new framework has two key elements: a movement control policy and a movement planning policy. The motion control component allows the robot to follow an arbitrary trajectory for the toe on its kicking leg. The motion planning policy, on the other hand, selects an optimal toe trajectory to shoot a nearby soccer ball (detected by an external camera) towards a target location (e.g., the goal post).

A reinforcement learning framework for improving soccer shooting skills of quadruped robots

Credit: Ji et al.

“Our design allows us to decouple the challenge of the precise soccer shooting task into two subtasks: control and planning,” Li said. robust control that can run on hardware and then reuse such a controller to learn planning strategy.To accurately shoot the bullet at targets in the real world, the planner is trained using real-world data when the robot shoots the real football.”

Li and his colleagues tested their framework in a series of real-world tests, using an A1 quadruped robot. They discovered that this allowed the robot to fire a deformable soccer ball at random targets with high accuracy. This is a very complex task to accomplish, as the robot must quickly swing its kicking leg and gain momentum without losing its balance.

“The soccer ball presents more challenges because the robot has to deal with not only the hard-to-model soft contact with the deformable ball, but also the uncertainties of the rolling friction between the ball and the ground,” Li said. that we have developed to solve such problems could potentially be useful for tasks where dynamic robots, such as legged robots, need to interact with soft objects, such as a ball, ropes, leash, clothes, etc.”

  • A reinforcement learning framework for improving soccer shooting skills of quadruped robots

    Credit: Ji et al.

  • A reinforcement learning framework for improving soccer shooting skills of quadruped robots

    Credit: Ji et al.

  • A reinforcement learning framework for improving soccer shooting skills of quadruped robots

    Credit: Ji et al.

In the future, the framework created by this team of researchers could be used to improve the performance of robots in football tournaments, in particular the Robocup. Meanwhile, Li and his colleagues plan to design other frameworks and machine learning models to improve robot performance in other elements of the soccer game.

“Our long-term goal is to develop quadruped robot footballers who could one day compete with humans,” Li added. “We are developing more complex football skills using quadruped robots and hope that in the future soon, we will be able to launch a fully autonomous football match using quadruped robots.”


A Q-learning algorithm to generate kicks for walking robots in football simulations


More information:
Yandong Ji et al, Hierarchical Reinforcement Learning for Accurate Soccer Shooting Skills Using a Quadruped Robot. arXiv:2208.01160v1 [cs.RO], arxiv.org/abs/2208.01160

© 2022 Science X Network

Quote: A reinforcement learning framework to improve the soccer shooting skills of quadruped robots (2022, August 22) Retrieved on August 22, 2022 from https://techxplore.com/news/2022-08-framework-soccer-skills -quadruped-robots.html

This document is subject to copyright. Except for fair use for purposes of private study or research, no part may be reproduced without written permission. The content is provided for information only.

Share.

Comments are closed.