Solving the challenges of robot pizza making

Could this be the beginning of the end for your favourite pizza crust chef? Probably not, but US scientists have worked out a way to teach robots how to deal with pliable substances such as pizza dough.

Robots find working with deformable objects like dough difficult for a variety of reasons. The shape changes continuously which is difficult to represent in an equation, while multiple steps or different tools are often required to do so.

It’s also difficult for robots to learn a manipulation task with a long sequence of steps – where there are many possible choices to make – since their learning often occurs through trial and error.

Now, researchers have come up with a better way to teach robots how to make pizza dough, by creating a new framework that uses a two-stage learning process.

This method – which they’ve called DiffSkill – could enable a robot to perform complex manipulation tasks over a long timeframe, like making pizza bases.

---

---

“This method is closer to how we as humans plan our actions,” says Yunzhu Li, a graduate student in the Computer Science and Artificial Intelligence Laboratory (CSAIL) at Massachusetts Institute of Technology (MIT) in the US and an author of a new paper presenting DiffSkill.

“When a human does a long-horizon task, we are not writing down all the details.

“We have a higher-level planner that roughly tells us what the stages are and some of the intermediate goals we need to achieve along the way, and then we execute them.”

DiffSkill works by first having a “teacher” algorithm solve each step the robot must take to complete the task within a differentiable physics simulator (a computer simulation that models the physics of the real world).

It’s what’s known as a trajectory optimisation algorithm that can solve short-horizon tasks where an object’s initial state and target location are close together. The “teacher” algorithm uses the information in the simulator to learn how the dough must move at each stage of the process, one at a time, and then outputs those trajectories.

Then it trains a “student” neural network that learns to imitate these actions. The student uses two camera images, one showing the dough in its current state and the other showing the dough at the end of the task, as inputs which then generate a high-level plan to link different skills in order to reach the end goal.

It then generates specific, short-horizon trajectories for each skill and sends commands directly to the tools.

The scientists tested this technique with three different simulated dough manipulation tasks and found that DiffSkill was able to outperform other popular machine learning techniques that rely on a robot learning through trial and error.

In fact, DiffSkill was the only method that was able to successfully complete all three dough manipulation tasks.

“Our framework provides a novel way for robots to acquire new skills,” says lead author Xingyu Lin, a graduate student in the Robotics Institute at Carnegie Mellon University (CMU) in the US. “These skills can then be chained to solve more complex tasks which are beyond the capability of previous robot systems.”

The researchers intend to improve DiffSkill’s performance by using 3D data as inputs instead of images (that can be difficult to transfer from simulation to the real world) and hope to apply the method to more diverse tasks like cloth manipulation.

In the future, this method could be applied in settings where a robot needs to manipulate deformable objects, such as a caregiving robot that feeds, bathes, or dresses someone who is elderly or who has motor impairments.

The research will be presented at the Tenth International Conference on Learning Representations (ICLR 2022), a machine learning and artificial intelligence conference held online from 25-29 April 2022.