DiffSkill: Skill Abstraction from Differentiable Physics
for Deformable Object Manipulations with Tools

ICLR 2022


Xingyu Lin, Zhiao Huang, Yunzhu Li, Joshua B. Tenenbaum, David Held, Chuang Gan

Paper Code (Coming soon)

We consider the problem of sequential robotic manipulation of deformable objects using tools. Previous works have shown that differentiable physics simulators provide gradients to the environment state and help trajectory optimization to converge orders of magnitude faster than model-free reinforcement learning algorithms for deformable object manipulation. However, such gradient-based trajectory optimization typically requires access to the full simulator states and can only solve short-horizon, single-skill tasks due to local optima. In this work, we propose a novel framework, named DiffSkill, that uses a differentiable physics simulator for skill abstraction to solve long-horizon deformable object manipulation tasks from sensory observations. In particular, we first obtain short-horizon skills using individual tools from a gradient-based optimizer, using the full state information in a differentiable simulator; we then learn a neural skill abstractor from the demonstration trajectories which takes RGBD images as input. Finally, we plan over the skills by finding the intermediate goals and then solve long-horizon tasks. We show the advantages of our method in a new set of sequential deformable object manipulation tasks compared to previous reinforcement learning algorithms and compared to the trajectory optimizer.

Task1: Lift and Spread


The task is to first lift the dough onto the cutting board and then spread it using a rolling pin.

Left below shows the plan generated by DiffSkill and the right shows the execution of the plan. The red line shows the normalized performance, while the green line shows the success threshold of the task.

DiffSkill Generated plan
DiffSkill Execution

Compare with baselines

Model-free RL (SAC)
Tool A only
Tool B only
Multi-tool
Trajectory Optimizer (Oracle)
Tool A only
Tool B only
Multi-tool

Task2: Gather and transport


The task is to first gather the scatthered dough and then trasport it onto the cutting board.

Left below shows the plan generated by DiffSkill and the right shows the execution of the plan. The red line shows the normalized performance, while the green line shows the success threshold of the task.

DiffSkill Generated plan
DiffSkill Execution

Compare with baselines

Model-free RL (SAC)
Tool A only
Tool B only
Multi-tool
Trajectory Optimizer (Oracle)
Tool A only
Tool B only
Multi-tool

Task3: Cut and rearrange


The task is to first cut the dough using the knife and the transport each part to targeted locations

Left below shows the plan generated by DiffSkill and the right shows the execution of the plan. The red line shows the normalized performance, while the green line shows the success threshold of the task.

DiffSkill Generated plan
DiffSkill Execution

Compare with baselines

Model-free RL (SAC)
Tool A only
Tool B only
Multi-tool
Trajectory Optimizer (Oracle)
Tool A only
Tool B only
Multi-tool

Bibtex


@inproceedings{lin2022diffskill, author = {Lin, Xingyu and Huang, Zhiao and Li, Yunzhu and Tenenbaum, Joshua B. and Held, David and Gan, Chuang}, title = {DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools}, journal={International Conference on Learning Representation (ICLR)}, year={2022}}