We are training and testing handkerchief folding on a dual-arm robot built from two SO-ARM101 arms. People operate the robot to collect example movements (demonstration data), and we check on the real robot whether a model trained on them can grasp, lift, and fold the cloth.
We first tried ACT, an imitation learning method that learns robot movements from demonstrations. An ACT model trained on 100 demonstrations folded the handkerchief into quarters in 37 out of 40 trials, using the same handkerchief and placement conditions as in training. However, it sometimes failed when the pattern or placement changed, revealing the challenge of handling different conditions.
We are also fine-tuning π0.5, a general-purpose model that produces robot actions from images and language, for handkerchief folding. After revisiting the training process and the robot’s control conditions, we confirmed successful folding on the real robot. Alongside numerical metrics during training, we observe actual movements and how failures unfold as we keep improving the model’s ability to perform the task reliably.
