Using household tasks from the RoboCup world competition, we are working on a domestic service robot that tidies up objects in a room. Grasping, carrying, and placing are not learned. Each is carried out by rule-based control that follows predefined steps, and we use an LLM (large language model) to update those rules in a simulation environment.
The movements are divided into stages such as approach, grasp, lift, carry, lower, and back away, and run in that order for each object. Because each stage follows rules with defined conditions and steps, it is easy to see what the robot is doing at any moment and to pinpoint the stage where something went wrong. We run a task in simulation, check which stage did not go well, and the LLM updates the rules based on those results before we test again. Because this cycle runs without moving the real robot, the rules can be improved safely and as many times as needed.
The video records a continuous test in a simulated home, in which the robot tidies up 14 objects, including fruit, tableware, drinks, and food boxes, one after another (played at 6× speed). The top of the screen shows the target object, the number of objects completed, and the current stage. For each object, the robot records whether placement succeeded and the final check before moving on.
