FLoRe: Learning Force-Reactive Humanoid Loco-Manipulation

Abstract

Many everyday tasks require controlling contact force as well as motion. On a humanoid, this force must often be coordinated across both hands and the whole body, and cannot be specified by kinematic trajectories alone. Learning such tasks by imitation therefore needs demonstrations that record contact force, and a whole-body controller that can deliver it. We present FLoRe, a framework for learning force-reactive humanoid loco-manipulation. Its robot-free force-annotated data collection system records the contact wrench applied through two UMI-style handheld grippers, together with hand, torso, and foot motion. A force-reactive diffusion policy takes vision, proprioception, and measured wrenches as input, then predicts actions and per-hand force commands. A hybrid compliance whole-body controller with force tracking executes these commands while maintaining balance: an integral of the force error measured by real force/torque (F/T) sensors lets each hand track force accurately along the commanded direction and comply along the others. In simulation, the controller tracks per-hand force commands of varied magnitude and direction, with little loss in whole-body motion accuracy. Across three contact-rich tasks on the robot, the force-reactive policy succeeds in 80% of rollouts, compared with 26.7% for a position-only policy trained on the same demonstrations and executed by the same controller.

Pipeline

Hybrid Compliance Whole-body Controller
with Force Tracking

The control law. Each hand's force command becomes a shift of its position target. Along the task direction $x$ , the target and the integral that shifts it are

\ptgt_x \;=\; \underbrace{p^{*}_x}_{\text{motion reference}} \;+\; \underbrace{\frac{\fcmd_x}{K}}_{\text{force command}} \;+\; \underbrace{\frac{\Ii_x}{K}}_{\text{force feedback}}

\Ii_x \;\leftarrow\; \underbrace{c}_{\text{contact phase}}\,\operatorname{clip}\Big(\Ii_x + K_I\,\underbrace{\big(\fcmd_x + \fmeas_x\big)}_{\text{force error}}\,\Delta t,\ \pm I_{\mathrm{max}}\Big)

Unified reference generator. The policy outputs only joint positions, so a force command has to become a position target. The motion's reference $p^{*}_x$ brings the hand to the surface, and the force command $\fcmd_x$ , the force the hand should exert on it, then moves the target into it by $\fcmd_x/K$ , with $K$ a nominal stiffness. The hand cannot follow the target there, so in tracking it the hand presses on the surface; its wrist F/T sensor measures $\fmeas_x$ , the force the surface exerts on the hand, and tracking means $\fmeas_x = -\fcmd_x$ .

How hard a shift presses depends on the stiffness of the contact, which the robot does not know. So $\Ii_x$ , the integral of the force error with gain $K_I$ , moves the target further in, or back, by $\Ii_x/K$ , until the measured force balances the command; the contact phase $c$ lets it build up only in contact and clears it on release. The policy is trained to reach $\ptgt_x$ while it balances.

The integral at work. An idealised press along

x

, run in your browser: the reference brings the hand to the surface, the command ramps up, holds and ramps down, and the reference leaves again. Drag the dot to change

K

and

K_I

: the integral makes up the difference, and

-\fmeas_x

heads for

\fcmd_x

, the faster the higher

K_I

and the lower

K

; at a low

K_I

it is still on its way when the command ends. It cannot pass its bound

I_{\mathrm{max}}

(dotted): towards a high

K

and

K_I

it reaches it and stays there, flat, and

-\fmeas_x

falls short. The ring marks the paper's gains.

$\fcmd_x$ $-\fmeas_x$ $\Ii_x$ $I_{\mathrm{max}}$

The chart runs with JavaScript.

Compliant behavior. On the robot the law runs on all three axes of each hand, and a measured force need not lie along the force direction. A push along $z$ , say, where nothing is commanded, is measured too: $\fmeas_z = \fext_z$ , the external force on the hand. In contact, with $\fcmd_z = 0$ , the integral then builds up with the push and moves the target along with it, for as long as the push lasts and as far as its bound allows: the hand gives way instead of resisting it.

\begin{aligned} x:&\quad \fcmd_x > 0 &&\Longrightarrow\quad \fmeas_x \to -\fcmd_x \qquad && \text{force tracking}\\[2pt] z:&\quad \fcmd_z = 0 &&\Longrightarrow\quad \dot{\Ii}_z = K_I\,\fext_z \qquad && \text{compliance} \end{aligned}

So the hand tracks the force along the force direction and complies along the other axes at once, with nothing switching between the two.

Compared with other learned controllers. Learned legged and humanoid controllers treat the force that the world exerts on the hand, the external force $\fext$ , in one of two ways. They reject it as a disturbance and hold the reference (FALCON), or give way to it with a commanded stiffness, which at rest makes the hand a spring (FACET, SoftMimic, CHIP):

\begin{aligned} \text{FALCON:}\quad \ptgt_x &= p^{*}_x \\[4pt] \text{FACET, SoftMimic, CHIP:}\quad \ptgt_x &= p^{*}_x + \frac{\fext_x}{K} \quad \text{at rest} \\[4pt] \text{FLoRe (ours):}\quad \ptgt_x &= p^{*}_x + \frac{\fcmd_x + \Ii_x}{K} \\[2pt] \dot{\Ii}_x &= K_I\,\big(\fcmd_x + \fmeas_x\big) \quad \text{in contact} \end{aligned}

A stiff hand holds its reference however hard it is pushed, and a compliant one gives way, but neither can be told how hard to press. Only FLoRe measures the force at each wrist and integrates its error, so that one whole-body policy tracks position in free space and force along the task direction, and complies along the other axes.

Try it below: the controller runs in your browser, with MuJoCo and the policy compiled to WebAssembly as in the code release. The left hand faces a virtual wall along x, with the compliant contact of training, and each arrow carries its value along x, so you can read how closely F^meas mirrors F^cmd: the same force, pushing back. Of the forces from outside, the simulated F/T sensor reads the wall's alone, as in training, where nothing else touches the hand: the orange F^ext is an extra push it does not read, which the policy has to reject.

▶ Start the interactive demo Loads about 55 MB and runs entirely in your browser; best on a computer

Command a force Set F^cmd (purple) with the upper slider, up to 20 N: the hand presses on the wall until F^meas (blue) reads −F^cmd, the same force pushing back.
Push the hand Push or pull it with F^ext (orange), the lower slider: F^meas returns to −F^cmd. Or drag the robot with the mouse.
Look around Drag to orbit and scroll to zoom; R resets. A panel folds away when you click its title.

Try the full demo in a new tab!

Compliant Behavior

Force Tracking (x direction −10 N)

Here F^cmd = −10 N along x, and the gauge reads the same force as the wrist's F/T sensor, F^meas. When the external force grows or shrinks, the reading strays from 10 N, and the controller gradually brings it back.

Real-world Experiments

Three contact-rich tasks on a Unitree G1. Across them, the force-reactive policy succeeds in 80% of rollouts, against 26.7% for a position-only policy trained on the same demonstrations and executed by the same controller.

Wipe Board

The robot presses an eraser against a whiteboard and wipes a marked strip. Success depends on the normal force, which the images do not show.

Force of the wiping hand over time: one stroke, measured and predicted per axis.

Force of the wiping hand over time: two strokes, measured and predicted per axis.

Baseline failure

Without a force command, the position-only baseline brings the eraser to the board but does not press it on: the eraser hovers over the mark and leaves it.

Rotate Box

The robot pinches the rim of a crate with both hands, rotates it by about 50°, drags it about 0.5 m across the table and releases it. The hands share an internal force: if one leads the other, the crate tips.

Force of the left and the right hand over time while the crate is rotated, measured and predicted per axis.

Force of the left and the right hand over time in a second rollout, measured and predicted per axis.

Baseline failure

Without a force command, the position-only baseline puts both hands on the rim but does not squeeze it, and the heavy crate does not turn.

Push Gantry

The robot places both hands on a heavy wheeled frame and walks it forward. The legs and torso produce the push while the arms keep both hands on the frame: too little force does not move it, too much makes the hands slip or the robot lose balance.

Force of the left and the right hand over time while the gantry is pushed, measured and predicted per axis.

Baseline failure

Without a force command, the position-only baseline reaches the demonstrated pose, but the frame does not move and the policy has no way to push harder; in the end the robot loses its balance.

Robot-free Force-annotated Data Collection System

A demonstrator holding the two handheld grippers and wearing a torso tracker, beside a close-up of one
gripper with its Vive Tracker 3.0, RealSense D405 camera, F/T sensor, YAM linear gripper and USB 2-key. — Two handheld UMI-style grippers, each carrying a 6-axis F/T sensor, and three trackers record hand and torso motion, gripper width and contact force – with no robot in the loop.

A Rotate Box demonstration: the two grippers pinch the crate’s rim, turn it and drag it across the table.

Abstract

Pipeline

Hybrid Compliance Whole-body Controller with Force Tracking

Real-world Experiments

Wipe Board

Baseline failure

Rotate Box

Baseline failure

Push Gantry

Baseline failure

Robot-free Force-annotated Data Collection System

Hybrid Compliance Whole-body Controller
with Force Tracking