Reproduction · Grandia et al., RSS 2024

BDX

Episode R / A New Gait

A small droid learned to walk in thousands of simulated worlds at once. This page shows how it was taught, how each result compares with Design and Control of a Bipedal Robotic Character, and, in a short film, where those first steps lead.

0.017rad standing MAE

1.8rad/s turning, as in the paper

0falls / 15,360 s each

Watch the film

Transmission · ten chapters

The film

A small droid wakes under two suns and follows a distant signal across the dunes. By nightfall no one has answered, until someone does. Every pose of both droids comes from a trained policy running in simulation; Blender adds only light, lens and paint. Each footprint and each puff of dust sits where a simulated foot touched down, and each footstep you hear is a real step on desert sand, taken from a field recording.

1280 × 720 · 24 fps · stereoBlender Cycles on an RTX 4090 · MuJoCo rollouts at 50 Hz

Prologue

EPISODE RA NEW GAIT

In the quiet of a simulator, thousands of copies of one small droid fall, reset and try again, all at the same time. From their mistakes a single policy is distilled: a way of walking.

Its gait was drawn by animators. One reward keeps it close to their drawing, and on its feet through pushes, slips and uneven ground.

Now the policy is given a body, and a desert under two suns. Somewhere past the dunes, a relay has begun to call.

Technical readout

A stand-in, fifteen kilograms

The paper's droid is private, so its stand-in is the open-source BDX-R, rebalanced to the paper's mass budget: 5.8 kg of torso, 3.6 kg in each leg, 2.4 kg of head. Every joint runs the paper's actuator model: encoder offset, backlash, friction, and a torque limit that falls as speed rises.

Line drawing of the droid from the front and from the side

Height (without antennas): 0.66 m
Mass (torso / leg / head): 5.8 / 3.6 / 2.4 kg
Total mass: 15.4 kg
Actuated joints: 14 (5 per leg, 4 neck/head)
Policy rate: 50 Hz
Setpoint path: FOH → 37.5 Hz LPF
Physics: 600 Hz, MuJoCo Warp
Network: 3 × 512 ELU, PPO
Parallel worlds: 8,192 × 24, 100,000 iterations

Leg drivesUnitree A1 on hip roll, hip pitch and knee. Unitree Go1 on hip yaw and ankle.

Head drivesGo1 neck pitch, then three Dynamixel XH540s whose PD acts on the derivative of the setpoint error (Eq. 16).

Paint shop

Scout livery

White enamel, orange-red armour, warm-grey trim and dark machined drives, after the field droids of the space-opera tradition. The eyes are not paint at all: each is a glass lens over a procedural iris, sharp at any distance, that blinks.

Three-quarter view of the painted droid — ¾

Enamel#ECE6D8 · face, torso, shins

Signal orange#D2502A · thighs, head cheeks

Warm grey#7F817D · feet, rear head

Gunmetal#2E3136 · drives, frame

Iris#5FFFF4 · emissive ring

Meshy painted 36 parts at 4K with full PBR maps: 20 shell panels and 16 machined parts, whose textures are reused for their mirrored twins. Cycles then baked each surface onto its own UV layout on the unchanged simulation mesh, so the paint changes how the droid looks and nothing about how it moves.

Mission log

Results

Each policy was trained for the paper's full 100,000 iterations, then walked, stood or danced through 256 parallel worlds for 60 seconds, pushed and jostled by the paper's disturbances. That is 15,360 simulated seconds per policy. Not one of them fell.

Joint tracking error

Table III, mean absolute error in rad. Lower is better.

PaperThis reproduction

Push recovery

0.1 s torso push in a random direction, 128 worlds per force.

Walking 0.3 m/sStanding

Velocity tracking

Fig. 7 analogue, measured in the path frame.

Command	Target	Measured
Forward	0.70 m/s	0.696 m/s
Strafe left	0.40 m/s	0.416 m/s
Strafe right	−0.40 m/s	−0.442 m/s
Turn	1.00 rad/s	0.947 rad/s
Fast turn left	1.80 rad/s	1.749 rad/s
Fast turn right	−1.80 rad/s	−1.754 rad/s
Arc, forward	0.40 m/s	0.386 m/s
Arc, turn	−1.00 rad/s	−0.932 rad/s

Policy switching

Stand → walk → stand at double support → happy dance → stand, with full state transfer.

Switch	Setpoint jump
Stand → walk	0.40 rad
Walk → stand	0.49 rad
Stand → dance	0.27 rad
Dance → stand	0.13 rad
Per-step p99 in normal motion	0.32 rad

The chapters

One walk, ten chapters

Each chapter shows one learned behaviour doing what it was trained to do. The line of text is written for the film; the line beneath it is measured.

I · AwakeningTwo suns stand low. Its eyes flicker on, and it tries its weight on the sand.standing policy · 8 command channels · joint error 0.017 rad

II · The SignalBeyond the dunes a red lamp blinks every two seconds. It turns its head to listen.animation engine · nod, gaze and posture over the standing policy

III · The CrossingIt walks toward the light, carefully at first, then with more conviction. The sand keeps every step.calm-gait walking policy · 0.15 to 0.4 m/s · about 2 to 3 steps/s · joint error 0.043 rad

IV · Against the WindGusts shove it sideways. It catches itself and walks on.90 N gusts for 0.1 s · calm-gait walking policy at 0.3 m/s

V · The LeapHalfway there, for no reason it could name, it jumps.jump policy · shown at 0.4x · 0.19 s in the air, feet 8 cm off the sand

VI · ArrivalIt stops short of the mast, settles onto both feet, and dances.stand > walk > stand > dance · full state handed over at each switch

VII · The Long NightNight falls. The lamp keeps blinking. Nothing answers, and it stamps its feet.tantrum policy · joint error 0.033 rad

VIII · An AnswerFar off on a dune, a second pair of eyes comes on, and nods.a second droid, 13 m away · the same standing policy

IX · A New HopeIt hurries across the sand, stops beside the other, and cannot keep still.walking at 0.7 m/s, about 4 steps/s · stop · excited policy

X · Binary StarsTwo small machines under a ringed planet, looking up.head pitch command +0.5 rad · every pose a policy rollout

Sec. VII-B

Why imitation

Four ways to ask a droid to walk, each given the same 5,000 iterations (on the earlier body) and judged at 0.3 and 0.6 m/s. Given the animator's reference, it steps at about 2 Hz and matches the drawn contacts 90 % of the time. Rewarded for velocity alone, it never lifts a foot.

Formulation	v (m/s)	Steps (Hz)	Clearance	Contact match	MAE vs ref	v error	Falls	Walks
Imitation + phase (ours)	0.3	1.82	3.4 cm	0.89	0.038	0.033	0	Yes
	0.6	2.21	3.4 cm	0.91	0.066	0.058	0	Yes
Reference as input	0.3	1.83	3.7 cm	0.92	0.044	0.036	0	Yes
	0.6	2.14	4.1 cm	0.94	0.052	0.061	0	Yes
Velocity reward only	0.3	≈0	—	0.59	0.42	0.29	7	No
	0.6	≈0	—	0.58	0.49	0.60	2	No
Phase + contact reward	0.3	≈0	—	0.59	0.20	0.30	1	No
	0.6	≈0	—	0.58	0.35	0.60	0	No

Incident log

What broke on the way

INC-01

The feet teleported

The gait planner integrated phase with Euler steps and overshot the short double-support segments. Advancing time exactly through the phase map, t ← pm(φ) + dt and φ ← pm⁻¹(t), let the droid walk for the first time.

INC-02

Standing still paid better

A survival weight of 20 made standing in place the best strategy. A curriculum from 5 to 20, generator-based reference velocities and reference-state resets with velocities fixed it.

INC-03

The head would not listen

A tiny head action scale let the neck's action penalties dominate, and the action noise collapsed. Scaling actions by each joint's reference range and adding a 0.005 entropy bonus brought standing MAE from 0.2 to 0.029 rad.

INC-04

The jump never left the ground

The clip was not ballistic, and a later sync silently overwrote the rebuilt one. A physically consistent jump, synced safely, plus a flight-frame airborne bonus got the droid off the ground.

INC-05

By the book, exploration collapsed

With the paper's entropy coefficient of 0, the action noise fell from 0.3 to 0.02 within 2,500 iterations: the walker learned to stand still and the jump never left the ground. With entropy 0.005 for both, and the jump's height reward sharpened to 1.5 cm, both worked: 1.8 rad/s turns and 0.19 s in the air.

Where this differs

Deviations

Topic	Paper	Here
Robot	BDX, in-house	BDX-R, re-massed to 15.4 kg, rebuilt to the paper's 0.66 m height and hip ranges
Episodic motions	Artist-animated	Procedural clips
Entropy coefficient	0	0.005 for walking, jump and tantrum; 0 elsewhere
Jump emphasis (Eq. 13)	Values not given	Height kernel σ = 1.5 cm, weights 12 / 3 / 12
Jump torques	Knees at the torque limit	Knee peak ≈ 14 Nm, below the limit
Gait	—	A calm gait whose cycle stretches at low speed