Shot‑5.1.1 · MuJoCo · causal live captioning · LeRobot episodes

robot learning arcade

Choose a seeded robot game, teleoperate the main character, and watch grounded captions and a goal-conditioned coach respond while you play.

connecting locally
Select a task family

Choose your mission

Every game shares the same teleoperation, temporal evidence, live-caption, recording, and replay interfaces.

The loop we are trying to make less painful
  1. 01Turn ona robot
  2. 02Learnthrough a game
  3. 03Finda broken policy
  4. 04See + checkwhat looks wrong · does it still jitter?
  5. 05Harmonizethe useful episodes
  6. 06Try againwith better evidence

WHY Make the boring middle of robot learning easier to see and work through together before another physical iteration.

  1. 01 · PLAY & RECORD Game run → local episode Capture actions, observations, timing, captions, and outcomes.
  2. 02 · HARMONIZE Raw data → auditable common view Map declared meaning; preserve the source; abstain when meaning is missing.
  3. 03 · SCREEN Evidence → candidate problems Join episode, policy, and controller contracts; flag what deserves human review.

THIS TOYrecord · compare · harmonize · flag · document · coordinate

EXPERIENCED OWNERchange weights · judge a fix · test the real robot

Harmonize episodes → Screen a policy →

Choose a playable mission to begin locally.

Live robot game arena

World—
Seed—
Time0.0s
Score1000
G1 PILOTshot-1 avatar · input-linked
MUJOCO · LIVE FORCES
SEEDED PROCEDURAL TASK
JIMINY · LIVEgoal + observation → local response
I’ll watch the goal and help you choose the next small move.

Goal critic: waiting for grounded simulator evidence.

OBSERVE
Waiting for the local simulation… Caption evidence will appear here.
Teleop View
Deterministic local instruction router—not a trained policy.
W ASD

Drag the humanoid, click a destination, or steer with WASD. Every game records the same bounded [x, y, rotate, gripper] action. Use Q/E to rotate and Space to grasp or release. A gamepad also works. Press R to replay the seed, N for the next one, P to pause, and Esc for a safe stop.