MirroS released Code-as-World: a paradigm that represents physical worlds through executable world representations. The argument is narrow and testable: pixels are evidence of a physical scene, not its ontology. A video model can predict plausible frames without ever representing mass, contact, or gravity. So instead of pixels, latents, or captions, Code-as-World represents a scene as executable code — a scene.json that MuJoCo can run, that an agent can verify against the source video, and that anyone can edit and re-simulate. An agentic loop recovers those programs from real footage in up to five rounds. The verified worlds then become training data with exact physical labels, which real video does not carry. Trained on that supervision, Code-as-World-VL-9B scores 55.4 MRA on QuantiPhy-validation, above Gemini-3.1 Flash at 54.8 and roughly 15 points above the strongest open-weight baseline.
Is it deployable?
Yes, at the research and internal-prototype tier. MirroS shipped the GitHub repo and two checkpoints — Code-as-World-VL-4B and Code-as-World-VL-9B — under Apache 2.0, fine-tuned from Qwen3.5-4B and Qwen3.5-9B. Both are BF16 safetensors served by vLLM behind an OpenAI-compatible /v1 endpoint, with 16 sampled frames per video and --max-model-len 4608.
MirroS · Code-as-World · interactive explainer
A world you can execute, edit, and check against the video
Four moving parts of the technical report. Click through them.
observed evidence
simulated rollout
keyframes compared
ROUND 1 / 5
Δ — first hypothesis proposed from depth, masks and tracks. Contacts land early; the ball drifts left of the observed path.
- ProposeAgent writes an EWR from the evidence and the previous Δ.
- InstantiateCompile to simulator-ready parameters.
- ExecuteMuJoCo rolls out states, contacts, collisions.
- RenderProject back to RGB, depth, masks, tracks.
- VerifyCompare at key frames; accept, or aggregate Δ.
Budget is K = 5 rounds. If nothing is accepted, the hypothesis is rejected. Animation is illustrative; the stages, the budget and the reject rule are from the report.
"objects": [
LW=lc.clientWidth,
EW=ec.clientWidth ]
"initial_state":
"contact_events": [ ]
"simulator": 640;lc.width=LW*2;lc.height=LH*2;lc.style.height=LH+'px';lx.setTransform(2,0,0,2,0,0);
"camera": EW=ec.clientWidth
"rendering": { "background_kind": "synthetic" }
"timeline": { "fps": 29.97, "frame_count": 60 }Fields abridged from the scene.json in Figure 13 of the report. Toggle a chip to see what each component owns.
- CompositionWhat exists. Objects, geometry, metric size, mass, friction, gravity. Floors and walls are static physical entities so they can support and collide.
- EvolutionHow it unfolds. Initial states, forces, contacts, collisions, termination, duration. Executing it expands composition into a full state trajectory.
- AppearanceHow it is seen. Camera, lighting, materials, background, frame rate, render config. Changing it never changes the physics.
Because the world is a program, a parameter can be changed while the rest of the structure holds. The report re-simulates a bowling ball under a new launch direction and re-renders the same collision from three cameras.
Mean Relative Accuracy on QuantiPhy-validation, macro-averaged over the 2S / 2D / 3S / 3D subsets. Table 1 of the report. The 27B row is a reasoning variant and is not a controlled comparison against the direct-answer 4B and 9B.