ZimaBlue: Evolving Generalizable World Action Models through Scalable Video Pre-training
1. Paper overview
In one sentence: ZimaBlue grounds video priors in robot actions, then reuses a slow world model’s cached features for fast corrective control, trading fresher feedback against stale guidance and distillation loss. e03e04e05e06e07e10e12e17
| At a glance | What to know |
|---|---|
| Research problem | Author claim Robot demonstrations are costly and limited in diversity, while abundant embodied videos lack motor labels. The paper asks how video-derived dynamics can transfer to executable manipulation, and how an iterative generative world model can provide that knowledge without delaying corrective control. e02 |
| Core mechanism | |
| A key reported result | Deployment acceleration: Slow: 449.6 ms, 63.9%; undistilled Dual-System: 145.6 ms, 77.8%; distilled and compiled Dual-System: 33.0 ms, 75.0%. Reported end-to-end inference latency (ms) and overall task-macro success (%). Real-robot evaluation; RTX 4090 deployment. The accelerated variant combines two-step distillation and compilation. The paper reports 13.6× overall acceleration and a 2.8-percentage-point success reduction relative to undistilled Dual-System. The approximately 30 Hz configuration has 75.0% success. Pairing 33.0 ms with the separate 77.8% result would conflate configurations. e12e13 |
| Reading caution | Reader analysis Physical evaluation covers one Franka platform and a small task suite. Ten rollouts per task provide limited precision, and Toys uses placement-level rather than whole-task success. The appendix establishes exclusion from DROID post-training but supplies no corpus-wide overlap audit for the large video mixtures. e05e10e11 |
Core contributions
Figure 2. Cached visual dynamics meet an updated observation in the Fast action tower. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the two towers from their inputs downward. The blue Slow tower receives observation, state, noisy action, and noisy future-video tokens. Its video and auxiliary action outputs carry separate flow-matching losses. The green Fast tower has its own updated observation and state, alongside noisy actions. Follow the purple K/V boxes across the center: features from the first 12 Slow layers enter Fast attention. This connection carries a representation rather than decoded video frames. The Fast action decoder on the right produces the commands used by the real robot. The legend distinguishes observation, state, future-video, and action tokens; their colors are not interchangeable modalities. e04e07iv01
What it supports. The diagram identifies two routes into a control decision: a reusable dynamics representation and a newly observed robot state. Slow’s action head supplies alignment supervision, while Fast supplies the deployed actions. This division explains how the system can refresh feedback more often than it refreshes the expensive world-model prediction.
Where the evidence stops. The arrows specify information flow, not a controlled demonstration that predictive features are necessary. A matched Fast controller with nonpredictive or shuffled cached features is not isolated by this architecture figure.
2. Motivation
2.1 The problem and the proposed response
Robot demonstrations are costly and limited in diversity, while abundant embodied videos lack motor labels. The paper asks how video-derived dynamics can transfer to executable manipulation, and how an iterative generative world model can provide that knowledge without delaying corrective control. e02
2.2 What this reading follows
A predictive robot model faces two distinct problems: learning useful dynamics from abundant video and producing corrections before the scene changes again. ZimaBlue addresses the first through video pretraining, action-grounded mid-training, and target-domain adaptation. It addresses the second with a 5B Slow transformer and a 0.5B Fast controller that reads both cached Slow features and fresh observations. These illustrations follow that transfer of information, then separate the evidence for data scaling, asynchronous control, and acceleration. The real-robot results support the combined recipe, while task-level regressions and simulation diagnostics show why a higher aggregate score should not be read as uniformly better control. e03e04e05e06e07e10e12e17
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | IDM |
| Quadrant | Q4 · Dual-system × IDM |
3.1 Evidence-based assessment
Conflicts with the recorded classification
WAM and Dual-system are directly supported: separate Slow and Fast transformers exchange cached video features and run asynchronously. The recorded IDM designation is not established by the described mechanism. Slow jointly predicts video and actions with mutual attention; its action loss is auxiliary for deployed dual-system control. Fast is an action flow model conditioned on fresh observations and predictive internal features, without an explicit generated-frame transition-to-action inverse-dynamics stage. Its behavior may be inverse-dynamics-like in a broad sense, but that does not verify IDM or Q4 as recorded. Preserve the snapshot for editorial review. e04e06e07e08e20
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 First give video dynamics a motor vocabulary
Action-free video can teach visual transitions without specifying how a particular robot realizes them. ZimaBlue therefore trains only Slow during video pretraining, then adds synchronized robot actions during mid-training. The shared 100-dimensional interface assigns semantic slots to end effectors, grippers, joints, torso, and base channels; masks remove undefined coordinates from supervision. Chunk-relative geometry further reduces dependence on a robot’s global coordinate frame: a target translation becomes Δp = R₀⁻¹(pₕ − p₀). Video and action flow losses then align predicted transitions with motor consequences. Only afterward does target-domain training introduce Fast with Slow frozen. My interpretation is that the curriculum separates representational coverage from deployment specialization: video supplies breadth, robot trajectories supply grounding, and the target controller learns how to use that representation under its own observation and action interface. e03e05e06e07
Figure 4. A cache can remain useful after its originating observation is old. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the blue observation markers and long Slow inference spans first. The hatched cache intervals indicate which available prediction guides the controller; a cache originates from an earlier observation and can therefore already be stale when used. Now follow the more frequent purple Fast requests. Each receives new observations as well as available Slow guidance. The lower action bars split an inherited prefix from the action portion that will be used next. RTC arrows connect successive requests through those committed prefixes. The two legends distinguish used and unused cache or action regions. Reading the timelines together explains why Fast need not block until a new Slow rollout finishes. e07e08iv02
What it supports. Asynchrony changes the age of information available to each module. Fast can correct using fresh sensing while sharing one Slow prediction across several requests. Preserving an already committed prefix prevents a new chunk from rewriting actions that the execution system has accepted, while the remaining suffix remains adjustable.
Where the evidence stops. This is a schematic schedule with no calibrated time axis. It establishes the intended execution protocol but cannot supply actual cache ages, scheduling jitter, command latency, or a measured stability guarantee.
5.2 Train Fast for the information it will actually receive
The cache consumed by Fast was computed before the newest observation, so ordinary synchronized training would present an easier problem than asynchronous execution. ZimaBlue samples a temporal offset within the action horizon, supplies the corresponding updated observation and state, and shifts the remaining action targets. It also keeps a short prefix clean to represent actions already committed by the previous request. Padding, inactive coordinates, and that prefix are excluded from the action loss. At runtime, Fast conditions on the fresh observation, the available Slow cache, and the committed prefix, replacing only the unexecuted suffix. The important causal distinction is that feedback can change the next action without waiting for another full world-model prediction. The cache provides guidance, while local sensing anchors the command to the robot’s current situation. e07e08
5.3 Read acceleration as a success–latency tradeoff
After adaptation, distribution matching distillation compresses each branch from eight to two transformer evaluations. Slow is distilled first; its frozen features then condition Fast distillation. Each stage uses a deployable student, a fixed teacher-derived score model, and a learned score model tracking the student distribution, together with supervised anchors. Those training-time score networks are not extra runtime modules. Table 4 measures the resulting system alongside the eight-step variants, with compilation included in the fastest configuration. My reading is that the useful outcome is the joint operating point: 33.0 ms and 75.0% success, compared with 145.6 ms and 77.8% for the undistilled dual system. The paper establishes that tradeoff on its hardware and task suite; neither the individual task rows nor the simulation transfer diagnostic supports a blanket claim that acceleration or adaptation improves every behavior. e09e12e13e17
5.4 Training and inference
During training
Stage I adapts Wan2.2-TI2V-5B using video-only flow matching. A broad mixture of human, simulated, and robot videos is followed by a curated manipulation-focused mixture that includes proprietary human video. The default sample has 33 frames across four temporal chunks. Blockwise noise and teacher forcing permit parallel training with causal context; state/action placeholders carry no supervision. e05
Stage II trains only Slow on synchronized video, state, instruction, and 24-step action chunks from DROID, AgiBot, Galaxea, and RoboMIND2-Franka. Video and action noise levels are coupled, and their masked flow losses train a shared backbone. Action prediction grounds the visual representation but is not the deployed Slow branch's control output. e04e06
Stage III first specializes Slow, then freezes it while training Fast. Fast weights are transferred from Slow's first 12 layers and interpolated to a narrower width. Random observation offsets expose Fast to partially executed trajectories; clean, loss-masked action prefixes teach continuity with previously committed commands. e07
Sequential distribution matching distillation reduces Slow and then Fast from eight to two DiT evaluations each. Each stage uses a student, frozen teacher-derived real-score network, and online fake-score network, with five fake-score updates per generator update. Supervised flow losses remain as anchors. Fast distillation uses frozen distilled Slow features matched by denoising-step index. e09
During inference
Slow and Fast run asynchronously. Fast reuses the available Slow cache while receiving newer observations and states. Real-Time Chunking preserves committed prefix actions and replaces only the unexecuted suffix. Decoding restores physical units and composes relative targets with robot state before execution. e03e08
Distilled deployment retains only the two student generators; score networks are discarded. Slow still predicts latent video and auxiliary actions, whereas Fast supplies robot commands. The simulation benchmarks instead use Slow alone, so their scores do not establish the performance of the accelerated dual-system controller. e09e13
5.5 Implementation flow
- Standardize observations and robot coordinates
State and action share 100 semantic slots covering end-effectors, grippers, joints, torso, base, and hands. Undefined coordinates are zero-filled and loss-masked. End-effector targets are relative to the chunk's initial pose; translations use robust-percentile normalization, while continuous 6D rotations remain unscaled. Camera padding is excluded from attention and video supervision. e03e05
- Predict dynamics and expose internal context
The 5B Slow DiT jointly denoises future video and action tokens after action grounding. Its block-causal mask allows mutual interaction within the current noisy video–action block while blocking access to clean future outcomes. It exports video key/value features that encode predictive context. e04e06e20
- Generate actions from predictive context and fresh feedback
The 0.5B Fast DiT receives updated observation and state tokens, language, and a noisy action chunk. Action queries also attend to cached features from the first 12 Slow layers. Fast has no future-video query stream; it generates executable actions without waiting for a newly completed Slow rollout or decoded future RGB sequence. e04e08e20
6. Experiments & results
ZimaBlue converts action-free embodied video into robot control through video pre-training, cross-embodiment video–action alignment, and target-domain adaptation. Its deployed controller combines a predictive Slow transformer with a smaller Fast action generator that consumes cached video features and fresh observations. The central evidence is executed Franka manipulation: expanding training raises reported task-macro success from 36.1% to 77.8%. A separately accelerated configuration reaches 33.0 ms inference latency with 75.0% success. The work supports video scaling and asynchronous feedback as useful ingredients, while leaving the specific necessity of predictive cached features insufficiently isolated.
6.1 Read the original evidence
Table 3. Additional egocentric video improves both suites, with a larger final gain under visual perturbations. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the four ZimaBlue rows; these form the paper’s cumulative training comparison. The baseline already inherits Wan weights. The next row adds 6K hours of action-labeled multi-embodiment data, followed by configurations with 60K and 120K hours of egocentric video. All configurations also receive the common DROID post-training omitted from this column. Read the Standard and Perturbed columns separately before their combined average: they contain eight and four tasks, respectively. The overall number is therefore a task-macro average across twelve tasks, not an equal average of the two displayed suite means. The external policy rows provide context with different pretraining histories. e10e11e18iv03
What it supports. The combined average rises from 36.1% to 46.1%, 66.9%, and 77.8% across the four configurations. Increasing egocentric video from 60K to 120K hours raises perturbed success from 35.0% to 57.5%, alongside a smaller standard-suite increase from 82.9% to 87.9%. The effect is especially visible under this suite’s appearance shifts.
Where the evidence stops. The table supports a cumulative recipe under eight-step inference and matched DROID adaptation. It does not equalize all external pretraining data or isolate video quality from data quantity, and the real-robot task samples are small.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Zero-shot physical manipulation under increasing pre-training DROID-adapted Franka; eight Standard and four Perturbed tasks; ten rollouts per task, with three separately scored placements per Toys rollout. All ZimaBlue variants share target-domain adaptation and eight-step inference. | Overall: 36.1 → 46.1 → 66.9 → 77.8 for the Wan-initialized baseline, +6K hours multi-embodiment data, +60K hours video, and +120K hours video. Full model: Standard 87.9; Perturbed 57.5. Task-macro success rate (%) | Official DROID-adapted π0.5: 54.4 overall; DreamZero: 53.6. These are reported executed robot outcomes. The cumulative comparison supports the training recipe, with matched downstream schedules, but does not equalize total pre-training compute or external baselines' upstream data. e10e11 |
| Contribution of asynchronous Slow–Fast control Same real-robot task suites; eight-step Slow versus eight-step dual-system deployment. | Slow → Dual-System: Standard 80.8 → 87.9; Perturbed 30.0 → 57.5; Overall 63.9 → 77.8. Task-macro success rate (%) | Reported gains are 7.1 percentage points on Standard and 27.5 on Perturbed. The larger perturbation gain is consistent with useful fresh feedback. Because the intervention changes architecture and control latency together, it does not isolate the benefit of predictive K/V content. e12 |
| Deployment acceleration Real-robot evaluation; RTX 4090 deployment. The accelerated variant combines two-step distillation and compilation. | Slow: 449.6 ms, 63.9%; undistilled Dual-System: 145.6 ms, 77.8%; distilled and compiled Dual-System: 33.0 ms, 75.0%. Reported end-to-end inference latency (ms) and overall task-macro success (%) | The paper reports 13.6× overall acceleration and a 2.8-percentage-point success reduction relative to undistilled Dual-System. The approximately 30 Hz configuration has 75.0% success. Pairing 33.0 ms with the separate 77.8% result would conflate configurations. e12e13 |
| LIBERO-Plus robustness Slow-only simulation evaluation on 10,030 episodes. Zero-shot transfer trains on standard LIBERO; supervised fine-tuning additionally uses LIBERO-Plus training data. | Zero-shot transfer: 86.7; supervised fine-tuning: 92.0. Unweighted mean success across seven perturbation categories (%) | Within the corresponding protocols: InternVLA-A1.5 scores 85.8 and CAC-VLA scores 90.1. Episode-weighted scores are separately 86.0 and 91.5. Camera success rises from 58.1 to 95.4, but robot-initial-state success falls from 88.9 to 81.1; aggregate improvement does not imply improvement in every category. e13e14e17 |
| RoboTwin 2.0 bimanual manipulation One Slow policy post-trained on clean and randomized demonstrations across all 50 tasks; 100 trials per task under each condition. | Clean 94.7; Randomized 94.3; their mean 94.5. Mean task success (%) | ABot-M0.5: Clean 94.0; Randomized 94.2; mean 94.1. This demonstrates simulated robustness on trained tasks under the benchmark's randomization. The average improvement is small, and the table supplies no uncertainty for assessing its significance. e13e15 |
| RoboCasa365 household manipulation and unseen compositions Slow-only policy trained on Human300; 18 Atomic-Seen, 16 Composite-Seen, and 16 Composite-Unseen evaluation tasks, each with 50 rollouts. | Atomic-Seen 78.1; Composite-Seen 50.4; Composite-Unseen 16.5; Overall 49.5. Split success and overall success (%) | ABot-M0.6: 46.6 overall and 7.9 Composite-Unseen. Xiaomi-Robotics-1: 57.4 overall and 32.1 Composite-Unseen. Overall weights the splits by their task counts, rather than averaging three split percentages equally. Unseen-composition performance exceeds the compared WAMs, but remains low in absolute terms and below the strongest compared VLA. e13e16e19 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 4. Faster control improves the average, but the per-task rows expose its tradeoffs. Original paper, p. 16 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare columns in two passes. Slow versus Dual-System tests the move to the responsive Fast controller, while Dual-System versus DMD-Distilled measures the accelerated configuration that also includes compilation. The upper rows are successes and denominators for individual standard tasks; the perturbed block repeats four tasks under changed visual conditions. Toys has a denominator of thirty placement opportunities, so normalize each task before reading the macro averages. The bottom two rows put overall success beside inference latency. Keep the individual rows visible: Bowls and Blocks in the standard suite move differently from Basket and Microwave, which prevents the aggregate from becoming a claim of universal improvement. e11e12e13iv04
What it supports. The dual system moves overall success from 63.9% to 77.8% and latency from 449.6 to 145.6 ms. The distilled, compiled system reaches 33.0 ms with 75.0% success, losing 2.8 percentage points relative to the undistilled dual system. The largest suite-level feedback gain appears in perturbed tasks, from 30.0% to 57.5%.
Where the evidence stops. The 33.0 ms result combines two-step distillation and compilation; this table cannot assign all speedup to distillation. Task counts are limited, and the RTX 4090 measurements should not be generalized to another deployment stack.
Table 12. Fine-tuning changes the shape of robustness, not just its average. Original paper, p. 28 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each cell is a difference in percentage points: supervised fine-tuning success minus zero-shot transfer success on the same evaluation episodes. Positive entries favor fine-tuning. Scan down Camera and Robot before moving across a row. Every camera entry is positive, whereas every robot-initial-state entry is negative. Then inspect Language and Layout, where several regressions accompany the positive camera and background results. The four rows partition LIBERO into Spatial, Object, Goal, and LIBERO-10. This is a diagnostic of distribution-specific adaptation, rather than a table of absolute success rates. The appendix uses the same 10,030 episodes for both protocols and preserves the official robot-state evaluation setup. e14e17iv05
What it supports. Camera gains range from 28.8 to 45.8 percentage points, while the Object suite loses 16.8 points for robot initial states. Those opposing directions explain how fine-tuning can improve overall performance yet weaken a specific robustness dimension. The aggregate improvement therefore leaves a concrete transfer failure mode to investigate.
Where the evidence stops. These are simulation results using the Slow branch alone. They do not measure the deployed dual controller, and category-macro results elsewhere differ from episode-weighted appendix averages because their aggregation rules differ.
7. Analysis & limitations
7.1 What the evidence leaves open
Physical evaluation covers one Franka platform and a small task suite. Ten rollouts per task provide limited precision, and Toys uses placement-level rather than whole-task success. The appendix establishes exclusion from DROID post-training but supplies no corpus-wide overlap audit for the large video mixtures. e05e10e11
The scaling experiments control DROID adaptation but add both experience and pre-training work. The baseline already inherits Wan video pre-training despite the appendix's 'From scratch' label. Cross-model RoboCasa comparisons vary architectures and datasets, so they cannot independently establish a causal video-volume scaling law. e05e10e16e18
No reported ablation separates future-predictive K/V guidance from ordinary visual features, cache reuse, or increased feedback frequency. Causal attention establishes permissible temporal information flow; the experiments do not directly measure interventional dynamics accuracy. e06e12e20
The authors identify failures to advance from correct intermediate states, lost contact, target displacement, and interaction with the tablecloth. They call for progress-aware replanning and broader evaluation. These remaining errors qualify the claim of generalizable physical knowledge. e21
Some presentation details require reconciliation: Table 4 calls a Standard task 'Basket', whereas Appendix A defines cup selection. Table 7's vague Average caption is clarified by the task-weighted protocol and Table 14. The supplied Figure 8 label removes both pre-training and mid-training, limiting attribution of that qualitative example specifically to video pre-training. e11e12e16e19e22
7.2 Questions for discussion
- Does predictive Slow guidance outperform observation-only guidance when Fast capacity and feedback frequency are matched? [e04, e12]
- How much of the video-scaling gain survives fixed-compute training and an explicit overlap audit across video and evaluation tasks? [e05, e10]
- What cache-age threshold causes Fast to lose the benefits of its predictive context under abrupt scene changes? [e07, e08]
- Why does LIBERO-Plus fine-tuning improve camera robustness while reducing robot-initial-state robustness? [e14, e17]
8. Reproducibility audit
8.1 Requirements and known gaps
The most concrete training specification is DROID adaptation: Slow runs for 60K steps with global batch 256, learning rate 1×10^-4, and 2% warmup; frozen-Slow Fast training runs for 160K steps with batch 64, the same learning rate, and 1% warmup. Both use action horizon 24. e11
Rebuilding the full recipe requires the Wan initialization, public and proprietary video mixtures, embodiment mappings and normalization statistics, causal/view masks, target-domain data, and sequential distillation. The supplied text does not fully specify Stage I/II mixture weights, optimization budgets, total training compute, or exact normalization percentiles, preventing an exact reconstruction. e03e05e06e09
A focused mechanism test would train matched Fast branches using predictive Slow caches, observation-only caches, and no Slow cache, holding action horizon, target data, and feedback rate fixed. Perturbed-suite success, latency, and cache age would test whether predicted dynamics add value beyond frequent visual correction. e07e08e12
The paper lists project and code links, but no repository contents were supplied or inspected and no experiments were reproduced. The release's completeness, proprietary-data accessibility, and reproducibility of the reported latency remain unverified. e01e05e13
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does predictive cache content matter after controlling feedback rate?
Proposed check: hold the Fast architecture, DROID adaptation, fresh observation cadence, RTC prefix, and inference budget fixed. Compare the normal Slow cache with a same-shaped cache from current observations only, a cache shuffled across matched task episodes, and no Slow guidance. Sweep cache age under the same controlled perturbations. Measure task success, recovery delay, and action discontinuities. A consistent advantage for the predictive cache at matched timing would support its dynamics contribution; equal performance would suggest that faster local feedback explains much of the reported gain. e04e07e08e12
Check 2: Does adaptation preserve robustness when perturbations are balanced?
Proposed check: reproduce the zero-shot and SFT comparison on the identical LIBERO-Plus episodes, retaining the official robot-state protocol. Add an SFT variant with balanced camera and robot-initial-state perturbations while matching training steps and total demonstrations. Report each category, each suite, and both category-macro and episode-weighted averages. The falsifiable prediction is that balancing reduces the negative Robot column without erasing the camera gains. Include paired confidence intervals to distinguish retention from task-sampling noise. e14e17
8.3 Reading coverage
Visual audit: Original PDF pages 6, 10, 14, 16, and 28 and all five final crops were visually inspected. The pass covers Figure 2 architecture, Figure 4 asynchronous timeline, Table 3 training comparison, Table 4 latency/task ablation, and Table 12 robustness deltas; axes, legends, row labels, and denominators were checked against source text. The PDF SHA-256 was recomputed and matches the base record. Other paper figures, rollout videos, released code, and actual robot behavior were not visually or experimentally audited in this illustrated pass. Base evidence additions record this later visual inspection without implying a new full-paper reproduction.
PDF pages inspected for this edition: 6, 10, 14, 16, 28. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1 Introduction
- 2 Related Work, including Sections 2.1–2.3
- 3 Model Architecture, including Sections 3.1–3.2
- 4 Training Pipeline, including Sections 4.1–4.4.2
- 5 Acceleration Schemes, including Sections 5.1–5.3
- 6 Experiments, including Sections 6.1–6.3.3
- 7 Conclusion and Future Work
- 8 Authors
- References
- A Real-Robot Evaluation Details, including Sections A.1–A.6
- B Additional Simulation Experiment Details, including Sections B.1–B.3
- C Attention Masks
Outside the original text pass
- Reading covered the supplied text extraction. PDF figure images and graphical attention masks were not visually inspected; only their captions and surrounding explanations were available.
- Separate supplemental material availability was not verified.
- The supplied artifact hash was retained as provenance; the PDF binary was not independently rehashed.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, arXiv version line, and Abstract links; p. 20, Section 8 Authors
The supplied title and identifier match the catalog. The text identifies arXiv:2609.00188v1 dated 31 August 2026, and Section 8 lists the catalog's authors. Project and code URLs are printed on page 1.
Go to primary source ↓e02PDF pp. 1–3, Section 1 Introduction
The motivation connects limited action-labeled experience, abundant action-free embodied video, mismatches in generic video pre-training, and the latency of generative robot control.
Go to primary source ↓e03PDF pp. 4–5, Section 3.1 Unified Representation, Table 1, Eq. (1)
The interface has 100 semantic coordinates with shared state/action layout, validity masks, anchor-relative poses and joints, percentile-normalized translation, unscaled 6D rotation, and inverse transformations for deployment.
Go to primary source ↓e04PDF pp. 5–6, Section 3.2 Slow-Fast Dual-System
Slow is a 5B DiT and Fast a 0.5B DiT. Fast uses updated observations and state plus video K/V from the first 12 Slow layers. Slow action supervision aligns representations; Fast generates deployed real-robot actions.
Go to primary source ↓e05PDF pp. 6–8, Sections 4.1, 4.2.1 and 4.2.2, Eqs. (2)–(3)
Stage I initializes from Wan2.2-TI2V-5B, uses broad then curated video mixtures including proprietary human video, and trains causal masked video flow matching. Default clips contain 33 frames in four chunks; the three-view interface masks absent cameras.
Go to primary source ↓e06PDF pp. 8–9, Sections 4.3.1–4.3.2, Eqs. (4)–(6)
Four embodiment families provide synchronized observations, state, language, and 24-step actions. Slow alone learns coupled video/action flow objectives, with validity masking, coupled noise levels, and mutual attention among aligned noisy video/action tokens.
Go to primary source ↓e07PDF pp. 9–10, Sections 4.4.1–4.4.2, Eqs. (7)–(8)
Target-domain Slow specialization precedes frozen-Slow Fast training. Fast initialization transfers and width-interpolates early Slow weights. Random temporal offsets, padded shifted chunks, and clean masked prefixes train asynchronous action refinement.
Go to primary source ↓e08PDF pp. 10–11, Section 5.2 Asynchronous Inference in Dual-System
Slow exports reusable predictive video caches. Fast concurrently predicts from fresh observations and state. Real-Time Chunking preserves committed actions and updates the remaining suffix.
Go to primary source ↓e09PDF pp. 11–12, Section 5.3 Step Distillation, Eqs. (9)–(13)
Sequential DMD reduces each branch from eight to two evaluations using student, real-score, and fake-score networks. Training uses truncated UniPC, five fake-score updates per generator update, supervised anchors, and matched-step frozen Slow guidance for Fast. Only students remain at deployment.
Go to primary source ↓e10PDF p. 12, Sections 6.1.1–6.1.2; p. 14, Table 3 and its caption
The real platform is a Franka with two external cameras and one wrist camera. Table 3 reports overall success of 36.1, 46.1, 66.9, and 77.8 across cumulative ZimaBlue configurations, versus 54.4 for π0.5 and 53.6 for DreamZero. All ZimaBlue variants use eight-step inference and matched DROID adaptation.
Go to primary source ↓e11PDF p. 26, Appendix A.1–A.3 and Table 8
Twelve tasks are held out from DROID post-training. Ten rollouts are used per task; Toys has 30 placement outcomes, and suite scores average task rates. Table 8 specifies Slow/Fast steps, batches, learning rates, warmup, and horizon.
Go to primary source ↓e12PDF pp. 14–15, Section 6.2; p. 16, Table 4, suite-average, overall-average, and latency rows
Slow, Dual-System, and DMD-distilled configurations score 63.9, 77.8, and 75.0 overall, with latencies 449.6, 145.6, and 33.0 ms. Standard averages are 80.8, 87.9, and 85.0; Perturbed averages are 30.0, 57.5, and 55.0. The table includes a task named Basket.
Go to primary source ↓e13PDF p. 3, Section 1 deployment paragraph; p. 15, Section 6.2 Acceleration Ablation and Section 6.3 opening paragraph; p. 16, Table 4 caption
The 33 ms deployment claim specifies RTX 4090. The accelerated configuration includes distillation and compilation and loses 2.8 success points relative to undistilled Dual-System. All three simulation benchmarks evaluate Slow alone.
Go to primary source ↓e14PDF pp. 16–17, Section 6.3.1 and Table 5, ZimaBlue, InternVLA-A1.5, and CAC-VLA rows
LIBERO-Plus category-macro scores are 86.7 for zero-shot transfer and 92.0 for SFT; corresponding comparison leaders score 85.8 and 90.1. ZimaBlue camera scores are 58.1 and 95.4, while robot-initial-state scores are 88.9 and 81.1.
Go to primary source ↓e15PDF p. 17, Section 6.3.2 and Table 6; p. 29, Appendix B.2 and Table 13 caption
RoboTwin uses one policy across 50 trained tasks and 100 episodes per task per condition. ZimaBlue scores 94.7 Clean, 94.3 Randomized, and 94.5 Average; ABot-M0.5 scores 94.0, 94.2, and 94.1.
Go to primary source ↓e16PDF pp. 17–18, Section 6.3.3 and Table 7
Human300 training is followed by 50 rollouts on each of 18 Atomic-Seen, 16 Composite-Seen, and 16 Composite-Unseen tasks. ZimaBlue scores 78.1, 50.4, 16.5, and 49.5 overall. ABot-M0.6 has 7.9 unseen and 46.6 overall; Xiaomi-Robotics-1 has 32.1 unseen and 57.4 overall.
Go to primary source ↓e17PDF p. 28, Appendix B.1, Evaluation scope and aggregation, Tables 11–12
Both LIBERO-Plus protocols use 10,030 episodes. Episode-weighted overall scores are 86.0 and 91.5, distinct from category-macro scores. Fine-grained results show camera gains alongside robot-initial-state regressions.
Go to primary source ↓e18PDF p. 27, Appendix A.4–A.5 and Tables 9–10
Per-task counts support the Standard and Perturbed macro averages. Initialization columns include a From scratch label, although Table 3 identifies the baseline as Wan-initialized.
Go to primary source ↓e19PDF p. 30, Appendix B.3, Table 14 caption, split-average rows, and Overall row
Each RoboCasa task uses 50 episodes. Split means are 78.1, 50.4, and 16.5; the overall 49.5 averages all 50 tasks, confirming task-count weighting across splits.
Go to primary source ↓e20PDF p. 31, Appendix C, Slow DiT and Fast DiT prose
Slow's mask blocks clean current/future outcomes from noisy transition queries. Fast has no future-video query stream; action queries access local inputs and read-only Slow caches. Its within-request action attention is non-causal, and Slow receives no gradients during Fast training.
Go to primary source ↓e21PDF p. 14, Section 6.1.2 Qualitative analysis; p. 19, Section 7; p. 27, Appendix A.6
The authors describe stalled task progress, lost contact, target displacement, tablecloth interaction, and failures to recover. Future directions include broader multi-embodiment evaluation and stronger reasoning and self-correction.
Go to primary source ↓e22PDF p. 18, Section 6.3.3 final paragraph; p. 19, extracted Figure 8 labels and caption
The narrative attributes a qualitative comparison to video pre-training, but the extracted ablation label states that both pre-training and mid-training are removed. This observation relies only on supplied text, not visual inspection.
Go to primary source ↓iv01PDF p. 6, Figure 2; original page and final crop visually inspected at 200 DPI
Figure 2 visibly distinguishes the 5B Slow tower, 0.5B Fast tower, first-12-layer video K/V cache, updated Fast observation/state, Slow auxiliary action head, and separate training losses. All boxes and the modality legend were inspected.
Go to primary source ↓iv02PDF p. 10, Figure 4; original page and final crop visually inspected at 200 DPI
Figure 4 was visually read as a schematic: Slow cache k-1 is used during a later rollout, cache k subsequently replaces it, Fast requests receive observations, and RTC carries action prefixes. The source caption explicitly calls the figure schematic.
Go to primary source ↓iv03PDF p. 14, Table 3; original page and final crop visually inspected at 200 DPI
Table 3 headers, all six policy/configuration rows, and all success columns were inspected. ZimaBlue averages are 36.1, 46.1, 66.9, and 77.8; the final Standard/Perturbed entries are 87.9/57.5. Caption context identifies hours, cumulative configurations, Wan initialization, common DROID post-training, and eight-step inference.
Go to primary source ↓iv04PDF p. 16, Table 4; original page and final crop visually inspected at 200 DPI
Table 4 was visually inspected, including every task denominator, both suite averages, overall success, and latency. Standard Bowls decreases 10/10 to 8/10 from Slow to Dual-System; Basket increases 6/10 to 10/10. DMD-Distilled is 75.0% overall and 33.0 ms; its caption includes compilation.
Go to primary source ↓iv05PDF p. 28, Table 12; original page and final crop visually inspected at 200 DPI
Table 12 was visually inspected: Camera deltas are +42.3,+28.8,+32.1,+45.8; Robot deltas are -6.9,-16.8,-1.5,-6.1. Headers and all four suite rows are preserved in the crop. The caption defines SFT minus zero-shot percentage points on identical episodes.
Go to primary source ↓8.5 Primary sources
ZimaBlue: Evolving Generalizable World Action Models through Scalable Video Pre-training ↗
PDF · 15,627 extracted words
Source fingerprint
42577f8b8c587b5f192a20dd2f41c79aea05f9733059bc73e9b5a5c10df7fcbe