World Pilot: Steering Vision-Language-Action Models with World-Action Priors
1. Paper overview
In one sentence: A frozen world-action model supplies future-scene features and a soft motion hint to a separate VLA, improving reported OOD success at the cost of an extra model pass per decision. identityarchitecturelatent-steeringaction-steeringtrainingsimulationrobotlatent-ablationaction-ablationlimits
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors argue that semantic grounding from static image-text pretraining leaves a VLA without sufficient anticipation of contact-rich scene evolution. The practical problem is how to inject video-derived dynamics into both perception and action generation without forcing the policy to reproduce a noisy predicted image or trajectory. This is a proposed explanation for OOD fragility, rather than a direct measurement that semantic pretraining lacks all dynamics knowledge. motivation |
| Core mechanism | Source description A modular fusion recipe separates residual Latent Steering of VLM states from single-token Action Steering of the flow-matching head, while freezing the external WAM. architecturelatent-steeringaction-steeringtraining |
| A key reported result | LIBERO-Plus zero-shot OOD manipulation: 84.7 Total; Camera 82.8 Task success rate (%). Train on LIBERO only; evaluate 10,030 perturbed tasks across seven axes. World Pilot Total is averaged over three random seeds. Total: Being-H0.7 82.1, ABot-M0 80.5, Cosmos Policy 79.7. Camera: next-best Cosmos Policy 69.6. The Total lead is 2.6 percentage points over the strongest reported baseline. Camera improves most; Robot, Language, and Layout do not lead. The paper treats Language as perturbation insensitivity. No per-seed dispersion is supplied. implementationsimulation |
| Reading caution | Author claim The authors acknowledge inherited WAM coverage limits, residual OOD failures, and extra per-decision latency. Freezing preserves modularity but prevents prior-policy co-adaptation; no high-frequency control timing is reported. limits |
Core contributions
- Source description
A modular fusion recipe separates residual Latent Steering of VLM states from single-token Action Steering of the flow-matching head, while freezing the external WAM. architecturelatent-steeringaction-steeringtraining
- Source description
Pathway, representation, and prior-source ablations test which information survives transfer; a world model without action post-training also supplies a useful latent prior. pathway-ablationworld-only-transferlatent-ablationaction-ablation
Figure 2. Two priors enter different parts of the VLA decision chain. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the shared input box and follow the blue semantic branch into the Vision Language Model. Its multimodal tokens enter Latent Steering. The orange WAM branch splits into a scene-evolution latent and an anticipated action sequence. Follow the upper orange path through the Dynamic Encoder to the cross-attention fusion; follow the lower path through the Action Encoder to the action head. The snowflake on the WAM agrees with the frozen-parameter rule in Section 3.4. The action head receives enhanced VLM states, the encoded motion condition, and noise. Only its rightmost output is the executable VLA action chunk. architecturelatent-steeringaction-steeringtraininglimits
What it supports. The architecture keeps prediction and execution distinct. A WAM future latent changes what the VLA representation can attend to, while a compressed trajectory changes the action generator’s context. Neither branch directly replaces the VLA output, and the WAM receives no policy-training gradients.
Where the evidence stops. This schematic omits the residual equation and prefix-token internals. Sections 3.2–3.4 verify that VLM states query future tokens and that the motion prefix is not denoised. Online WAM inference still adds latency; Figure 2 gives no timing measurement.
2. Motivation
2.1 The problem and the proposed response
The authors argue that semantic grounding from static image-text pretraining leaves a VLA without sufficient anticipation of contact-rich scene evolution. The practical problem is how to inject video-derived dynamics into both perception and action generation without forcing the policy to reproduce a noisy predicted image or trajectory. This is a proposed explanation for OOD fragility, rather than a direct measurement that semantic pretraining lacks all dynamics knowledge. motivation
2.2 What this reading follows
World Pilot asks where a robot policy should receive knowledge about the future. Its answer is two interfaces: scene-evolution latents modify the VLM representation, and an anticipated trajectory becomes one conditioning token for the action head. The frozen predictor supplies both signals, while expert demonstrations train the VLA to use them. Read the architecture first, then compare the simulation and physical-robot evidence, and finally inspect the ablations that change each interface. The reported gains support this particular fusion recipe; they leave the proposed explanations about dynamics preservation and trajectory-noise resistance only partially tested. This edition follows the supplied arXiv v1. identityarchitecturelatent-steeringaction-steeringtrainingsimulationrobotlatent-ablationaction-ablationlimits
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | IDM |
| Quadrant | Q4 · Dual-system × IDM |
3.1 Evidence-based assessment
Conflicts with the recorded classification
Dual-system is supported: a frozen WAM and separately trained VLA exchange latent and trajectory conditions. IDM and therefore the recorded Q4 assignment conflict with the described mechanism: the WAM jointly predicts future latents and anticipated actions, and the VLA generates actions from noisy trajectories conditioned on both. No inverse-dynamics stage recovering actions from a planned future state is described. This also does not make the complete system One Model. Latent prediction and action alignment are relevant, while no JEPA objective is specified. architectureaction-steeringtraining
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Preserve present-scene tokens while adding an anticipated future
The perception interface starts with current-image and instruction tokens in the VLM. World Pilot does not replace those tokens with a generated image. It projects the WAM future latent through a dynamics encoder, labels the result with a future temporal embedding, and lets the existing VLM tokens query those future tokens. Equation (2) adds the cross-attention result back to the original hidden states. This preserves the token sequence expected by the action head while supplying another source of information. The temporal tag distinguishes the future condition from present-scene content; the text says removing it weakens the prior, but provides no numerical row for that claim. Table 5 supports this latent interface against the tested decoded-image route, without proving that artifact removal explains the difference. latent-steeringlatent-ablation
5.2 Use the anticipated trajectory as context for supervised action generation
The action interface first aligns the WAM trajectory with the VLA horizon and compresses it into one token. During training, the generator receives a noisy interpolation between Gaussian noise and the expert action chunk. The motion token remains a prefix, alongside optional state and future-query tokens, while enhanced VLM states provide cross-attention conditioning. The target is still the expert clean chunk, with the time-dependent weighting in Equation (4); there is no additional loss requiring the output to copy the WAM trajectory. The WAM is frozen and its predictions can be cached, but the VLA and fusion modules learn from action supervision. At deployment both branches run on live observations, so caching saves training-loop work without eliminating the online cost of obtaining each new prior. action-steeringtrainingimplementation
5.3 Separate transfer of a world prior from learning a complete policy
Table 4 provides a useful boundary on the mechanism claim. With Action Steering disabled, Cosmos-Predict supplies only future-scene latents and improves ABot-M0 from 80.5% to 82.6% on LIBERO-Plus, from 54.0% to 62.7% on RoboCasa, and from 81.2% to 85.3% on RoboTwin2.0 clean. These are different benchmarks, not interchangeable evaluation splits. The Cosmos-Policy latent-only counterpart reaches 83.7% on LIBERO-Plus, suggesting further action adaptation can improve the prior. Reader interpretation: a video-derived predictor can contribute useful conditions before its own action post-training, but the receiving VLA still learns from expert actions. This is evidence for transfer through a latent interface, not evidence that video-only training yields the complete executable policy. The modular construction also supports dual-system classification while leaving the catalog’s IDM label inconsistent with the described joint WAM outputs. world-only-transferpathway-ablationtrainingarchitecture
5.4 Training and inference
During training
Freeze the WAM and cache its forward outputs. Update the VLM, dynamics encoder, steering cross-attention, action encoder, and flow head using expert-action supervision; no separate prior loss is introduced. Cached and online priors have matching shapes. training
The reported system uses ABot-M0 with Qwen3-VL and a DiT action head, Cosmos Policy with five denoising steps, 0.3 dropout on both WAM conditions, and eight RTX PRO 6000 GPUs. These hardware counts do not specify training duration or a minimum compute requirement. implementation
During inference
Both models run online from each live observation. Enhanced VLM states provide action-head cross-attention; the fixed trajectory prefix conditions self-attention while the noisy trajectory is denoised. The VLA action chunk controls the robot, and new observations supply subsequent decisions. The paper does not specify the executed chunk fraction, control frequency, or complete numerical integration schedule. trainingaction-steeringarchitecture
5.5 Implementation flow
- Predict paired priors
From the current observation, instruction, and optional state, the WAM jointly predicts a future-scene latent and coarse action trajectory from a shared encoder. The semantic VLM separately encodes the current images and instruction. This is an external prediction branch, not an inverse-dynamics decoder operating on a selected future image. architecture
- Let current tokens query future tokens
A VAE/DiT pathway supplies a per-view future latent. The dynamics encoder projects it and adds a temporal embedding identifying future-scene tokens. Cross-attention queries that representation from the VLM hidden states, then adds a residual without changing their order or downstream interface. latent-steering
- Summarize anticipated motion
Resampling aligns the WAM trajectory with the VLA horizon K. An action encoder compresses it into one prefix token, supplied to the action head alongside optional state, future-query tokens, and noisy actions. The prior is soft context; the head determines the executed chunk. action-steering
6. Experiments & results
World Pilot adds two frozen world-action-model priors to an ABot-M0 VLA: future-scene latents update visual-language hidden states, while one encoded trajectory token conditions the action generator. The VLA still predicts executable actions under expert supervision. Reported gains concentrate on OOD manipulation, including 84.7% LIBERO-Plus success. The design improves modular reuse of video-derived knowledge but requires an additional world-model pass at every decision step.
6.1 Read the original evidence
Table 1. The strongest reported aggregate OOD result does not imply a win in every setting. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read LIBERO, LIBERO-Plus, and RoboCasa as separate evaluations. Within LIBERO-Plus, begin with the yellow Total column, which aggregates all 10,030 perturbed tasks rather than merely presenting one favorable axis. Then compare the World Pilot row with ABot-M0 and Cosmos Policy to see what fusion adds to its constituent systems. Camera is the clearest per-axis improvement; Robot and Layout retain stronger baselines. The paper also cautions that Language reflects insensitivity to instruction perturbations. Finally inspect RoboCasa: the original-benchmark ABot-M0 rerun is the relevant comparator, rather than its earlier GR1 result. Dashes preserve unreported entries. implementationsimulation
What it supports. World Pilot reaches 84.7% Total against Being-H0.7 at 82.1% and ABot-M0 at 80.5%. Camera reaches 82.8% versus Cosmos Policy at 69.6%. On RoboCasa, however, World Pilot’s 65.5% remains below Cosmos Policy’s 67.1%, bounding the claim of complementary benefits.
Where the evidence stops. The table combines author reruns with previously reported baselines. World Pilot Total is averaged over three seeds, but no dispersion or confidence interval appears. These are task-success results, not direct measurements of future-prediction accuracy or matched end-to-end compute.
Table 2. Physical trials show smaller OOD drops across four manipulation tasks. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each task as a three-column group: its ID reference followed by two OOD variants. The retained footnote identifies color and height for blocks, direction and instance for towels, category and layout for fruit, and object category and lid pose for closure. Red parentheses are absolute drops from the same method’s ID score, not uncertainty estimates. Section 4.2 specifies 100 ID demonstrations per task, 10,000 fine-tuning steps, and 20 trials in each setting. Figure 3 on page 6 supplies the PiPER/D435 platform and task scenes. Success concerns the executed robot reaching the instructed final state within the allowed time. robot
What it supports. World Pilot’s OOD drops are 10–20 percentage points, compared with 25–50 for these baselines. On novel-container and lid-pose tests it succeeds at 70% and 65%, while ABot-M0 reaches 30% and 15%. The retained footnote makes clear that successful closure requires both rim alignment and a fully closed lid.
Where the evidence stops. With 20 trials per setting, one outcome changes the rate by five percentage points. The authors avoid interpreting differences below ten points. Exact time limits, perturbation ranges, and numerical optimizer settings are not supplied, despite the stated matched training protocol.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO-Plus zero-shot OOD manipulation Train on LIBERO only; evaluate 10,030 perturbed tasks across seven axes. World Pilot Total is averaged over three random seeds. | 84.7 Total; Camera 82.8 Task success rate (%) | Total: Being-H0.7 82.1, ABot-M0 80.5, Cosmos Policy 79.7. Camera: next-best Cosmos Policy 69.6. The Total lead is 2.6 percentage points over the strongest reported baseline. Camera improves most; Robot, Language, and Layout do not lead. The paper treats Language as perturbation insensitivity. No per-seed dispersion is supplied. implementationsimulation |
| RoboCasa kitchen manipulation Original RoboCasa benchmark; ABot-M0 is rerun because its cited result used GR1. | 65.5 Task success rate (%) | Cosmos Policy 67.1; ABot-M0 rerun 54.0. Competitive transfer to kitchen tasks, with a lower score than the WAM baseline. The paper does not establish universal improvement over either constituent model. simulation |
| Physical robot manipulation across four tasks 100 ID demonstrations per task; 10,000 training steps; 20 trials per method in each ID/OOD setting. | Stack Blocks (ID/color/height): 70/55/50; Fold Towel (ID/direction/novel towel): 85/75/70; Fruit-to-Plate (ID/novel fruit/layout): 90/75/70; Container-Lid Alignment (ID/novel object/lid pose): 80/70/65. Success within allowed time (%) | World Pilot leads all 12 cells. For novel container/lid pose, ABot-M0 achieves 30/15 and Cosmos Policy 25/10. These are executed robot outcomes. Container success requires complete aligned closure. Consistency across settings matters more than small cell differences; authors treat gaps below 10 points as trial-level variance. robot |
| LIBERO-Plus pathway ablation ABot-M0 baseline, either steering pathway alone, and both together. | Latent only 83.7; action only 83.1; both 84.7. Success rate (%) | Baseline 80.5. Both pathways contribute; the combined gain is 4.2 points, not the sum of the isolated gains. No uncertainty establishes statistical separation of the close variants. pathway-ablation |
| World-model-only latent transfer Cosmos-Predict without action post-training; Latent Steering enabled and Action Steering disabled. RoboTwin2.0 uses clean split. | LIBERO-Plus 82.6; RoboCasa 62.7; RoboTwin2.0 clean 85.3. Success rate (%) | ABot-M0: 80.5, 54.0, and 81.2 respectively. An action-trained WAM is unnecessary for a useful latent pathway. This does not establish an executable policy from video pretraining alone; VLA action training remains. world-only-transfertraining |
| LIBERO-Plus future representation ablation Intermediate Cosmos latents at different denoising depths versus a decoded future image through the VLA image encoder. | One/three/five-step latents: 84.6/84.5/84.7. Success rate (%) | Decoded image: 83.5. The latent interface performs better in this configuration. The image comparison also changes decoding and encoding paths, so it does not isolate visual artifacts as the cause. latent-ablation |
| LIBERO-Plus action-prior interface ablation Vary trajectory granularity and entry point into the flow generator. | Single encoded token 84.7. Success rate (%) | Per-step tokens 83.6; trajectory-based flow initialization 84.1; raw trajectory 83.0. Soft trajectory context ranks best. Noise propagation is the authors’ explanation, not a separately measured diagnostic. action-ablation |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 3. Each pathway helps; their combined gain is not additive. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with ABot-M0 at the top as the shared reference. The middle rows enable one prior pathway at a time, so the first tests future-latent conditioning without the trajectory token and the second tests trajectory conditioning without the latent update. The final row enables both. The green numbers are differences from 80.5%, measured in percentage points. To evaluate the value of combining priors, compare the final row with each middle row, not only with the baseline. That comparison gives an additional 1.0 point over latent-only steering and 1.6 points over action-only steering. pathway-ablationlatent-steeringaction-steering
What it supports. Both individual conditions improve reported success, and the full system ranks highest at 84.7%. Reader inference: the 4.2-point combined gain, smaller than the sum of the isolated gains, is consistent with overlapping benefits. The table alone cannot identify precisely which information the pathways share.
Where the evidence stops. No confidence intervals or parameter-matched sham-prior controls appear. The ranking supports using both pathways in the reported setup, but does not establish statistical significance or exclude extra capacity and regularization as contributors to the gains.
Table 5. Intermediate future latents work across several denoising depths. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare the first three rows before looking at the decoded image. Their step counts refer to Cosmos future-latent denoising states, not robot action timesteps or the VLA action horizon. The small spread asks whether a useful control condition requires a thoroughly denoised prediction. The final row changes the route: a fully decoded future image goes through the VLA image encoder instead of using the latent interface. All entries report LIBERO-Plus task success, so this comparison evaluates downstream control utility. It does not score image quality, latent prediction error, or physical consistency directly. latent-ablation
What it supports. The latent variants span only 84.5–84.7%, while the decoded-image variant reaches 83.5%. This supports retaining a latent interface in the tested system. The authors attribute the gap to decoding artifacts and diluted dynamics cues, but the table measures only the resulting success rates.
Where the evidence stops. The image row changes the decoding and encoding path as well as the representation. It therefore does not isolate artifacts as a cause. Missing variance also prevents a firm ranking of the three denoising depths or a claim of statistically equivalent performance.
Table 6. A trajectory summary performs better than more direct action-prior interfaces. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the rows as different ways of presenting the same kind of WAM anticipated trajectory to a flow-matching generator. The default compresses the trajectory into a single conditioning token. Per-step tokens expose its temporal structure more directly, while the raw-prior row removes the encoded summary. The flow-initialization row changes the entry point by starting the flow from the WAM trajectory. In the default formulation of Section 3.3, the prior is instead a fixed prefix outside the noisy trajectory and is not denoised. Thus the comparison changes both granularity and how strongly the prior constrains generation. action-steeringaction-ablation
What it supports. The single encoded token reaches 84.7%, ahead of per-step tokens at 83.6%, trajectory-based initialization at 84.1%, and the raw prior at 83.0%. The authors interpret this ranking as evidence that a motion hint leaves more room for VLA corrections when the WAM trajectory is approximate.
Where the evidence stops. The table does not directly vary or measure trajectory corruption. Consequently, resistance to noise propagation remains a mechanism hypothesis. Token count, encoding, and initialization differ across variants, and no reported uncertainty establishes the reliability of the smallest margin.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors acknowledge inherited WAM coverage limits, residual OOD failures, and extra per-decision latency. Freezing preserves modularity but prevents prior-policy co-adaptation; no high-frequency control timing is reported. limits
Reported baseline provenance mixes reruns and published results. Tables omit confidence intervals, and the ablations do not directly measure dynamics fidelity or trajectory-noise amplification. Success-rate rankings support interface choices more strongly than their proposed causal explanations. simulationpathway-ablationlatent-ablationaction-ablation
7.2 Questions for discussion
- Does an observation-mismatched prior retain gains, suggesting capacity or regularization rather than useful prediction?
- How much action-prior corruption can a single token tolerate before steering hurts the baseline?
- Can adaptive WAM querying reduce latency without losing OOD success?
8. Reproducibility audit
8.1 Requirements and known gaps
Reconstruction requires compatible ABot-M0/Qwen3-VL and Cosmos checkpoints, LIBERO training data, correct per-view latent extraction, horizon resampling, both fusion modules, condition dropout, and the weighted clean-action objective. Exact checkpoint variants, encoder sizes, resampling rules, software versions, optimizer values, and flow-time sampling/clipping are not supplied here. architecturelatent-steeringaction-steeringtrainingimplementation
For robot replication, match the PiPER/D435 setup, ID demonstrations, four task definitions, and closure criterion. The paper states matched optimization and 20 trials per setting but omits numerical optimizer settings, precise perturbation ranges, and allowed-time limits. Proposed checks should log these choices and distinguish reproduction from an independently specified implementation. robot
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does the latent pathway depend on the correct anticipated scene?
Reader-proposed check: reproduce the Latent-Steering-only Cosmos-Predict setting on LIBERO-Plus using the same training split, fusion capacity, dropout, and schedule for three conditions: the correct future latent, a latent shuffled across observations, and current-scene VAE features projected into the same token shape. Keep Action Steering disabled and include ABot-M0. Repeat with multiple documented seeds and report Total, Camera, and uncertainty. A reproducible advantage for correctly paired future latents would support anticipation-specific information; equal gains from shuffled latents would weaken that account and implicate capacity or regularization. Resolve the missing checkpoint and encoder specifications before treating this as an exact replication. world-only-transferlatent-steeringimplementationsimulation
Check 2: Does a single motion token resist controlled trajectory errors?
Reader-proposed check: train the single-token and per-step variants with identical WAM predictions, latent pathway, data, dropout, and action supervision. At evaluation, inject the same progressively stronger temporal shifts or additive action perturbations into the WAM trajectory before each variant’s encoder, while retaining uncorrupted and no-action-prior controls. Evaluate the same LIBERO-Plus tasks and record success degradation and deviation from expert actions with repeated seeds. More gradual degradation for the single token would support the paper’s noise-resistance explanation; parallel degradation curves would weaken it. Report parameter counts and retain the same VLA integration settings so interface size and solver changes do not silently explain the comparison. action-steeringaction-ablationpathway-ablationtrainingimplementation
8.3 Reading coverage
Visual audit: Visually inspected the title/authors/version and Figure 1 on p. 1; motivation on p. 2; Figure 2 and problem formulation on p. 3; latent/action steering and Eqs. (1)–(2) on p. 4; action-generator inputs, training objective, online inference, hardware and evaluation protocol on p. 5; Table 1 and Figure 3 on p. 6; Table 2, its footnote, robot protocol and ablation setup on p. 7; Tables 3–6 and their explanations on p. 8; and limitations on p. 9. Figure 2 arrows and frozen-WAM icon were cross-checked against Sections 3.1–3.4: VLM states query future tokens, and the motion prefix conditions rather than joins the denoised trajectory. No claim-relevant conflict was found. All six final original crops were inspected; narrow ablation tables were rendered at 400 DPI and dense result tables at 300 DPI. The full 15-page text, including acknowledgments and references, was read in five chunks. Reference pages 10–15 were not visually inspected; no appendix is present and separate supplements remain unverified.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, abstract, and Figure 1 (p. 1)
- 1 Introduction (p. 2)
- 2 Related Work (p. 3)
- 3.1 Problem Formulation (pp. 3–4)
- 3.2 Latent Steering (p. 4)
- 3.3 Action Steering (pp. 4–5)
- 3.4 Policy Training (p. 5)
- 4.1 Main Experiments (pp. 5–6)
- 4.2 Real-World Experiments (pp. 6–7)
- 4.3 Ablations (pp. 7–8)
- 5 Conclusion and Limitations (p. 9)
- Acknowledgments and References (pp. 10–15)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Reviewed the supplied arXiv:2606.12403v1 dated 10 June 2026. Title, seven authors, and affiliations agree with the catalog; no alternate revision or edition was supplied for comparison.
- Acquisition omission: “Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.” This edition addresses that limitation through inspection of the original PDF pages and final crops.
- Separate supplemental material availability has not been fully verified.
- All five supplied text chunks were read, covering all 15 pages. Visual inspection covers pp. 1–9; reference pages 10–15 were read as text. No appendix is present in this PDF.
- Project website and code were not inspected. No paper code was executed and no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title block and arXiv margin stamp
The exact title and all seven catalog authors appear on the title page. The stamp identifies arXiv:2606.12403v1 [cs.RO], 10 Jun 2026. Affiliations are CASIA, Nanjing University, and Beihang University; the first three authors share equal contribution and Lue Fan and Zhaoxiang Zhang are corresponding authors.
Go to primary source ↓motivationPDF p. 2, Section 1, paragraphs 1–5
The authors motivate supplementing image-text semantic representations with scene-evolution and anticipated-trajectory priors, injected at different stages of a VLA. They attribute OOD fragility partly to missing dynamics information.
Go to primary source ↓architecturePDF p. 3, Figure 2 and Section 3.1; p. 4, Section 3.1, Eq. (1)
Observations, instruction, and optional proprioception feed a WAM that jointly predicts a scene-evolution latent and anticipated action trajectory from a shared encoder. A separate VLA consumes these priors and produces the executable action chunk. Figure 2 shows separate VLM and frozen WAM branches, dynamics and action encoders, and the action head.
Go to primary source ↓latent-steeringPDF p. 4, Section 3.2, Eq. (2)
The WAM uses a VAE and diffusion transformer to produce a per-view future-scene latent. A dynamics encoder projects it and adds a future temporal embedding. VLM hidden states query these future tokens by cross-attention and receive a residual update preserving their sequence. The text asserts diminished benefit without the temporal tag, without giving a numerical ablation.
Go to primary source ↓action-steeringPDF pp. 4–5, Section 3.3
The WAM trajectory is resampled to the VLA horizon K and encoded into one prior token. This token enters the flow generator as a prefix alongside an optional state token, learned future-query tokens, and noisy actions. It conditions self-attention without being denoised; enhanced VLM states condition cross-attention. Executed actions remain the VLA output.
Go to primary source ↓trainingPDF p. 5, Section 3.4, Eqs. (3)–(4)
WAM parameters remain frozen; VLM, fusion encoders, cross-attention, and action-generator parameters are updated. WAM outputs can be cached during training; both branches run online at every decision step. A Gaussian/expert interpolation is supervised by a clean-action squared loss weighted by 1/(1−tau)^2, with no separate prior loss.
Go to primary source ↓implementationPDF p. 5, Section 4.1, paragraphs 1–3
The implementation uses ABot-M0, Qwen3-VL, a DiT flow-matching action head, and Cosmos Policy with five denoising steps. WAM-condition dropout is 0.3 and fine-tuning uses eight RTX PRO 6000 GPUs. LIBERO-Plus contains 10,030 perturbed tasks across seven axes; training uses LIBERO only, with zero-shot perturbation evaluation.
Go to primary source ↓simulationPDF p. 5, Section 4.1, final paragraph; p. 6, Table 1, caption and World Pilot/Being-H0.7/Cosmos Policy/ABot-M0 rows; Section 4.1 continuation
World Pilot reports LIBERO 98.5%, LIBERO-Plus Total 84.7% averaged over three random seeds, and RoboCasa 65.5%. Total baselines are Being-H0.7 82.1%, ABot-M0 80.5%, and Cosmos Policy 79.7%. Camera is 82.8% versus Cosmos Policy 69.6%; RoboCasa is 65.5% versus Cosmos Policy 67.1% and rerun ABot-M0 54.0%. World Pilot leads Camera, Light, Background, and Noise, but not Robot, Language, or Layout. Baseline provenance mixes author reruns and cited results; the RoboCasa ABot-M0 rerun uses the original benchmark instead of GR1. Language is described as instruction-perturbation insensitivity, not straightforward robustness.
Go to primary source ↓robotPDF p. 6, Figure 3; p. 7, Table 2 with footnote and Section 4.2
Figure 3 labels PiPER and D435. Each task uses 100 ID teleoperated demonstrations, 10,000 fine-tuning steps, and 20 trials per method per setting under matched optimizer, batch size, and learning-rate schedule. Success requires the instructed final state within allowed time. World Pilot ID/two-OOD percentages are Stack Blocks 70/55/50, Fold Towel 85/75/70, Fruit-to-Plate 90/75/70, and Container-Lid Alignment 80/70/65. For novel containers and lid pose, ABot-M0 scores 30/15 and Cosmos Policy 25/10. Closure requires alignment with the rim and a fully closed lid. Authors caution against interpreting differences below 10 percentage points.
Go to primary source ↓pathway-ablationPDF p. 7, Section 4.3, Each pathway contributes; p. 8, Table 3
LIBERO-Plus success is 80.5% for ABot-M0, 83.7% with Latent Steering alone, 83.1% with Action Steering alone, and 84.7% with both. Green parenthetical values are absolute percentage-point gains over ABot-M0.
Go to primary source ↓world-only-transferPDF pp. 7–8, Section 4.3, world-model-only transfer paragraphs; p. 8, Table 4
Cosmos-Predict supplies only future-scene predictions without action post-training, with Action Steering disabled. ABot-M0 versus this Latent Steering variant scores 80.5/82.6 on LIBERO-Plus, 54.0/62.7 on RoboCasa, and 81.2/85.3 on RoboTwin2.0 clean. Cosmos Policy Latent Steering alone scores 83.7 on LIBERO-Plus under matched projection head, dropout, and training schedule.
Go to primary source ↓latent-ablationPDF p. 8, Table 5 and Section 4.3, Latent Steering: latent injection over decoded future images
One-, three-, and five-step Cosmos intermediate future latents yield LIBERO-Plus success of 84.6%, 84.5%, and 84.7%; the decoded future image passed through the VLA image encoder yields 83.5%. The authors attribute the decrease to visual artifacts and dilution of dynamics structure, without a direct diagnostic quantifying those mechanisms.
Go to primary source ↓action-ablationPDF p. 8, Table 6 and Section 4.3, Action Steering: how the trajectory prior conditions the generator
LIBERO-Plus success is 84.7% for one encoded trajectory token, 83.6% for per-step tokens, 84.1% for flow initialization from the anticipated trajectory, and 83.0% for a raw trajectory prior. The text attributes differences to sensitivity to approximate WAM actions and reduced room for VLA corrections.
Go to primary source ↓limitsPDF p. 9, Section 5, Limitations
The authors identify inherited WAM pretraining coverage, uneven OOD improvement, absence of joint WAM–VLA co-adaptation, and an extra online WAM forward pass limiting high-frequency reactive control. They propose uncertainty-aware gating, co-tuning, and distillation or adaptive querying as future directions.
Go to primary source ↓8.5 Primary sources
World Pilot: Steering Vision-Language-Action Models with World-Action Priors ↗
PDF · 7,911 extracted words
Source fingerprint
f0e24d4801d22320173ba938b378996ba2419fe19ee95a5a0fa4e5ea124448df