Awesome-WAMs
GitHub
PAPER REPORTENAll readings ↗

AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routing

English reading report: Method, equations, original figures, experiments and reproducibility.

Authors: Jisong Cai; Long Ling; Shiwei Chu; Zhongshan Liu; Jiayue Kang; Zhixuan Liang; Wenjie Xu; Yinan Mao; Weinan Zhang; Xiaokang Yang; Ru Ying; Ran Zheng; Yao Mu

Affiliations: Shanghai Jiao Tong University; Shanghai AI Laboratory; Baidu AI Cloud; The University of Hong Kong

Source: 2606.09811 ↗ · Project page ↗ · Catalog record

Reading: 176 / 558 · 6 original figures & tables · ~20 min ·

1. Paper overview

In one sentence: AHA-WAM reuses a slow video planner's latent context for fast, observation-conditioned action updates, gaining speed through both its asynchronous interface and substantial deployment optimization. e03e04e05e06e11e13e16e19

At a glanceWhat to know
Research problem
Source description

A shared short temporal horizon makes world modeling expensive at every control update. The authors propose slower, longer-horizon visual modeling alongside reactive action generation, but context reuse creates stale visual information and planner–executor phase mismatch. “Horizon-adaptive” means training for variable phase offsets, not choosing horizons online. e02e03

Core mechanism
Source description

A dual-DiT interface couples a slow video planner and fast action executor through layerwise attention while removing explicit future-frame generation from deployment. e03e04

A key reported resultRoboTwin 2.0: 50-task manipulation: 93.40 clean; 92.20 randomized; 92.80 overall.

Task-averaged success (%). Multi-task clean/randomized training; 100 evaluation episodes per task per setting; no embodied pretraining for AHA-WAM. External baseline results are published values when available.

Fast-WAM: 91.88/91.78/91.83; LingBot-VA: 92.90/91.50/92.20. The reported overall gain is 0.97 percentage points over Fast-WAM and 0.60 over LingBot-VA. No seed-level uncertainty is supplied. Figure 1 labels LingBot-VA 92.24; this report uses Table 1's 92.20. e10e11e18e24

Reading caution
Author claim

The authors identify planner frequency, video horizon and action chunk size as task-dependent choices and call for dedicated long-horizon evaluation. The paper does not demonstrate dynamic horizon selection. e02e25

Core contributions

  • Source description

    A dual-DiT interface couples a slow video planner and fast action executor through layerwise attention while removing explicit future-frame generation from deployment. e03e04

  • Source description

    OVCR, randomized action-start offsets and rolling planner memory address observation alignment, temporal phase and historical context, respectively. e06e07e08

  • Reader analysis

    CUDA deployment and action-sampler distillation produce a measured speed–success tradeoff; architectural asynchrony alone is not the entire acceleration story. e11e16e19e21

Figure 2. Current observations adapt the planner's cached context before the action expert uses it. Original paper, p. 4 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Start at the bottom left: the VAE and rolling K/V memory feed the blue video planner. Move to the current image beneath the green branch: its visual features form observation-guided queries, which read the cached video context. The circled cross marks attention; the circled plus marks the residual edit before joint attention in the action DiT. The state encoder supplies a separate proprioceptive pathway. On the right, compare the training mask's causal video block with the smaller inference mask. Section 3.1 explains the removed tokens: future-video supervision is available during training, but actions are masked from those future tokens. e03e04e06e08

What it supports. The connection between world modeling and action execution survives removal of future-video generation. The reusable object is layerwise latent K/V context, adapted using current visual feedback. This supports a dual-system interpretation: two distinct transformers interact through attention while serving different temporal roles.

Where the evidence stops. The diagram is a structural explanation, not a complete router specification. Equations (8)–(10) define observation pooling and gated residual edits, but the precise layerwise router and gate implementation remain insufficiently specified for exact reconstruction.

2. Motivation

2.1 The problem and the proposed response

Source description

A shared short temporal horizon makes world modeling expensive at every control update. The authors propose slower, longer-horizon visual modeling alongside reactive action generation, but context reuse creates stale visual information and planner–executor phase mismatch. “Horizon-adaptive” means training for variable phase offsets, not choosing horizons online. e02e03

2.2 What this reading follows

A robot can benefit from visual dynamics without generating a new future video before every action. AHA-WAM trains a video planner together with an action executor, then deploys them at different rhythms. The key interface is reusable layerwise attention context: current images help the executor retrieve and edit that context between planner refreshes. Read this paper by separating three questions: what video prediction teaches during training, how fresh feedback reaches an action chunk, and which engineering changes produce the reported speed. The simulation results, physical trials and latency tables answer different questions and use different pretraining or measurement conditions. e03e04e05e06e11e13e16e19

3. Research context

We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.

Catalog dimensionRecorded classification
Major categoryWAMs
ArchitectureDual-system
Prediction paradigmJoint prediction
QuadrantQ3 · Dual-system × Joint prediction

3.1 Evidence-based assessment

Supports the recorded classification

Reader analysis

The recorded Dual-system × Joint prediction classification fits two distinct DiTs coupled by layerwise attention and jointly supervised action/video prediction. This is not inverse-dynamics extraction from decoded future frames. Joint prediction describes training; deployment retains video-trained latent context while removing explicit future prediction. FIFO memory and the asynchronous executor support the recorded memory and efficiency subcategories. e03e04e05e08e09

This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.

4. Problem formulation

4.1 Inputs and outputs

InputsOutputs
  • Language instruction; visual observation history and current images
  • Proprioceptive robot state; noisy action tokens during denoising
  • Executable short action chunks
  • Future video latents during training; reusable layerwise planner K/V context during inference

4.2 Equations and their role

yρ=(1−ρ)y+ρϵ,LFM(y)=Ey,ϵ,ρ ⁣[∥fθ(yρ,ρ,Otv,st,l)−(ϵ−y)∥22],L=La+λLvy_\rho=(1-\rho)y+\rho\epsilon,\qquad \mathcal{L}_{\mathrm{FM}}(y)=\mathbb{E}_{y,\epsilon,\rho}\!\left[\left\|f_\theta(y_\rho,\rho,O_t^v,s_t,l)-(\epsilon-y)\right\|_2^2\right],\qquad \mathcal{L}=\mathcal{L}_a+\lambda\mathcal{L}_v
The target y is an action chunk or future video latents; epsilon is Gaussian noise and rho is flow time. The network f_theta predicts velocity conditioned on visual history O_t^v, state s_t and language l. Action and video losses instantiate the same objective; lambda balances them and is effectively one for joint optimization. e05e18
K~tp,ℓ=Kτ(t)p,ℓ+αtℓΔKtp,ℓ,V~tp,ℓ=Vτ(t)p,ℓ+αtℓΔVtp,ℓ\widetilde K_t^{p,\ell}=K_{\tau(t)}^{p,\ell}+\alpha_t^\ell\Delta K_t^{p,\ell},\qquad \widetilde V_t^{p,\ell}=V_{\tau(t)}^{p,\ell}+\alpha_t^\ell\Delta V_t^{p,\ell}
K and V are planner keys and values, p denotes the planner, ell the transformer layer, t the action update and tau(t) the latest planner refresh. The router predicts residual updates Delta K and Delta V; learned gate alpha controls their addition. Tildes identify the adapted context. e06
δ∼U{0,1,…,ha−1},La=Eδ ⁣[LFM(Aτδ)]\delta\sim\mathcal{U}\{0,1,\ldots,h_a-1\},\qquad \mathcal{L}_a=\mathbb{E}_{\delta}\!\left[\mathcal{L}_{\mathrm{FM}}(A_\tau^\delta)\right]
Delta is the sampled action-grid offset, h_a the chunk horizon and A_tau^delta an offset-aligned action chunk within the planner window beginning at tau. The loss trains across chunk phases without changing either horizon online. e07

5. Method in detail

5.1 Train on futures, deploy through latent context

Reader analysis

During training, AHA-WAM receives dense visual-dynamics supervision alongside action supervision. The same flow-matching construction corrupts either future video latents or action chunks and trains a velocity predictor; Appendix A gives equal weights when the branches are jointly optimized. The crucial deployment preparation is the attention mask: action tokens cannot rely on future video tokens during training. At inference, the future-video path can therefore disappear while the planner still supplies a representation shaped by video learning. Reader interpretation: the term world planner here denotes learned, future-oriented latent context, not an explicit search over candidate action sequences. Physical action comes from the action DiT's denoising process and execution interface. Keeping that distinction clear prevents attributing test-time imagined rollouts or inverse-dynamics decoding to this architecture. e03e04e05e18

Figure 3. Offset training varies where the action grid begins inside a fixed planner window. Original paper, p. 7 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Follow the blue timeline first, then the more frequent green action updates below it. The delta bracket at the left shifts the green grid relative to the planner's starting point. The caption and Equation (11) specify a discrete offset drawn from zero through one less than the action chunk size. This is a training-time shift of the action grid, not an online decision to lengthen or shorten a horizon. The drawing is schematic: use Section 4.1 and Table 4 for the configured planning horizon, chunk size and temporal settings rather than estimating implementation ratios from the artwork. e02e07e09e12e17

What it supports. Randomized alignment exposes the executor to different phases of a reused planner context. Together with current-observation routing, it supplies a plausible training mechanism for asynchronous operation. The planner horizon stays longer than an individual executable action chunk; the sampled offset changes their relative alignment.

Where the evidence stops. Table 2 does not include a standalone removal of offset training. The figure explains the intended mechanism, but the reported component ablation cannot establish its independent contribution or robustness to arbitrarily long planner delays.

5.2 Keep three kinds of temporal information separate

Source description

The system handles history, current evidence and relative timing with different mechanisms. Rolling K/V memory belongs inside the slow planner and lets a new refresh incorporate historical observations. OVCR works between refreshes: current visual tokens form compact queries, those queries retrieve planner features, and a gated residual update changes the layerwise keys and values consumed by the action expert. Proprioception follows a direct state-encoding path. Offset training changes the action chunk's starting phase within the longer planner window, exposing the executor to different alignments during learning. None of these operations selects a new horizon at test time. A useful reading order is therefore memory before planner refresh, routing before action denoising, and randomized offsets before deployment. Table 2 measures memory and routing additions, leaving the independent offset-training effect open. e04e06e07e08e12

5.3 Read the speed claim as a measured tradeoff

Reader analysis

Asynchrony moves video prefill outside the action-update critical path, but the fast branch still requires image encoding, context routing and repeated action denoising. Table 8 shows why implementation matters: the same ten-step deployment path becomes much faster after TensorRT compilation, CUDA Graph capture, redundant-work elimination and a runtime upgrade. Flash then reduces the action sampler to two steps, with lower RoboTwin success in Table 1. Reader interpretation: a useful reproduction should report both success and action-update latency, while also checking that background planner refresh remains timely. Table 9 reinforces this point: the compile mode with the shortest prefill latency increases chunk latency, so optimizing the slow branch in isolation can hurt the fast path. The external Fast-WAM speedup should remain separate from these within-model comparisons. e09e11e16e19e21e22

5.4 Training and inference

During training

Source description

Joint flow matching supervises actions and future video latents with logit-normal noise times and equal video/action loss weights. Offset training samples every phase within a chunk. RoboTwin uses 50 clean and 500 randomized demonstrations per task, global batch 512, five epochs, AdamW at 10^-4, weight decay 0.01 and cosine scheduling with 5% warmup. e05e07e17e18

Source description

Flash freezes the video DiT and regresses from sampled noisy trajectory anchors to the teacher's final action. Table 5 specifies a 16-step teacher, two-step student, learning rate 2×10^-5 and five epochs. This conflicts with prose describing a 10-step teacher; the exact teacher configuration remains unresolved. e09e17e20

During inference

Source description

The configured planning horizon is 64 and executable chunk size is 16; history holds at most six frames and OVCR uses 32 queries. Table 4 separately labels the aggregate action horizon as 64, comprising four chunks. Default action inference uses ten denoising steps and CFG 1.0. e09e17

Reader analysis

Reported action-update latency includes image encoding, context access, routing and denoising. Video prefill runs in the background. These frequencies describe chunk-inference updates, not an independently measured low-level actuator servo rate. e09e16e22

5.5 Implementation flow

  1. Build the slow planner

    Wan2.2 initializes the video DiT, text encoder and VAE. Language conditions both branches. The video planner uses a causal video mask and exposes layerwise keys and values after one forward. The action branch cannot attend to future video tokens. e03e04e09

  2. Separate memory from the executor interface

    A fixed FIFO retains historical K/V states inside the planner. Each refresh attends to this memory to produce new context; the executor consumes only the latest adapted context, rather than directly reading the historical FIFO. e04e08

  3. Route current visual feedback

    Learnable queries pool projected current visual tokens, attend to cached planner features and generate gated residual K/V edits at each layer. Proprioception enters the action DiT directly; visual feedback enters through OVCR. e06

  4. Denoise and execute

    Action queries attend jointly to the action branch's own K/V and the adapted planner K/V. The planner refreshes asynchronously while successive action updates obtain fresh observations and reuse its context. e04e06e09

6. Experiments & results

AHA-WAM separates a video-trained world planner from a fast action executor. The planner exposes reusable latent context; current images route and edit that context before each action chunk. Future-video prediction supervises training but is removed at deployment. The evidence supports strong manipulation performance with substantial systems acceleration, subject to different pretraining and timing protocols across comparisons.

6.1 Read the original evidence

Table 1. AHA-WAM leads the reported RoboTwin average, while Flash exchanges some success for speed. Original paper, p. 9 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read the robot-pretraining flag before comparing the success columns. AHA-WAM, Flash and Fast-WAM are marked as having no embodied pretraining before RoboTwin training; this still permits pretrained video initialization. Clean and randomized columns summarize different scene settings, and the last column is their reported average. Section 4.2 uses 50 tasks and 100 test episodes per task in each setting. Start with Fast-WAM as the closest interface comparison, then inspect LingBot-VA and the generalist policies. Appendix B says external baseline values come from official published results when available, so these rows are not all new runs in one controlled implementation. e09e10e11e18e24

What it supports. AHA-WAM reports 92.80% overall, compared with 91.83% for Fast-WAM and 92.20% for LingBot-VA. Its 93.40% clean and 92.20% randomized results remain close. Flash reports 90.20% overall, exposing a measurable success cost alongside its faster sampler.

Where the evidence stops. No seed-level uncertainty accompanies these success rates. Figure 1 prints LingBot-VA as 92.24, whereas this table and the main comparison use 92.20. The report preserves that inconsistency and uses the table value.

Figure 4. Physical success depends on the task and shift; aggregate progress scores require a separate interpretation. Original paper, p. 11 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. The left panels summarize original and shifted conditions; the right panels pair each task with one visual change. Keep the model colors aligned with the bottom legend. Store Plate varies lighting, Organize Desktop changes object configuration, Fold Towel changes texture, and Prepare Soy Milk changes environment. Success bars are percentages. The panels labeled Score require caution: Appendix C defines an individual rollout score from zero to three, but the plotted aggregate values use an unexplained scale. Appendix C states 30 trials per model and task, with success requiring completion of the third subtask. Section 4.4 supplies the pretraining and physical deployment conditions. e13e14e15

What it supports. The reported original-condition average is 78.3% for AHA-WAM, 76.7% for pi0.5 and 68.3% for Fast-WAM. Under the illustrated shifts, AHA-WAM reaches 53.3%, behind pi0.5 at 55.0%. Thus the physical evidence supports competitive robustness across these four tasks, with no uniform advantage across conditions.

Where the evidence stops. AHA-WAM and Fast-WAM receive RoboCOIN pretraining for these experiments. The printed progress score of 35 cannot be treated as a mean on the defined 0–3 scale; its aggregation is unresolved. The small success differences lack reported uncertainty.

6.2 Results and evaluation conditions

Task & protocolReported resultComparison & interpretation
RoboTwin 2.0: 50-task manipulation

Multi-task clean/randomized training; 100 evaluation episodes per task per setting; no embodied pretraining for AHA-WAM. External baseline results are published values when available.

93.40 clean; 92.20 randomized; 92.80 overall.

Task-averaged success (%)

Fast-WAM: 91.88/91.78/91.83; LingBot-VA: 92.90/91.50/92.20.

The reported overall gain is 0.97 percentage points over Fast-WAM and 0.60 over LingBot-VA. No seed-level uncertainty is supplied. Figure 1 labels LingBot-VA 92.24; this report uses Table 1's 92.20. e10e11e18e24

RoboTwin 2.0: Flash speed–success tradeoff

Same reported clean/randomized benchmark; distilled two-step action sampler.

90.48 clean; 89.92 randomized; 90.20 overall.

Task-averaged success (%)

Ten-step AHA-WAM: 92.80 overall.

Flash loses 2.60 percentage points overall; faster action inference does not preserve every task's performance. e11e17e21e23

RoboTwin 2.0: asynchronous component ablation

Clean/randomized task averages in Table 2; memory and OVCR are described as separate additions to Naive-Async.

Naive-Async 88.60; + KV Memory 91.01; + OVCR 91.47; full AHA-WAM 92.80.

Overall success (%)

Fast-WAM 91.83.

Memory and routing recover the naive asynchronous drop. The table does not independently isolate offset training or provide capacity-matched controls. e12

Physical bimanual manipulation: original conditions

AgileX Piper; head RGB, state and language; four tasks; about 120 demonstrations per task. AHA-WAM and Fast-WAM receive RoboCOIN pretraining: 24,600 trajectories, about 165 hours. Appendix C states 30 trials per model/task.

78.33

Average task success (%)

Fast-WAM 68.33; pi0.5 76.67; Motus 21.67.

This is executed robot behavior. It is not evidence for robot-data-free real-world learning; slower WAM baselines use RTC-style execution and interpolation. e13e14e15

Physical bimanual manipulation: generalization shifts

Lighting, towel texture, object configuration and environment shifts for the four physical tasks.

53.3

Average task success (%)

pi0.5 55.0; Fast-WAM 46.7; Motus 16.7.

AHA-WAM ranks second in success under these shifts. The printed progress-score aggregation is unclear, so it is not converted to a mean 0–3 score. e14e15

Closed-loop action-chunk inference latency

RoboTwin timing; single RTX 5090D except Fast-WAM's official latency. Warmup episode discarded; bf16; asynchronous planner prefill.

AHA-WAM 41.37 ms / 24.17 Hz; Flash 17.56 ms / 56.95 Hz.

Latency (ms); action-update frequency (Hz)

Fast-WAM 190.00 ms / 5.26 Hz: reported speedups 4.59× and 10.82×. AHA-WAM eager-to-optimized latency: 415.77 to 41.37 ms.

The internal optimization comparison substantiates systems gains; the external speedup is not a fully hardware-matched experiment. e16e19e21e22

6.3 Ablations and diagnostic examples

Read component removals and qualitative examples within their stated evaluation conditions.

Table 2. Naive context reuse loses success; memory and observation-guided routing each recover part of it. Original paper, p. 9 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Use Naive-Async as the intervention baseline: it separates planner and executor but reuses the latest context without memory or OVCR. Section 4.3 says the next two mechanisms are added separately, then jointly in the full model. Accordingly, the '+ OVCR' row should not be read as necessarily including the preceding '+ KV Memory' row. Read across clean and randomized columns before comparing averages, since a mechanism could help one condition more than the other. The accompanying discussion assigns historical continuity to memory and current-state alignment to routing; those functional explanations should be distinguished from what the table directly measures. e06e07e08e12

What it supports. Naive-Async achieves 88.60% overall, below Fast-WAM's 91.83%. Memory alone raises the reported average to 91.01%, and routing alone to 91.47%; the full system reaches 92.80%. The observed pattern supports combining historical context with fresh visual conditioning.

Where the evidence stops. This is not a capacity-matched factorial study of every design choice. It does not isolate offset training, query count, memory length or planner refresh schedule, and no uncertainty estimates accompany the component differences.

Table 8. The reported action-update speed requires a cumulative compiler and runtime optimization path. Original paper, p. 18 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read downward: each row adds an optimization to the previous stage. L_chunk measures one action update, while L_prefill measures the background video-planner call. TRT denotes TensorRT and CG denotes CUDA Graph capture. The first large drop follows compiling and capturing the action DiT; later rows add context modules, prefill compilation, redundant-work removal, a runtime upgrade, buffer-copy removal and a TensorRT VAE encoder. A dash in the prefill column means no value is reported there, not zero planner cost. Preserve the runtime footnote: the A-to-B change updates PyTorch and Triton as part of the measured cumulative path. e09e16e19e22

What it supports. The table reports a decrease from 415.77 ± 0.33 ms in eager execution to 41.37 ± 0.03 ms after all stages. The first action-DiT optimization already reaches 83.87 ± 0.51 ms. These within-model results show that deployment engineering contributes substantially to the final latency.

Where the evidence stops. Cumulative rows cannot isolate order-independent effects, and the paper does not define the displayed ± statistic. Prefill is separate from action latency. The headline Fast-WAM speedup additionally uses an external published timing rather than this same cumulative experiment.

7. Analysis & limitations

7.1 What the evidence leaves open

Author claim

The authors identify planner frequency, video horizon and action chunk size as task-dependent choices and call for dedicated long-horizon evaluation. The paper does not demonstrate dynamic horizon selection. e02e25

Reader analysis

Aggregate success hides failures: Open Microwave is 40% clean and 46% randomized for AHA-WAM, versus 9%/23% for Flash. Success tables lack seed-level uncertainty, and the component ablation does not separate offset training from other changes. e11e12e23

Reader analysis

Figure 4 prints progress scores such as 35, whereas Appendix C defines per-trial scores from 0 to 3. The aggregation is unspecified. Neither those bars nor the small success advantage over pi0.5 establishes statistical superiority. e14e15

7.2 Questions for discussion

  1. Does offset training improve robustness to delayed planner refreshes when routing and memory are held fixed?
  2. Which task failures increase when two-step distillation replaces the default sampler?

8. Reproducibility audit

8.1 Requirements and known gaps

Reader analysis

Recreate three RoboTwin camera views at 384×320, 14-dimensional state/action inputs, 30 routed layers, six-frame history and 32 queries. The model totals about 7.23B parameters. Training GPU count, wall-clock cost, exact RoboCOIN subset selection, router/gate implementation and precise noisy-anchor sampling probabilities remain insufficiently specified. e06e09e13e17e20

Reader analysis

Resolve the 10-versus-16-step teacher discrepancy before reproducing Flash. Record the exact runtime: Table 8 changes PyTorch 2.7.1/cu128 and Triton 3.3.1 to PyTorch 2.12.0/cu130 and Triton 3.7.0, with TensorRT 10.16.1.11 throughout. Measure chunk latency and planner prefill separately. e09e17e19e20e22

8.2 Proposed reproduction checks

The following checks are proposals motivated by the paper. They have not been run as part of this reading.

Check 1: Separate offset robustness from observation routing

Reader-proposed experiment, not performed: train matched replicas with fixed versus randomized action-grid offsets, crossed with OVCR enabled versus disabled, while holding planner memory, horizons, demonstrations and training budget constant. Keep the disabled router's architecture present where possible to control parameter count. Evaluate identical held-out RoboTwin episodes while varying planner refresh delay and applying a controlled object displacement between refreshes. Record success by delay, planner-context age and action latency over multiple seeds. Offset training should particularly reduce sensitivity to phase shifts, while OVCR should help after changed visual evidence. If neither advantage appears under matched delays, the claimed separation of temporal alignment and observation correction would be weakened. e06e07e08e10e12e17

Check 2: Reproduce the latency–success curve with a fixed teacher and runtime

Reader-proposed experiment, not performed: first resolve and record whether the teacher trajectory uses 10 or 16 steps; if clarification is unavailable, test both as separate configurations. On the same RTX 5090D, data split and fixed bf16 runtime, compare eager ten-step inference, optimized ten-step inference and the distilled two-step student. Reuse identical task seeds and log chunk latency, prefill latency, context age and per-task success. Follow the paper's warmup exclusion and per-episode averaging, and add clearly labeled latency quantiles. Open Microwave is a useful diagnostic because Table 12 shows a large Flash drop. A speedup without the reported success profile, or with accumulating stale context, would fail to reproduce the full deployment tradeoff. e09e11e16e17e19e20e22e23

8.3 Reading coverage

Visual audit: The title/author/version page, all five figures, all twelve tables, method equations, training settings, physical scoring definitions and timing appendices were rendered and visually inspected. Six faithful crops were inspected at their final asset sizes; the component-table crop was adjusted to retain its full top rule and inspected again. Figure 2's arrows and mask blocks were checked against its caption and Sections 3.1–3.2; Figure 3's offset was checked against Equations (11)–(12). The declared pages cover all retained numerical, method, training, evaluation and reproduction claims. References on pp. 12–14 were read as text; those pages contain no retained scientific claims requiring visual evidence. Unresolved source inconsistencies remain explicit: Figure 1 versus Table 1's LingBot-VA average, the real-world progress-score aggregation, and the distillation teacher schedule. Separate supplements and external code remain outside this reading.

PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 15, 16, 17, 18, 19, 20. Appendix coverage: reviewed.

Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.

Text reading scope & known omissions
  • Abstract; 1 Introduction (pp. 1–3)
  • 2 Related Work (pp. 3–4)
  • 3 Asynchronous Horizon-Adaptive World-Action Modeling, including 3.1–3.5 (pp. 4–8)
  • 4 Experiments, including 4.1–4.5 (pp. 8–10)
  • 5 Conclusion, limitations and acknowledgments (p. 11)
  • References (pp. 12–14)
  • A Implementation Settings (pp. 15–16)
  • B RoboTwin Evaluation Details (p. 16)
  • C Real-World Task Execution and Scoring Criteria (pp. 16–17)
  • D Inference Speedup Details (pp. 16–19)
  • E Per-Task RoboTwin Success Rates (p. 20)

Outside the original text pass

  • Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
  • Separate supplemental material availability has not been fully verified.
  • Identity: the inspected title page identifies arXiv:2606.09811v1 [cs.RO], 8 June 2026. Its title and all 13 authors agree with the catalog after name-order normalization. No alternative revision or edition was supplied or substituted.
  • The acquisition warning that text extraction does not reconstruct figure images was addressed by inspecting the retained PDF's figures, equations and tables. All six text chunks were read individually without truncation.
  • Separate supplemental material availability has not been fully verified; no separate supplement was supplied.
  • The source explicitly omits logging, checkpoint and dataset paths, run-specific bookkeeping, and inactive legacy annealing options in Appendix A.
  • Code, project pages and external baseline sources were not inspected; no experiments were reproduced. Reference-list text was read, but the cited works were not independently reviewed.

The visual audit above records the subsequent illustrated pass.

8.4 Traceable evidence

e01PDF p. 1, title, author/affiliation block and arXiv margin stampInspect

The observed title matches the supplied title; arXiv:2606.09811v1 is dated 8 June 2026. Authors in order are Jisong Cai, Long Ling, Shiwei Chu, Zhongshan Liu, Jiayue Kang, Zhixuan Liang, Wenjie Xu, Yinan Mao, Weinan Zhang, Xiaokang Yang, Ru Ying, Ran Zheng and Yao Mu. Affiliations are Shanghai Jiao Tong University, Shanghai AI Laboratory, Baidu AI Cloud and The University of Hong Kong.

Go to primary source ↓
e02PDF pp. 2–4, Introduction and opening of Section 3Inspect

The authors motivate different world/action temporal roles and explicitly define horizon-adaptive as decoupled horizons with offset robustness, not online horizon selection.

Go to primary source ↓
e03PDF p. 4, Figure 2 and captionInspect

The architecture shows separate video/action DiTs, observation-guided routing, residual K/V adaptation, rolling planner memory and training/inference attention masks.

Go to primary source ↓
e04PDF p. 5, Section 3.1, Eqs. (1)–(3) and masking paragraphInspect

The planner emits reusable layerwise K/V; action attention concatenates its own K/V with adapted planner context. Video prediction uses a causal mask; actions cannot attend to future video tokens, allowing removal of the future-video path during inference.

Go to primary source ↓
e05PDF p. 6, Section 3.1, Eqs. (4)–(7)Inspect

Gaussian interpolation defines the flow-matching velocity target epsilon minus y; action and future-video losses are combined with weight lambda. Explicit future-frame decoding is removed at deployment.

Go to primary source ↓
e06PDF p. 6, Section 3.2, Eqs. (8)–(10)Inspect

Base queries attention-pool current visual tokens. Layerwise routing reads planner features, predicts K/V residuals and applies a learned gate; proprioception is encoded directly for the executor.

Go to primary source ↓
e07PDF p. 7, Figure 3, Section 3.3 and Eqs. (11)–(12)Inspect

The action grid is shifted by a discrete uniform offset from zero through h_a minus one inside a longer planner horizon; the action flow-matching objective averages over offsets.

Go to primary source ↓
e08PDF p. 7, Section 3.4 and Eq. (13)Inspect

A fixed FIFO of historical video K/V is internal to the planner. It informs each new context refresh and is not directly consumed by the action DiT.

Go to primary source ↓
e09PDF p. 8, Sections 3.5 and 4.1Inspect

Non-blocking streams separate planner refresh and action updates. Wan2.2 initializes the planner; reported sizes are 4.99B video, 1.02B action and 1.22B memory/routing parameters. Horizons are 64/16, history six frames and routing 32 queries. Flash prose describes 10-to-2-step distillation with a frozen video DiT.

Go to primary source ↓
e10PDF p. 9, Section 4.2, benchmark protocolInspect

RoboTwin has 50 tasks, 50 clean and 500 randomized training demonstrations per task, global batch 512 and 100 test trials per task per setting.

Go to primary source ↓
e11PDF p. 9, Table 1, AHA-WAM, Flash, Fast-WAM and LingBot-VA rows; Section 4.2Inspect

Clean/randomized/average success is 93.40/92.20/92.80 for AHA-WAM, 90.48/89.92/90.20 for Flash, 91.88/91.78/91.83 for Fast-WAM and 92.90/91.50/92.20 for LingBot-VA. Robot-pretraining flags differ. No success uncertainty is shown.

Go to primary source ↓
e12PDF pp. 9–10, Table 2 and Section 4.3Inspect

Overall success is Fast-WAM 91.83, Naive-Async 88.60, + KV Memory 91.01, + OVCR 91.47 and full AHA-WAM 92.80. The text describes memory and routing as separate and joint additions; no standalone offset-training ablation is reported.

Go to primary source ↓
e13PDF p. 10, Section 4.4, Setup, Implementation and Results; compare p. 1 AbstractInspect

Physical tests use AgileX Piper, head RGB, state/language and four tasks with about 120 demonstrations per task. AHA-WAM and Fast-WAM are pretrained on 24,600 RoboCOIN trajectories (about 165 hours), qualifying the abstract's broad no-robot-pretraining language. Original success is 78.33 versus 68.33 Fast-WAM, 76.67 pi0.5 and 21.67 Motus. Slower baselines use RTC-style execution and interpolation.

Go to primary source ↓
e14PDF p. 11, Figure 4, original/generalization summaries and four task panelsInspect

Generalization success is AHA-WAM 53.3%, pi0.5 55.0%, Fast-WAM 46.7% and Motus 16.7%. Panels show lighting, texture, object configuration and environment shifts. The graphic prints AHA-WAM generalization score 35 and original score 38 despite the caption referring to 0–3 scores.

Go to primary source ↓
e15PDF pp. 16–17, Appendix C, Figure 5 and Table 6Inspect

Appendix C states 30 independent trials per model/task, binary success at score 3, and ordered subtask progress scores 0–3. Table 6 defines plate transfer/racking, towel folding/basket placement, desktop organization and soy-milk preparation. Figure 5 calls the final task Spoon Powder; Table 6 calls it Prepare Soy Milk. Figure 4's larger score aggregates are not defined here.

Go to primary source ↓
e16PDF p. 10, Table 3 and Section 4.5Inspect

AHA-WAM is 41.37 ms/24.17 Hz/4.59×, Flash 17.56 ms/56.95 Hz/10.82× and Fast-WAM 190.00 ms/5.26 Hz/1.00×. Fast-WAM uses official latency; the remaining methods are timed on a single RTX 5090D.

Go to primary source ↓
e17PDF p. 15, Appendix A, Tables 4–5 and omission statementsInspect

Table 4 gives three cameras, 384×320 resolution, six history frames, frequency ratio and RoPE stride 8, 30 routed layers, 14-dimensional states/actions, aggregate action horizon 64 and four 16-action chunks, 32 queries, AdamW 10^-4, weight decay 0.01, 5% warmup, batch 512, five epochs, ten default inference steps and CFG 1.0. Table 5 specifies 16 teacher steps, two student steps, anchors 0/1/2/4/8/12/16 and learning rate 2×10^-5. Run paths, bookkeeping and legacy options are explicitly omitted.

Go to primary source ↓
e18PDF p. 16, Appendix A Training configuration and Appendix BInspect

Joint video/action flow matching uses equal weights and logit-normal noise times. RoboTwin uses 2,500 clean and 25,000 randomized demonstrations; 100 test episodes per task/setting. External baselines use official published results when available.

Go to primary source ↓
e19PDF p. 18, Appendix D, Table 8 including runtime footnote, and Table 9Inspect

Cumulative CUDA stages reduce L_chunk from 415.77±0.33 to 41.37±0.03 ms; Action DiT TRT/CUDA Graph gives 83.87±0.51 ms. Runtime A is PyTorch 2.7.1/cu128, Triton 3.3.1, TensorRT 10.16.1.11; B is PyTorch 2.12.0/cu130, Triton 3.7.0, same TensorRT. Default prefill compile gives 34.59±0.08 ms with 71.45 ms chunk latency; reduce-overhead gives 25.70±0.22 ms prefill but 75.08 ms chunk latency. The ± statistic is not defined.

Go to primary source ↓
e20PDF pp. 18–19, Appendix D, ODE distillation; compare p. 15 Table 5 and p. 8 Section 3.5Inspect

The video DiT is frozen and the student regresses from non-final anchors to the teacher's final action, favoring noisy anchors. Appendix D refers both to a 10-step teacher and a 16-step teacher trajectory; Table 5 specifies 16 steps. Exact noisy-anchor probabilities are not given.

Go to primary source ↓
e21PDF p. 19, Table 11Inspect

One/two/four/ten distilled sampling steps have chunk latencies 14.67/17.56/23.45/41.37 ms and rounded frequencies 68.3/56.9/42.6/24.2 Hz. The table contains no corresponding one- or four-step task-success results.

Go to primary source ↓
e22PDF pp. 16–17, Appendix D Measurement protocol and Table 7Inspect

L_chunk includes image encoding, context access, OVCR and action denoising; L_prefill is a separate asynchronous video call. Timing uses RoboTwin tasks, discards the first warmup episode, averages remaining per-episode means and uses bf16 for baseline and optimized pipelines.

Go to primary source ↓
e23PDF p. 20, Appendix E, Table 12, Open Microwave row and Average rowInspect

Open Microwave clean/randomized success is 40/46 for AHA-WAM, 9/23 for Flash, 62/45 for Fast-WAM and 82/86 for LingBot-VA. The printed average row repeats Table 1's clean/randomized aggregates.

Go to primary source ↓
e24PDF p. 2, Figure 1 RoboTwin chart; compare p. 9 Table 1Inspect

Figure 1 labels LingBot-VA 92.24; Table 1 and its surrounding comparison use 92.20, giving a 0.60-point advantage for AHA-WAM.

Go to primary source ↓
e25PDF p. 11, Section 5, Limitations and future workInspect

The authors identify task/embodiment-dependent temporal hyperparameters and call for dedicated long-horizon benchmarks and stronger future planner representations.

Go to primary source ↓

8.5 Primary sources

Scroll across the image to inspect details. Press Esc to close.