Dreaming when Necessary: Advancing World Action Models with Adaptive Multi-Modal Reasoning
1. Paper overview
In one sentence: AdaWAM gates subtask-language updates and visual foresight independently to improve difficult manipulation, trading modest per-step overhead against more successful task execution. e02e03e04e06e08e13
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors argue that always generating video wastes computation during simple motion, while action-only prediction loses useful physical foresight during precise manipulation. Static language also underspecifies progress through long tasks. Their proposed remedy is to learn when each reasoning modality should be invoked. e02 |
| Core mechanism | Source description A trajectory-guided annotation pipeline supplies manipulation-phase and subtask-transition supervision for adaptive routing. e03 |
| A key reported result | LIBERO-Long manipulation: 99.1; overall LIBERO 98.5 Success rate (%). Table 1 simulation evaluation; Appendix A identifies LIBERO-10 as the hard subset. Main-table rollout counts and training splits are not specified. Long: LingBot-VA 98.5, Fast-WAM 95.2, without visual reasoning 96.6, without text reasoning 97.4. Best listed Long result; overall 98.5 ties ACoT-VLA and LingBot-VA. Missing non-Long text-ablation cells remain unreported. e07e08 |
| Reading caution | Source description The authors identify RGB-only perception and dependence on heuristic routing supervision as limitations. Tactile/depth sensing and reinforcement-learned routing are future directions. e14 |
Core contributions
- Source description
A trajectory-guided annotation pipeline supplies manipulation-phase and subtask-transition supervision for adaptive routing. e03
Figure 2. Adaptive computation begins with two kinds of annotated temporal structure. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the episode strip: blue markers represent fine manipulation, while red markers mark subtask transitions. Below it, follow the arrows from robot trajectories through motion parsing and manipulation cues. These cues support the green binary manipulation-label strip and the candidate windows on the right. The right-hand branch combines windows with sampled visual observations, a subtask description, a completion criterion and neighboring context. Qwen verifies a boundary, then boundary fusion imposes temporal ordering and segment validity. Section 3.1 identifies this verifier as Qwen3-VL-8B and describes completion as visually stable. The resulting labels supervise the router; this is an offline annotation pipeline. e03e05e15
What it supports. AdaWAM's adaptivity depends on labeled task structure. Motion cues supply a physical basis for deciding when precise manipulation occurs, while local semantic verification associates visual progress with the next subtask. This separates the supervision for visual reasoning from the supervision for text updating, even though both originate in demonstrations.
Where the evidence stops. The diagram does not establish annotation accuracy. Exact thresholds, sampling rules and verifier prompts are unreported. Qwen3-VL-8B here is the annotation verifier; Appendix B names a separate Qwen3-VL-4B model for runtime text reasoning.
2. Motivation
2.1 The problem and the proposed response
The authors argue that always generating video wastes computation during simple motion, while action-only prediction loses useful physical foresight during precise manipulation. Static language also underspecifies progress through long tasks. Their proposed remedy is to learn when each reasoning modality should be invoked. e02
2.2 What this reading follows
A robot stacking bowls does not face the same problem throughout its trajectory. Reaching can require little anticipation; grasping can require precise physical guidance; switching to the next bowl requires an updated subtask. AdaWAM turns this distinction into two supervised routing decisions. A text model updates the current subtask, a video model supplies optional future latents, and an action model generates the next movement chunk. Read the annotation pipeline first, then the inference mechanism and evaluation tables. The evidence supports selective benefits on long-horizon and compositional tasks, while leaving the optimality of the routing schedule and broad generalization unresolved. e02e03e04e06e08e13
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Joint prediction |
| Quadrant | Q3 · Dual-system × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
Separate VideoDiT and ActionDiT modules, plus a text specialist, support the recorded Dual-system interpretation rather than One Model. Joint prediction applies to the video-action capability and co-training; inference also permits action-only chunks. Long-horizon support comes from subtask updating/history conditioning, not a demonstrated memory bank, and efficiency results do not establish a hard real-time guarantee. e04e05e06e12e15
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Convert task progress into two supervised decisions
AdaWAM's starting point is a distinction between knowing what to do next and anticipating how an object will move. The annotation pipeline gives these questions different targets. Robot-state changes identify manipulation intervals; candidate subtask windows are checked by a semantic verifier and fused into an ordered sequence. During Stage 1, these annotations train the router alongside the video and action flow models. The router therefore learns to imitate the supplied labels, rather than discovering computation allocation solely from task reward. This matters when interpreting the word adaptive: deployment decisions depend on current context, but their supervision inherits the assumptions of the trajectory heuristics and verifier. The authors explicitly identify this dependence as a limitation and leave reward-based routing to future work. e03e04e05e14
Figure 3. Two optional reasoning branches condition an action generator that remains active. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read upward from the overall instruction, current subtask and observation encoders. Gray tokens represent language, orange tokens represent visual information and blue tokens represent actions; striped tokens indicate noise. The router sits beneath the text and visual blocks, with separate TR and VR outputs above them. The figure is schematic, so use Equations (3)–(5) to recover deployment order: predict both routing tokens, optionally update subtask text, optionally generate future visual latents, then sample the action chunk. Section 3.2 explicitly makes the token decisions independent. Both reasoning branches can therefore be enabled in one chunk, and skipping visual generation still leaves observation conditioning available. e04e05e06e15
What it supports. The action policy can use refreshed semantic guidance, predicted physical foresight, both or neither newly generated intermediate. Separate VideoDiT and ActionDiT modules support the catalog's Dual-system assessment. The visual branch affects inference-time action conditioning; its role is more than an auxiliary video loss during training.
Where the evidence stops. Figure 3 shows future observations and an action sequence entering encoders. These are consistent with training representations in Equation (1); the inference equations instead start from observation history and noise. The schematic alone must not be read as requiring future ground-truth observations at deployment.
5.2 Follow one action chunk through the gates
For each chunk, the router sees pooled current visual information and embeddings of the overall task and retained subtask. It predicts both activation tokens before either reasoning update. If the text token is active, the text VLM uses visual history and the global instruction to produce updated subtask text; otherwise that text persists. If the visual token is active, VideoDiT generates future latents using the observation history and updated text. ActionDiT then samples the continuous action chunk from its flow under the available conditions. This ordering explains how text can influence a dreamed future and ultimately the action. It also explains why action-only chunks retain perception and language conditioning. The method specifies conditional generation, without introducing candidate-rollout scoring or an inverse-dynamics action extraction stage. e04e06
5.3 Separate useful reasoning from optimal scheduling
Reader analysis: the experiments answer several related questions with different strength. LIBERO-Long and the selected RoboTwin hard tasks support the usefulness of the full architecture. The unseen soup-and-butter test gives more focused evidence that removing textual reasoning damages compositional success. Neither comparison holds capabilities constant while changing only when reasoning is invoked. Likewise, Figure 6 demonstrates a useful operating point, but AdaWAM is slightly slower per inference step than Fast-WAM. Its shorter reported duration could reflect better actions and fewer retries, as the authors argue, rather than cheaper individual decisions. A stronger test of the central scheduling claim would freeze the generators and compare learned routing with always-on and matched-budget schedules. This distinction preserves the reported gains while identifying what remains unestablished. e07e08e09e12e13
5.4 Training and inference
During training
Stage 1 jointly trains VideoDiT, ActionDiT and the router using continuous-time flow matching plus binary cross-entropy on annotated routing labels. Stage 2 freezes the generative backbone and router, training only the text VLM with next-subtask token likelihood on explicit transition segments. e05
The PyTorch implementation uses eight NVIDIA A100 80GB GPUs. Stage 1 runs 50,000 steps with learning rate 3×10⁻⁵ and weight decay 0.005; Stage 2 runs 10,000 steps with learning rate 10⁻⁵ and weight decay 0.01. Training wall time is unreported. e15
During inference
First predict both routing tokens. If text reasoning is enabled, generate an updated subtask; otherwise retain the previous text. Next generate visual foresight only if the visual token is enabled. Finally integrate the action flow under the assembled conditioning. Future ground-truth observations shown in the architecture schematic are not deployment inputs in Equations (3)–(5). e04e06
The generated action chunk is the policy output used for task execution, while imagined visual latents provide conditioning. The paper does not describe a rollout-scoring planner or inverse-dynamics action extractor. Exact chunk execution and feedback cadence remain unspecified. e06e07
5.5 Implementation flow
- Construct reasoning supervision
End-effector motion, orientation changes, local variation and gripper activity identify fine manipulation intervals and candidate subtask windows. Qwen3-VL-8B verifies the earliest stable subtask completion within sampled windows. Boundary fusion enforces ordering and segment validity; temporal refinement removes isolated manipulation labels. e03
- Separate physical prediction from semantic updating
A pretrained encoder maps observations to latent tokens. VideoDiT predicts future visual latents; ActionDiT generates continuous actions conditioned on observations, subtask text and optional foresight. A separate VLM predicts the next subtask. Appendix B specifies Wan2.2-5B, a compressed 1B action module with hidden dimension 1024, and Qwen3-VL-4B for runtime text reasoning. e04e15
- Route each action chunk
The router concatenates pooled current visual features with global-task and current-subtask embeddings. It independently predicts the text and visual activation tokens. Thus both branches may be active; this is not a mutually exclusive three-class selector. e04e06
6. Experiments & results
AdaWAM spends computation selectively: it updates subtask language when needed, predicts future visual latents when needed, and always generates an action chunk. Separate text, video and action modules are coordinated by a supervised router. The clearest evidence is improved long-horizon and compositional task success, with a qualified efficiency tradeoff rather than uniformly faster action generation.
6.1 Read the original evidence
Table 1. The strongest LIBERO result is concentrated in the Long suite. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Locate AdaWAM in the final row and begin with the Long column. Its 99.1 exceeds LingBot-VA's 98.5 and Fast-WAM's 95.2. Then compare the two immediately preceding ablation rows: removing visual reasoning gives 96.6, while removing text reasoning gives 97.4. Section 4.1 defines these as module removals, so neither row is a direct experiment on routing timing alone. Finally move right to Overall: AdaWAM's 98.5 ties ACoT-VLA and LingBot-VA. The dashes in the text-ablation row represent missing results outside Long; they are not zeros and should not enter any average. e07e08
What it supports. The Long column supports contributions from both reasoning capabilities: the full model exceeds the visual ablation by 2.5 percentage points and the text ablation by 1.7 points. The overall tie gives a narrower conclusion than a claim that AdaWAM dominates every LIBERO setting.
Where the evidence stops. The table supplies no uncertainty intervals, main-evaluation rollout counts or exact training split. Small differences near the success ceiling cannot establish statistical significance. Module-removal ablations also change model capability, so they do not isolate the router's scheduling quality.
Tables 2 and 3. Hard-task gains coexist with a Random-overall deficit and a limited physical evaluation. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the Clean and Random sections independently. Within each, Hard SR aggregates the seven displayed difficult tasks, while Overall SR concerns the broader benchmark described in Appendix A. Compare AdaWAM's rightmost column with Fast-WAM: hard success rises from 83.43 to 88.43 in Clean and from 84.57 to 86.57 in Random. Next check Overall before generalizing: AdaWAM leads Clean but scores 91.35 against Fast-WAM's 91.78 in Random. The lower table changes setting entirely to physical ALOHA/PiPER tasks. Its cleanup and wiping rows are real execution evidence, separate from the simulated RoboTwin aggregates. e07e09e10
What it supports. AdaWAM's gains are strongest on the selected hard tasks, rather than uniformly across all randomized tasks. Physical cleanup reaches 70 compared with FastWAM's 30. Wiping reaches 60, but both AdaWAM ablations also reach 60, so this task does not establish a benefit from combining both reasoning modules.
Where the evidence stops. Do not pool simulation and physical scores. The main tables omit trial counts, uncertainty and detailed baseline training parity. Seven selected hard tasks and two physical tasks support specific manipulation findings, not unrestricted robustness under domain shifts.
Figure 6. Per-step speed and total task duration tell different stories. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the legend to locate green AdaWAM and blue Fast-WAM, then read horizontal position as inference seconds per step and vertical position as success rate. Section 4.3 describes 100 trajectories per task. AdaWAM appears at approximately 0.46 seconds for stacking and 0.48 for cabinet placement, slightly to the right of Fast-WAM at approximately 0.40 and 0.42. MM-ACT lies beyond the horizontal axis breaks at 1.04 and 1.51. Bubble size represents total duration according to the caption; AdaWAM carries 327 in both panels. Keep this experiment distinct from Table 2: several plotted success values differ from the main benchmark table. e09e12
What it supports. AdaWAM combines higher plotted success and smaller duration annotations with modestly slower inference steps than Fast-WAM. It is much faster per step than MM-ACT. The authors attribute shorter task completion to fewer retries, but this figure does not separately count retries or quantify their causal contribution.
Where the evidence stops. Both horizontal axes are broken, and the left vertical axis is also broken. Duration units, inference hardware, Clean/Random setting and the red dashed line's meaning are unspecified. Do not infer a calibrated frontier or merge these success rates with Table 2.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO-Long manipulation Table 1 simulation evaluation; Appendix A identifies LIBERO-10 as the hard subset. Main-table rollout counts and training splits are not specified. | 99.1; overall LIBERO 98.5 Success rate (%) | Long: LingBot-VA 98.5, Fast-WAM 95.2, without visual reasoning 96.6, without text reasoning 97.4. Best listed Long result; overall 98.5 ties ACoT-VLA and LingBot-VA. Missing non-Long text-ablation cells remain unreported. e07e08 |
| RoboTwin 2.0 hard-subset manipulation Seven selected hard tasks, reported separately under Clean and Random conditions; benchmark has 50 scenarios. | Hard: Clean 88.43, Random 86.57; Overall: Clean 93.11, Random 91.35 Success rate (%) | Fast-WAM hard: 83.43/84.57; overall: 91.88/91.78, respectively. AdaWAM leads both hard aggregates and Clean overall, but trails Fast-WAM on Random overall. The hard-task selection is not the full benchmark. e07e09 |
| Physical table cleaning AgileX Split-Type ALOHA with PiPER 6-DoF arms; Table 3 gives no trial counts or uncertainty. | Clean Up Trash On Table 70; Wipe Table Clean 60 Reported task success (%) | FastWAM 30/50; π0.5 60/50. Both AdaWAM ablations also score 60 on wiping. Evidence includes physical action execution, but only two tasks; wiping does not distinguish the full model from its ablations. e10 |
| Unseen LIBERO soup-and-butter composition 100 runs on soup&butter, recombining subtasks from seen soup&cheese and cheese&butter instructions. | 61 Success rate (%) | FastWAM 0; MM-ACT 24; without text reasoning 3; without visual reasoning 38. Supports compositional benefit in one specified recombination. The 58-point gap to the text ablation is not a broad out-of-distribution guarantee. e13 |
| StackThreeBowls and PutObjectCabinet inference timing Separate Figure 6 study with 100 trajectories per task; timing hardware and Clean/Random condition are not specified. | AdaWAM approximately 0.46 / 0.48 Approximate inference seconds per step, read from plotted positions | Fast-WAM approximately 0.40 / 0.42; MM-ACT 1.04 / 1.51. Faster than MM-ACT but slower per step than Fast-WAM. Duration annotations favor AdaWAM; their units are not labeled. Figure 6 success values differ from Table 2 and must remain separate. e09e12 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 4. One unseen object combination provides a focused test of textual and visual reasoning. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the task labels before comparing scores. The seen instructions pair soup with cheese and cheese with butter; the unseen instruction pairs soup with butter. Section 4.4 evaluates the recombined task over 100 runs. Compare the last row across models: FastWAM scores 0, MM-ACT 24, AdaWAM without text reasoning 3, without visual reasoning 38, and full AdaWAM 61. The first two rows establish strong reported performance on the familiar combinations; they are not additional unseen tests. The sharp drop for the text ablation is consistent with subtask-language updating helping recombination, while the remaining visual-ablation gap motivates a separate physical-control question. e07e13
What it supports. The full model exceeds the text ablation by 58 percentage points and the visual ablation by 23 on this specific recombination. Both capabilities appear useful in the reported setup. The 61% result also leaves substantial failure, despite AdaWAM's much higher success on the two seen combinations.
Where the evidence stops. One recombination cannot establish general compositional or out-of-domain competence. The experiment does not isolate whether text helps through better subtask content, better transition timing or both; confidence intervals and a detailed leakage audit are absent.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors identify RGB-only perception and dependence on heuristic routing supervision as limitations. Tactile/depth sensing and reinforcement-learned routing are future directions. e14
Removing a whole reasoning module changes capability as well as compute. These ablations do not isolate the benefit of learned timing against always-on or equal-budget random routing. Figure 5 supplies an illustrative fixed-seed trajectory, not phase-specific statistical evidence. e07e08e09e11e13
Tables omit confidence intervals; most main evaluations omit rollout counts, exact splits and baseline retraining details. One compositional test and two physical tasks limit generalization claims. Figure 6 has broken axes, an unexplained dashed line and no duration-unit definition. e08e09e10e12e13
7.2 Questions for discussion
- Does learned routing outperform an equal-compute schedule when policy weights and evaluation seeds are fixed?
- How much unseen-combination failure comes from subtask prediction, routing timing or physical action generation?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the named pretrained modules, demonstrations with robot-state trajectories, subtask descriptions/completion criteria, annotation verification, and the two-stage schedule. The 8B annotation verifier and 4B runtime text model have distinct roles. e03e05e15
Missing details include demonstration counts, exact splits, annotation thresholds/prompts, router architecture and decision thresholds, λ, batch size, optimizer identity, action representation, horizons, flow solver/steps, software versions and latency hardware. Appendix B specifies module sizes and learning rates but does not close these gaps. e03e04e05e06e07e12e15
Proposed checks: compare learned routing with always-on and equal-budget randomized routing on matched seeds; then repeat unseen recombination while controlling whether subtask text is predicted, retained or supplied correctly. Neither check has been run here. e06e12e13
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Hold the generators fixed and test routing timing
Reader-proposed experiment, not performed: on StackThreeBowls and PutObjectCabinet, freeze one trained checkpoint and compare learned routing, always-on reasoning, action-only generation and randomized text/visual gates matched to the learned policy's activation frequencies. Use the same held-out initial states, hardware, action horizon and flow solver. Report success, each branch's activation count, inference time per step, total duration with explicit units, and retry counts over at least the paper's 100 trajectories per task. Supply uncertainty across repeated seeds. If budget-matched randomized gates retain the same success and duration, the claim that learned timing is essential is weakened; a repeatable advantage would distinguish scheduling from merely having the extra modules. e04e06e07e12e15
Check 2: Separate subtask content from transition errors
Reader-proposed diagnostic, not performed: rebuild the seen soup&cheese and cheese&butter split and explicitly exclude soup&butter demonstrations. On matched unseen episodes, hold the action model fixed and compare predicted subtask text, retained text and oracle subtask text supplied at verified boundaries. Cross these conditions with visual generation enabled or disabled, keeping sampling settings fixed. The oracle condition is a diagnostic upper bound, not a deployable policy. Record wrong-subtask choices, late transitions and failures after a correct subtask. If oracle text closes most of the success gap, semantic content or transition timing is the likely bottleneck; persistent failure under correct text motivates testing visual foresight and action generation separately. e03e06e07e13
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version block; Figures 1–6; Tables 1–4; method, training and inference equations on pages 4–6; evaluation text on pages 6–8; limitations on page 9; and benchmark and implementation details in Appendices A–B on page 13. All six final original crops were visually inspected with their labels, axes and table boundaries intact. Figure 3's training-style encoder inputs were checked against Equations (1) and (3)–(5); Figure 6's broken axes and separate evaluation values are disclosed. Reference-only pages 10–12 were read in the complete text and are not used as scientific evidence here. Separate supplements and external code remain outside the supplied reading.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 13. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, authors and abstract
- 1 Introduction
- 2 Related Work
- 2.1 Vision Language Action Models
- 2.2 World Action Models
- 3 Method
- 3.1 Multimodal Reasoning Data Annotation Pipeline
- 3.2 Model Architecture
- 3.3 Multi-task Training
- 3.4 Test-time Inference with Adaptive Multi-modal Reasoning
- 4 Experiments
- 4.1 Benchmarks
- 4.2 Evaluation Results
- 4.3 Inference Time Analysis
- 4.4 Generalization for Adaptive Textual Reasoning
- 5 Conclusion
- 6 Limitations
- References
- A Detailed Information of Benchmarks
- B Implementation Details
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction warning above was addressed by visually inspecting all six figures and all four tables, with supporting PDF pages 1–9 and 13. References on pages 10–12 were read as text.
- Identity note: the inspected title page is arXiv:2606.07089v1 [cs.RO], dated 5 June 2026. Its exact title and seven authors match the catalog. Wei Wu is affiliated with Manifold AI; the other six authors are affiliated with Tsinghua University, so the catalog affiliation string is incomplete. No other edition or revision comparison was supplied.
- The project link is an author statement of code/demo availability. Code, external project material and separate supplements were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/author block and arXiv margin
Exact title and arXiv:2606.07089v1, dated 5 June 2026. Authors: Yinzhou Tang, Jingbo Xu, Yu Shang, Zihao Song, Chen Gao, Wei Wu and Yong Li. The first three share equal contribution. Wei Wu: Manifold AI; others: Tsinghua University.
Go to primary source ↓e02PDF pp. 1–2, Section 1 and Figure 1
Contrasts always-on video-action prediction, action-only prediction and adaptive multimodal reasoning; motivates text at task transitions and visual foresight for precise manipulation.
Go to primary source ↓e03PDF p. 3, Figure 2 and Section 3.1; p. 4, both annotation paragraphs
Trajectory cues generate manipulation intervals and candidate windows. Qwen3-VL-8B verifies stable completion; ordering/validity fusion creates subtask labels. Fine intervals receive temporal refinement; exact thresholds and prompts are absent.
Go to primary source ↓e04PDF p. 4, Figure 3 and Section 3.2; p. 5, text-module and router paragraphs
Separate VideoDiT, ActionDiT and text VLM; action conditioning includes optional future latents. Router uses pooled current observation plus global/subtask embeddings and independently predicts text and visual tokens.
Go to primary source ↓e05PDF p. 5, Section 3.3, Equations (1)–(2) and Stage 2 paragraph
Stage 1 combines video/action flow matching with binary routing supervision; λ is symbolic. Stage 2 freezes backbone/router and fits next-subtask token likelihood on transition segments.
Go to primary source ↓e06PDF p. 5, Section 3.4, Equations (3)–(4); p. 6, Equation (5)
Text token 1 updates the subtask; token 0 retains it. Visual token 1 generates foresight; token 0 yields an empty foresight set. Action flow is conditioned on history, updated text and optional foresight.
Go to primary source ↓e07PDF p. 6, Section 4.1 and variant definitions; p. 13, Appendix A
Evaluates LIBERO, RoboTwin 2.0 and physical tasks. Appendix A identifies LIBERO-10 as hard and seven selected hard tasks among 50 RoboTwin scenarios. Ablations remove either reasoning module; detailed splits and main-table trial counts are absent.
Go to primary source ↓e08PDF p. 6, Table 1, Long and Overall columns and AdaWAM ablation rows
Long: AdaWAM 99.1, LingBot-VA 98.5, Fast-WAM 95.2, no visual reasoning 96.6, no text reasoning 97.4. Overall AdaWAM/ACoT-VLA/LingBot-VA each 98.5. Non-Long text-ablation cells are dashes.
Go to primary source ↓e09PDF p. 7, Table 2, Clean/Random Hard SR and Overall SR rows
AdaWAM hard 88.43/86.57 and overall 93.11/91.35 for Clean/Random. Fast-WAM hard 83.43/84.57 and overall 91.88/91.78. AdaWAM does not lead Random overall. Per-task and ablation results have no uncertainty columns.
Go to primary source ↓e10PDF p. 6, Figure 4; p. 7, Table 3; p. 13, Appendix A, Real-World Testbed
Split-Type ALOHA uses PiPER 6-DoF arms. Clean-up/wiping scores: AdaWAM 70/60, FastWAM 30/50, π0.5 60/50. Both AdaWAM ablations score 60 on wiping. Trial counts are not stated.
Go to primary source ↓e11PDF p. 7, Section 4.2, Performance on Fine-grained Tasks; p. 8, Figure 5
A fixed-seed trajectory illustration contrasts successful AdaWAM HangingMug manipulation with an action-only failure; it does not supply aggregate phase-level success or routing accuracy.
Go to primary source ↓e12PDF p. 7, Section 4.3; p. 8, Figure 6, both panels and caption
100 trajectories per timing task. Approximate plotted step times are AdaWAM 0.46/0.48 s and Fast-WAM 0.40/0.42 s; MM-ACT is plotted at 1.04/1.51 s. AdaWAM duration labels are 327 in both panels; units and timing hardware are not specified. Axes include breaks; the dashed red line is unexplained.
Go to primary source ↓e13PDF p. 8, Section 4.4 and Table 4, UnSeen soup&butter row
100 runs on a recombination of seen soup&cheese and cheese&butter tasks. Unseen success: FastWAM 0, MM-ACT 24, no text 3, no visual 38, AdaWAM 61. AdaWAM seen scores are 98 and 96.
Go to primary source ↓e14PDF p. 8, Section 5; p. 9, Section 6
RGB-only perception and heuristic supervised routing are acknowledged limitations; depth/tactile inputs and unsupervised or reward-based routing are proposed future work.
Go to primary source ↓e15PDF p. 13, Appendix B, Implementation Details
Wan2.2-5B visual module; compressed 1B action module, hidden dimension 1024; Qwen3-VL-4B text module. PyTorch on 8×A100 80GB; stages use 50,000/10,000 steps, learning rates 3×10⁻⁵/10⁻⁵ and weight decays 0.005/0.01. Batch size, optimizer identity, software version and solver configuration are not given.
Go to primary source ↓8.5 Primary sources
Dreaming when Necessary: Advancing World Action Models with Adaptive Multi-Modal Reasoning ↗
PDF · 6,048 extracted words
Source fingerprint
78a15386e02b0c6c1fd4f57ff70d5e5670c603cb3f3c804454d497b82cf89909