LiLa-WAM: Lightweight Latent Reasoning World-Action Model for Robotic Manipulation
1. Paper overview
In one sentence: LiLa-WAM makes future prediction an auxiliary teacher inside a compact action transformer, gaining control performance while relying on a fixed demonstration-derived task cue. e03e04e05e11e14e16
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors target the training burden of world-action models: pixel generation spends capacity on appearance, while some latent approaches require separately trained stages. They ask whether a compact visual policy can acquire useful foresight through joint training under a single-GPU budget. e02 |
| Core mechanism | |
| A key reported result | RoboTwin 2.0: 50-task manipulation: 90.48 clean; 89.04 randomized Average success rate (%). One jointly trained policy; 50 clean and 500 randomized demonstrations collected per task; one 320×240 third-person view; 50 evaluation rollouts per task. Clean and Random are separate reported columns. Being-H0.7: 90.20/89.60; Motus: 88.66/87.02, respectively. Competitive at 0.5B parameters, but the small clean lead over Being-H0.7 reverses under randomization. These comparisons do not control backbone pretraining or total training compute. e08e09 |
| Reading caution |
Core contributions
Figure 2. One shared expert connects compact visual reasoning to actions and a training-only future-feature target. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the current frame on the left. The snowflake marks frozen DINOv3; its features move upward into fusion and then into the orange query adapter. Follow the compressed blue reasoning tokens into the green expert, alongside the orange VTT, yellow proprioceptive token and green action tokens. The upper foresight tokens lead rightward to the decoder inside the dashed Training Only region. The future image supplies frozen target features for cosine supervision. Equation 3 on page 4 specifies the actual input groups, and Section 3.2 specifies the two readouts; the schematic does not depict a separate planner or an inverse-dynamics stage. e03e04e06e07
What it supports. The architectural reason for the One Model assessment is the shared transformer stream: action velocity and the future latent are read from different positions after joint processing. Feature-space supervision can therefore shape action computation without requiring the policy to render future pixels. The frozen backbone still supplies substantial pretrained visual capacity.
Where the evidence stops. Figure 2's caption says foresight tokens are discarded at inference, while Section 3.2 says the latent is inherently produced and the decoder is discarded. Read this as unused future outputs, not removal of the shared visual-token computation. Action sampling still requires repeated flow steps.
2. Motivation
2.1 The problem and the proposed response
The authors target the training burden of world-action models: pixel generation spends capacity on appearance, while some latent approaches require separately trained stages. They ask whether a compact visual policy can acquire useful foresight through joint training under a single-GPU budget. e02
2.2 What this reading follows
The central question is how to give a small manipulation policy useful foresight without making it generate video. LiLa-WAM compresses frozen visual features, mixes them with task and action tokens, and trains the resulting stream to produce both actions and a future representation. The future decoder teaches the policy during training and disappears at deployment. Read the evidence in three layers: how the shared representation is built, whether its auxiliary supervision improves executed actions, and whether its future features actually respond to action changes. The results support this compact design, while leaving counterfactual prediction accuracy and unseen-task generalization unresolved. e03e04e05e11e14e16
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | One Model |
| Prediction paradigm | Joint prediction |
| Quadrant | Q1 · One Model × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
The shared DiT token stream and two readouts support One Model × Joint prediction at the latent-state level. This is architectural evidence beyond joint losses alone; there is no separate inverse-dynamics action extraction stage. Foresight decoding is auxiliary during training, not an inference-time planner. The recorded joint video-action subcategory is broader than the literal output here: the model predicts features, not video pixels. Efficiency is supported within the reported hardware/settings. e03e04e07e15
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Follow the information bottleneck before asking what is predicted
DINOv3 starts with dense visual evidence, but the expert never receives the whole patch grid directly. Layer features are fused, then a fixed set of learnable queries compresses them into Z_v. This makes representation size an explicit design variable. The chosen implementation uses 64 queries, while Table 4 shows that increasing to 96 raises training memory from 18.0 to 21.3 GB and reduces ten-task success from 70.0% to 67.6%. Table 5 also shows that feature choice matters: blocks 17 and 21 together outperform the final block alone. Reader interpretation: compactness works here through selective visual information, not simply by reducing every component. The ablations motivate the particular bottleneck but do not prove its optimality across resolutions, tasks or hardware. Frozen DINOv3 pretraining remains outside the reported end-to-end policy-training budget. e03e07e12
Figure 3. A task is represented by what its demonstrations change in visual feature space. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The left panel groups initial and final demonstration frames in a visual feature space. The subtraction symbol alone does not name its operand order; Equation 4 fixes it as final embedding minus initial embedding, averaged over episodes of the same task. An MLP then maps that average into the expert's task-token width. On the right, each point is a per-episode transition embedding rather than the final averaged task vector. Colors distinguish ten RoboTwin tasks, and the plot includes clean and randomized episodes. Read the grouping qualitatively: it shows structure in a two-dimensional t-SNE projection, without a numerical separation or generalization metric. e05e07e11
What it supports. The task cue can be computed offline and reused, so a deployment episode needs neither a language instruction nor a new goal image. The visualization is consistent with task-specific transition structure. It does not show that the policy can infer an unknown requested task from the current scene alone.
Where the evidence stops. The target task's existing VTT is selected at deployment. Demonstration dependence and task selection remain part of the interface. t-SNE clusters are illustrative, and the source does not compare this figure against a learned task-ID representation.
5.2 Separate a teacher for the latent from an instruction for the task
Two uses of future imagery have different jobs. The VTT uses the initial and final frames of task demonstrations offline; their average feature difference identifies which transition the task should accomplish. Future supervision instead uses the actual observation at the look-ahead horizon as a per-training-example target for the decoder. The first provides task conditioning, while the second trains the shared representation. At deployment, the selected fixed VTT is still supplied, but that example's future image is unavailable and the decoder is discarded. The expert predicts the action velocity through repeated flow evaluations and inherently produces future-aware visual outputs. The architecture caption's wording about discarding foresight tokens should therefore be read alongside Section 3.2: the future readout is unused, while the shared visual-token computation continues to contribute to action generation. e03e04e05e06e07
5.3 Distinguish better control from verified counterfactual dynamics
The evidence answers progressively stronger questions. Table 3 shows that future supervision helps the ten-task policy. Table 7 carries that comparison onto a physical arm with shared data handling and execution. Figure 6 then asks whether a different action changes the predicted future features, and Table 6 disables a specific action-information pathway. That pathway uses visual queries with action keys and values: although the table calls it visual-to-action attention, the information being removed travels from actions into visual updates. Reader interpretation: these results make a purely action-insensitive representation less plausible. They still do not show that an alternate predicted future is correct. A feature can change under an inconsistent action without forecasting the result of executing it. Evaluating such accuracy requires matched action interventions and observed future outcomes, beyond the supplied sensitivity plot. e11e14e15e16e17
5.4 Training and inference
During training
The encoder is DINOv3-ViT-L/16, using blocks 17 and 21. The adapter has 64 queries and four layers; the expert has 12 layers, eight heads and width 768; the decoder has four layers. Of 0.5B parameters, 0.2B are trainable and 0.3B frozen. End-to-end training refers to the trainable pipeline, excluding encoder pretraining. e07
AdamW uses betas (0.9, 0.99), weight decay 0.01, batch size 128, clipping norm 1.0 and foresight weight 0.5. Flow times are logit-normal. Stage-wise cosine scheduling uses early/later peak rates 2e-4 and 5e-5. The source reports a 24 GB budget and about 110 GPU hours on one RTX 5090 for 50-task RoboTwin training. e07
During inference
Select the target task's VTT and integrate the action velocity from noise with ten Euler ODE steps. The decoder is discarded; future-aware hidden outputs still arise inside the expert. Thus a shared forward pass is not one-step action sampling. Reported latency is 85 ms on an RTX 4090. e04e07
The main configuration predicts 32 actions and executes the first 16, with future target Delta=32. LIBERO instead uses 12-action chunks. Physical control uses 32-action chunks at 30 Hz; an asynchronous queue triggers new inference when eight actions remain and refreshes upon completion. e07e08e17
5.5 Implementation flow
- Compress spatial features
Frozen DINOv3 supplies patch features. Features from multiple layers are concatenated and linearly projected, then learnable queries pass through interleaved self- and cross-attention in the adapter. The resulting visual latent Z_v has a fixed query count, bounding downstream attention size independently of image patch count. e03
- Mix task, observation and action information
The DiT expert receives [Z_a; Z_q; c_tau; Z_v]: action, proprioception, task and visual tokens. Bidirectional self-attention allows all groups to exchange information. A linear head reads action velocity at action positions; observation-side outputs form the future latent after look-ahead interval Delta. e03e15
- Supervise foresight in feature space
A lightweight query-based decoder maps the future latent to frozen DINOv3 patch features of the true future observation. Per-token cosine loss supplies auxiliary supervision. Both objectives update the shared trainable representation; the future image is a target, not a deployment input. e04e06
6. Experiments & results
LiLa-WAM trains action generation and future-feature prediction inside one compact transformer stream built on frozen DINOv3 features. A demonstration-derived Visual Transition Token selects the task. Its strongest evidence combines competitive simulation success with controlled foresight-loss ablations, including physical execution; it does not establish general-purpose future simulation or unseen-task instruction following.
6.1 Read the original evidence
Tables 1 and 2. Small parameter count accompanies competitive reported success, with distinct evaluation protocols. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each table independently. The left table averages success over 50 RoboTwin tasks, separating Clean and Random columns; LiLa-WAM uses one joint policy and is evaluated over 50 rollouts per task. On the right, Spatial, Object, Goal and Long are LIBERO suites, with a separate trained policy per suite and 50 rollouts per task. Params reports model size in billions, not trainable parameter count or measured compute. The bottom Ours rows identify LiLa-WAM. Compare within columns before reading the aggregate: the right-hand average can conceal differences on Goal and Long, just as the left-hand clean result cannot stand for randomized evaluation. e07e08e09e10e18
What it supports. LiLa-WAM reports 90.48% clean and 89.04% randomized RoboTwin success. Being-H0.7 is slightly lower on clean scenes at 90.20% but higher on randomized scenes at 89.60%. LIBERO's reported 97.1% average matches OpenVLA-OFT, while LiLa-WAM uses 0.5B parameters versus 7B. This supports competitiveness rather than a uniform win.
Where the evidence stops. These tables do not equalize pretraining, training compute or all baseline data choices, and provide no seed-level uncertainty. Appendix Table 8 also shows substantial per-task failures despite the strong RoboTwin average. Parameter efficiency alone does not establish total training-cost superiority.
Table 7. Foresight supervision also improves this controlled comparison on a physical robot. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The upper row disables the future-feature loss; the lower row is the full model. Button means pushing the green button, Block placing the red block on the box, Banana placing the banana on the pink plate, and Basketball putting the basketball in the hoop. Each column uses 60 demonstrations and 50 evaluation rollouts per variant with randomized object placements. Read this as executed control on the PiPer arm, not a video-generation score. Appendix A matters to interpretation: both variants share preprocessing and postprocessing, and both use the asynchronous action-queue execution scheme. This keeps those deployment choices from explaining the difference between rows. e16e17
What it supports. The average rises from 74.0% to 82.0%, an eight-percentage-point gain. Every task improves, with the largest increase on Button, from 72% to 86%. Basketball remains the lowest at 56% for the full model, showing that useful foresight supervision does not remove task difficulty.
Where the evidence stops. The control is the same model without future supervision, not a broad comparison against external robot policies. Four tasks on one platform, without seed-level intervals, support a focused physical validation rather than a general deployment guarantee.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| RoboTwin 2.0: 50-task manipulation One jointly trained policy; 50 clean and 500 randomized demonstrations collected per task; one 320×240 third-person view; 50 evaluation rollouts per task. Clean and Random are separate reported columns. | 90.48 clean; 89.04 randomized Average success rate (%) | Being-H0.7: 90.20/89.60; Motus: 88.66/87.02, respectively. Competitive at 0.5B parameters, but the small clean lead over Being-H0.7 reverses under randomization. These comparisons do not control backbone pretraining or total training compute. e08e09 |
| LIBERO: four-suite manipulation Separate policy per suite; ten tasks per suite and 50 rollouts per task; third-person plus wrist views at 256×256; chunk length 12. | Spatial 98.0; Object 98.8; Goal 97.2; Long 94.2; reported average 97.1 Success rate (%) | OpenVLA-OFT averages 97.1 at 7B parameters; LiLa-WAM uses 0.5B. OpenVLA-OFT Goal/Long scores are 97.9/94.5. Matches the reported aggregate with fewer parameters, without leading every suite. Do not reinterpret suite-specific training as one jointly trained LIBERO model. e08e10 |
| RoboTwin 2.0: ten-task mechanism ablations Selected ten-task subset in Section 4.2; the exact ablation clean/random data mixture and training schedule are not fully specified. | Full model 70.0; no foresight loss 54.4; CLIP-language conditioning 61.4; Qwen3VL-language variant 61.0 Average success rate (%) | Reader-computed full-minus-ablated differences: +15.6 percentage points for foresight and +8.6 for VTT versus CLIP-language. Supports the two design choices within this experiment. The backbone comparison also changes task conditioning, and 70.0 is not directly comparable with the 50-task headline. e08e11 |
| RoboTwin 2.0: action-to-visual information pathway Ten-task attention-mask comparison; visual queries are prevented from reading action keys/values. | 64.8 with the pathway masked Average success rate (%) | 70.0 with default bidirectional attention: a 5.2-percentage-point decrease. The queried direction is visual-to-action attention, but the removed information flows from actions into visual states. The intervention supports functional coupling, not a calibrated dynamics simulator. e15 |
| Real robot: four manipulation tasks AgileX PiPer with third-person D435 RGB; randomized object placements; 60 demonstrations and 50 evaluation rollouts per task for each variant; shared preprocessing/postprocessing. | Button 86; Block 90; Banana 96; Basketball 56; average 82.0 Success rate (%) | Without foresight: 72, 84, 92, 48; average 74.0. An eight-percentage-point aggregate gain extends the foresight ablation to executed physical control. Basketball remains difficult, and four tasks do not establish broad robot generalization. e16e17 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 3. Removing future supervision produces the largest success drop in this component comparison. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the bottom row as the reference: DINOv3, foresight enabled, VTT conditioning and 70.0% average success. Compare it with the first row, which retains DINOv3 and VTT but disables foresight, to examine the auxiliary objective. Compare it with the second row to examine the replacement of VTT by CLIP-encoded language. The third row uses Qwen3VL and language at 2.2B parameters, so it changes more than backbone size relative to the full system. All averages belong to the selected ten-task RoboTwin experiment. Section 4.2 names that subset; it is a different evaluation context from the 50-task headline table. e08e11
What it supports. The full model exceeds the no-foresight row by 15.6 percentage points, calculated from 70.0 minus 54.4. VTT exceeds CLIP-language conditioning by 8.6 points under this comparison. Together these results support the usefulness of future supervision and the chosen task cue within the reported setup.
Where the evidence stops. The source does not fully specify the ablation's clean/random training mixture, exact schedule or seed variability. The Qwen3VL row changes both backbone and task-conditioning configuration, so it does not isolate the effect of model scale.
Table 6. Blocking action information from visual-token updates reduces ten-task control success. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the Masked Query–KV column, because the direction is easy to misread. Z_v denotes compressed visual-context tokens, and Z_a denotes action tokens. In the second row, visual queries cannot read action keys and values. The source calls this a visual-to-action attention mask, describing which tokens attend to which; the information removed from the visual updates originates in the action tokens. The first row preserves the default bidirectional self-attention over the shared sequence. Compare the rightmost column for average task success on ten RoboTwin tasks. This table measures control performance after a connectivity intervention; the separate Figure 6 probe instead measures similarity between original and perturbed future-feature predictions. e03e08e14e15
What it supports. Masking the specified pathway reduces success from 70.0% to 64.8%, a reader-computed decrease of 5.2 percentage points. Section 4.5 also reports that replacing actions late in denoising lowers decoded-feature similarity to approximately 0.85. These complementary observations support useful action dependence in the shared representation, while neither directly establishes counterfactual prediction accuracy.
Where the evidence stops. The mask changes information mixing throughout the shared expert, so a success decrease does not isolate accurate dynamics prediction. Figure 6 remains supporting page-level evidence: its axis titles are unreadable in the supplied rendering, and its shaded-band statistic is undefined.
7. Analysis & limitations
7.1 What the evidence leaves open
VTT removes deployment text and goal-image requirements but retains task identity and task demonstrations. Unseen-task composition or arbitrary language following is not evaluated. Qualitative attention and t-SNE separation do not establish causal task understanding. e05e08e13
Figure 6's replacement probe lowers future-feature similarity to about 0.85 late in denoising, while added noise has little effect. This demonstrates action sensitivity, not prediction accuracy under counterfactual execution; the shaded band's statistical definition is unstated. e14
Success tables provide no seed-level uncertainty. Strong averages conceal failures: Table 8 reports Hanging Mug at 56/44 and Place Dual Shoes at 60/54 for clean/randomized settings. There is no dedicated limitations section. e09e10e11e16e18e20
7.2 Questions for discussion
- Does foresight still improve control when visual tokens cannot read action tokens?
- Does VTT outperform a matched task-ID embedding when demonstrations and optimization are controlled?
- How well do predicted features match actual alternate-action futures rather than merely change under perturbation?
8. Reproducibility audit
8.1 Requirements and known gaps
Recover exact update counts, schedule stage boundaries, logit-normal parameters, data allocation for ablations, normalization and software versions before claiming replication. Section 4.1 gives substantial architecture and optimizer detail, but these remaining choices are not fully specified there or in Appendix A. e07e08e11e17
Preserve the query budget and feature layers: Table 4 pairs 64 queries with 70.0% and 18.0 GB versus 96 queries with 67.6% and 21.3 GB. Table 5 gives 48.0% for block 24 alone versus 70.0% for blocks 17+21. These are ten-task measurements, not full-benchmark memory guarantees. e12
Reader-proposed checks are specified in the illustrated edition: cross foresight supervision with the visual-query/action-key mask, and compare VTT against matched task-ID and shuffled-vector controls. Neither check has been run. e05e11e15
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Cross future supervision with the action-information pathway
Reader-proposed, not run: train a two-by-two comparison on the paper's ten named RoboTwin tasks, crossing foresight loss enabled/disabled with the Q:Z_v, K/V:Z_a pathway enabled/masked. Fix demonstrations, DINOv3 layers, VTTs, query count, updates and evaluation scenes; repeat training seeds and report task-level success with uncertainty. As a separate diagnostic, retain the trained decoder in the foresight-enabled variants and apply the same replacement actions. The mechanism is strengthened if foresight's control gain is substantially larger when visual tokens can read actions and if decoded features track matched executed futures. Similar gains with the pathway blocked would weaken the specific action-conditioned explanation, while remaining consistent with a generic auxiliary representation benefit. e03e06e08e11e14e15
Check 2: Test whether transition content adds more than task identity
Reader-proposed, not run: compare the real VTT with a learned per-task embedding of matched output width and with fixed random task vectors passed through the same conditioning interface. Use only training demonstrations to compute VTTs, identical scene splits, optimization and parameter-budget accounting. Add a deliberately shuffled assignment of VTT vectors to task labels, used consistently during training and evaluation, to test whether the vector's specific semantics matter beyond providing a stable identifier. Evaluate success on held-out randomized scenes across repeated seeds. A reproducible VTT advantage over both identity controls would support transition content as useful conditioning. Similar performance would suggest that the reported language comparison may largely reflect efficient task identification rather than uniquely informative visual transition geometry. e05e07e08e11
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version block; every method and evaluation page; Figures 1–10; Tables 1–8; and the real-robot hardware and asynchronous-inference appendix. Cross-checked Figure 2 with Equations 1–3 and the decoder description, Figure 3's subtraction with Equation 4, and Table 6's query/key-value direction with its prose. All six final original crops were inspected: Figures 2–3, Tables 1–2 together, Table 3, Table 6 and Table 7. Figure 6 was inspected on page 9 and at higher resolution; its axis titles remain unreadable, so the final diagnostic crop uses the fully legible Table 6. The perturbation result is retained, with its protocol and metric supported by Section 4.5 on pages 8–9. Reference-only pages 12–14 were read in the complete text pass; they are not needed for retained method or numerical claims and were not visually inspected. No separate supplement, linked code or external baseline paper was inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 15, 16. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1 Introduction
- 2 Related Work (2.1–2.3)
- 3 Method (3.1–3.4), Equations 1–6
- 4 Experiments (4.1–4.6, including 4.4 attention analysis)
- 5 Conclusion
- References, pages 11–14
- Appendix A: Real-Robot Experiment Details
- Appendix figures and Table 8, pages 15–16
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Identity: title and all ten authors match the catalog, with name-order and capitalization normalization, including Jiahong Ning. The inspected title page carries arXiv:2608.03701v1 [cs.RO], 4 Aug 2026, and a front-matter date of August 5, 2026. Both dates are preserved; no later revision was supplied or substituted.
- The acquisition omission that text extraction does not reconstruct figure images was addressed by inspecting the retained PDF: pages 1–11 and 15–16, all ten figures, all eight tables, and six final crops. Reference-only pages 12–14 were read as text but not visually rendered.
- Separate supplemental material availability has not been fully verified. No separate supplement was supplied; the included Appendix A was read in full.
- Code and external links were not inspected, and experiments were not reproduced. The paper's code-availability statement is not an independently verified release.
- Figure 6 has legible ticks, curves and legend, but its displayed axis titles remain clipped in the supplied PDF rendering at higher resolution. Section 4.5 and its caption identify the metric and intervention; no missing labels were recreated. The illustrated edition uses the readable Table 6 as its action-conditioning diagnostic crop and preserves the Figure 6 findings with pages 8–9 as supporting evidence.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/author/affiliation block and arXiv margin stamp
The title matches the supplied observed title. Credits are Fan Yang, Yuting Su, Xiaobo Wang, Yuncheng You, Fugui Fan, Yuting Wu, Minghui Wu, Chenxu Zhao, Jiahong Ning and Peiguang Jing. The stamp is v1, 4 Aug 2026; front matter says August 5, 2026. Five affiliations are printed.
Go to primary source ↓e02PDF pp. 1–2, Introduction and contribution bullets
Motivation concerns pixel-level appearance overhead, multi-stage latent pipelines and large backbones; the proposed contributions are a lightweight shared latent model and a demonstration-derived VTT.
Go to primary source ↓e03PDF p. 3, Figure 2; p. 4, Sections 3.1–3.2, Equations 1–3
Frozen visual features are fused and query-compressed. The single DiT sequence contains action, proprioception, VTT and visual tokens. Action positions output velocity and observation positions output the future latent.
Go to primary source ↓e04PDF p. 3, Figure 2 caption; p. 4, Section 3.2, Future-state prediction
The future latent is decoded to frozen-encoder features during training. The caption says foresight tokens are discarded at inference; the prose clarifies that the expert still produces future-aware latent outputs and the decoder is discarded.
Go to primary source ↓e05PDF p. 5, Figure 3 and Section 3.3, Equation 4
VTT averages final-minus-initial global visual embeddings over task demonstrations, then an MLP creates a fixed task token. Figure 3 shows its construction and ten-task transition-embedding t-SNE.
Go to primary source ↓e06PDF p. 5, Section 3.4, Equations 5–6 and following paragraph
Training combines conditional flow-matching velocity regression with per-token cosine supervision of decoded future features. Gaussian noise and demonstrated actions define a linear flow interpolation.
Go to primary source ↓e07PDF p. 6, Section 4.1, all three paragraphs
Implementation specifies DINOv3 blocks 17/21, 64 queries, 4/12/4 adapter/expert/decoder layers, eight heads, width 768, 16-dimensional proprioception, 14-dimensional actions, 32-step chunks with 16 executed, Delta=32, 0.5B total/0.2B trainable parameters, optimizer settings, 24 GB budget, 110 GPU hours on RTX 5090, ten Euler steps and 85 ms on RTX 4090.
Go to primary source ↓e08PDF p. 6, Section 4.2, RoboTwin 2.0 and LIBERO protocol paragraphs
RoboTwin has 50 tasks, 50 clean and 500 randomized demonstrations per task, one third-person view and 50 rollouts per task; ten named tasks define the ablation subset. LIBERO uses separate suite policies, two views, 12-action chunks and 50 rollouts per task.
Go to primary source ↓e09PDF p. 6, Table 1, Ours/Being-H0.7/Motus rows, Params/Clean/Random columns
Ours is 0.5B and 90.48/89.04; Being-H0.7 is 3B and 90.20/89.60; Motus is 8B and 88.66/87.02. Table entries have no uncertainty intervals.
Go to primary source ↓e10PDF p. 6, Table 2, Ours and OpenVLA-OFT rows
Ours reports 98.0/98.8/97.2/94.2 with average 97.1 at 0.5B. OpenVLA-OFT reports 97.6/98.4/97.9/94.5 with average 97.1 at 7B; uncertainty is not tabulated.
Go to primary source ↓e11PDF p. 7, Section 4.3 first paragraph and Table 3, all rows
Ten-task averages are 54.4 without foresight, 61.4 with CLIP-language, 61.0 with Qwen3VL/language and 70.0 for the full DINOv3/VTT model. The Qwen variant changes both backbone and task-conditioning configuration.
Go to primary source ↓e12PDF p. 7, Tables 4–5 and associated paragraphs
Query counts 8/32/64/96 yield 66.2/65.2/70.0/67.6 success and 14.7/16.0/18.0/21.3 GB training memory. Feature blocks 24/23/21/17/17+21 yield 48.0/53.2/57.4/67.2/70.0 on the ten-task subset.
Go to primary source ↓e13PDF pp. 7–8, Section 4.4, Figures 4–5 and captions
Selected attention visualizations suggest VTT responds to task objects/destinations while action tokens emphasize manipulation regions. Per-layer place-fan maps cover 12 expert blocks; action-token maps average denoising steps.
Go to primary source ↓e14PDF p. 8, Section 4.5 Action perturbation probe; p. 9, continuation and Figure 6/caption
The probe adds Gaussian noise or replaces actions with another trajectory's chunk and compares decoded future-feature cosine similarity. Replacement begins reducing similarity after approximately s=0.5 and reaches about 0.85 at s=1. The rendered graph has clipped axis titles; the text identifies axes, and the shaded-band statistic is not defined.
Go to primary source ↓e15PDF p. 9, Section 4.5 Attention mask analysis, Table 6 and Attention dynamics paragraph
Default self-attention is bidirectional and unmasked. Masking Q:Z_v, K/V:Z_a reduces ten-task success from 70.0 to 64.8, blocking action information from entering visual-query updates.
Go to primary source ↓e16PDF p. 9, Section 4.6 setup; p. 10, continuation, Figure 8 and Table 7
The PiPer/D435 setup evaluates four tasks under randomized placements with 60 demonstrations and 50 rollouts per task. Button/Block/Banana/Basketball scores are 86/90/96/56 with foresight versus 72/84/92/48 without it; averages are 82.0 and 74.0.
Go to primary source ↓e17PDF p. 15, Appendix A, Hardware platform and Asynchronous inference paragraphs
The six-DoF PiPer has a parallel gripper and absolute joint-position actions with continuous gripper control. Third-person D435 RGB, 32-action chunks and 30 Hz control are used. Inference starts when eight queued actions remain; both variants share pre/postprocessing.
Go to primary source ↓e18PDF p. 16, Table 8, LiLa-WAM Clean/Rand. columns, Hanging Mug, Place Dual Shoes and Average rows
Hanging Mug scores 56/44 and Place Dual Shoes 60/54, while overall averages are 90.48/89.04. Per-task results reveal substantial variation behind the headline average.
Go to primary source ↓e20PDF p. 11, Section 5 Conclusion and transition to References; p. 15, Appendix A
The paper concludes after experiments and proceeds to references and a real-robot appendix; it contains no dedicated limitations section.
Go to primary source ↓8.5 Primary sources
LiLa-WAM: Lightweight Latent Reasoning World-Action Model for Robotic Manipulation ↗
PDF · 8,182 extracted words
Source fingerprint
3a936110b132dbb91b7e98a8649f24fa4afd96fa060d472806de1af4ce314ead