Light-WAM: Efficient World Action Models with State-Fusion Action Decoding
1. Paper overview
In one sentence: Light-WAM retains future-video supervision during training and directly decodes actions from pooled backbone states, gaining efficiency while giving up success on harder tasks. e03e05e06e10e11e12e13e14e17
| At a glance | What to know |
|---|---|
| Research problem | Source description Can future-video supervision improve policy representations without making control depend on expensive future generation? The authors target large generative WAMs’ training cost and latency, retaining temporal supervision while replacing iterative action generation with direct decoding. e02e03 |
| Core mechanism | Source description Separate resolution budgets: downsample future-video latents for co-training while preserving the current observation’s original latent resolution for actions. e05 |
| A key reported result | RoboTwin inference efficiency: 72.03 ms; 4.1 GiB; action branch 2.1 ms. Overall latency; peak GPU memory. Per query with cached language on the paper-designated single NVIDIA RTX 4090 48GB GPU; includes VAE/policy, excludes simulator/I/O. Fast-WAM: 404.62 ms, 12.7 GiB; action branch 356.8 ms. Model-query latency does not establish end-to-end robot feedback latency. The table’s π0.5 timing comes from another cited study. e14 |
| Reading caution | Author claim The authors acknowledge weaker performance in harder multi-task settings and no evaluation on dedicated robustness benchmarks such as LIBERO-Plus. e18 |
Core contributions
- Source description
Separate resolution budgets: downsample future-video latents for co-training while preserving the current observation’s original latent resolution for actions. e05
Figure 1. One adapted backbone supplies a training video objective and a direct action decoder. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the two inputs on the left. The blue video path uses downsampled video latents; the red action path uses the current observation at original latent resolution. Black arrows mark shared processing, not simultaneous future generation at deployment. In the central block, context C conditions the transformer, and the adapter contributes to the adapted state. Equation (4) and Appendix A specify this residual as the block output plus its adapter output. Follow the red state connection into MHA: learned queries attend to the selected layer’s tokens as keys and values. The compressed layer states are averaged and fused before action decoding. e03e04e05e06e07e10
What it supports. The architecture explains why video co-training can survive removal of the runtime video rollout: both objectives train adaptation modules in a shared representation pipeline, while the action head reads current-observation states directly. The diagram therefore supports a training-time world-model role rather than inference-time planning through imagined futures.
Where the evidence stops. The sketch does not show all implementation detail, including LoRA placements or step embeddings. Its action curve is schematic; the specified decoder produces an action chunk, not a plotted probability density or an iterative generative trajectory.
2. Motivation
2.1 The problem and the proposed response
Can future-video supervision improve policy representations without making control depend on expensive future generation? The authors target large generative WAMs’ training cost and latency, retaining temporal supervision while replacing iterative action generation with direct decoding. e02e03
2.2 What this reading follows
Light-WAM asks how much of a video-generative model a robot policy needs at deployment. Its answer separates learning from acting: a downsampled future-video objective shapes adapted representations during training, while the deployed policy reads only the current observation, language and proprioception. Learned queries compress several backbone layers into a state for direct action decoding. This reading follows that interface, then compares the reported gains in training throughput and model-query latency with the task results. The tradeoff is uneven: LIBERO remains competitive, RoboTwin exposes a sizable gap to Fast-WAM, and the three physical tasks favor π0.5. All results here are the paper’s reported measurements. e03e05e06e10e11e12e13e14e17
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Supports the recorded classification
Dual-system is supported as a modular video-backbone/action-expert architecture with a separate video head. This is one jointly optimized policy stack, not two independent runtime agents. Other mechanisms and Outside quadrants fit direct action regression with auxiliary video supervision: inference neither jointly samples future video/actions nor obtains actions by inverse dynamics from imagined futures. Efficiency is measured directly. e03e05e06e07e10e13e14
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Keep future supervision in training while preserving current-observation detail
The two branches share an adapted backbone but receive different latent inputs. In the RoboTwin training algorithm, three camera streams form a canvas video; frames 0 through 32 are subsampled at stride four and encoded with the Wan VAE. The video branch spatially downsamples these latents, perturbs them for flow matching, and keeps the first latent frame fixed as an observation anchor. Separately, the action branch uses the original-resolution first latent frame. This is the essential separation: expensive temporal supervision can be reduced without applying that additional spatial reduction to the action observation. Algorithm 2 retains only current-observation encoding, backbone features and action decoding during deployment. It does not send generated future frames to a planner or inverse-dynamics model. The benefit of future prediction is therefore sought through learned representations. e03e05e08e10
5.2 Trace a dense video-token state into a direct action chunk
The action expert begins with adapted states from layers 8, 16 and 24. At each depth, a distinct learned query set attends to that layer’s tokens as keys and values. Sixteen query outputs are averaged and normalized into one layer state; the representation is then projected, concatenated with the other selected states and fused. Appendix A specifies a 6144-dimensional fused state and one residual MLP block. This compression means the temporal decoder need not repeatedly attend over the full video-token grid. Projected sinusoidal step embeddings distinguish positions within the action horizon, and an output MLP produces the action for each position. RoboTwin uses a 24-by-14 chunk. The information reaches actions through the pooled backbone states; there is no iterative action denoising in the described inference algorithm. e04e06e07e10
5.3 Separate the measured tradeoff from the proposed explanation
Reader analysis: the strongest evidence concerns the complete system’s efficiency, while the causal role of its temporal supervision remains less isolated. Table 3 changes multiple components and shows that the compact backbone alone is slower in the tested configuration. Table 4 localizes a major inference saving to the action branch. Table 5 then tests downsampling, adapter count and query count, but never removes the video loss. Consequently, the results support a useful recipe without proving how much success comes from future supervision. Task outcomes constrain the recipe’s value: the LIBERO average is competitive, RoboTwin trails Fast-WAM, and all three physical tasks trail π0.5. Figures 3 and 5 show learned future structure, but plausible frames and attention maps cannot close the missing causal comparison. e11e12e13e14e15e16e17
5.4 Training and inference
During training
The video branch uses 2× spatial latent downsampling and fixes its first latent frame as the observation anchor during flow matching. The action branch reads the original-resolution first latent frame. Joint learning updates LoRA, adapters, video head, proprio encoder and action expert; pretrained backbone and VAE remain frozen. e05e08e09e10e19
Cached-latent training uses AdamW, learning rate 1e-4, weight decay 1e-2, cosine scheduling and 1,000 warmup steps on four H100 GPUs. Global batches are 64/128 for LIBERO/RoboTwin. Selected LIBERO checkpoints are 60K steps for Spatial/Goal, 12.5K for Object and 80K for Long; RoboTwin uses 460K. e09
During inference
Encode current camera input online, combine language with current proprioception, run the adapted backbone once, pool selected states, predict a chunk and execute actions. Algorithm 2 omits future-video generation; it does not specify the number of actions executed before reobservation or controller frequency. e10
5.5 Implementation flow
- Encode observations and context
Wan VAE latents become video tokens through patch embedding. Language and a projected proprioceptive token form cross-attention context. RoboTwin training constructs a three-camera canvas and samples frames 0 through 32 at stride 4. e03e10
- Adapt frozen video features
Pretrained Wan2.1-T2V-1.3B stays frozen. LoRA updates attention and feed-forward projections; adapters at layers 8, 16 and 24 apply residuals to transformer-block outputs, using bottleneck width 256 and scale 1.0. e04
- Compress and fuse selected states
Each selected layer uses 16 learned queries and eight attention heads. Query outputs are averaged, normalized, projected to width 4608, concatenated, fused to width 6144 and processed by one residual MLP block. e06
- Decode the action horizon
Projected sinusoidal step embeddings of width 256 are added to the fused state. A normalized output MLP predicts each step’s action without iterative denoising. e06e07
6. Experiments & results
Light-WAM learns robot actions from a video backbone adapted with LoRA and sparse residual adapters. Future-video flow matching supplies training supervision at reduced latent resolution; inference pools current-observation states from several layers and directly regresses an action chunk. The reported efficiency gains accompany competitive LIBERO performance but lower RoboTwin and real-world success than key baselines.
6.1 Read the original evidence
Table 1. Competitive LIBERO averages coexist with a weaker Long-suite score. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the EPT column before the rankings: it distinguishes embodied pretraining from the downstream evaluation, not all forms of pretraining. Light-WAM still uses pretrained video weights. The four suite columns describe different manipulation demands, so inspect the Long column before accepting the average as a complete description. Light-WAM’s final row gives 98.2, 99.6, 97.8 and 93.0 percent, with a reported average of 97.2. The nearest non-EPT WAM row, Fast-WAM, averages 97.0 but reaches 94.8 on Long. The Params column reports rounded model sizes; it must not be confused with the smaller trainable-parameter budget analyzed elsewhere. e04e08e09e11e19
What it supports. Within this table, Light-WAM ranks first among methods without embodied pretraining and third overall. Its small average advantage over Fast-WAM does not extend to the Long suite. These results support the claim of competitive LIBERO performance, with a task-dependent tradeoff rather than uniform superiority.
Where the evidence stops. The table mixes pretraining histories and supplies no confidence intervals or seed variation. A 0.2-percentage-point average difference is a reported ranking, not an established statistically reliable improvement under a fully matched training budget.
Table 3. The complete efficiency recipe matters more than backbone size alone. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read downward as a sequence of configurations measured on four H100 GPUs at effective global batch 64. Switching to the compact backbone while keeping a DiT action head yields 0.43 steps/s, below Fast-WAM’s 0.49. Replacing the head with StateFusion raises throughput to 0.56; enabling cached latents gives 0.86; applying 2× video downsampling reaches 2.08. Loaded parameters and trainable parameters answer different questions, while memory is reported per GPU. Keep the starred-row footnote in view: those two intermediate StateFusion configurations use batch eight per GPU with gradient accumulation two to avoid out-of-memory failures. e13e09
What it supports. The final configuration reports 4.25× Fast-WAM’s normalized step throughput, 43.1 rather than 70.7 GiB per GPU, and 0.44B rather than 6.02B trainable parameters. The progression shows why a smaller backbone by itself is insufficient: the head, latent cache and supervision resolution contribute to the measured outcome.
Where the evidence stops. These are combined engineering changes, not a controlled estimate of one module’s causal contribution. Equal effective batch size does not equal identical microbatch execution, and the table does not include a separate accounting of cache-creation cost.
Table 4. Direct action decoding removes the dominant action-branch cost in this comparison. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with Prediction Scope: some baselines generate video and actions, while Fast-WAM and Light-WAM use action-only inference. Then read across the two action-only WAM rows. Light-WAM’s visual branch takes 56.5 ms versus Fast-WAM’s 36.0 ms, but its action branch takes 2.1 ms versus 356.8 ms. The overall latency column reports 72.03 versus 404.62 ms and peak memory is 4.1 versus 12.7 GiB. The caption defines a per-query measurement with cached language on a single paper-designated RTX 4090 48GB GPU. VAE encoding is included; simulator and I/O time are excluded. The retained footnote identifies the separate source of π0.5 timing. e06e07e10e14
What it supports. The large reported reduction is concentrated in action decoding, even though the visual branch is slower. This supports the intended benefit of a direct StateFusionActionExpert for model querying. It does not establish that the complete robot loop runs at the reciprocal of the table’s overall latency.
Where the evidence stops. Prediction scopes differ, and π0.5 was not timed in the same reported experiment. Displayed VAE-plus-policy component times do not exactly sum to the overall totals; the source does not explain this gap, so the reported totals are preserved.
Figure 4. Real-world demonstrations establish physical deployment, with lower success than π0.5 on every shown task. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the task labels beneath the three scenes: clear a paper ball into a trash bin, stack bowls and place them in a basket, then hand off a water bottle into a basket. T1, T2 and T3 in the chart refer to these scenes in the same order. Read the legend carefully: yellow is π0.5 and pink Ours is Light-WAM. Their success pairs are 80 versus 67, 93 versus 87, and 60 versus 53 percent. Section 4.7 reports 50 training demonstrations for each task and describes the comparison as using the same setting. Those demonstration counts are not evaluation trial counts. e14e17
What it supports. These bars substantiate an actual physical-robot evaluation, extending the evidence beyond simulated manipulation or generated video. They also place a clear boundary on the performance claim: Light-WAM is below π0.5 on all three tasks, despite the favorable efficiency measurements reported for RoboTwin inputs.
Where the evidence stops. Evaluation trial counts, uncertainty and physical-loop latency are not supplied. Appendix D’s rollout and future-prediction images add qualitative context, but generated future frames are separate from executed robot actions and cannot establish a success rate.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO four-suite manipulation Official suites; Light-WAM uses no embodied pretraining and suite-specific checkpoints. | Spatial 98.2; Object 99.6; Goal 97.8; Long 93.0; reported average 97.2. Success rate (%) | Fast-WAM: 97.0 average, 94.8 Long, also without EPT. LingBot-VA: 98.5 average with EPT. First among listed non-EPT methods and third overall. The 0.2-percentage-point advantage over Fast-WAM has no reported uncertainty; pretrained video weights are still used. e08e09e11 |
| RoboTwin 2.0 50-task manipulation One policy; 2,500 clean and 25,000 randomized training demonstrations; clean/randomized evaluations. | Clean 76.4; randomized 76.3; reported average 76.4. Success rate (%) | Fast-WAM: 91.9/91.8/91.9, also without EPT; Motus without EPT: 74.9 average. The average gap to Fast-WAM is 15.5 percentage points. Similar condition averages hide task variation: Hanging Mug scores 25/17, versus Adjust Bottle’s 100/100. e08e12e20 |
| Training efficiency Four NVIDIA H100 GPUs; effective global batch 64; Table 3 variants. | 2.08 steps/s; 43.1 GiB/GPU; 0.44B trainable and 1.99B loaded parameters. Steps/s; peak memory per GPU; trainable parameters | Fast-WAM: 0.49 steps/s, 70.7 GiB/GPU, 6.02B trainable and 6.73B loaded; reported throughput ratio 4.25×. The gain combines backbone/head changes, caching and downsampling. Two intermediate variants use gradient accumulation; this is not a single-factor speedup. e13 |
| RoboTwin inference efficiency Per query with cached language on the paper-designated single NVIDIA RTX 4090 48GB GPU; includes VAE/policy, excludes simulator/I/O. | 72.03 ms; 4.1 GiB; action branch 2.1 ms. Overall latency; peak GPU memory | Fast-WAM: 404.62 ms, 12.7 GiB; action branch 356.8 ms. Model-query latency does not establish end-to-end robot feedback latency. The table’s π0.5 timing comes from another cited study. e14 |
| LIBERO-Spatial design ablations Default: 2× downsampling, layers {8,16,24}, 16 queries; change one listed design. | Default 98.2; full-resolution video supervision 99.0; five adapters 98.0; eight queries 95.4. Success rate (%) | Changes: +0.8, −0.2 and −2.8 percentage points. Query capacity matters here; extra adapters show no clear gain. These runs do not isolate video supervision’s benefit. e15 |
| Real-world dual-arm manipulation IMETA Y1; three tasks, 50 training demonstrations each; authors describe the π0.5 comparison as using the same setting. | Clear paper ball into trash bin: 67; stack bowls then place in basket: 87; handoff water bottle into basket: 53. Success rate (%) in Figure 4 | π0.5: 80, 93 and 60, respectively. Physical execution is demonstrated, but Light-WAM is lower on all three tasks. Evaluation trial counts and uncertainty are not supplied. e17 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 5. The query bottleneck shows a larger success penalty than the tested resolution compromise. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the bold default row as the reference: 2× spatial downsampling for video co-training, adapters at layers 8, 16 and 24, and 16 queries. Each subsequent row changes one listed design. Removing downsampling raises success from 98.2 to 99.0 percent. Adding adapters at layers 4 and 20 gives 98.0 percent. Reducing the learned queries to eight gives 95.4 percent. DS affects video supervision; Section 3.3 keeps the action observation at original latent resolution. Read this table together with the training-efficiency table, because success alone does not reveal the computational cost of keeping full-resolution future supervision. e05e13e15
What it supports. The reported query reduction costs 2.8 percentage points, larger than the 0.8-point gain from full-resolution video supervision in this suite. Extra adapters bring no observed improvement. These local comparisons motivate the default bottleneck capacity and sparse layer selection, while leaving their behavior on harder tasks open.
Where the evidence stops. Only LIBERO-Spatial is tested here, without reported uncertainty or a video-loss removal control. The rows do not establish that future supervision is necessary, nor that 16 queries is optimal across tasks.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors acknowledge weaker performance in harder multi-task settings and no evaluation on dedicated robustness benchmarks such as LIBERO-Plus. e18
Figures 3 and 5 show smoother predicted futures and layer-dependent attention maps. These qualitative examples neither quantify dynamics fidelity nor establish that highlighted regions causally improve control. e16
Tables 1–5 provide no seed variation or confidence intervals. Table 5 lacks an action-only training control; similar aggregate RoboTwin scores do not establish individual-task robustness. e11e12e13e14e15e20
7.2 Questions for discussion
- Would matched-budget training without the video loss preserve success or remove the claimed representation benefit?
- Would the eight-query deficit persist across seeds and harder RoboTwin tasks with other settings fixed?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the pretrained Wan backbone/VAE, robot demonstrations and appendix architecture. Table 6 assigns 351.03M trainable parameters to the action expert and 87.49M to LoRA: 0.44B is trainable, not total, model size. e04e06e08e09e19
Missing settings include LoRA rank/scaling, exact canvas resolution/layout, regression weights, action normalization, LIBERO action horizon, evaluation trials/seeds and software versions. Equation (12)’s lambda versus Algorithm 1’s unweighted sum needs clarification; arbitrary defaults would not constitute exact reproduction. e03e07e08e09e10e17
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Measure the contribution of future-video supervision
Reader-proposed, not executed: train the LIBERO-Spatial default alongside an action-only training control with the video loss removed. Keep demonstrations, pretrained weights, adapted layers, query count, action observation resolution, optimizer and checkpoint rule fixed. Compare both equal-update and equal-training-time budgets, since deleting a branch changes throughput. Use several seeds and the same evaluation initial states; report success, uncertainty, memory and samples processed. A repeatable success advantage for co-training at equal updates would support a representation benefit. If that advantage disappears at equal time, the evidence would favor a narrower accuracy–compute tradeoff. Resolve the paper’s unspecified regression weighting before treating either run as a reproduction. e05e07e09e10e13e15
Check 2: Test whether query capacity and supervision resolution interact
Reader-proposed, not executed: run a 2×2 comparison of eight versus sixteen queries and 1× versus 2× video-latent downsampling, retaining adapters at {8,16,24}. Keep current-observation resolution, demonstrations, initialization protocol and checkpoint selection fixed. Start on LIBERO-Spatial, then repeat on a prespecified RoboTwin task subset that includes low-success tasks such as Hanging Mug. Report success across seeds together with training throughput, peak memory and inference branch timings. If the query penalty persists at both video resolutions, it supports an action-interface bottleneck independent of supervision resolution. If full-resolution supervision removes that penalty, the two design choices interact and the paper’s separate ablations do not capture the full tradeoff. e05e06e09e13e14e15e20
8.3 Reading coverage
Visual audit: Visually inspected the title/version/author page, all method and experiment pages, Appendix A/B configuration text, Algorithms 1–2, parameter breakdown, all per-task results and Appendix D. Figures 1–5 and Tables 1–7 were viewed on original PDF renders. All six final crops were separately inspected; the architecture crop was widened to retain its complete right-hand label. Figure 1’s shared/video/action arrows, query/key/value roles and residual adapter routing were cross-checked against its caption, Equations (4) and (9), Appendix A and Algorithms 1–2; the enlarged view resolves the adapter branch as leaving the transformer block, with no retained contradiction. References on pages 9–11 were read in the complete text but were not visually rendered. Separate supplements and code remain outside this review.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 12, 13, 14, 15. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, author credits, abstract and version stamp (p. 1)
- Sections 1–2: Introduction and Related Work (pp. 1–2)
- Sections 3.1–3.4: complete methodology and Equations (1)–(12) (pp. 3–5)
- Sections 4.1–4.7: setup, benchmarks, efficiency, ablations, qualitative and real-world evaluation (pp. 5–8)
- Section 5: Conclusion and Limitations (p. 8)
- References (pp. 9–11)
- Appendix A: Algorithmic Details (p. 12); Algorithms 1–2 (p. 13)
- Appendix B: Training and Implementation Details (p. 12); Table 6 (p. 14)
- Appendix C: Full RoboTwin 2.0 Results (p. 14)
- Appendix D: Real-World Rollouts (p. 15)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Identity: title and all eight authors match the catalog. The inspected title page identifies arXiv:2606.08242v1 [cs.CV], 6 June 2026. No other revision was supplied or compared; no title or author discrepancy was observed.
- Acquisition note preserved: Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This gap was addressed by inspecting the retained PDF pages and original crops.
- Separate supplemental material availability has not been fully verified.
- No separate supplement was supplied. Linked code was not inspected, and no experiments were reproduced. References were read as supplied text; their cited works were not independently opened.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, author/affiliation block and arXiv margin stamp
Exact observed title and eight authors match the catalog; the artifact identifies arXiv:2606.08242v1 [cs.CV], 6 June 2026, and five institutions.
Go to primary source ↓e02PDF pp. 1–2, Section 1 and contributions; p. 2, Section 2
The paper motivates retaining future prediction as training supervision while reducing WAM training and inference costs.
Go to primary source ↓e03PDF p. 3, Figure 1 and caption; Sections 3.1–3.2, Equations (1)–(3)
A shared adapted video backbone supports training video prediction and current-observation action decoding; inputs include observation, language and proprioception.
Go to primary source ↓e04PDF p. 4, Section 3.2, Equations (4)–(5); p. 12, Appendix A, Backbone adaptation
Frozen Wan2.1-T2V-1.3B receives LoRA across self/cross-attention and feed-forward projections. Layers {8,16,24} apply adapters to block outputs, using bottleneck 256 and scale 1.0.
Go to primary source ↓e05PDF p. 4, Section 3.3, Equations (6)–(7)
The flow objective uses downsampled latent video with its first frame fixed; action prediction uses the original-resolution first latent frame.
Go to primary source ↓e06PDF p. 4, Section 3.4, Equations (8)–(10); p. 12, Appendix A, State-fusion
Layer-specific queries pool adapted states; default 16 queries/eight heads, projected widths 4608 and 6144, one residual MLP, sinusoidal step width 256, and RoboTwin output 24×14.
Go to primary source ↓e07PDF p. 5, Section 3.4, Equations (11)–(12)
Step embeddings condition direct action output; joint loss combines video and action regression with explicit lambda in Equation (12).
Go to primary source ↓e08PDF p. 5, Section 4.1, Benchmarks and data; Implementation details; p. 6, Tables 1–2, EPT column
LIBERO uses official suites; RoboTwin trains one policy on 50 tasks with 2,500 clean plus 25,000 randomized demonstrations. Defaults include 2× video downsampling; Light-WAM has no embodied pretraining.
Go to primary source ↓e09PDF p. 12, Appendix B, Training setup and Checkpoint selection
AdamW, learning rate 1e-4, decay 1e-2, cosine schedule, 1,000 warmup steps, four H100s, global batches 64/128, caching, trainable module list and suite-specific/460K checkpoints are specified.
Go to primary source ↓e10PDF p. 13, Algorithm 1, steps 1–11; Algorithm 2, steps 1–5
RoboTwin uses three cameras, stride 4 over frames 0:32, 128 language tokens plus proprioception, and weighted squared action regression. Algorithm 1 omits lambda in the total; Algorithm 2 encodes current input, predicts a chunk and executes actions without video rollout.
Go to primary source ↓e11PDF p. 6, Table 1, suite/EPT/rank columns; p. 5, Section 4.2
Light-WAM: 98.2/99.6/97.8/93.0, average 97.2; Fast-WAM average 97.0 and Long 94.8; LingBot-VA average 98.5. Light-WAM ranks first without EPT and third overall.
Go to primary source ↓e12PDF p. 6, Table 2, Clean/Randomized/Avg. and EPT columns; Figure 2
Light-WAM: 76.4/76.3/76.4; Fast-WAM: 91.9/91.8/91.9; Motus without EPT averages 74.9. LingBot-VA without EPT has no randomized score.
Go to primary source ↓e13PDF p. 7, Table 3, every variant and footnote; p. 6, Section 4.4
At batch 64 on four H100s, final throughput is 2.08 versus 0.49 steps/s, normalized 4.25×. Memory is 43.1 versus 70.7 GiB/GPU. Intermediate StateFusion variants use batch 8/GPU and accumulation 2.
Go to primary source ↓e14PDF p. 7, Table 4, caption, timing columns and footnote; Section 4.4 continuation
Light-WAM reports VAE 12.7 ms, visual 56.5 ms, action 2.1 ms, policy 58.6 ms, overall 72.03 ms and 4.1 GiB. Fast-WAM reports overall 404.62 ms, action 356.8 ms and 12.7 GiB. Cached-language timing excludes simulator/I/O; π0.5 timing is externally reported.
Go to primary source ↓e15PDF p. 7, Table 5 and Section 4.5
Spatial success: 98.2 default, 99.0 without downsampling, 98.0 at layers {4,8,16,20,24}, 95.4 with eight queries. No video-loss removal control or uncertainty is provided.
Go to primary source ↓e16PDF p. 8, Figure 3 and Section 4.6; p. 15, Figure 5, lower prediction panels
Predicted future frames are compared with references; query attention maps differ across layers 8/16/24. Appendix examples separately label predicted and ground-truth futures at +24.
Go to primary source ↓e17PDF p. 8, Section 4.7 and Figure 4, task labels, legend and bars; p. 15, Figure 5, rollout panels
Three IMETA Y1 tasks use 50 training demonstrations each. Yellow π0.5 bars are 80/93/60; pink Ours bars are 67/87/53. Rollout images are supplied; evaluation trial counts are not stated.
Go to primary source ↓e18PDF p. 8, Section 5, Conclusion and Limitations
Larger models/embodied-pretrained policies do better in complex manipulation; no dedicated generalization/robustness benchmark such as LIBERO-Plus is tested.
Go to primary source ↓e19PDF p. 14, Table 6, component rows and Total
Total parameters: 1986.82M; trainable: 441.03M; frozen: 1545.79M. StateFusionActionExpert has 351.03M and LoRA 87.49M. Backbone and VAE are frozen.
Go to primary source ↓e20PDF p. 14, Appendix C, Table 7, Adjust Bottle, Hanging Mug and Average rows
Light-WAM clean/randomized: 100/100 for Adjust Bottle, 25/17 for Hanging Mug, 76.4/76.3 overall; per-task variation remains substantial.
Go to primary source ↓8.5 Primary sources
Light-WAM: Efficient World Action Models with State-Fusion Action Decoding ↗
PDF · 6,769 extracted words
Source fingerprint
7028480b1f8cee5674cdc9fd7f9c6619b65e5cc8c8cb6d76558567a11a0928c1