WAM-OPD: On-Policy Distillation for World Action Models
1. Paper overview
In one sentence: WAM-OPD trains actions under the student's own video plan using dense teacher labels, recovering capability in a two-task pilot while leaving target mismatch and stale training histories unresolved. e2e3e5e8e10e11e6e14
| At a glance | What to know |
|---|---|
| Research problem | Source description Offline acceleration can leave task-specific capability gaps and expose a student to unfamiliar histories after its own errors. A second mismatch arises inside a video-first policy: deployment actions consume student-generated plans, whereas teacher-conditioned action training would expose them to teacher plans. The authors seek dense supervision for both interfaces without sparse-reward policy-gradient learning. e2e5 |
| Core mechanism | Source description The proposed training graph preserves the student's deployed video-to-action condition while a frozen LingBot-VA teacher supplies coherent video/action targets. e5 |
| A key reported result | HANDOVER MIC: 58.3% Official latched eval_success rate; higher is better. RoboTwin 2.0 demo_clean; selected epoch-3 checkpoint; six held-out scene seeds × two fixed noise banks. Released and adapted policies share instruction, initial snapshot, seed and bank. Released one-video/one-action-step Flash-WAM: 0.0%; reported improvement +58.3 percentage points. Measured microphone-transfer success in simulation. Twelve paired units reuse six scenes, so they are not twelve independent scene samples. No uncertainty interval is reported. e9e10 |
| Reading caution | Reader analysis Figure 1 contains a source-diagram inconsistency: its orange Teacher Video z_T connector reaches the a_T target box, and Teacher Action a_T reaches z_T. The loss labels and Eq. (9) instead pair z_S with z_T and a_S with a_T. This report follows the explicit objective and preserves the unmodified figure with this warning; the connectors cannot verify the intended supervision routing. e6e14 |
Core contributions
- Source description
The proposed training graph preserves the student's deployed video-to-action condition while a frozen LingBot-VA teacher supplies coherent video/action targets. e5
Figure 1. The student execution loop is shown clearly, but the orange teacher-target connectors conflict with Eq. (9). Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the left loop: observations enter history h_S, the student generates video z_S, and action a_S returns to the environment. The stop-gradient arrow marks the detached plan condition. The dashed student box names rank-8 LoRA in shared blocks 0–29. Inspect the orange lines literally: the outer connector from Teacher Video z_T ends at a_T, while the inner connector from Teacher Action a_T ends at z_T. These destinations contradict the loss-box labels and Eq. (9) below the figure. That equation defines video supervision using z_S and z_T, and action supervision using a_S and a_T. Use Sections 4.1–4.3 for the intended training computation; the original crop preserves this diagram inconsistency. e3e5e6e7e11e12e14
What it supports. The student-plan action condition agrees with Eqs. (7)–(8). The method text specifies that both modality losses update shared adapters while released weights remain frozen, with video, action and flow-matching weights 1, 1 and 0.2 on page 7. These training details follow the equations and prose; the reversed orange connectors do not verify them.
Where the evidence stops. The source figure's orange teacher-target routing is inconsistent with its loss labels and Eq. (9). Separately, the text states that teacher actions use teacher plans, leaving a teacher/student conditioning gap. Neither the diagram nor the pilot isolates a component's causal contribution.
2. Motivation
2.1 The problem and the proposed response
Offline acceleration can leave task-specific capability gaps and expose a student to unfamiliar histories after its own errors. A second mismatch arises inside a video-first policy: deployment actions consume student-generated plans, whereas teacher-conditioned action training would expose them to teacher plans. The authors seek dense supervision for both interfaces without sparse-reward policy-gradient learning. e2e5
2.2 What this reading follows
An accelerated robot policy can fail because its own decisions carry it into unfamiliar states, and because its action generator sees imperfect video plans. WAM-OPD addresses both interfaces: Flash-WAM collects histories, a frozen LingBot-VA teacher labels them, and the student learns while conditioning actions on its own predicted future. Read the architecture figure alongside Eq. (9): its orange teacher-target connectors are reversed relative to the objective. The success table shows what the small clean-simulation pilot establishes, while the planned-evidence table identifies the comparisons still missing. The result is task-specific recovery under a shared-backbone update, with no completed ablation demonstrating which loss or collection choice is necessary. e2e3e5e8e10e11e6e14
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | IDM |
| Quadrant | Q4 · Dual-system × IDM |
3.1 Evidence-based assessment
Conflicts with the recorded classification
The recorded IDM paradigm is supported: actions condition on predicted future video and history. The Dual-system architecture and Q4 assignment conflict with explicit evidence of one shared 30-block Transformer with modality-specific surrounding modules. This supports One Model × IDM as a reader assessment, independently of joint training. Separate teacher/student instances during distillation do not constitute two deployed planning/control systems. Post-training is appropriate; no reward-based RL is used. e2e3e5e7
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Follow the history before following the teacher
A video-first policy first predicts a future latent from its observation–action history and language instruction, then conditions its action chunk on that latent. After execution, new observations change the history available to the next prediction. This creates the environment-level distribution problem motivating WAM-OPD: errors made by a fast student can move later decisions away from offline training contexts. The released student therefore collects the trajectories, and the stronger frozen teacher labels exactly those histories. This pilot uses eight trajectories and 160 labeled contexts per task, followed by a three-epoch update. Its on-policy qualification matters: the package belongs to the released collector. Once the student changes, those same histories no longer represent fresh interaction by the current policy. Iterative recollection is a proposed extension. e2e3e8e9
5.2 Separate the action condition from the action target
Follow Eqs. (7)–(9) for this computation; Figure 1 reverses the orange teacher-target connections relative to the written objective. At training time the student computes video plan z_S and feeds a detached copy into its action branch. The teacher separately produces z_T and a_T, with its action conditioned on its own plan. Deployment consistency therefore applies to the student's computation, while a residual teacher/student target mismatch remains. The video endpoint loss is intended to align plans and reduce that gap. Detachment prevents action-loss gradients through the generated-plan tensor, but leaves the shared Transformer trainable during action prediction. Video and action losses can still modify the same rank-8 adapters across all 30 blocks. The action flow-matching term supplies a boundary velocity target. The source limits this objective to endpoint alignment plus a boundary auxiliary, without claiming full generative-path matching. e3e4e5e6e7e14
5.3 Distinguish a paired gain from a mechanism test
The measured comparison asks whether the complete adapted checkpoint beats its released initialization under matched evaluation conditions. Sharing instruction, snapshot, scene seed and noise bank controls important variation between the two policies. Separating the tasks also exposes the larger handover gain and smaller cabinet gain. However, repeating each scene with two banks does not create twelve independent scenes. Reader analysis: the comparison establishes a small combined-recipe capability result, while it cannot attribute the gain specifically to video supervision, shared adaptation or fresh occupancy. The source's planned matrix makes those missing controls explicit. Official latched success is also narrower than a demonstration of stable contact: restored snapshots omit the simulator contact manifold. A useful reproduction should preserve the reported task endpoint and evaluate any contact-mechanics claim separately. e10e11e12e13
5.4 Training and inference
During training
Each task uses eight student trajectories, 160 labeled macro-step contexts and four disjoint calibration trajectories. AdamW uses learning rate 2 × 10⁻⁵ and batch size 4 for three epochs, totaling 120 optimizer steps. e9
A predeclared ordering of task success, semantic progress and calibration loss screens checkpoints on a disjoint split. Epoch 3 is selected for both tasks; screening data are excluded from held-out evaluation. e9
The fixed package is on-policy for its released collector only. Reusing it as parameters change makes occupancy stale; alternating fresh collection, labeling and optimization is proposed but not evaluated. e8
During inference
The shared model generates and caches its video plan, then generates an action chunk from that cache and the history. The student executes actions and incorporates subsequent observations. The pilot starts from one-video/one-action-step Flash-WAM; the teacher supplies training labels rather than deployment actions. e3e5e9
5.5 Implementation flow
- Collect student histories
Released Flash-WAM alone acts in RoboTwin. Its closed-loop trajectories determine the contexts labeled for this post-training package; the teacher does not control collection. e5e8
- Generate teacher targets
Frozen, slower LingBot-VA produces a video plan z_T and an action a_T on the same history. The teacher action uses z_T, so its target condition still differs from the student's plan. e5
- Follow the deployed student computation
Generate z_S first, then predict a_S conditioned on h_S and stop-gradient z_S. Detachment blocks action-loss gradients through the plan tensor; the action forward pass still updates shared adapters. e5
- Update shared representations
Separate modality input, time and output modules surround one 30-block Transformer. Rank-8 JointLoRA spans blocks 0–29; released weights remain frozen. Joint adaptation permits, but does not establish, beneficial gradient interaction. e3e7
6. Experiments & results
WAM-OPD adapts an accelerated video-first robot policy using frozen-teacher labels on histories collected by the student. Actions are trained under the student's own predicted video, and shared adapters receive video, action and flow-matching losses. A small two-task simulation pilot improves success, while component causality and continuing on-policy adaptation remain untested.
The supplied v1 paper contains one architecture figure, one measured quantitative table and one planned-evidence table, so this edition has three original visuals. Table 2 explicitly marks every performance entry TBD, and Sections 5.4 and 6 leave controlled ablations, broader evaluation and retention for future work. There is no completed ablation or diagnostic-result graphic to include as a fourth visual. The planned table is shown only to explain this evidence gap. Figure 1 also reverses its orange teacher-target connections relative to its loss labels and Eq. (9); the unmodified crop documents that inconsistency, while the written method supports the intended pairings. e5e10e11e12e6e14
6.1 Read the original evidence
Table 1. The same adaptation recipe improves two clean RoboTwin tasks by different amounts. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each row as a separate task, moving from the released Flash-WAM success rate to the selected WAM-OPD checkpoint. Improvement is a percentage-point change, not a relative percentage increase. The surrounding protocol uses six held-out scene seeds under two fixed noise banks for each task. Within a comparison, both policies share instruction, initial simulator snapshot, scene seed and bank. Page 7 specifies disjoint checkpoint screening, which selects epoch 3 for both tasks. HANDOVER MIC concerns transfer between robot hands; PUT OBJECT CABINET requires placing and releasing the object in the drawer, without requiring the drawer to close. e9e10e13
What it supports. Reported success rises from 0.0% to 58.3% on microphone handover and from 16.7% to 33.3% on cabinet placement. The table reports gains of 58.3 and 16.7 percentage points. These are executed simulated-task outcomes from the official latched success predicate, and their differing magnitudes limit claims of uniform benefit.
Where the evidence stops. Twelve paired units per task reuse six scenes, so they are not independent scene samples. No uncertainty interval is supplied. The rounded cells and reported changes are preserved exactly. Success-latch results do not verify sustained contact mechanics.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| HANDOVER MIC RoboTwin 2.0 demo_clean; selected epoch-3 checkpoint; six held-out scene seeds × two fixed noise banks. Released and adapted policies share instruction, initial snapshot, seed and bank. | 58.3% Official latched eval_success rate; higher is better | Released one-video/one-action-step Flash-WAM: 0.0%; reported improvement +58.3 percentage points. Measured microphone-transfer success in simulation. Twelve paired units reuse six scenes, so they are not twelve independent scene samples. No uncertainty interval is reported. e9e10 |
| PUT OBJECT CABINET Same clean RoboTwin protocol: six held-out scenes × two noise banks, with exact pairing against Released and disjoint checkpoint screening. | 33.3% Official latched eval_success rate; higher is better | Released Flash-WAM: 16.7%; reported improvement +16.7 percentage points. Measured object placement and release in the designated drawer; closing it is not required. The smaller gain supports task dependence. Preserve the source's rounded percentages and reported change; do not pool both tasks. e9e10 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2 (planned evidence; no measured ablations). The source explicitly reserves mechanism controls and broader evaluations for future work. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the short warning above the table: every TBD entry is a placeholder. The first two rows would extend coverage across tasks and randomized settings. The action-only versus joint-update row would test whether supervising video adds value beyond action learning. The fixed-package versus refreshed-OPD row would test whether collecting new histories matters after the policy changes. Cross-task retention would assess what adaptation does to capabilities outside its target task. Read these rows as a map of missing evidence. The Released and WAM-OPD columns name intended comparisons, but neither column contains a measured value in this table. e7e8e11e12
What it supports. The v1 source contains no completed controlled ablation of video supervision or refreshed occupancy. Its two-task improvement therefore supports the combined recipe's capability proof without establishing that each proposed ingredient is necessary. The planned rows provide concrete follow-up questions, not additional benchmark wins.
Where the evidence stops. Placement in the ablation section identifies the subject of the missing controls; it does not turn this table into ablation evidence. JointLoRA gradient interaction and matched SFT, RL or pathwise-flow comparisons also remain unresolved in Section 6.
7. Analysis & limitations
7.1 What the evidence leaves open
Figure 1 contains a source-diagram inconsistency: its orange Teacher Video z_T connector reaches the a_T target box, and Teacher Action a_T reaches z_T. The loss labels and Eq. (9) instead pair z_S with z_T and a_S with a_T. This report follows the explicit objective and preserves the unmodified figure with this warning; the connectors cannot verify the intended supervision routing. e6e14
Only two clean simulation tasks and a small held-out set are measured. Broader tasks, domain randomization, action-only comparisons, refreshed occupancy and cross-task retention remain TBD in Table 2; no real-robot transfer is demonstrated. e11e12
Teacher-plan/student-plan target mismatch remains after video alignment. Shared video/action gradient interaction is unmeasured, and compute-matched SFT, RL, action-only and full pathwise-flow comparisons are absent. Thus success gains cannot identify the necessary mechanism. e5e7e12
Snapshot replay preserves pose and the success latch but not the simulator contact manifold. Auxiliary restored-snapshot contact diagnostics cannot establish sustained contact; official task success is the supported endpoint. e13
7.2 Questions for discussion
- Would querying the teacher under the student's exact plan improve action supervision beyond video alignment?
- How much of the two-task gain survives action-only controls and repeated collection from the updated policy?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the referenced LingBot-VA and released one-step Flash-WAM checkpoints, RoboTwin 2.0 clean tasks, teacher-labeled contexts, calibration split, simulator snapshots and fixed noise banks. The paper specifies update counts and learning rate but omits exact seed lists, checkpoint fingerprints, pseudo-Huber δ, detailed adapter target projections, hardware, runtime and teacher-labeling cost. e3e6e7e9e10
Reader-proposed checks: hold histories, labels, optimizer budget and paired evaluation fixed while comparing joint versus action-only supervision; separately compare a reused package with refreshed collection under matched teacher-query and update budgets. These would test mechanism and occupancy claims; neither comparison was run here. e8e11e12
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does video supervision add value under the same student-plan condition?
Reader-proposed experiment, not run: initialize both arms from the same released checkpoint and use identical histories, teacher targets, batch order, adapter scope and 120-step budget. Keep student-plan action conditioning and the action flow-matching weight fixed. Compare the reported joint objective against an arm with only the video endpoint weight set to zero. Screen checkpoints on the same disjoint data, then use paired held-out scenes and banks. Measure official success and plan-target discrepancy separately. If removing video supervision preserves success across repeated runs despite worse plan alignment, the claim that video alignment is necessary for these gains would be weakened. e5e6e7e9e11
Check 2: Does recollecting from the updated policy improve adaptation?
Reader-proposed experiment, not run: compare the published fixed package with a version that alternates bounded updates and fresh student collection. Match total labeled contexts, teacher-query budget, optimizer steps and initial checkpoint; use the same context-selection rule. Keep calibration and held-out scene seeds disjoint from collection, and report simulator interaction and wall time separately. Evaluate task success on paired scene/bank units and teacher-target discrepancy on newly visited histories. Refreshed collection would support the occupancy argument if it improves held-out success and reduces discrepancy on current-policy histories across repeated runs; equal performance would limit the case for refreshing at this budget. e8e9e10e11
8.3 Reading coverage
Visual audit: All 11 supplied PDF pages were visually inspected, including the title/version/author block, shared architecture and equations, training protocol, measured table, planned-evidence table and contact limitation. Figure 1 and both tables were cropped from the original PDF and every final PNG was visually checked. On page 6, Figure 1's outer orange Teacher Video connector reaches a_T and its inner Teacher Action connector reaches z_T, contrary to the loss labels and Eq. (9). The figure is preserved exactly and this inconsistency is disclosed in its explanation and base evidence e14. Pages 5–7 supply the intended method details; pages 7–9 supply the result protocols and limitations. Table 2's short warning is retained; it contains no experimental results. No appendix is present. Separate supplements, referenced papers and code remain outside this inspection.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, author block and Abstract (p. 1)
- 1. Introduction (pp. 1–3)
- 2. Related Work (pp. 3–4)
- 3. Preliminaries (pp. 4–5)
- 4. Method, including Sections 4.1–4.4 (pp. 6–7)
- 5. Experiments, including Sections 5.1–5.4 (pp. 7–8)
- 6. Discussion and Limitations (pp. 8–9)
- 7. Conclusion (p. 9)
- References [1]–[27] (pp. 9–11)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Identity/version: the inspected title page identifies arXiv:2608.22364v1 [cs.AI], 23 Aug 2026. Title and all four authors match the catalog after name-order normalization. No different edition was supplied or compared.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This acquisition limitation was addressed by visually inspecting all 11 PDF pages and all three final crops.
- Separate supplemental material availability has not been fully verified.
- No appendix appears in the supplied 11-page PDF. Referenced papers and code were not independently inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e1PDF p. 1, title/author block and arXiv margin stamp
The title is WAM-OPD: On-Policy Distillation for World Action Models; authors are Liuhaichen Yang, Zhuang Jiang, Chenchao Sheng and Zezhi Tang. The stamp identifies arXiv:2608.22364v1, 23 Aug 2026. Affiliations are UCL Computer Science and Mechanical Engineering.
Go to primary source ↓e2PDF pp. 2–3, Section 1, distribution-shift discussion, Eq. (1) and contributions
The paper distinguishes student-induced environment histories from the video-plan condition seen by the action branch; on-policy refers to supervision support and does not require policy-gradient RL.
Go to primary source ↓e3PDF pp. 4–5, Section 3, World Action Model policies, Eq. (2) and following paragraph
History includes observations, past actions and language. Future video conditions inverse-dynamics action generation. Teacher and student each use modality-specific modules around one shared 30-block Transformer; deployment caches video before actions.
Go to primary source ↓e4PDF p. 5, Section 3, Flow matching, Eqs. (3)–(5)
The straight path is (1−σ)x₀+σε with velocity ε−x₀. Sampling runs from σ=1 to 0; conditioning includes language and robot history.
Go to primary source ↓e5PDF p. 6, Figure 1 caption and Section 4.1, Eqs. (7)–(8) and following paragraphs
The caption and method text state that only the student acts. Teacher action labels use its own video plan. Student actions consume stop-gradient student video; detachment does not freeze the shared action computation. Figure 1's inconsistent orange connectors are separately recorded in e14.
Go to primary source ↓e6PDF pp. 6–7, Section 4.2, pseudo-Huber definition and Eq. (9)
Video and action endpoint penalties combine with action flow matching at σ=1, weighted 1, 1 and 0.2. Exact reverse-KL and full trajectory matching are explicitly disclaimed; no numerical δ is given.
Go to primary source ↓e7PDF p. 7, Section 4.3
Rank-8 JointLoRA spans shared blocks 0–29 and released weights are frozen. Per-modality gradient interaction is left for controlled ablation.
Go to primary source ↓e8PDF p. 7, Section 4.4
One fixed released-student trajectory package is labeled and reused for three epochs. It becomes stale as the student changes; iterative recollection is future work.
Go to primary source ↓e9PDF p. 7, Sections 5.1–5.2
Two clean RoboTwin tasks use eight trajectories, 160 contexts, four disjoint calibration trajectories, batch size 4, AdamW at 2 × 10⁻⁵, three epochs and 120 steps. Disjoint screening selects epoch 3. Cabinet success does not require drawer closing.
Go to primary source ↓e10PDF p. 8, Section 5.3 and Table 1, both task rows and all performance columns
Six scene seeds and two noise banks form twelve exact-paired units per task. Released/adapted success is 0.0%/58.3% for HANDOVER MIC and 16.7%/33.3% for PUT OBJECT CABINET; reported gains are 58.3/16.7 pp. Scene samples are dependent across banks.
Go to primary source ↓e11PDF p. 8, Section 5.4 and Table 2, all five planned rows
Broader tasks, domain randomization, action-only versus joint update, fixed versus refreshed OPD and cross-task retention are planned. Every performance entry is TBD, explicitly not an experimental result.
Go to primary source ↓e12PDF pp. 8–9, Section 6, limitations through matched-comparison paragraph
Evidence is limited to two clean simulated tasks. Plan mismatch and shared-gradient effects remain unresolved; matched SFT, RL, action-only and full pathwise-flow comparisons are absent.
Go to primary source ↓e13PDF p. 9, Section 6, snapshot replay/contact paragraph
Official latched eval_success is the task endpoint. Restored snapshots omit the contact manifold, making auxiliary contact diagnostics unreliable for claims of stable contact mechanics.
Go to primary source ↓e14PDF p. 6, Figure 1, orange connectors from the two Frozen Teacher blocks to the right-hand target boxes; compare loss labels and Eq. (9)
The outer orange connector runs from Teacher Video z_T to the a_T target box, while the inner connector runs from Teacher Action a_T to the z_T target box. These destinations conflict with the figure's video/video and action/action loss labels and with Eq. (9), which penalizes z_S−z_T and a_S−a_T. The original graphic is retained without correcting its connectors.
Go to primary source ↓8.5 Primary sources
WAM-OPD: On-Policy Distillation for World Action Models ↗
PDF · 4,688 extracted words
Source fingerprint
7c44edc0fb37c33d7dd1c5308b52781ba48e11f7efce7036d8fc9bfca1bbf287