Enhancing Policy Learning with World-Action Model
1. Paper overview
In one sentence: Inverse-action supervision can improve the frozen features used by a diffusion policy, but this paper's inconsistent result summaries and absent ablations leave the size and cause of that benefit uncertain. e03e04e06e11e12e13e16
| At a glance | What to know |
|---|---|
| Research problem | Author claim A representation that reconstructs camera images may omit distinctions needed to choose actions. The authors propose supervising the action responsible for each observed transition so that the encoder retains controllable structure useful to a downstream policy. Their causal explanation is a hypothesis supported indirectly by performance comparisons, rather than a measured property of the latent space. e02e04e11 |
| Core mechanism | |
| A key reported result | Behavioral cloning on eight CALVIN manipulation tasks: Table III average: 61.7%; close drawer 89.7±3.1%; move slider right 82.8±3.9%. Task success rate (%). 50 expert demonstrations per task; 29 held-out initial configurations per task; four-step chunks, 72-step maximum and CALVIN task-checker success. DiWA average 45.8%; close drawer 58.6±4.2%; move slider right 51.7±4.5%. The displayed average gain is 15.9 percentage points, with seven of eight rows improved. Turn on LED decreases from 44.8±3.9% to 41.4±4.1%. Abstract averages differ; uncertainty aggregation and seed counts remain unspecified. e09e11e16 |
| Reading caution | Reader analysis Abstract/Introduction BC means are 71.2% versus 59.4%, while Table III, results and conclusion use 61.7% versus 45.8%. PPO prose claims all-task improvement and open drawer versus 70.0%; Table IV shows one regression and a 74.44% baseline. These are internal v1 conflicts, not verified revision differences. e16 |
Core contributions
Figure 2. Action supervision enters at adjacent encoder embeddings and is intended to affect the prior used for imagination. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the bottom images, labeled x in this figure and o in the equations. Each encoder supplies the corresponding posterior z. The arrows between adjacent encoders meet at an estimated action, matching Equation (6)'s concatenated embeddings; they are not actions chosen by the diffusion policy. Move upward to the recurrent h states and observation decoders, then left to the prior states marked with hats. The dashed “min KL” links represent posterior–prior matching. The downward arrows identify prior-based imagination for policy fine-tuning. The text specifies that the policy actually consumes the combined feature f=[h;z], so the diagram's prior-only arrows simplify its input. e03e04e05e06e15
What it supports. The important architectural separation is between supervised transition explanation and policy action generation. The inverse head sees both ends of an observed transition during training. The diffusion policy later uses latent features from the frozen model. The authors' proposed benefit is that action supervision survives the posterior-to-prior transfer and improves imagined policy learning.
Where the evidence stops. The diagram and Equations (1)–(7) support the stated information paths, but do not measure how much action information reaches the prior. No isolated test of the proposed KL-mediated transfer is reported.
2. Motivation
2.1 The problem and the proposed response
A representation that reconstructs camera images may omit distinctions needed to choose actions. The authors propose supervising the action responsible for each observed transition so that the encoder retains controllable structure useful to a downstream policy. Their causal explanation is a hypothesis supported indirectly by performance comparisons, rather than a measured property of the latent space. e02e04e11
2.2 What this reading follows
WAM asks whether a world model becomes more useful for control when it must explain which action connected two observations. It adds a small inverse-dynamics head to a DreamerV2-style encoder, then freezes the world model and trains a diffusion policy through imitation and imagined PPO rollouts. The most directly traceable aggregate is Table III's behavioral-cloning result: 61.7% versus 45.8% average success across eight CALVIN tasks. Read the architecture as a training pathway, then compare visual prediction with executed benchmark actions. The original tables and curves also expose disagreements with the abstract and PPO prose; those disagreements are part of the evidence. e03e04e06e11e12e13e16
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is WAMs. View the current classification.
3.1 Evidence-based assessment
Insufficient evidence to decide
The snapshot is entirely unassigned. Architecture evidence supports an RSSM world model with an auxiliary inverse-dynamics head plus a separately trained diffusion policy. Joint observation/action losses do not establish a single model that jointly generates future observations and executable actions. Inverse dynamics describes the auxiliary mechanism; a One Model quadrant cannot be inferred from the WAM name. e03e04e06e09
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Why predict actions before the recurrent state?
Follow a single demonstrated transition from observation o_t to o_{t+1}. The encoder produces two embeddings, and the inverse head must infer the recorded a_t from their concatenation. This makes action-relevant distinctions part of the encoder's training target, alongside image reconstruction. The placement matters: the authors argue that consecutive RSSM features would expose the action through recurrent state updates and make prediction too easy. In Figure 2, the inverse-action arrows therefore originate at neighboring encoders, while recurrent transitions separately receive action inputs. A reader interpretation is that this placement tries to remove a direct copying shortcut. It does not prove that the representation learns causal structure, filters every distractor, or retains all information needed for control; those are stronger claims than the architectural choice alone establishes. e02e03e04e05e15
5.2 Trace training supervision into observation-free imagination
The posterior can use an observed embedding, but the prior used for imagination cannot. WAM's proposed cascade bridges that gap: inverse-action loss shapes the encoder, the encoder conditions the posterior, and KL matching links posterior and prior. After world-model training, the entire world model is frozen. Behavioral cloning trains a DiffusionMLP on expert latent features, and optional PPO then generates trajectories from the prior and scores them with task-reward classifiers. The classifiers are retrained because WAM changes their feature inputs. This explains why the auxiliary inverse head is not the action policy: its training requires a future observed embedding, whereas the diffusion policy denoises actions from available latent features. The inferred benefit of KL-mediated transfer remains a hypothesis because the paper does not ablate that path separately. e03e04e05e06e09e10e15
5.3 Separate image prediction, policy completion and sample counts
Start with Table I: ground-truth actions drive predicted images, so success there means better visual prediction under supplied controls. Move to Table III: a learned policy acts on held-out CALVIN configurations and the task checker decides completion. Table IV adds PPO inside the learned model, while its DPPO columns count environment steps needed to match DiWA. Finally, the 8.7× claim compares world-model gradient steps, not any of those success metrics or measured runtime. This separation prevents unlike quantities from becoming one efficiency claim. It also exposes source conflicts: the BC abstract differs from its table, PPO aggregates disagree with displayed task means, and one PPO task regresses. The sound reading retains the favorable table rows while withholding an unqualified headline about the exact PPO average gain. e07e08e11e12e16
5.4 Training and inference
During training
World-model training uses CALVIN D play data: about six hours, described as roughly 500K transitions and about 512K frames. Settings are sequence length 50, batch size 500, AdamW learning rate 3×10^-4, weight decay 0.05, KL balance 0.8, and loss weights 3, 1 and 1000 for KL, reconstruction and action prediction. WAM uses 230K gradient steps; the stated baseline uses 2M. e07
BC uses 20 denoising steps, action horizon four, 5,000 epochs, batch 256, learning rate 10^-4 decaying to 10^-5, weight decay 10^-6 and EMA 0.995. PPO uses 50 parallel imagined rollouts, batch 7,500, ten update epochs, actor/critic rates 10^-5/10^-3, discount 0.999, GAE 0.95 and BC regularization 0.025; it runs 800 iterations with evaluation every 25. e09e10
Reward training featurizes play data, pairs features and raw states from expert episodes using seed 42, replays expert actions in CALVIN for labels, adds imagined rollouts, balances classes and trains contrastive classifiers. Reported precision ≥0.97 and recall 1.00 are explicitly training-data results. e10
During inference
BC evaluation encodes the current observation with the frozen model and samples a four-step DDPM action chunk. Episodes allow up to 18 decisions, or 72 steps, and use CALVIN's task checker. Imagined PPO rollouts instead use the prior because new observations are unavailable, with ten denoising steps. These are distinct feedback regimes. e06e11
Inverse-action prediction is a training auxiliary, not the deployed action-selection rule. The future embedding required by its head is supplied by an observed transition during training; the diffusion policy produces actions from current latent features. e04e06e11
5.5 Implementation flow
- Encode observations and maintain dynamics
A dual-stream CNN fuses the two cameras with proprioception into a 1,554-dimensional embedding e_t. The RSSM combines recurrent state h_t with a 32×32 categorical stochastic variable z_t; f_t=[h_t;z_t] has 2,048 dimensions and feeds decoding and policy learning. e03e07
- Regularize the encoder with inverse dynamics
The MLP predicts a_t from e_t and e_{t+1}. The authors avoid consecutive RSSM features because recurrent dynamics already receive actions, creating an action-copying shortcut. They argue that encoder supervision shapes the posterior and transfers through KL matching to the imagination prior; no isolated gradient-path experiment verifies that transfer. e04e05e15
- Freeze the world model and learn behavior
Extract latent features from 50 expert episodes per task, fit the diffusion policy, then optionally optimize it through clipped PPO using prior-generated latent trajectories and learned binary task rewards. Reward classifiers are retrained for WAM's changed feature space. e06e09e10
6. Experiments & results
WAM adds inverse action prediction between consecutive encoder embeddings to a DreamerV2 world model, then trains a separate diffusion policy on its frozen latent features. Table III supports 61.7% versus 45.8% average behavioral-cloning success on eight CALVIN tasks. The mechanism is plausible, but inconsistent headline numbers, baseline names and PPO summaries limit stronger conclusions.
The supplied paper contains no dedicated ablation table or figure for the inverse head, loss weights or posterior-to-prior transfer. Figure 4 is included in the ablation section only as an original learning-curve diagnostic, not as evidence of an isolated mechanism test. No physical deployment or held-out-domain experiment is supplied. All four tables and four figures were inspected; the six crops select the architecture, qualitative prediction, three quantitative tables and training curves. e13e15
6.1 Read the original evidence
Figure 3. The baseline static view includes an extra colored blob; WAM more closely resembles the shown scene but retains prediction errors. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read columns as selected times from t=1 to t=49. Within each camera block, compare ground truth, baseline and WAM at the same time. At t=40 and t=49 in the static view, the baseline contains both a red object and a separate blue blob that is absent from the corresponding ground-truth and WAM images. The gripper view reveals remaining blur and shape errors in both models. Section IV-A describes the prediction evaluation as open-loop imagination using the first observation and ground-truth actions. These rows compare visual predictions under supplied controls; they do not show actions selected by a policy or measured task completions. e07e08e14
What it supports. The shown WAM static views more closely match ground-truth scene appearance, including the absence of the baseline's extra colored blob. Both models still depart from ground truth, with early gripper-view blur and altered object shapes. These selected examples illustrate prediction errors; aggregate comparison rests on Table I's separately reported validation metrics.
Where the evidence stops. The extra blob does not establish that the red object was recolored. The baseline row says DreamingV2, whereas the caption and Table I say DreamerV2. These selected stills establish neither continuous-video quality nor robot-execution success.
Table I. All four reported prediction metrics favor WAM under the stated open-loop validation protocol. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the direction arrow before comparing each pair: higher PSNR and SSIM are preferred, while lower LPIPS and FVD are preferred. The columns are WAM and the baseline labeled DreamerV2. The three image-metric rows contain plus/minus quantities, while FVD supplies only one number per model. Section IV-A gives the supporting protocol: 100 random CALVIN validation sequences, each imagined for 50 steps from the first observed frame under ground-truth actions. Keep that conditioning in mind when moving from this table to policy results. A model can generate better images for supplied actions without necessarily selecting better actions from its own observations. e07e08e14e15
What it supports. WAM reports PSNR 22.10 versus 21.66, SSIM 0.814 versus 0.807, LPIPS 0.144 versus 0.149, and FVD 10.82 versus 12.13. These directional improvements support better prediction under this protocol. They do not isolate the inverse head's contribution or quantify downstream policy benefit on their own.
Where the evidence stops. The paper does not define the displayed uncertainty aggregation or provide FVD uncertainty. Training uses unequal world-model step counts, and the DreamerV2/DreamingV2 naming conflict prevents treating baseline identity as fully resolved.
Table III. The BC table supports a 15.9-percentage-point average improvement, with one task-level regression. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the bottom average row, then inspect which tasks produce the difference. Close drawer and move slider right each improve by 31.1 percentage points. Next check the exception: turn_on_led favors DiWA, so this table cannot support an all-task improvement claim. The paper describes identical DiffusionMLP policy architecture and BC settings, with each model supplying its own frozen features. Evaluation uses 29 held-out initial configurations per task, four-step action chunks, a maximum of 72 steps and CALVIN's built-in task checker. These are benchmark completion percentages; they measure a policy acting in CALVIN rather than visual similarity inside the world model. e09e11e16
What it supports. The supported average is 61.7% for WAM versus 45.8% for DiWA. Close drawer reaches 89.7±3.1% versus 58.6±4.2%, but turn_on_led decreases to 41.4±4.1% from 44.8±3.9%. The result supports useful representation changes for several tasks while exposing task dependence.
Where the evidence stops. The abstract instead gives 71.2% versus 59.4%. No reconciliation is supplied. The paper attributes the LED exception to evaluation variance, but does not report enough seed or uncertainty details to verify that explanation.
Table IV. Read task rows and measurement units carefully: the policy columns and DPPO columns answer different questions. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The first numerical column is the base diffusion policy. The next two columns give DiWA and WAM success after fine-tuning; the two DPPO columns instead report environment steps required to match DiWA. The bottom interaction row must stay with these headers. Although the WAM header says “Online Fine-Tuning,” Section IV-B.3 says optimization occurs entirely in the frozen world model. Inspect the lightbulb-off row before accepting the prose's all-task claim: 75.9 is below 77.01. Finally, compare the actual DiWA open-drawer entry, 74.44, with the prose's 70.0. The table and prose cannot both be used as one consistent numerical record. e07e10e12e16
What it supports. WAM reaches 100.0±0.0% on lightbulb-on and LED-off and exceeds DiWA on seven displayed tasks. Reader arithmetic on the eight means yields approximately 92.9% for WAM and 83.7% for DiWA. Those computed summaries conflict with the source prose's 92.8% versus 79.8%; no corrected author aggregate is available.
Where the evidence stops. Zero listed physical interactions concerns fine-tuning, not absence of offline demonstrations or simulator replay for reward labels. DPPO interaction totals are a different measure from the 230K-versus-2M world-model training-step comparison.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| CALVIN D open-loop visual prediction 100 random validation sequences; 50-step imagination from the first ground-truth observation with ground-truth actions. | WAM: 22.10±2.22; 0.814±0.061; 0.144±0.072; 10.82. PSNR ↑; SSIM ↑; LPIPS ↓; FVD ↓ | Table I's DreamerV2 column: 21.66±2.20; 0.807±0.067; 0.149±0.073; 12.13. All four displayed metrics favor WAM. This tests action-conditioned prediction, not autonomous control. The meaning of ± and FVD uncertainty are not specified; baseline naming elsewhere conflicts. e07e08e14 |
| Behavioral cloning on eight CALVIN manipulation tasks 50 expert demonstrations per task; 29 held-out initial configurations per task; four-step chunks, 72-step maximum and CALVIN task-checker success. | Table III average: 61.7%; close drawer 89.7±3.1%; move slider right 82.8±3.9%. Task success rate (%) | DiWA average 45.8%; close drawer 58.6±4.2%; move slider right 51.7±4.5%. The displayed average gain is 15.9 percentage points, with seven of eight rows improved. Turn on LED decreases from 44.8±3.9% to 41.4±4.1%. Abstract averages differ; uncertainty aggregation and seed counts remain unspecified. e09e11e16 |
| CALVIN policy success after model-based PPO 800 fine-tuning iterations inside the frozen world model; Table IV's task-specific policy-success columns, separate from its DPPO interaction columns. | WAM: open drawer 96.7±2.4; close drawer 96.6±1.8; slider left 87.5±3.7; slider right 89.7±3.2; lightbulb on 100.0±0.0; lightbulb off 75.9±4.3; LED on 96.6±2.1; LED off 100.0±0.0. Task success rate (%) | DiWA in the same row order: 74.44±1.92; 91.95±1.99; 83.33±1.80; 82.76±3.45; 91.92±1.75; 77.01±1.99; 86.21±3.45; 82.33±6.53. Seven rows favor WAM; lightbulb off does not. Reader arithmetic on the displayed means gives about 92.9% versus 83.7%, conflicting with the prose's 92.8% versus 79.8%. The exact aggregate remains unresolved. e10e12e16 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 4. Learning curves provide a diagnostic view, but no controlled ablation of inverse-action supervision. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the shared legend to identify blue DiWA and orange WAM, then read each task separately. The horizontal axis is BC epoch, not PPO iteration or environment interaction count. Close drawer and the two slider tasks show clear separation over much of the displayed training, while open drawer and turn-on-LED largely overlap. The lightbulb tasks have changes in direction rather than uniformly rising curves. The plot ends at 4,000 epochs although the BC settings state 5,000; the source does not specify how these checkpoints map to Table III. Treat the lightly filled areas as visual fills, since no uncertainty-band definition is supplied. e09e11e13e15
What it supports. The diagnostic supports the narrower observation that learning trajectories differ by task. It does not support uniform superiority at every point, and it does not explain why several curve levels differ from the BC table. This is useful evidence to audit when reconstructing evaluation logs and checkpoint selection.
Where the evidence stops. No action-loss toggle, shuffled-label control or matched-budget ablation appears here. The caption's broad claim of consistently higher success exceeds what the overlapping panels visibly establish, and curve–table correspondence remains unresolved.
7. Analysis & limitations
7.1 What the evidence leaves open
Abstract/Introduction BC means are 71.2% versus 59.4%, while Table III, results and conclusion use 61.7% versus 45.8%. PPO prose claims all-task improvement and open drawer versus 70.0%; Table IV shows one regression and a 74.44% baseline. These are internal v1 conflicts, not verified revision differences. e16
Table I and the method name DreamerV2, but Figure 3 and parts of IV-A say DreamingV2; IV-A cites [17], which is the distinct DreamingV2 reference. Figure 4 is labeled BC but its plotted trends do not cleanly reconcile with Table III. Neither discrepancy can be resolved from this PDF. e08e13e14
Only CALVIN D and eight tasks are evaluated. No dedicated ablations, generalization suite or physical deployment are reported. The 8.7× claim concerns unequal world-model update counts, not measured wall time or a matched-budget causal test. Training-only reward accuracy does not establish reliability on policy-generated states. e07e10e15
7.2 Questions for discussion
- Does correct transition-action supervision improve prior features beyond what a matched training budget achieves?
- How much of PPO improvement comes from the representation versus the retrained reward classifier?
- Which checkpoint and evaluation aggregation reconcile the conflicting tables, curves and prose?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires CALVIN D play data, the selected expert episodes and held-out configurations, the RSSM/encoder and inverse head, DiffusionMLP, reward-label replay and DPPO. Resolve the DreamerV2/DreamingV2 identity conflict first. Hardware, runtime, software versions, inverse-head widths, action preprocessing, full seed protocol and checkpoint/uncertainty aggregation are not specified. e03e04e07e09e10e11e14e15
Proposed checks: compare action supervision against no-action-loss and shuffled-action controls at matched update budgets; separately test reward classifiers on held-out simulated policy trajectories against CALVIN task labels. Neither check was performed for this report. e04e07e10e15
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test whether correct action labels improve the prior at matched budgets
Reader-proposed, not performed: resolve baseline implementation identity, then train three otherwise identical RSSMs with action-loss weight zero, the reported weight 1000 with correct labels, and weight 1000 with shuffled transition labels. Use the same data order and several recorded seeds; compare checkpoints at both 230K and 2M updates. Train identical BC policies from each frozen model on the same 50 episodes/task and evaluate the same held-out configurations. Add held-out action-prediction probes on posterior and prior features. The cascade hypothesis predicts a reproducible correct-label advantage in prior features and policy success. A gain shared by shuffled labels, or present only in the posterior, would weaken that explanation. e03e04e05e07e09e11e14e15
Check 2: Check imagined reward reliability against simulator task completion
Reader-proposed, not performed: freeze each model and its retrained reward classifiers, then collect separate held-out CALVIN trajectories from BC and progressively fine-tuned policies. At aligned states, compare classifier outputs with simulator task-checker labels, reporting precision and recall by task, policy checkpoint and imagination horizon. Keep training episodes out of this evaluation and evaluate actual simulator task success alongside predicted return. Use the same label threshold and seeds across models. If imagined reward rises while held-out precision or task completion falls, the reported training precision ≥0.97 and recall 1.00 have not transferred to policy-generated states; that would identify reward exploitation as an alternative explanation needing control. e06e10e11e12
8.3 Reading coverage
Visual audit: All seven supplied PDF pages were rendered and visually inspected, covering identity/version and affiliation (p. 1), conceptual diagram and RSSM equations (p. 2), architecture, objectives and policy training (p. 3), data/settings and qualitative predictions (p. 4), Tables I–III and evaluation/reward settings (p. 5), Table IV and Figure 4 with conflicting result prose (p. 6), and the conclusion continuation plus baseline references (p. 7). Every final crop was separately viewed. Figure 2's encoder-action arrows and KL links were cross-checked with Eqs. (1)–(7); its policy arrows simplify the full feature input described in text. No appendix is present, and separate supplements remain unverified. No code execution, experiments or external source inspection formed part of this reading.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, author affiliation, abstract and arXiv version (p. 1)
- I. Introduction (p. 1)
- II. Related Works, A–B (pp. 1–2)
- III. Method, A. World-Action Model and B. Enhancing Downstream Policy Learning with WAM (pp. 2–3)
- IV. Experiments, A. Action Regularization Enhances WAM and B. Enhancing Policy Learning with WAM (pp. 4–6)
- V. Conclusion (pp. 6–7)
- References [1]–[19] (p. 7)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- The inspected source is arXiv:2603.28955v1 [cs.AI], dated 30 March 2026. Its title and both authors match the supplied catalog. No different revision or edition was supplied or compared.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This acquisition limitation was addressed by visually inspecting all seven PDF pages and every final crop.
- Separate supplemental material availability has not been fully verified.
- No appendix occurs in the supplied seven-page PDF. No external code, supplements, linked references or project material were inspected; no experiments were reproduced.
- All three supplied text chunks were read individually, including continuations across chunk boundaries. Internal numerical and baseline-label conflicts remain unresolved and are disclosed below.
- The PDF has overlapping body text in places, particularly at the Training Objective transition on p. 3 and subsection transitions on pp. 4 and 6. Retained claims are also supported by legible equations, captions, tables or repeated prose; obscured fragments are not reconstructed.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, byline, affiliation footnote and arXiv margin
Title: Enhancing Policy Learning with World-Action Model. Authors: Yuci Han and Alper Yilmaz. Both are affiliated with the Photogrammetry and Computer Vision Lab, The Ohio State University, Columbus, OH 43210, USA. The margin identifies arXiv:2603.28955v1 [cs.AI], 30 Mar 2026.
Go to primary source ↓e02PDF p. 1, I. Introduction, representation motivation and proposed inverse-dynamics objective
The authors argue that reconstruction-oriented latents may omit action-relevant structure and propose action prediction from consecutive encoder embeddings to improve downstream control.
Go to primary source ↓e03PDF p. 2, III-A.1–2, Eqs. (1)–(5)
The dual-camera/proprioception encoder produces 1,554-dimensional embeddings. The DreamerV2 RSSM has recurrent state, 32×32 categorical stochastic variables, a posterior conditioned on embeddings and a prior without them; concatenated policy/decoder features have dimension 2,048.
Go to primary source ↓e04PDF p. 3, Figure 2 and III-A, Eq. (6) with cascading-effect paragraph
A three-layer MLP predicts an action from two consecutive encoder embeddings. The authors choose embeddings over recurrent features to avoid access to action inputs and describe encoder-to-posterior-to-prior transfer through KL. Figure 2's inverse-action arrows meet between adjacent encoders.
Go to primary source ↓e05PDF p. 3, III-A, Training Objective, Eq. (7)
The objective combines posterior-to-prior KL, squared observation reconstruction error and L1 action prediction error with separate loss coefficients.
Go to primary source ↓e06PDF p. 3, III-B, Behavioral Cloning and Offline Policy Fine-tuning, Eqs. (8)–(10)
A frozen world model supplies features from 50 expert episodes per task to a denoising DiffusionMLP. PPO uses prior-generated imagined states, a binary reward classifier, 50 parallel rollouts, ten denoising steps and BC regularization 0.025.
Go to primary source ↓e07PDF p. 4, IV opening and IV-A.1–2, dataset, evaluation and training settings
CALVIN D supplies two 64×64 camera views and roughly six hours/500K play transitions, also described as about 512K frames. Prediction evaluation uses 100 random validation sequences and 50-step open-loop rollouts with initial observation and ground-truth actions. Training settings include sequence length 50, batch 500, AdamW rate 3×10^-4, decay 0.05, KL balance 0.8, loss weights 3/1/1000 and 230K versus 2M gradient steps.
Go to primary source ↓e08PDF p. 5, Table I, all metric rows; PDF p. 4, Figure 3 and caption
Table I labels the baseline DreamerV2 and reports WAM/baseline PSNR 22.10±2.22/21.66±2.20, SSIM 0.814±0.061/0.807±0.067, LPIPS 0.144±0.072/0.149±0.073 and FVD 10.82/12.13. Figure 3 compares GT, DreamingV2-labeled and WAM images from both cameras at t=1,5,10,15,20,30,40,49. At t=40 and t=49, the baseline static view shows a red object and a separate blue blob; the corresponding ground-truth and WAM views lack that extra blob. Both generated gripper-camera rows show blur and shape differences from ground truth.
Go to primary source ↓e09PDF p. 5, Table II and IV-B.1–2
DiWA and WAM use DiffusionMLP with different world-model features. BC settings: 50 demonstrations/task, 20 denoising steps, four-step action horizon, 5,000 epochs, batch 256, learning rate 10^-4 to 10^-5, weight decay 10^-6, EMA 0.995.
Go to primary source ↓e10PDF p. 5, IV-B.3, policy fine-tuning and reward-classifier paragraphs
PPO is performed entirely within frozen latent dynamics: 50 parallel rollouts, batch 7,500, ten update epochs, actor/critic rates 10^-5/10^-3, discount 0.999, GAE 0.95, ten denoising steps, BC weight 0.025, 800 iterations and evaluation every 25. Reward preparation uses expert seed 42, simulator replay, imagined data and class balancing; ≥0.97 precision and 1.00 recall are measured on training data.
Go to primary source ↓e11PDF p. 5, Table III, all rows and average; IV-B.4 evaluation protocol
BC uses 29 held-out configurations/task, four-step DDPM chunks, at most 18 decisions/72 steps and CALVIN's checker. Displayed averages are DiWA 45.8% and WAM 61.7%. WAM improves seven rows; turn_on_led is 41.4±4.1 versus 44.8±3.9. Close drawer and slider right each improve by 31.1 percentage points.
Go to primary source ↓e12PDF p. 6, Table IV, all policy-success columns and Total Physical Interactions row
The eight WAM means are 96.7,96.6,87.5,89.7,100.0,75.9,96.6,100.0; DiWA means are 74.44,91.95,83.33,82.76,91.92,77.01,86.21,82.33. Displayed errors are retained in the result record. DPPO columns are environment steps to match DiWA, not success percentages. The footer lists DiWA/WAM interactions as 0/0 and DPPO totals about 2.5M/8M; its WAM header says Online Fine-Tuning.
Go to primary source ↓e13PDF p. 6, Figure 4, axes, eight panels, legend and caption; PDF p. 5, Table III and IV-B.2
Figure 4 plots BC success against epochs through 4,000, with blue DiWA and orange WAM. Open-drawer and turn-on-LED curves largely overlap, unlike the caption's broad higher-success claim. Some curve levels differ substantially from Table III. The settings specify 5,000 BC epochs; checkpoint correspondence is not explained.
Go to primary source ↓e14PDF pp. 2 and 4, III-A.1 and IV-A.1–3; PDF p. 4, Figure 3 labels; PDF p. 5, Table I; PDF p. 7, References [4] and [17]
The method and Table I identify DreamerV2; IV-A cites [17] for its baseline and uses DreamingV2 in training/results prose, also printed in Figure 3. Reference [4] is Mastering Atari with Discrete World Models, while [17] is DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction.
Go to primary source ↓e15PDF pp. 4–6, complete IV. Experiments, Tables I–IV and Figures 3–4; PDF pp. 6–7, V. Conclusion and end of paper
Experiments cover prediction quality, BC and PPO for eight CALVIN D tasks. There is no dedicated loss-weight, action-head or gradient-path ablation, held-out-domain suite, hardware/runtime report or physical deployment. The paper ends with references and provides no appendix or detailed implementation configuration beyond the reported settings.
Go to primary source ↓e16PDF p. 1, Abstract and I. Introduction; PDF p. 5, Table III and IV-B.4; PDF p. 6, Table IV, IV-B.5 and V. Conclusion
Abstract/Introduction BC averages are 59.4% to 71.2%, versus 45.8% to 61.7% in Table III and later prose. PPO prose reports 92.8% versus 79.8%, improvement on every task, and an open-drawer baseline of 70.0%; Table IV lists 74.44 for that baseline and a lightbulb-off regression. Arithmetic means of its displayed task means are 92.875 and 83.74375, not the prose aggregates.
Go to primary source ↓8.5 Primary sources
Enhancing Policy Learning with World-Action Model ↗
PDF · 4,319 extracted words
Source fingerprint
e5daab4fd095f6a1300f4f9949826c163589b2f44428d992a5e9c9bf50cc5b0d