EVA: Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewards
1. Paper overview
In one sentence: EVA improves a video planner by rewarding smooth, limit-respecting IDM-decoded actions, but this kinematic proxy can favor motion that never completes the task. e-overviewe-rewarde-visual-resultse-sim-resultse-real-resultse-hackinge-limitations
| At a glance | What to know |
|---|---|
| Research problem | Source description A plausible manipulation video can contain deformed arms, ambiguous joints or temporal jumps that an inverse dynamics model translates into unstable commands. EVA targets this executability gap by changing the generator’s distribution during training, avoiding reliance on expensive inference-time rejection sampling. e-overview |
| Core mechanism | Source description The central intervention repurposes a supervised action decoder as a reward model: generated videos are evaluated through their implied joint trajectories, and GRPO updates the video generator while the IDM stays frozen. e-idme-rewarde-backbone |
| A key reported result | Real-robot execution on five OOD tasks: 60.0 Average executed task success (%). CobotMagic; PlaceBlock, PourWater, WipeTray, FoldCloth and PlaceToy; 20 trials per task. Success requires completion without safety interruption or human intervention. Unaligned EVA 42.0; Vidar 34.0; pi0 11.0; GE-Act 3.0; ACT not reported. An 18-percentage-point gain over unaligned EVA on this specific task set. No confidence intervals or repeated-training-seed results are supplied. e-real-protocole-real-results |
| Reading caution | Source description The reward omits forces, friction and torques; expensive video diffusion limits fast reactive control. Prolonged GRPO can produce high-reward static behavior, wrong interactions and unrealistic link lengths because task completion is not directly rewarded. e-limitationse-hacking |
Core contributions
- Source description
The central intervention repurposes a supervised action decoder as a reward model: generated videos are evaluated through their implied joint trajectories, and GRPO updates the video generator while the IDM stays frozen. e-idme-rewarde-backbone
- Source description
The authors support alignment with human-rated rollouts, executed simulation and physical-robot trials, reward diagnostics, and an IDM spatial-softmax ablation. These tests address different links in the video-to-action pipeline. e-visual-resultse-sim-resultse-real-resultse-reward-validitye-ablation
Figure 1. The action decoder also supplies the generator’s post-training feedback. Original paper, p. 2 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow each row from left to right: the current observation and stapler-placement instruction condition a video model, its imagined sequence enters the IDM, and the decoder outputs an action. The lower row introduces a second path: an arrow rises from the IDM to the reward block, then returns left and down into the video generator. Sections 4.2 and 5.1 clarify what the schematic compresses: the IDM is frozen, action penalties produce the reward, and generator parameters are updated through GRPO. The separate generator and decoder remain present during deployment. The feedback arrow depicts post-training alignment, rather than a robot sensor loop. e-overviewe-idme-rewarde-backbonee-inferencee-limitationse-hacking
What it supports. The central architectural change is how the video generator is trained. EVA uses the action decoder’s implied trajectory to shape future video samples. This supports a dual-system, inverse-dynamics classification: post-training connects the objectives of two models without combining future-video and action prediction into one architecture.
Where the evidence stops. The check mark is conceptual, not a guarantee of collision-free contact or task completion. Segment-wise replanning from recent camera frames is specified separately on p. 7 and is absent from this simplified diagram.
2. Motivation
2.1 The problem and the proposed response
A plausible manipulation video can contain deformed arms, ambiguous joints or temporal jumps that an inverse dynamics model translates into unstable commands. EVA targets this executability gap by changing the generator’s distribution during training, avoiding reliance on expensive inference-time rejection sampling. e-overview
2.2 What this reading follows
A robot can fail even when its imagined future looks convincing: the video may bend a link, blur a joint, or imply a sudden motion that becomes unstable when decoded into commands. EVA uses that same decoder to judge the generator during post-training. Read the evidence as a sequence of tests: the diagram locates the feedback, human ratings measure visible artifacts, robot trials measure task execution, and the appendix examines the decoder and its reward. The strongest lesson is conditional: action-space alignment helps this embodiment and task set, while reward hacking and missing contact dynamics prevent smoothness from becoming a general certificate of success. e-overviewe-rewarde-visual-resultse-sim-resultse-real-resultse-hackinge-limitations
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | IDM |
| Quadrant | Q4 · Dual-system × IDM |
3.1 Evidence-based assessment
Supports the recorded classification
Architecture and inference support Q4, Dual-system × IDM: a separate video generator predicts future observations, and an independently trained IDM extracts actions. IDM-derived post-training feedback couples their objectives without making them one joint future/action predictor. The reward optimizes training; the IDM also remains an inference-time action decoder. e-overviewe-idme-rewarde-backbonee-inference
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 1. Translate visual mistakes into an action-space signal
EVA relies on an IDM trained from paired videos and executed commands. A short window around a frame supplies temporal context; a convolutional backbone and spatial softmax retain information about where robot features occur, and an MLP predicts the corresponding command. In deployment, the future portion of that window comes from the generated plan, so it does not require observing the real robot’s future. The same decoder can expose a malformed visual trajectory as a rough command sequence. EVA then applies hand-specified motion penalties to those commands. This is a learned visual-to-action mapping followed by a designed kinematic objective, not a learned contact simulator. The spatial-softmax ablation supports the decoder choice, while leaving its reliability on unusual generated artifacts an additional question. e-idme-rewarde-ablatione-limitations
5.2 2. Optimize the video model while keeping the decoder fixed
The frozen IDM assigns actions to each candidate video. Finite differences turn those commands into velocities, accelerations and jerks; Huber smoothness penalties and excess-limit penalties are aggregated across the sequence. Equation (9) maps this aggregate penalty to a scalar reward. Thus the signal uses motion throughout a rollout even though the final reward is video-level. Flow-GRPO samples eight candidates for a prompt, compares rewards within the group and applies clipped, KL-regularized updates to rank-32 LoRA parameters. The base generator weights and IDM stay frozen. During inference the updated model returns to flow sampling, then the IDM supplies executable commands; longer tasks refresh the visual plan from recent camera frames after each segment. Missing reward constants and the plotted-score scaling discrepancy remain reproduction gaps. e-rewarde-flowe-backbonee-traininge-inferencee-reward-scale
5.3 3. Follow the evidence chain without treating smoothness as success
Reader analysis: the experiments support different propositions rather than one universal feasibility claim. Human ratings show fewer visible kinematic defects; executed RoboTwin and physical trials establish that the aligned pipeline can complete more tasks in the evaluated settings. Ground-truth-video IDM execution tests whether the decoder can recover useful commands when the visual input is valid. Reward box plots address association between its score and success, but their overlap leaves high-scoring failures possible. Appendix C explains how prolonged optimization can exploit precisely this gap through static or otherwise incorrect behavior. Consequently, checkpoint selection is part of the effective method, not just presentation. A reproduction should monitor held-out execution alongside reward, preserve the reported table inconsistencies, and avoid treating synthetic video examples as demonstrated downstream data-augmentation gains. e-visual-resultse-sim-resultse-real-resultse-idm-executione-reward-validitye-hackinge-synthetic
5.4 Training and inference
During training
Simulation training uses 1,050 trajectories across 21 tasks, or 50 demonstrations per task. Real training uses 250 teleoperated trajectories across five tasks for the video model and IDM. The visual-quality comparison uses held-out instruction paraphrases. e-eval-protocole-real-protocol
SFT uses 49-frame clips, eight NVIDIA A800 GPUs, FSDP/bf16 and effective batch size 32. Simulation uses about 4,500 steps at 640×480; real data uses about 2,580 at 832×480. AdamW learning rates are 0.000008 for SFT and 0.0002 for GRPO. e-training
GRPO uses eight candidates, LoRA rank 32, clipping 0.001 and KL coefficient 0.004. Simulation runs 136 optimization steps with four inner epochs, taking about six days; real alignment uses 46 steps with two inner epochs, taking about three days. Validation-based checkpoint selection and early stopping mitigate reward hacking. e-traininge-hacking
During inference
The updated generator returns to the original flow sampler. The IDM decodes generated temporal windows into actions for execution. For long tasks, each new rollout conditions on the latest four camera frames after the previous segment executes. Thus deployment includes observation feedback between segments, despite the earlier description of an open-loop IDM extractor. e-flowe-inferencee-open-loop
5.5 Implementation flow
- Generate a latent visual plan
EVA starts from the LVP checkpoint with a Wan2.1-14B Diffusion Transformer and diffusion forcing. Embodiment-specific supervised fine-tuning produces the unaligned comparison model. A pretrained 3D VAE connects pixel videos to the flow-matching latent representation. e-flowe-backbone
- Decode video windows into actions
The IDM stacks temporal frames, extracts convolutional features, converts each channel to an expected 2D coordinate using spatial softmax, and predicts an action through an MLP. Supervised regression uses ground-truth robot commands. e-idm
- Score the implied motion
Finite differences of decoded commands yield velocity, acceleration and jerk. Huber penalties discourage acceleration and jerk; squared excess penalties enforce velocity and acceleration bounds. Weighted penalties become a decreasing reward. The displayed objective contains no explicit task-completion, contact-force or joint-position-limit term. e-rewarde-limitationse-hacking
- Align the generator
Flow-GRPO introduces stochastic sampling for policy optimization, normalizes rewards within each prompt’s candidate group, and applies clipped updates with KL regularization. The learned change is in the generator’s LoRA parameters; the base weights and trained IDM remain frozen. e-flowe-backbonee-training
6. Experiments & results
Executable Video Alignment (EVA) post-trains a video planner using a frozen inverse dynamics model to penalize rough or limit-violating decoded actions. Reported robot success improves, but smoothness remains an imperfect proxy for completing a task (e-overview, e-reward, e-real-results, e-hacking).
6.1 Read the original evidence
Table 1. Alignment improves visible kinematics more than instruction adherence. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the four columns as distinct human judgments of generated videos. Kinematic asks whether the arm preserves its structure and temporal continuity; interaction concerns plausible contact and object motion; instruction asks whether the language goal is followed. Perfect requires visual task completion while satisfying the other three conditions. The comparison contains 210 prompts, ten for each selected RoboTwin task, using held-out instruction paraphrases. Videos were anonymized and shuffled. The two EVA rows isolate adding reward-based post-training to the same supervised starting point. Vidar is an additional comparison with a different backbone, so it is a weaker control for the alignment mechanism. e-eval-protocole-visual-resultse-backbone
What it supports. EVA’s kinematic rating increases from 70.5% to 91.4%, a 20.9-percentage-point gain. Its perfect visual execution rating increases from 68.1% to 83.8%. Instruction adherence changes from 90.5% to 89.5%, so the table supports substantial artifact reduction with a small observed decline on that separate criterion.
Where the evidence stops. These are visual ratings, not simulator or physical-robot successes. The paper supplies neither rater count and agreement nor uncertainty estimates, so the one-point instruction change cannot be assigned statistical significance from this table.
Table 2. Executed simulation success rises on average, with task-level exceptions. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the shaded Average column: it summarizes all 21 tasks, whereas the individual columns show only a representative subset. Every task cell counts successful executions out of 20 randomized episodes. Compare the two EVA rows to assess reward alignment; they share a multi-task setup trained on the same per-task demonstrations. ACT, DP, RDT and pi0 use separate single-task policies, each with 50 demonstrations, so their averages compare different training organizations. After reading the average, inspect individual tasks such as StampSeal to see whether the improvement is uniform. Appendix Table S1 provides the full task list and exposes a reporting discrepancy. e-eval-protocole-sim-results
What it supports. The reported overall success changes from 46.2% without RL to 52.6% with RL, a 6.4-percentage-point gain. The result concerns executed IDM commands, giving stronger control evidence than visual ratings alone. Improvement is not universal: StampSeal changes from five successes to four out of twenty episodes.
Where the evidence stops. Aligned OpenLaptop is 12/20 here but 10/20 in Table S1. S1 also labels its caption real-robot despite Section A identifying simulation. These source discrepancies remain unresolved; the crop preserves Table 2 exactly.
Table 3. Physical trials support improved execution on the five tested novel tasks. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the table in two halves. The left contains the five tasks used to collect 250 teleoperated training trajectories; the right contains five additional OOD tasks. Each task has 20 physical trials, and a success requires completing the objective without safety interruption or human intervention. Keep the two averages separate rather than treating them as one benchmark. The right-side tasks include placing a block, pouring water, wiping a tray, folding clothing and placing a toy; Table S4 defines their goals. The EVA rows provide the direct alignment comparison. ACT’s OOD cells are unavailable results, not zero successes. e-real-protocole-real-results
What it supports. OOD execution increases from 42% to 60%, an 18-percentage-point improvement over unaligned EVA. PourWater changes from 11/20 to 15/20. The aligned seen-task average is 64%; HangCable remains 8/20 in both EVA rows. These outcomes demonstrate physical execution on the specified tasks rather than video-only generalization.
Where the evidence stops. The unaligned seen average is printed as 52%, but its five counts sum to 51/100. Preserve that inconsistency when comparing seen averages. Twenty trials per task and absent uncertainty/seed reporting also limit precision and generality.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Human-rated visual rollout quality 210 prompts: ten per RoboTwin task, with anonymized, shuffled videos and held-out paraphrases. | 91.4 / 86.2 / 89.5 / 83.8 Kinematic / interaction / instruction / perfect rating rates (%) | Without RL: 70.5 / 83.3 / 90.5 / 68.1. Vidar: 67.6 / 66.7 / 87.6 / 62.9. Kinematic plausibility rises 20.9 percentage points; instruction adherence falls 1.0 point. “Perfect” is a human judgment of generated video, not executed robot success. e-eval-protocole-visual-results |
| RoboTwin 2.0 executed manipulation 21 randomized bimanual tasks, 20 episodes each. EVA is multi-task; ACT, DP, RDT and pi0 are trained separately per task. | 52.6 reported in Table 2; 52.62 in Table S1 Average executed task success (%) | Unaligned EVA 46.2; pi0 45.7; RDT 37.1; DP 29.5; ACT 29.0 in Table 2. The printed EVA gain is 6.4 percentage points. OpenLaptop disagrees between Table 2 (12/20) and S1 (10/20); S1’s caption says real-robot although its section describes simulation. StampSeal decreases from 5/20 to 4/20. e-eval-protocole-sim-results |
| Real-robot execution on five OOD tasks CobotMagic; PlaceBlock, PourWater, WipeTray, FoldCloth and PlaceToy; 20 trials per task. Success requires completion without safety interruption or human intervention. | 60.0 Average executed task success (%) | Unaligned EVA 42.0; Vidar 34.0; pi0 11.0; GE-Act 3.0; ACT not reported. An 18-percentage-point gain over unaligned EVA on this specific task set. No confidence intervals or repeated-training-seed results are supplied. e-real-protocole-real-results |
| Real-robot execution on five seen tasks Same physical protocol; StackBowl, HangCable, Place2Basket, Place2Tray and FoldTowel. | 64.0 Average executed task success (%) | Unaligned EVA is printed as 52.0, but its counts 12+8+12+14+5 sum to 51 successes in 100 trials. The baseline average is internally inconsistent; both printed average and row arithmetic are preserved. HangCable stays at 8/20. e-real-protocole-real-results |
| Inverse-dynamics decoding and spatial-softmax ablation Ground-truth video decoding and simulated execution across 21 tasks, 20 episodes each; prediction accuracy uses the ±0.05-radian tolerance. | 0.9864 / 89.52 Action test accuracy (fraction) / execution success (%) | Replacing spatial softmax with global average pooling: 0.7738 / 84.29. The ablation supports spatial localization for action decoding. Ground-truth-video performance does not establish reward reliability on arbitrary generated artifacts. e-idm-executione-ablation |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table S2. Spatial localization improves the decoder before its reward role is assessed. Original paper, p. 23 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The first row uses the IDM described in Section 4.1: convolutional channels are converted into expected spatial coordinates before an MLP predicts actions. The second replaces spatial softmax with global average pooling. Read Test Accuracy as a fraction, using the paper’s ground-truth action tolerance of ±0.05 radians per dimension; read Test Success Rate as a percentage from simulated execution. These are distinct metrics. Section 5.5 evaluates action decoding from ground-truth demonstration videos, which separates decoder performance from video-generation errors. This experiment therefore tests the value of the spatial representation before asking whether its outputs make a reliable reward for generated rollouts. e-idme-idm-executione-ablation
What it supports. Spatial softmax gives 0.9864 test accuracy versus 0.7738 with pooling, and 89.52% execution success versus 84.29%. The execution difference is 5.23 percentage points. This supports the chosen decoder representation, while also showing that high prediction accuracy and actual execution success are not interchangeable measurements.
Where the evidence stops. The ablation changes the IDM architecture; it does not isolate individual reward terms or prove reward robustness under generated artifacts. The accuracy-test sample count and split are not specified here, and no uncertainty is reported.
Figure S1. Reward is associated with quality, but successful and failed rollouts overlap. Original paper, p. 21 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the legends to distinguish the two questions. Panel (a) compares visually perfect rollouts with those showing kinematic artifacts; panel (b) compares successful with failed simulator executions. Each panel repeats the comparison for OpenLaptop, PlaceObjStand and PlaceBurger. Dots are individual rollouts, with ten samples in each task-and-outcome group; the boxes summarize their distributions. Follow the pair within a task before comparing across tasks, since reward levels differ. In the right panel, several failed PlaceObjStand rollouts have scores near successful examples. The selected groups demonstrate separation and overlap; they do not estimate how often either outcome occurs in unrestricted deployment. e-reward-validitye-reward-scalee-hacking
What it supports. The distributions generally place artifact-free and successful rollouts at higher rewards, supporting the motivation for an IDM-based signal. However, overlapping execution groups show why a high score cannot certify task completion. The appendix’s separate reward-hacking examples strengthen the need to track actual execution while optimizing this proxy.
Where the evidence stops. Reward scores above one in this plot are not reconciled with Equation (9)’s bounded form under nonnegative penalties and positive scale/decay. Figure 2 likewise reports 7.94 and 3.04. No scale conversion is supplied; do not invent one.
7. Analysis & limitations
7.1 What the evidence leaves open
The reward omits forces, friction and torques; expensive video diffusion limits fast reactive control. Prolonged GRPO can produce high-reward static behavior, wrong interactions and unrealistic link lengths because task completion is not directly rewarded. e-limitationse-hacking
Reward-validity plots compare ten examples per group for three tasks, with overlapping success/failure distributions. They support an association, not calibrated success probabilities. Figure 2 reports rewards 7.94 and 3.04 and appendix plots exceed one; Equation (9), with nonnegative penalties and positive scale/decay, would be at most one. No conversion is specified. e-reward-validitye-reward-scale
Different backbones and single-task versus multi-task training complicate cross-method attribution. The within-EVA comparison is more direct. Appendix F illustrates synthetic-scene video generation, but supplies no downstream policy-training benefit or execution benchmark for that augmentation pipeline. e-backbonee-eval-protocole-synthetic
7.2 Questions for discussion
- Can a held-out task-completion signal detect when higher kinematic reward stops predicting better execution? (e-hacking)
- Does spatial softmax improve reward ranking on generated videos as well as decoding ground-truth demonstrations? (e-ablation, e-reward-validity)
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the LVP initialization, paired embodiment-specific trajectories, frozen IDM, RoboTwin setup and matching physical platform. The supplement identifies AgileX PiPER arms with six arm degrees of freedom plus grippers; this distinguishes the hardware from Figure 2’s seven plotted channels including the gripper. e-backbonee-real-protocole-hardwaree-reward-scale
Unspecified details include numeric IDM context radius, control interval, reward weights, Huber thresholds, acceleration limits, P_0/gamma values, IDM optimizer schedule, inference sampling/segment settings and software versions. Joint maximum speeds are listed, but their exact mapping into reward configuration is not. Rater count/agreement, validation partition and checkpoint-selection rules remain incomplete. e-idme-rewarde-eval-protocole-traininge-hackinge-hardware
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does better IDM decoding also yield better reward ranking?
Proposed check, not performed: train spatial-softmax and global-pooling IDMs on the same paired data and matched schedules. Freeze one video generator and evaluate both decoders on the same held-out generated-video bank, including static, visibly distorted and apparently plausible rollouts. Keep reward terms and normalization calibration fixed using training data only. Execute each decoder’s commands in matched simulator initial states; measure action reconstruction on genuine demonstrations separately from within-task reward ranking of execution outcomes. The discriminating observation is whether the spatial-softmax advantage in ground-truth decoding also improves ranking on generated videos. An accuracy gain without a ranking gain would weaken using Table S2 alone to justify the reward model. e-idme-rewarde-idm-executione-ablatione-reward-validity
Check 2: Locate when reward improvement stops predicting task completion
Proposed check, not performed: start aligned and no-RL controls from the same SFT checkpoint and hold the IDM, prompts, sampler and execution horizon fixed. Save GRPO checkpoints at a predetermined schedule. On a validation set separate from final evaluation, track reward components, movement magnitude, visual contact errors and executed task success using matched simulator seeds. Then compare validation-selected early checkpoints with later checkpoints on held-out scenes. Rising reward accompanied by more static trajectories and flat or falling task success would reproduce the documented proxy failure; parallel improvement in reward and success would support alignment within that optimization range. Report selection rules and all checkpoints rather than choosing by final-test success. e-backbonee-rewarde-eval-protocole-traininge-hacking
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version page; all method, equation, experimental and limitation pages; Figures 1–6; Tables 1–4; and the embedded supplement’s Figures S1–S5 and Tables S1–S4. This includes supporting training, hardware, task-definition, reward-hacking and synthetic-generation pages even when uncropped. All six final crops were inspected at their original rendered resolution, with table headers, plot axes and legends retained. Figure 1’s reward-arrow direction was checked against Sections 4.2 and 5.1; it denotes generator post-training. Reward-score scaling and the Table 2/S1 and Table 3 discrepancies are disclosed in the relevant cautions. Reference-only pp. 15–19 were read as text, not visually inspected. External videos, code, linked resources and any separate supplements are outside this pass.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 20, 21, 22, 23, 24, 25, 26. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- 1 Introduction
- 2 Related Works
- 3 Preliminaries
- 3.1 Flow-matching-based Video Generation
- 3.2 Group Relative Policy Optimization
- 4 Method
- 4.1 Inverse Dynamics Model
- 4.2 IDM-based Executability Reward
- 5 Experiments
- 5.1 Experimental Settings
- 5.2 Visual Rollout Quality
- 5.3 Simulation Policy Execution on RoboTwin
- 5.4 Real-World Deployment
- 5.5 IDM evaluation
- 5.6 Failure Modes
- 6 Conclusion
- References
- A Detailed Experimental Results
- B Reward Validity
- C Detailed Training Analysis
- D Ablation Study on the IDM
- E Real-World Experimental Setup
- E.1 Robot Platform Setup
- E.2 Task Descriptions
- F Scaling Embodied Data via Zero-Shot Generation
- Abstract
- Limitations and Future Work
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The complete supplied text, including the embedded supplement on PDF pp. 20–26 (S1–S7), was read. Figures and tables were inspected from the PDF; reference-only pp. 15–19 were read as text without page-image inspection. No separate supplement was supplied.
- Identity/version note (e-identity): the title page prints “Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewards”; the catalog and supplied primary-source title prepend “EVA:”. The descriptive title, exact identifier, six authors, and the abstract’s explicit EVA acronym establish the same work. This prefix discrepancy is preserved without inferring an undocumented title revision. The inspected artifact is arXiv:2603.17808v2, dated 24 March 2026; the catalog submission date is 18 March 2026. Version 1 and revision history were not supplied, so differences between revisions cannot be assessed.
- The supplied source fields are preserved verbatim. Metadata retains the supplied observedTitle and identifies the printed-title prefix discrepancy in its verified location note.
- Code, linked project pages, external citations and external accounts were not inspected. No model training or robot experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author/affiliation block, arXiv margin and abstract
Printed title: Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewards. Authors: Ruixiang Wang, Qingming Liu, Yueci Deng, Guiliang Liu, Zhen Liu, Kui Jia. Affiliations: The Chinese University of Hong Kong, Shenzhen; DexForce Technology Co., Ltd. Margin: arXiv:2603.17808v2, 24 Mar 2026. Abstract defines Executable Video Alignment (EVA).
Go to primary source ↓e-overviewPDF pp. 2–3, Figure 1 and Section 1
The separate video model and inverse dynamics model form a visual-planning/action-decoding pipeline. Figure 1 directs the IDM reward back to the generator for post-training; the introduction defines the executability gap and discusses rejection-sampling cost.
Go to primary source ↓e-open-loopPDF p. 4, Section 2, Embodied Visuomotor Policies
The discussion calls the IDM an open-loop extractor unable to correct upstream generated-video errors; this describes the decoder, while p. 7 specifies segment-wise observation feedback.
Go to primary source ↓e-flowPDF pp. 5–6, Sections 3.1–3.2, Equations (1)–(2)
Video latents come from a pretrained 3D VAE. Flow matching learns noise-to-data transport; Flow-GRPO uses a stochastic process for group-relative clipped optimization with KL regularization, then returns to the original flow sampler.
Go to primary source ↓e-idmPDF p. 6, Section 4.1, Equations (3)–(4)
A temporal-window IDM is trained by squared action error. A convolutional backbone, spatial softmax expected coordinates and MLP predict actions. The context radius k is symbolic without a supplied numeric value.
Go to primary source ↓e-rewardPDF p. 7, Section 4.2, Equations (5)–(9)
Huber acceleration/jerk penalties and squared excess velocity/acceleration penalties form a weighted sum, mapped to reward by a negative power. P_0 is estimated from pretrained rollouts; gamma controls decay. Numerical reward configuration is not specified.
Go to primary source ↓e-inferencePDF p. 7, Section 5 opening paragraph
At deployment a generated visual rollout is decoded into per-step actions. Long-horizon execution conditions the next rollout on the most recent four camera frames after the previous segment.
Go to primary source ↓e-backbonePDF p. 8, Section 5.1, Base model and Baselines
EVA uses LVP initialization, Wan2.1-14B, diffusion forcing and embodiment-specific SFT before GRPO with a frozen IDM. Vidar uses Wan2.2-5B with the same fine-tuning data/protocol.
Go to primary source ↓e-eval-protocolPDF p. 9, Sections 5.2–5.3
The RoboTwin subset has 21 tasks and 1,050 training trajectories. Visual evaluation uses ten prompts per task, held-out paraphrases and anonymized/shuffled human ratings. Simulation baselines train separate policies using 50 demonstrations per task; EVA uses one multi-task policy.
Go to primary source ↓e-visual-resultsPDF p. 10, Table 1, all rows and four metric columns; p. 9, rubric
EVA with RL: 91.4/86.2/89.5/83.8 for kinematic/interaction/instruction/perfect ratings. Without RL: 70.5/83.3/90.5/68.1. Vidar: 67.6/66.7/87.6/62.9. Perfect requires a visually completed task satisfying the other criteria.
Go to primary source ↓e-sim-resultsPDF p. 11, Table 2, Average/OpenLaptop/StampSeal columns; p. 20 (S1), Section A and Table S1
Table 2 averages are ACT 29.0, DP 29.5, RDT 37.1, pi0 45.7, unaligned EVA 46.2, aligned EVA 52.6 percent. S1 reports 46.19 and 52.62 for EVA. Aligned OpenLaptop is 12/20 in Table 2 versus 10/20 in S1; StampSeal falls 5/20 to 4/20. S1’s caption says real-robot, contradicting Section A’s simulation description.
Go to primary source ↓e-real-protocolPDF pp. 10–11, Section 5.4; p. 25 (S6), Table S4
CobotMagic training uses 50 demonstrations for each of five seen tasks. Five additional OOD tasks are evaluated. Twenty physical trials per task count success only without safety interruption or human intervention. Table S4 lists task goals and arm configurations.
Go to primary source ↓e-real-resultsPDF p. 11, Table 3, seen/OOD averages and EVA task rows
Aligned EVA averages 64.0 percent seen and 60.0 OOD; unaligned averages are printed as 52.0 and 42.0. Unaligned seen counts are 12,8,12,14,5 out of 20 each, totaling 51/100 rather than the printed 52 percent. Aligned OOD counts are 10,15,11,12,12. Vidar OOD is 34, pi0 11, GE-Act 3 percent; ACT OOD is N/A.
Go to primary source ↓e-idm-executionPDF p. 12, Table 4 and Section 5.5
The IDM decodes ground-truth demonstration videos and executes predicted actions in RoboTwin, reporting 89.52 percent average success over 21 tasks, with 20 episodes per task.
Go to primary source ↓e-failuresPDF p. 13, Figure 5 and Section 5.6
Unaligned rollout failures include implausible kinematics (deformation, joint ambiguity, temporal discontinuity), wrong contact and incorrect goals.
Go to primary source ↓e-limitationsPDF p. 14, Limitations and Future Work
The authors identify absent explicit contact dynamics (forces, friction, torques) and costly diffusion sampling that limits high-frequency reactive control.
Go to primary source ↓e-reward-validityPDF p. 21 (S2), Figure S1 and Appendix B
Box plots compare reward by visual artifacts and by simulation outcome for OpenLaptop, PlaceObjStand and PlaceBurger, with ten rollouts per group per task. Higher rewards generally accompany plausible/successful rollouts, but distributions overlap.
Go to primary source ↓e-trainingPDF p. 9, implementation details; pp. 21–23 (S2–S4), Appendix C
Training uses eight A800 GPUs, batch 32, FSDP/bf16 and 49-frame SFT clips. SFT: 4,500 simulation steps at 640×480, 2,580 real steps at 832×480, AdamW 0.000008, batch one per device with four accumulation steps. GRPO: AdamW 0.0002, eight candidates, rank-32 LoRA only, clip 0.001, advantage clip 5, gradient norm 1, KL 0.004; 136 steps/four inner epochs/about six days in simulation, 46/two/about three days on real data.
Go to primary source ↓e-hackingPDF p. 22 (S3), Figure S2; p. 23 (S4), Appendix C, Reward Hacking
Prolonged optimization can produce high-reward wrong interactions, unrealistic links and static behavior without task completion. The authors use early stopping and checkpoint selection based on validation rollout quality and downstream execution.
Go to primary source ↓e-ablationPDF p. 23 (S4), Table S2 and Appendix D
Spatial softmax achieves test accuracy 0.9864 and execution success 89.52 percent; global average pooling achieves 0.7738 and 84.29 percent. Accuracy uses a ±0.05-radian ground-truth action tolerance per dimension; success follows the main simulator protocol.
Go to primary source ↓e-hardwarePDF pp. 23–24 (S4–S5), Appendix E.1, Figure S4 and Table S3
The physical setup uses AgileX Cobot Magic with PiPER arms, six degrees of freedom per arm and two-finger grippers. Table S3 gives joint ranges and maximum speeds but no acceleration-limit configuration for the reward.
Go to primary source ↓e-syntheticPDF p. 25 (S6), Appendix F; p. 26 (S7), Figure S5
A text-to-image generator synthesizes initial scenes, then EVA generates videos. Examples illustrate synthetic OOD rollouts; no downstream augmentation-training performance is quantified.
Go to primary source ↓e-reward-scalePDF p. 7, Equation (9); p. 8, Figure 2 and caption; p. 21, Figure S1; p. 22, Figure S3
Equation (9) has no displayed multiplicative rescaling. Figure 2 assigns 7.94 and 3.04 to two examples and labels seven left-arm action channels including the gripper. Appendix reward plots also exceed one. The source provides no mapping reconciling these plotted/captioned scores with the bounded formula.
Go to primary source ↓8.5 Primary sources
EVA: Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewards ↗
PDF · 9,226 extracted words
Source fingerprint
28ae84944459f6b19195eccf574e49c235252b865121eb54df535342e046861c