EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fields
1. Paper overview
In one sentence: Camera-aligned kinematic fields and event-supervised fusion improve simulated robot-video rollouts, but the visual action representation remains difficult to invert into precise numerical control. e02e03e04e05e08e12e15e18
| At a glance | What to know |
|---|---|
| Research problem | |
| Core mechanism | Source description KVAFs rasterize robot geometry into the target camera plane, giving action-derived information an explicit spatial correspondence with RGB video. e04 |
| A key reported result | EA-WM video generation on selected WorldArena metrics: 76.60; Interaction Quality 0.682, Trajectory Accuracy 0.430, Depth Accuracy 0.959, Perspectivity 0.838, Instruction Following 0.792, Semantic Alignment 0.895. P3CScore: mean of six normalized, higher-is-better metrics ×100. RoboTwin data and segmentation following WorldArena; Table 1 representative video/world-model comparison. Exact split size is not reported. CogVideoX: P3CScore 71.08; interaction 0.594, trajectory 0.353, instruction 0.727, semantic alignment 0.898. Reported gain is 5.52 score points; EA-WM leads five of six columns, with semantic alignment below CogVideoX. This evaluates generated video, not task execution. e07e08e18 |
| Reading caution | Source description The authors identify dependence on robot kinematics, calibrated cameras and synchronized logs, with vulnerability to calibration error, occlusion, sensor noise and embodiment shifts. Object/contact state is represented indirectly through fusion and EDLS. e14 |
Core contributions
- Source description
KVAFs rasterize robot geometry into the target camera plane, giving action-derived information an explicit spatial correspondence with RGB video. e04
Figure 2. Geometry supplies the spatial interface; event supervision trains how the two streams exchange information. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read from left to right. Forward kinematics and camera projection convert numerical controls and states into the black-background KVAF sequence. The middle column distinguishes RGB latents, KVAF latents and frame-difference targets encoded by the VAE. On the right, follow the separate video and KVAF branches and their cross-stream arrows. Equations (4)–(6) and Algorithm 2 clarify the central block: a shared event representation produces an event prediction and a gate, and that gate scales cross-attention residuals in both directions. The MSE connection is supervision, not an observed future supplied at deployment. Fire symbols on DiT blocks must be read with Section 4.1: the specified backbone adaptation uses LoRA. e03e04e05e06e07e19
What it supports. The architectural contribution connects action geometry to image-space generation without compressing all motion into a small numerical token. EDLS supervises a representation shared with the gate, providing a route by which temporal-change supervision can alter information exchange. The diagram and equations establish this mechanism; they do not independently establish its empirical benefit.
Where the evidence stops. The graphic says “Z-WM architecture” although its caption names EA-WM. Its inference box lists only text and first frame, while Section 3 also names actions. The exact Table 1/Table 4 inference distinction remains unresolved; stage labels describe training.
2. Motivation
2.1 The problem and the proposed response
Raw joint or end-effector vectors leave a video generator to infer where the robot should appear and how its motion affects objects. The authors target action-consistent future video synthesis, emphasizing geometry and interaction fidelity rather than policy success. e02e03
2.2 What this reading follows
A commanded motion tells a robot how to move, but does not directly tell a video generator where each joint or gripper belongs in the image. EA-WM renders that missing spatial information as Structured Kinematic-to-Visual Action Fields, or KVAFs. A second denoising stream processes these fields alongside RGB video, while event-supervised gates exchange geometry and scene information. This reading follows the construction, the evidence for its two components, and the limits of the resulting representation. The strongest support comes from simulated WorldArena video metrics. Separate action-recovery errors and broader appendix metrics keep the interpretation grounded: coherent-looking future frames do not establish a usable robot policy. e02e03e04e05e08e12e15e18
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Foundational work |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
This table preserves the labels recorded at reading time. The current major category is Benchmarks & simulators. View the current classification.
3.1 Evidence-based assessment
Supports the recorded classification
The recorded foundational-work placement fits a neural simulator with explicit 3D-to-image geometry. Keeping the control-quadrant axes not applicable is defensible: the architecture jointly models video and visual action fields through two full-depth streams, while numerical action recovery is diagnostic and no control policy is executed. Joint training alone does not establish a One Model action-policy classification. e04e05e12e18
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Make the action occupy the same image plane as the prediction
Begin with the robot state, not with the diffusion model. A joint vector determines link transforms through the robot's kinematic chain; the camera transform and perspective projection then determine where those links appear. EA-WM uses this geometry to render both arms, gripper opening, end-effector heatmaps and orientation axes into an RGB field. The appendix further encodes relative depth through an episode-normalized color map. Passing this field through the same VAE as the video places both streams in a shared latent representation. A useful reader interpretation is that projection supplies a spatial correspondence the generator would otherwise have to infer from numerical controls. The important boundary is what the field contains: it explicitly describes robot-side geometry. Object motion and contact state are not directly rendered, motivating a second mechanism that can exchange information with the RGB stream. e04e03e05e14
5.2 Follow the supervision into the fusion gate
An auxiliary loss influences generation only through the parameters and features it trains. Here, consecutive-frame absolute differences are encoded with the video VAE to create an event target. An event MLP consumes the current video and KVAF tokens, forming a representation shared by an event-prediction head and a gate head. EDLS trains the prediction, while the gate scales the two cross-attention residuals. That shared representation connects temporal-change supervision to the mechanism used for information exchange. Algorithm 1 clarifies the order: update the KVAF block, fuse at selected layers, then update the video block. Stage one stabilizes the LoRA-adapted streams with fusion frozen; stage two unfreezes fusion. This explains the intended causal route, but Table 2 removes the broader event-aware mechanism and cannot determine whether EDLS or gating alone accounts for the improvement. e05e06e07e10
5.3 Separate rollout quality, conditioning strength and action precision
The evaluation asks several different questions. Table 1 compares EA-WM with representative video/world models; Table 4 separately compares KVAF-conditioned generation with action- or trajectory-conditioned models. Their scores should remain attached to those settings because the paper does not specify a complete inference switch between them. Tables 6 and 7 broaden each comparison to sixteen video-perception metrics, revealing tradeoffs concealed by the selected six-metric score. Table 3 asks a different question again: whether generated visual fields can be inverted into numerical relative actions using a heuristic. Its lower detection coverage and larger errors show why visual alignment is not interchangeable with control precision. Finally, the generated overlays and ablation frames illustrate image-domain behavior. Appendix A.6 explicitly says the work evaluates simulation and does not execute actions on real robots. e08e13e15e16e12e17e18e19
5.4 Training and inference
During training
Absolute differences between consecutive training RGB frames, with an all-zero first difference, are VAE-encoded as event targets. The objective sums weighted RGB/KVAF flow-matching errors and mean event-latent error across fusion layers. The first RGB frame is a condition and excluded from video prediction loss. e06
Stage one freezes fusion modules and trains the main DiT LoRA, KVAF-branch LoRA and KVAF head; stage two unfreezes fusion. Figure 2 marks the VAE and video head frozen. Reported settings are LoRA rank 32, learning rate 8×10⁻⁵, batch size 32 and 32 H100 GPUs. e03e07
During inference
The intended generation flow is initial image/text plus structured action guidance through coupled denoising streams. EDLS difference videos are training targets; the learned gate regulates generation. Section 3 names an action sequence as input, but Figure 2 labels only text and first frame as inference inputs. Table 4 separately reports KVAF-conditioned generation; its exact conditioning/sampling distinction from Table 1 is not specified. e03e05e06e13e19
Heuristic inversion detects heatmap centers and pose axes, solves PnP, associates bimanual tracks, fills missing detections and smooths trajectories before computing relative actions. Appendix A.3 explicitly excludes this diagnostic from EA-WM training/inference. There is no demonstrated robot execution or closed-loop control. e12e18
5.5 Implementation flow
- Lift and project robot geometry
Forward kinematics recovers arm and gripper keypoints for both arms. Camera extrinsics and intrinsics project them into image coordinates; non-positive-depth points are discarded. Render depth-colored skeletons, landmarks, gripper geometry, Gaussian end-effector heatmaps and RGB pose axes on a black canvas. Object states are not explicitly rasterized. e04
- Denoise two aligned latent streams
The same pretrained video VAE encodes RGB and KVAF sequences. Both latents are noised for flow matching and patchified. The video stream uses Wan2.2-TI2V DiT blocks; the KVAF stream uses a full-depth copy. This couples two visual streams without establishing a numerical action policy. e03e05e06e18
- Exchange geometry and scene information
At selected fusion layers, an event MLP derives a shared representation from video and KVAF tokens. Separate outputs predict an event latent and a per-token gate. The gate multiplies cross-attention residuals in both directions. Algorithm 1 updates the KVAF block before fusion and the video block afterward; other layers update independently. e05
6. Experiments & results
EA-WM turns robot kinematics into camera-aligned visual action fields and couples their denoising stream to a Wan2.2 video generator through event-supervised gates. It improves simulated rollout metrics while leaving precise action recovery and deployment unresolved.
6.1 Read the original evidence
Table 1. EA-WM leads the selected six-metric average, with semantic alignment as the exception. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the far-right P3CScore column, then inspect the six underlying columns before accepting the ranking. The table caption defines these scores as normalized to [0,1], with higher values better, and the aggregate as their arithmetic mean multiplied by 100. Compare the EA-WM and CogVideoX rows for the strongest aggregate baseline. Interaction Quality and Trajectory Accuracy address different aspects of generated motion; Instruction Following and Semantic Alignment are also separate columns. This is the main representative-model comparison. The additional KVAF-conditioned row belongs to Table 4 on page 9 and has a different reported score, so it should not be inserted into this table. e07e08e13e18e19
What it supports. EA-WM reports 76.60 versus CogVideoX's 71.08, a gain of 5.52 score points. Interaction Quality rises from 0.594 to 0.682 and Trajectory Accuracy from 0.353 to 0.430. Five columns favor EA-WM; Semantic Alignment is 0.895 versus CogVideoX's 0.898. The pattern supports better robot-centric rollout quality within this selected metric set.
Where the evidence stops. These are generated-video scores, not manipulation success percentages. The paper refers to WorldArena's RoboTwin segmentation but omits evaluation counts, confidence intervals and enough conditioning details to establish identical inference protocols across all rows.
Table 3. Camera-aligned fields retain action cues, but this heuristic inversion loses considerable numerical precision. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Here the direction reverses: smaller errors are better. The three columns measure different action groups and should not be averaged together or assigned invented physical units. Appendix A.3 defines translation and rotation errors as vector norms and gripper error as absolute error against ground-truth action CSVs. The inversion detects a heatmap center and RGB pose axes, solves PnP with known camera parameters and axis length, associates left/right tracks, and converts recovered poses into relative actions. Read the detection-rate footnote together with the errors: nearest-neighbor filling and temporal smoothing supply a dense trajectory despite missed detections. This is a diagnostic procedure, explicitly outside normal training and inference. e12e18
What it supports. KVAF recovery reports translation, rotation and gripper errors of 0.0155, 0.110 and 0.039, compared with 0.004, 0.009 and 0.013 for direct numerical prediction. Detection succeeds at approximately 0.45. The result supports partial recoverability while showing that image-domain alignment is insufficient for this heuristic to match numerical action precision.
Where the evidence stops. The source gives no error units, detection thresholds or median-filter width. Imputation complicates the relation between detection coverage and aggregate error. These values do not measure closed-loop control, and the heuristic is not the model's deployed action decoder.
Table 6. The broader evaluation preserves the aggregate lead while exposing tradeoffs hidden by the selected score. Original paper, p. 17 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the group headers first: the appendix adds visual quality, motion quality and content consistency to the main robot-centric dimensions. Avg16 includes all sixteen columns, including the separately listed Action Following column, and is scaled by 100. It is therefore a different aggregate from P3CScore. Compare EA-WM with CogVideoX for the overall ranking, then compare EA-WM with Wan2.2 for a counterexample to uniform improvement. Keep the paper's abbreviated column labels when their detailed evaluator definitions are unavailable. Table 7 on page 18 repeats the broader evaluation for the separate action-conditioned comparison and should remain a distinct result rather than an extra row here. e15e16e07
What it supports. EA-WM's Avg16 is 60.38 versus CogVideoX's 57.91. However, EA-WM's Action Following is 0.048 versus Wan2.2's 0.051, and its Photo score is 0.220 versus 0.478. Thus the broader aggregate remains favorable while some perceptual dimensions and the separately measured action-following score do not improve over the backbone.
Where the evidence stops. Normalized values from different evaluators are not a common physical-error scale. All columns are declared higher-is-better, but this paper does not provide their full evaluator definitions or uncertainty. An aggregate lead cannot establish improved performance on every downstream use.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| EA-WM video generation on selected WorldArena metrics RoboTwin data and segmentation following WorldArena; Table 1 representative video/world-model comparison. Exact split size is not reported. | 76.60; Interaction Quality 0.682, Trajectory Accuracy 0.430, Depth Accuracy 0.959, Perspectivity 0.838, Instruction Following 0.792, Semantic Alignment 0.895. P3CScore: mean of six normalized, higher-is-better metrics ×100 | CogVideoX: P3CScore 71.08; interaction 0.594, trajectory 0.353, instruction 0.727, semantic alignment 0.898. Reported gain is 5.52 score points; EA-WM leads five of six columns, with semantic alignment below CogVideoX. This evaluates generated video, not task execution. e07e08e18 |
| KVAF and event-aware fusion ablation Same six selected WorldArena metrics; numerical actions replace KVAFs, or EAF is removed while bidirectional attention remains. | Full model 76.60; without KVAFs 70.97; without EAF 74.80; Wan2.2 60.83. P3CScore; higher is better | Full-model gains: 5.63 points over numerical actions and 1.80 over non-event fusion. Supports both components in this setup. The EAF ablation does not separately isolate event supervision from the gating mechanism; uncertainty and seed variation are unreported. e10 |
| KVAF-conditioned generation against action-conditioned baselines Separate Table 4 action/trajectory-conditioned comparison; do not combine its row with Table 1 EA-WM. | 78.13; trajectory 0.494, depth 0.977, perspectivity 0.850. P3CScore; higher is better | CtrlWorld 74.03; IRASim 69.75; Cosmos-Predict 2.5 (action) 66.12; RoboMaster 61.63. Leads all six selected metrics in this comparison. The exact protocol change from the main experiment remains unclear. e13e19 |
| Full WorldArena video perception evaluation Appendix A.4; Table 6 repeats the main comparison, Table 7 the action-conditioned comparison. | EA-WM 60.38 in Table 6; KVAF-conditioned generation 60.84 in Table 7. Avg16: mean of all 16 normalized metrics ×100; higher is better | CogVideoX 57.91 in Table 6; CtrlWorld 59.70 in Table 7. Best reported aggregate in each table does not mean every metric improves. EA-WM Action Following is 0.048 versus Wan2.2 0.051, and its Photo column is 0.220 versus 0.478. e15e16 |
| Heuristic numerical action recovery Generated KVAFs converted to RoboTwin-style relative actions and compared with ground-truth action CSVs; Table 3 and Appendix A.3. | KVAF recovery: 0.0155 / 0.110 / 0.039; detection rate approximately 0.45. Translation/rotation vector-norm errors and gripper absolute error; lower is better; units not stated | Raw-action baseline: 0.004 / 0.009 / 0.013. Visual action information is partly recoverable, but this heuristic is substantially less accurate than direct numerical prediction. Missing detections are imputed; low coverage complicates interpretation. e12 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2. Both the visual action representation and event-aware fusion contribute to the reported aggregate. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read “w/o KVAFs” as a numerical-action replacement, not an experiment with no action information. Read “w/o EAF” as retaining bidirectional cross-attention while removing the event-aware fusion mechanism. These definitions come from Section 4.3 and matter more than the abbreviated row labels. Compare each row with the full model using both P3CScore and individual columns. The largest representation-related change is evident in Trajectory Accuracy, from 0.298 to 0.430. Removing event-aware fusion leaves trajectory relatively closer, at 0.407, while Interaction Quality and Perspectivity fall to 0.656 and 0.800. All six normalized metrics are higher-is-better; the aggregate multiplies their mean by 100. e10e05e06
What it supports. The full model gains 5.63 P3CScore points over numerical action conditioning and 1.80 over fusion without the event-aware mechanism. This is consistent with complementary geometric and interaction benefits. It is not a uniform improvement: the non-event variant's Semantic Alignment is 0.896, slightly above the full model's 0.895.
Where the evidence stops. Removing EAF changes the event-aware mechanism as a whole. It does not isolate supervision from learned gating, and no seed variation or error bars are supplied. A separate gate/EDLS factorial comparison would sharpen the causal claim.
Figure 4. The ranking_block_size example makes trajectory and object-consistency errors visually distinct. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the bottom GT row as the temporal reference and compare vertically at corresponding columns. The upper rows show EA-WM, numerical-action replacement, removal of EAF, and Wan2.2. Track the gripper's path separately from the purple blocks' sizes and arrangement: a plausible route can coexist with inconsistent objects, while stable-looking objects do not guarantee the correct route. Section 4.3 attributes the numerical-action variant's difficulty mainly to grasping trajectory and the non-event variant's difficulty to object consistency. The figure illustrates that explanation with successive images rather than a measured error curve. Neither frame times nor a quantitative contact-error scale appear in the panel. e11e10e18
What it supports. The sequence provides a concrete visual companion to the aggregate ablation. The authors interpret the full model as combining better spatial motion with more consistent block geometry and interactions. The displayed trajectory is compatible with that account, but a single task example cannot determine how frequently either failure mode occurs.
Where the evidence stops. This is one illustrated task, not a distribution of failures or an executed manipulation trial. The GT reference and selected frames support local visual comparisons; they cannot verify forces, contact stability or continuous-time dynamics between frames.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors identify dependence on robot kinematics, calibrated cameras and synchronized logs, with vulnerability to calibration error, occlusion, sensor noise and embodiment shifts. Object/contact state is represented indirectly through fusion and EDLS. e14
The results concern simulation and video perception. Qualitative comparisons and aligned overlays illustrate plausible interactions but establish neither contact dynamics quantitatively nor executed manipulation success. e09e11e17e18
The paper omits evaluation counts, uncertainty and sufficient sampling/conditioning details to establish matched inference protocols across all baselines. Broad superiority should therefore remain bounded to the reported tables. e07e08e13e15e16e19
7.2 Questions for discussion
- Does EDLS improve interaction modeling because of temporal-change supervision, or because learned gates add capacity?
- How robust are camera-aligned fields to calibration drift and unseen embodiments?
- Which inference conditions distinguish the main EA-WM experiment from KVAF-conditioned generation?
8. Reproducibility audit
8.1 Requirements and known gaps
A reproduction requires Wan2.2-TI2V/VAE, paired RoboTwin RGB/state sequences, URDFs, embodiment configuration, camera calibration and WorldArena segmentation/evaluation. Appendix A.2 specifies rendering components and depth normalization, but not numerical heatmap radius/width, axis length, palette or raster thickness. e03e04e07
Unspecified choices include fusion-layer indices, EDLS weight, stage lengths, optimizer/schedule details, training duration, clip resolution/horizon, random seeds and sampler settings. Recovery additionally needs detection thresholds, median-filter width and error units. Resolve these before claiming an exact reproduction. e04e05e06e07e12e19
Proposed checks: isolate EDLS with fixed-gate and unsupervised-gate controls; separately perturb camera calibration while holding RGB/actions fixed. Report per-metric changes and repeated-seed uncertainty, keeping main and KVAF-conditioned protocols distinct. e05e10e14e19
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Disentangle event supervision from the gate
Reader-proposed experiment, not performed: train matched KVAF/video models with (1) the reported event-supervised learned gate, (2) the same learned gate and event-prediction branch but EDLS weight zero, and (3) EDLS retained while fusion gates are fixed to one. Keep data, fusion locations, LoRA rank, stage lengths and compute matched, and use paired seeds. Measure all six selected metrics and Avg16 on the same held-out clips. If model (1) improves Interaction Quality and Perspectivity over both controls reproducibly, that supports the claimed supervision-to-gating mechanism. If (2) matches it, the event targets may not explain the gain; if (3) matches it, learned gate modulation may be unnecessary. Resolve the unspecified EDLS weight and fusion schedule before interpreting this comparison. e05e06e07e10e15
Check 2: Test whether accurate projection is the source of the KVAF advantage
Reader-proposed experiment, not performed: first define the exact Table 4 conditioning protocol, then freeze a trained model and hold RGB observations, instructions and robot trajectories fixed. Render KVAFs with the correct camera calibration, followed by controlled extrinsic perturbations such as 1, 2 and 5 degrees of rotation or 1, 2 and 5 centimeters of translation; these are proposed test levels, not paper settings. Include numerical-action conditioning and a correct-calibration rerender control. Report projected-keypoint displacement together with Trajectory Accuracy, Depth Accuracy and Interaction Quality over repeated generation seeds. A systematic decline with increasing misregistration, approaching the numerical-action baseline, would support dependence on precise camera alignment. Insensitivity would challenge that explanation or suggest the model is ignoring the perturbed fields. e04e10e13e14e19
8.3 Reading coverage
Visual audit: The title/byline and five affiliations were checked on p. 1. Figures 1–10, Tables 1–7, the method equations and Algorithms 1–4 were visually read on the declared original PDF pages. The appendix inspection includes full-metric results, recovery details and low-text generated-sequence pages 19–22. All six final crops were separately viewed; their labels, row/column headers and the recovery detection-rate footnote are retained. Figure 2 was cross-checked against Eqs. (4)–(8) and Algorithms 1–2: gates scale both cross-attention residuals, event differences are supervision, and the unexplained Z-WM heading/inference-input discrepancy is disclosed. Numerical claims in the main-table guide about the separate conditioned setting are supported by inspected p. 9; full-metric comparisons include inspected pp. 16–18. Reference-only pp. 10–11 were read as text. No external code, separate supplement or benchmark evaluator implementation was inspected, and no experiments were run.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; Sections 1–2.2: introduction and related work (pp. 1–4)
- Sections 3–3.2: method, KVAFs and event-aware fusion (pp. 4–6)
- Sections 4–4.4: experiments, ablations and additional analyses (pp. 6–9)
- Sections 5–6: limitations and conclusion (p. 9)
- References (pp. 10–11)
- Appendix A.1: architecture and Algorithms 1–2 (pp. 12–13)
- Appendix A.2: KVAF construction, Algorithm 3 and Table 5 (pp. 13–15)
- Appendix A.3: heuristic recovery and Algorithm 4 (pp. 14–17)
- Appendix A.4: full-metric evaluation and Tables 6–7 (pp. 16–18)
- Appendices A.5–A.6: visualizations and broader impacts, including Figures 6–10 (pp. 16–22)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Pages with little extracted text require visual inspection: 19, 20, 21, 22
- Identity/version note: the title page verifies arXiv:2605.06192v1, 7 May 2026, with the exact catalog title and the same five authors. PDF reading order places Kai Chen before Cong Huang; the catalog reverses them. The PDF additionally lists DeepCybo, omitted from the catalog affiliation string. These are metadata discrepancies, not evidence of a different work or later revision.
- All five supplied text chunks were read individually without truncation. All substantive PDF pages and all figures/tables were visually inspected; the reference-only pages 10–11 were read as text.
- Source-manifest caution: text extraction does not reconstruct figure images. This was addressed by inspecting original PDF pages and final crops.
- Separate supplemental material availability has not been fully verified; no separate supplement was supplied.
- Source-manifest caution: pages 19–22 contain little extracted text. All four pages were rendered and visually inspected as generated KVAF/video sequences.
- Code and external benchmark documentation were not inspected; no experiments were reproduced. Earlier retained HTML was not separately read or treated as an additional scientific source.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, byline, affiliations and arXiv margin
Exact title; authors Zhaoyang Yang, Yurun Jin, Lizhe Qi, Kai Chen, Cong Huang; five affiliations including DeepCybo; arXiv:2605.06192v1, 7 May 2026.
Go to primary source ↓e02PDF pp. 1–3, Abstract, Section 1 and Figure 1
Authors motivate camera-grounded action representations for robot-video geometry and interactions, contrasting compact numerical conditioning.
Go to primary source ↓e03PDF p. 4, Figure 2 and Section 3
Wan2.2-TI2V uses RGB and KVAF branches with shared VAE encoding and event fusion. Figure labels text/first frame as inference inputs, marks VAE/video head frozen, and uses the inconsistent heading Z-WM architecture; caption names EA-WM.
Go to primary source ↓e04PDF p. 5, Section 3.1, Eqs. (1)–(3); pp. 13–15, Appendix A.2, Algorithm 3 and Table 5
URDF/state/camera inputs support bimanual forward kinematics, projection and five rendering components; episode depth normalization is given. Object states are not directly rasterized.
Go to primary source ↓e05PDF p. 5, Section 3.2, Eqs. (4)–(6); pp. 12–13, Appendix A.1, Algorithms 1–2
Full-depth KVAF DiT copy; sparse fusion; event representation produces gate and latent prediction; gate multiplies both cross-attention residuals. Algorithm 1 fixes the KVAF/fusion/video update order.
Go to primary source ↓e06PDF p. 6, Eqs. (7)–(8) and preceding/following text; pp. 12–13, Appendix A.1
Zero initial difference and absolute consecutive RGB differences form VAE event targets. Both streams have flow losses plus layer-averaged EDLS; first RGB frame excluded from video loss.
Go to primary source ↓e07PDF p. 6, Section 4.1
Two-stage LoRA/fusion training; rank 32, learning rate 8×10⁻⁵, 32 H100 GPUs and batch 32; RoboTwin data/segmentation follows WorldArena. Training duration and detailed split sizes are not given.
Go to primary source ↓e08PDF p. 6, Table 1, EA-WM and CogVideoX rows, all six metrics/P3CScore; Section 4.2
EA-WM 76.60 versus CogVideoX 71.08; five selected columns improve, while semantic alignment is 0.895 versus 0.898. All selected metrics normalized to [0,1], higher better.
Go to primary source ↓e09PDF p. 7, Figure 3 and Section 4.2 qualitative analysis
Four RoboTwin 2.0 tasks compare EA-WM, Wan2.2 and GT at corresponding temporal steps; red boxes mark local interaction regions.
Go to primary source ↓e10PDF pp. 7–8, Section 4.3 and Table 2, all variant rows
Numerical-action replacement scores 70.97, removal of EAF retaining bidirectional attention 74.80, full model 76.60; trajectory 0.298/0.407/0.430; depth 0.884/0.947/0.959; semantic alignment 0.889/0.896/0.895.
Go to primary source ↓e11PDF pp. 7–8, Section 4.3 and p. 8 Figure 4
ranking_block_size sequences compare full model, without KVAFs, without EAF, Wan2.2 and GT. Authors describe trajectory versus object-consistency failure modes.
Go to primary source ↓e12PDF pp. 8–9, Section 4.4 and Table 3; pp. 14–17, Appendix A.3 and Algorithm 4
Recovery is analysis-only. PnP, track association, nearest-neighbor filling, median filtering and relative-action conversion precede vector-norm/absolute errors. KVAF errors 0.0155/0.110/0.039 versus 0.004/0.009/0.013, with detection rate about 0.45; units and thresholds are unspecified.
Go to primary source ↓e13PDF p. 9, Table 4 and KVAF-conditioned video generation paragraph
Separate conditioned-generation row scores 78.13 versus CtrlWorld 74.03, IRASim 69.75, Cosmos action 66.12 and RoboMaster 61.63; leads all six selected columns.
Go to primary source ↓e14PDF p. 9, Section 5
Requirements and limitations cover calibration, synchronized logs, robot kinematics, noise, occlusions and embodiment shifts; object/contact-aware fields are future work.
Go to primary source ↓e15PDF p. 16, Appendix A.4; p. 17, Table 6, Avg16, Action and Photo columns
Avg16 averages 16 metrics including Action Following: EA-WM 60.38, CogVideoX 57.91. EA-WM/Wan2.2 Action scores 0.048/0.051 and Photo scores 0.220/0.478 show that aggregate gains are not universal.
Go to primary source ↓e16PDF p. 18, Table 7, KVAF-conditioned generation and CtrlWorld rows
Full-metric conditioned comparison reports Avg16 60.84 versus 59.70. KVAF Action Following 0.048 remains below IRASim 0.053.
Go to primary source ↓e17PDF p. 9, Figure 5; p. 16, Appendix A.5; pp. 18–22, Figures 6–10
Generated sequences and colored KVAF overlays illustrate image-domain alignment across manipulation scenes; panels lack per-sequence error statistics.
Go to primary source ↓e18PDF pp. 16–17, Appendix A.6
Authors explicitly state simulation-only evaluation, no direct action execution and no real-robot deployment; downstream planning and policy uses are prospective.
Go to primary source ↓e19PDF p. 4, Figure 2 versus Section 3; p. 9, Section 4.4/Table 4; pp. 12–13, Appendix A.1
Section 3 includes action sequences in the generation goal, while the diagram inference box only names text/first frame. The appendix supplies training/fusion algorithms, but no sampler or explicit protocol switch explaining Table 1 versus Table 4.
Go to primary source ↓8.5 Primary sources
EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fields ↗
PDF · 8,658 extracted words
Source fingerprint
a37cec6fb4aa03ef872bdfd6e2616a98583b3cc4a400767832c6e22de9a92cf2