FlowWAM: Optical Flow as a Unified Action Representation for World Action Models
1. Paper overview
In one sentence: Encoding motion as a flow video lets one visual generator support control and prediction, while a separate action expert and carefully constructed flow targets remain essential. e-motivatione-architecturee-modese-experte-preprocesse-ablation
| At a glance | What to know |
|---|---|
| Research problem | Author claim How can a pretrained video generator represent actions in its native visual format while retaining motion information useful for control? The authors argue that numerical actions create a modality gap and static image-space cues underspecify cross-frame displacement. e-motivation |
| Core mechanism | Source description HSV-encoded optical flow serves both as a policy prediction target and as a world-model conditioning input inside the RGB video generator. e-encodinge-modes |
| A key reported result | RoboTwin 2.0 manipulation: With pretraining: 92.94 Clean / 92.14 Random. Mean task success rate (%). All 50 tasks; jointly trained Clean/Random demonstrations; 100 evaluation rollouts per task per setting. Fast-WAM: 91.88 / 91.78. FlowWAM without EgoDex pretraining: 82.40 / 80.80. Calculated improvements over Fast-WAM are 1.06 / 0.36 percentage points; pretraining adds 10.54 / 11.34 points. Baselines retain their own pretraining protocols, so this is not a compute-matched representation comparison. e-robotwin-protocole-robotwin-results |
| Reading caution | Reader analysis Robot-only head-view supervision excludes object motion and wrist-flow dynamics. Combined with clipping and thresholding, this narrows what the supposedly unified flow representation actually preserves. e-preprocesse-encoding |
Core contributions
- Source description
HSV-encoded optical flow serves both as a policy prediction target and as a world-model conditioning input inside the RGB video generator. e-encodinge-modes
- Source description
Action-unlabeled human video pretraining learns visual motion before a separate action expert is attached for robot supervision. e-pretraine-expert
Figure 2. One shared visual generator, two flow roles, and a separate path to robot control. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the upper arrows from the RGB and flow patch embeddings into joint self-attention. Separate RoPE labels indicate positional encoding for each stream; they do not indicate separate transformer backbones. The black feature arrow leads toward the action expert. The caption and Sections 3.3/A.1 clarify that its actual inputs are per-layer RGB/flow hidden states. In the lower-left panels, the grayed expert is inactive during video-only pretraining and attached for joint robot training. In the lower-right panels, policy inference denoises both streams, whereas world-model inference receives clean flow and generates RGB only. Thus the colored flow thumbnails switch from prediction targets to conditioning inputs. e-architecturee-modese-experte-objectives
What it supports. The architecture makes the flow stream useful during inference, beyond an auxiliary training loss. However, the numerical controller remains a distinct expert with its own action flow-matching process. Sharing RGB/flow transformer blocks supports visual alignment; it does not make action decoding disappear or establish that the expert needs flow alone.
Where the evidence stops. The schematic omits proprioceptive and expert text conditioning described in Appendix A.1. Its blank first flow thumbnail should not independently establish zero-flow initialization: Section 3.2 specifies clean reference conditioning. Read the schematic together with these descriptions.
2. Motivation
2.1 The problem and the proposed response
How can a pretrained video generator represent actions in its native visual format while retaining motion information useful for control? The authors argue that numerical actions create a modality gap and static image-space cues underspecify cross-frame displacement. e-motivation
2.2 What this reading follows
A robot command is useful to a controller, but a pretrained video generator has learned to model images changing over time. FlowWAM connects these domains by turning optical flow into color images and generating them alongside RGB futures. The key distinction is whether flow is unknown and must be predicted, or is supplied as a desired motion condition. A separate expert converts predicted visual features into executable actions. Read the architecture first, then compare simulated execution, video prediction and physical execution as different kinds of evidence. The ablations probe why the representation helps, while the appendix reveals the preprocessing assumptions that make the interface work. e-motivatione-architecturee-modese-experte-preprocesse-ablation
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | IDM |
| Quadrant | Q4 · Dual-system × IDM |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded Dual-system × IDM classification is defensible at the control-pipeline level: a video/flow generator supplies predicted-future features to a distinct action expert that recovers low-level controls. RGB and flow themselves share one DiT; two streams do not constitute two systems. The IDM label describes feature-conditioned inverse decoding, not an explicitly named two-frame IDM. Joint training does not remove the separate decoder. e-architecturee-modese-expert
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Understand what the colored motion image preserves
Optical flow records two-dimensional displacement between adjacent frames. Equation (1) turns its direction into hue and magnitude into saturation, with value fixed, allowing the frozen image VAE to encode motion without a new numerical-action tokenizer. That compatibility is the architectural motivation, but the appendix changes how literally one should interpret a dense motion plan. EgoDex uses flow extracted from ordinary video; RoboTwin uses robot-only replay, with flow supervision restricted to the head camera and constant wrist placeholders. Magnitudes are capped at 25 pixels and displacements below 0.5 pixels disappear. As a reader's deduction, the representation is therefore a filtered embodiment-motion signal, not a lossless description of every robot–object interaction. The stated invertibility applies to the idealized encoding, not arbitrary original displacements after preprocessing. e-encodinge-preprocesse-architecture
5.2 Follow prediction through to the next executed action
In policy mode, clean reference conditioning anchors a future in which both RGB and flow latents must be generated. The action expert reads intermediate features from both streams at corresponding transformer layers, alongside language and current proprioception. Appendix A specifies a 32-step action chunk decoded from nine pixel frames at temporal stride four; RoboTwin evaluation executes a 25-step window before replanning. Training the expert exclusively on clean encoded demonstrations would expose it to a different feature distribution from generated futures. Equation (2) addresses that mismatch by corrupting the visual latents on half of training steps and supplying the sampled noise level. Optical flow is the motion representation here, while flow matching is the learning and sampling procedure used for visual and action predictions; the two uses of the word flow describe different objects. e-modese-experte-objectivese-config
5.3 Separate evidence for execution, prediction and mechanism
The three evaluations answer different questions. RoboTwin tests executed actions across trained bimanual tasks under canonical and randomized scenes. The physical suite tests execution after task-specific demonstrations on two real platforms. WorldArena tests generated videos under a supplied trajectory, with an aggregate mixing appearance, motion, geometry and controllability metrics. A higher trajectory score therefore cannot be read as a higher robot success rate. Figure 4 provides a more focused representation comparison, but its WorldArena validation set differs from the main benchmark and its policy full-model score has an unspecified relationship to the headline setting. My interpretation is that these results jointly support a useful visual motion interface, while leaving stronger claims unresolved: unique recovery of actions from flow, fully isolated causal benefit, and long-horizon planning were not established. e-robotwin-protocole-reale-world-protocole-world-metricse-ablatione-experte-future
5.4 Training and inference
During training
Stage 1 updates the dual-stream DiT on EgoDex using the video objective. Videos are sampled at 15 fps, resized to 320×256, with frame buckets {17,33,49,65,81}. Stage 2 jointly updates the DiT, flow pathway and action expert on RoboTwin. The VAE and text encoder remain frozen. e-pretraine-config
RGB and flow use a shared sampled diffusion time and velocity MSE losses. The first latent frame is conditioned and loss-masked, with small Gaussian conditioning augmentation. Motion reweighting emphasizes changing flow regions. Half the action-expert training steps use stochastically corrupted visual latents and a noise-level embedding to reduce mismatch with generated inference features. e-objectives
Stage 2 combines 50 Clean and 500 Random demonstrations per task. Clean fixes layout and lighting; Random varies object poses, distractors, lighting and background. This evaluates robustness within a trained task suite, not unseen-task transfer. e-robotwin-protocol
During inference
Policy inference jointly denoises initially noisy future RGB and flow latents under clean reference conditioning, then decodes actions. RoboTwin uses 9 pixel frames at temporal stride 4, a 32-step action chunk, 25 video and 50 action denoising steps, and a 25-step execute-and-replan window. e-modese-config
World-model inference holds the desired flow latents fixed and denoises RGB only. This renders a requested motion trajectory; the reported evaluation does not establish a deployed search or planning algorithm. e-modese-world-protocol
5.5 Implementation flow
- Construct motion targets
RAFT extracts flow from EgoDex videos. RoboTwin instead replays recorded joint actions with the robot URDF in SAPIEN, renders only the embodiment against a static background, and extracts robot-only flow. Head-camera flow is supervised; wrist regions contain constant placeholders. e-preprocess
- Share the visual generator
RGB and HSV-flow videos use the same frozen VAE. Separate patch embeddings and output heads feed shared transformer blocks; independently positioned RGB/flow tokens interact through joint self-attention. Appendix A.1 also specifies a learnable flow-token identity embedding. e-architecturee-expert
- Decode a motion plan
A separate AdaLN action transformer cross-attends to paired, per-layer RGB/flow hidden states and instruction tokens, with proprioception appended to the instruction context. It predicts action flow-matching velocity. Thus actions depend on both visual streams and robot state, rather than a numerical inversion of flow alone. e-expert
6. Experiments & results
FlowWAM generates RGB futures and optical-flow plans with a shared video transformer, then decodes their hidden states into robot actions. Providing flow instead enables controlled video prediction. Its strongest evidence combines manipulation success, improved trajectory scores and representation ablations, with important preprocessing and evaluation qualifications.
6.1 Read the original evidence
Table 1. Pretraining substantially improves FlowWAM; the margin over the strongest reported baseline is smaller. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each method's Clean and Rand. columns as two evaluation settings of models trained using both demonstration sets. PT denotes action-unlabeled EgoDex pretraining. The displayed task rows are only a representative subset; the Average row covers all 50 tasks, whose complete values appear in Table 4 on page 19. Start with the two rightmost FlowWAM groups to measure the effect of pretraining within the method, then compare pretrained FlowWAM with Fast-WAM. Each setting uses 100 evaluation rollouts per task, following training with 50 Clean and 500 Random demonstrations per task. These are simulated action-execution success percentages. e-robotwin-resultse-robotwin-protocole-pretrain
What it supports. FlowWAM moves from 82.40 to 92.94 in Clean and from 80.80 to 92.14 in Random: calculated gains of 10.54 and 11.34 percentage points. Against Fast-WAM's 91.88 and 91.78, the gains are 1.06 and 0.36 points. The large pretraining effect and small top-baseline margin answer different comparison questions.
Where the evidence stops. All WAM baselines retain their own pretraining protocols, so the table does not isolate representation under equal data and compute. No confidence intervals or training-seed variability are reported, and high aggregate success does not imply winning every task.
Table 2. Dense flow conditioning improves trajectory fidelity in the reported video-prediction benchmark. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with Cond., which identifies the information supplied to each generator: text, numerical actions, image-space actions or flow. Then inspect Traj. Acc. before the aggregate EWMScore. Appendix D.2 defines trajectory accuracy through SAM3 robot-arm detections and normalized dynamic time warping against reference execution. EWMScore averages 16 normalized metrics, although this compact table displays only six components; Table 5 on page 20 supplies the full profile. The protocol generates 121 frames at 24 fps from an initial frame, instruction and recorded joint trajectory. This evaluates how a generated video follows prescribed motion, separately from whether a robot policy completes a task. e-world-protocole-world-resultse-world-metricse-preprocess
What it supports. The trajectory score rises from GigaWorld-1's 54.27 to 64.26, a calculated 9.99-point increase, approximately 18.4% relative. FlowWAM also has the highest reported EWMScore, 63.71, versus 62.63 for ABot-PhysWorld (text). Subject and background consistency remain competitive, but FlowWAM does not lead those columns.
Where the evidence stops. Different conditioning interfaces and model families prevent a fully controlled causal comparison. Appendix Table 5 reports Action Following at only 3.50; that separate instruction-diversity metric should not be conflated with trajectory fidelity. The exact WorldArena action-to-flow conversion remains underspecified.
Figure 3. Physical execution supports the policy claim, with task-dependent gains and a towel-folding tie. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the legend to compare peach π0.5, green Motus and yellow FlowWAM bars within each task. The first four groups are single-arm Franka tasks; the last three are dual-arm ARX tasks. Appendix E, on pages 21–22, identifies the platforms and task definitions. All methods use the same 100 demonstrations per task, followed by 10 randomized-pose trials, so each ten-point bar increment corresponds to one trial. Notice Put in Plate at 100% and Fold Towel at 40%. The latter ties π0.5 rather than giving FlowWAM a strict win. Photographs identify scenes; the measured outcomes are the bars. e-real
What it supports. The authors report mean success of 75.7% for FlowWAM, 61.4% for π0.5 and 57.1% for Motus. This is direct physical execution evidence after task-specific training on both platforms. It complements simulated success and establishes a different outcome from attractive or trajectory-aligned generated video.
Where the evidence stops. The prose's larger dual-arm advantage is baseline-dependent: calculated gains over π0.5 average 17.5 points on single-arm tasks and 10 on dual-arm tasks. Only ten trials per task are reported, without uncertainty estimates; broad coordination or zero-shot transfer claims exceed this evidence.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| RoboTwin 2.0 manipulation All 50 tasks; jointly trained Clean/Random demonstrations; 100 evaluation rollouts per task per setting. | With pretraining: 92.94 Clean / 92.14 Random. Mean task success rate (%) | Fast-WAM: 91.88 / 91.78. FlowWAM without EgoDex pretraining: 82.40 / 80.80. Calculated improvements over Fast-WAM are 1.06 / 0.36 percentage points; pretraining adds 10.54 / 11.34 points. Baselines retain their own pretraining protocols, so this is not a compute-matched representation comparison. e-robotwin-protocole-robotwin-results |
| WorldArena action-conditioned video prediction Initial image, instruction and recorded joint trajectory; 121-frame rollouts at 24 fps; main benchmark results. | EWMScore 63.71; Trajectory Accuracy 64.26. EWMScore and Trajectory Accuracy (0–100; higher is better) | ABot-PhysWorld (text) has EWMScore 62.63; GigaWorld-1 has trajectory score 54.27. Trajectory gain is 9.99 score points, or approximately 18.4% relative. EWMScore averages 16 metrics; trajectory accuracy measures alignment of detected arm trajectories. Neither is robot execution success. e-world-protocole-world-resultse-world-metrics |
| Physical-robot manipulation Seven Franka/ARX tasks; 100 shared teleoperated demonstrations and 10 randomized-pose trials per task. | 75.7. Mean task success rate (%) | π0.5: 61.4; Motus: 57.1. Fold Towel ties π0.5 at 40%. Physical execution supports transfer to both platforms after task-specific training. Ten trials per task and absent uncertainty estimates limit precision; this is not zero-shot embodiment transfer. e-real |
| Policy representation and training ablation Figure 4(a), RoboTwin ablation; its 89.8 full-model score is distinct from the main comparison. | Full 89.8; numerical actions 69.8; raw flow 72.3; no reweighting 83.9; no stochastic expert conditioning 82.1. Success rate (%) | Removing stochastic conditioning loses 7.7 percentage points within this ablation. The results support compatible flow encoding and robust expert conditioning. The exact policy-ablation split and the source of its different full-model score are unspecified. e-ablation |
| World-model conditioning ablation Custom WorldArena validation split; not the official main-table evaluation. | Full 65.23; text 49.31; numerical actions 54.18; raw flow 56.72; image masks 57.84. EWMScore | Full HSV-flow exceeds image masks by 7.39 score points. The authors hold the video generator fixed while varying conditioning. The 65.23 value must remain separate from the main benchmark's 63.71. e-ablation |
| Flow quality versus policy success 50 RoboTwin tasks; RAFT pseudo-ground-truth from 50 newly generated demonstrations per task. | r = −0.81 between flow error and success. Pearson correlation | Lower flow error accompanies higher task success. Task-level association supports the motion-plan hypothesis but does not exclude task difficulty or decoder shortcuts as explanations. e-correlation |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 4(a–c). Representation, loss weighting and noisy expert conditioning all matter in the reported ablations. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read panel (a) within its own policy-ablation setting: the full score is 89.8, and removing stochastic action-expert conditioning yields 82.1. Panel (b) changes the world-model conditioning interface; HSV flow reaches 65.23 versus 57.84 for masks. Its superscript 1 refers to the footnote on page 8: these scores use a custom validation split because official WorldArena scoring requires remote submission and does not provide local ground-truth video. Finally, panel (c) places task success on the horizontal axis and flow error in pixels on the vertical axis. Both the visible annotation and accompanying text specify a negative correlation, r = −0.81. e-ablatione-objectivese-correlatione-experte-world-resultse-robotwin-results
What it supports. The policy variants support both a video-compatible flow representation and training that exposes the expert to imperfect visual latents. The world-model variants favor HSV flow over masks and raw displacement tensors. Across tasks, better flow predictions accompany better execution, providing converging evidence for the motion-plan interpretation without proving its causal exclusivity.
Where the evidence stops. Do not merge 65.23 with the official-table 63.71, or 89.8 with the main policy averages. The policy-ablation split is not specified. Cross-task correlation against RAFT pseudo-labels cannot rule out task difficulty, RGB cues or decoder shortcuts.
7. Analysis & limitations
7.1 What the evidence leaves open
Robot-only head-view supervision excludes object motion and wrist-flow dynamics. Combined with clipping and thresholding, this narrows what the supposedly unified flow representation actually preserves. e-preprocesse-encoding
The action expert sees RGB, flow, language and proprioception. Ablations and correlation therefore do not establish that flow alone uniquely determines robot actions or causes all gains. e-experte-ablatione-correlation
The paper proposes internet-scale action-free pretraining and longer-horizon flow planning as future work; the reported study remains simulation and laboratory manipulation. e-future
7.2 Questions for discussion
- Does the expert still follow the motion plan when only flow features are perturbed while RGB and proprioception remain fixed?
- How much of the world-model advantage persists under a fully specified, shared action-to-visual-conditioning pipeline?
8. Reproducibility audit
8.1 Requirements and known gaps
The reported setup uses Wan2.2-TI2V-5B, a frozen Wan2.2 VAE and UMT5-XXL, plus an approximately 780M-parameter, 30-layer action expert. Training uses 32 H100 GPUs across four nodes, AdamW with weight decay 0.01, stage learning rates 5×10⁻⁵ / 10⁻⁴ and per-GPU batches 1 / 16. e-experte-config
Reconstruction requires camera tiling, robot URDF replay, RAFT preprocessing and proprioceptive normalization. Training duration, EgoDex sample count, exact wrist-placeholder values, normalization statistics, software versions and measured inference latency are not specified in the implementation appendix. e-preprocesse-confige-expert
WorldArena's exact action-to-flow conversion, training/adaptation configuration and custom validation membership remain unclear. Physical-robot flow preprocessing is also not detailed separately. These gaps require resolution before a faithful comparison, despite the documented evaluation horizons and trial counts. e-world-protocole-ablatione-preprocesse-real
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Intervene on the expert's flow features
Reader-proposed check, not performed: compare experts trained with and without the paper's stochastic latent conditioning using matched demonstrations, initialization, update budgets and repeated seeds. On the same held-out RoboTwin episodes, evaluate clean demonstration features, normally generated features and generated features whose flow portion is temporally shuffled within the episode. Hold the RGB features, instruction and proprioception fixed for the shuffled condition. Measure action error and executed success under the same replanning window. Greater degradation on generated features for the clean-only expert would support the noise-mismatch explanation. Little effect from shuffling flow would weaken the claim that the expert actually relies on the predicted motion plan. e-experte-objectivese-confige-ablatione-correlation
Check 2: Control the information supplied to world-model variants
Reader-proposed check, not performed: first publish a held-out validation split and an explicit action-to-flow pipeline using only the recorded joint trajectory, robot geometry and camera calibration. Compare HSV flow, raw displacement and embodiment masks derived from the same robot-only replay, with matched generator initialization, training budget, prompts and rollout horizon. Reserve reference future RGB exclusively for evaluation. Report the full WorldArena metric profile and repeat the comparison with temporally perturbed flow while keeping the initial observation fixed. A trajectory advantage that survives this control would strengthen the representation claim; disappearance after matching information, or insensitivity to incorrect flow, would weaken it. Keep custom-split scores separate from official submissions. e-world-protocole-world-metricse-ablatione-preprocesse-modes
8.3 Reading coverage
Visual audit: Inspected the title/author/version page; Figures 1–10; Tables 1–5; Equations (1)–(5); method, training, preprocessing, hyperparameter, evaluation and platform pages. All five final crops were individually viewed at their original extracted dimensions after rendering directly from the supplied PDF at 300 DPI. Figure 2 arrows and inactive expert blocks were checked against its caption, Sections 3.2–3.4 and Appendix A.1; the expert consumes hidden states and additional conditioning omitted from the schematic. Figure 4's correlation is visibly negative. Its footnote marker is retained; the explanatory footnote is on inspected page 8. The Fold Towel tie and baseline-dependent single-/dual-arm margins were checked against Figure 3. Reference-only pages 11–13 were read in the supplied text but not visually rendered. No separate supplement, code, linked website or independent experimental run was inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title page and Abstract
- 1 Introduction
- 2 Related Work
- 3 Method: 3.1–3.4
- 4 Experiments: 4.1–4.4
- 5 Conclusion
- References
- A Implementation Details and Training Pipeline: A.1–A.5
- B Baseline Details: B.1–B.2
- C Additional RoboTwin Results
- D WorldArena Evaluation Details: D.1–D.3
- E Real-World Experimental Setup: E.1–E.2
- F Visualization Results: F.1–F.3
- G Broader Impacts
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Reviewed the supplied arXiv:2607.13017v1 preprint, dated 14 July 2026. The observed title and all 16 authors match the catalog. The title page additionally lists FiveAges and MBZUAI, omitted from the catalog affiliation string. No other revision was supplied or compared.
- All eight supplied text chunks were read individually, including references and appendices. Text extraction does not reconstruct figure images; this omission was addressed by inspecting the retained PDF's figures, tables and relevant equation layouts.
- Separate supplemental material availability has not been fully verified; none was supplied.
- Code, project websites and external resources were not inspected. No paper scripts were executed and no experiments were reproduced. Reference-only pages 11–13 were read as text but not rendered.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author/affiliation block and arXiv margin
The title matches FlowWAM; the margin identifies arXiv:2607.13017v1, 14 July 2026. Sixteen authors are credited, with FiveAges and MBZUAI among five numbered affiliations.
Go to primary source ↓e-motivationPDF pp. 2–3, Introduction, Figure 1 and Related Work
The authors motivate dense, video-formatted displacement as a bridge between video priors, action prediction and motion-conditioned generation.
Go to primary source ↓e-architecturePDF p. 4, Figure 2 and Section 3.2, Dual-Stream Architecture
Shared frozen VAE and transformer blocks process separate RGB/flow streams, with independent patch embeddings, output heads and RoPE, and concatenated joint-attention tokens.
Go to primary source ↓e-encodingPDF p. 4, Section 3.2, Equation (1); p. 15, A.3–A.4, HSV Encoding
Hue encodes displacement direction, saturation magnitude, and value is fixed. The idealized text calls encoding invertible; the implementation caps magnitude at 25 px and thresholds below 0.5 px.
Go to primary source ↓e-modesPDF pp. 4–5, Section 3.2, Two Operating Modes; Figure 2
Policy sampling generates both future latent streams. World-model sampling fixes clean desired flow and denoises RGB. The first latent frame of each stream uses clean reference conditioning.
Go to primary source ↓e-expertPDF p. 5, Section 3.3; p. 14, A.1 Architectural Components
The approximately 780M, 30-layer AdaLN expert reads paired DiT-layer RGB/flow features and instruction context with normalized 14D proprioception; it predicts action velocity. A.1 identifies Wan2.2-TI2V-5B and frozen VAE/text modules and adds a learnable flow-token embedding.
Go to primary source ↓e-objectivesPDF p. 5, Sections 3.3–3.4, Equations (2)–(4); p. 6, Equation (5)
Half of expert training steps corrupt visual latents. RGB and flow velocity MSEs share a sampled time; conditioning frames are loss-masked and augmented. Motion-aware weighting modifies flow loss; labeled data add independently scheduled action flow matching.
Go to primary source ↓e-pretrainPDF p. 14, A.2 Two-Stage Training Strategy; p. 15, A.3 EgoDex Clip Sampling
EgoDex trains the video generator without robot labels before joint robot training; clips use 15 fps, 320×256 resolution and frame buckets 17, 33, 49, 65 and 81, without additional task-level filtering.
Go to primary source ↓e-preprocessPDF p. 15, A.3 Optical-Flow Extraction, Multi-View Flow Targets and A.4
EgoDex uses adjacent scene frames; RoboTwin uses RAFT on robot-only SAPIEN replays. Only the head-view flow is supervised, with constant wrist placeholders. Head and wrist RGB views form a 320×384 T-shaped tile.
Go to primary source ↓e-configPDF p. 14, A.2 Stage 2; p. 16, A.5 and Table 3
The appendix specifies 9 pixel frames, stride 4, 32 action steps, loss weights 0.1/1.0 and motion boost 2.0; 32 H100 GPUs on four nodes; AdamW and stage rates/batches; 25/50 video/action denoising steps and a 25-step replanning window.
Go to primary source ↓e-robotwin-protocolPDF p. 7, Section 4.1 Setup; p. 15, A.3 RoboTwin Demonstrations
Fifty tasks use 50 Clean plus 500 Random demonstrations per task in joint training, then 100 rollouts per task per setting. Compared WAMs retain their respective pretraining protocols.
Go to primary source ↓e-robotwin-resultsPDF p. 6, Table 1, Average row; p. 19, Table 4, full task rows and Average row
FlowWAM with pretraining reports 92.94/92.14 Clean/Random, without 82.40/80.80, versus Fast-WAM 91.88/91.78. Table 4 supplies all 50 rows; their pretrained FlowWAM means agree with the reported averages.
Go to primary source ↓e-world-protocolPDF p. 7, Section 4.2 Setup; p. 18, D.1 Metric Grouping and Aggregate Scoring
WorldArena uses initial frame, instruction and recorded joint-action trajectory for 121-frame, 24-fps predictions. EWMScore is the arithmetic mean of 16 metrics normalized to 0–100.
Go to primary source ↓e-world-resultsPDF p. 7, Table 2, FlowWAM/ABot-PhysWorld/GigaWorld-1 rows; p. 20, Table 5
FlowWAM EWMScore is 63.71 and trajectory accuracy 64.26; ABot-PhysWorld text EWMScore is 62.63 and GigaWorld-1 trajectory accuracy 54.27. FlowWAM depth accuracy is 98.97, subject consistency 82.46 and background consistency 89.97; Table 5 gives its Action Following score as 3.50.
Go to primary source ↓e-world-metricsPDF p. 20, D.2 Physics and 3D Accuracy; p. 21, D.2 Controllability
Trajectory Accuracy uses SAM3 arm bounding boxes and normalized dynamic time warping. Action Following measures CLIP-feature dissimilarity among generated videos for different instructions sharing an initial frame; it is distinct from reference-trajectory fidelity.
Go to primary source ↓e-realPDF p. 8, Figure 3 and Section 4.3; pp. 21–22, Appendix E
Seven tasks on Franka Research 3 and ARX 5 use 100 demonstrations and 10 trials each. FlowWAM/π0.5/Motus average 75.7/61.4/57.1%; Fold Towel is 40/40/30%. Figure 3 gives FlowWAM 90/90/100/60 on four single-arm tasks and 40/90/60 on three dual-arm tasks.
Go to primary source ↓e-ablationPDF p. 8, Section 4.4 and footnote 1; p. 9, Figure 4(a–b) and Section 4.4 continuation
Policy full/numerical/raw/no-reweighting/no-stochastic-conditioning scores are 89.8/69.8/72.3/83.9/82.1. World full/text/numerical/raw/mask scores are 65.23/49.31/54.18/56.72/57.84; footnote 1 identifies a custom validation split due to remote evaluation and unavailable local GT videos.
Go to primary source ↓e-correlationPDF p. 9, Figure 4(c) and Section 4.4 Flow decodability
Both the plot and prose report Pearson r=−0.81 across 50 tasks, comparing flow prediction error against RAFT pseudo-ground-truth from 50 new demonstrations per task with policy success.
Go to primary source ↓e-qualitativePDF p. 9, Figure 5; pp. 22–23, Appendix F; pp. 24–27, Figures 7–10
The figures juxtapose model RGB/flow predictions with simulated and physical policy rollouts and show additional flow-conditioned world-model examples. These are selected qualitative examples rather than extra trials or uncertainty estimates.
Go to primary source ↓e-futurePDF p. 10, Section 5 Conclusion; p. 23, G Broader Impacts
Internet-scale action-free pretraining and longer-horizon flow planning are future directions; the study is framed as simulated and laboratory-scale manipulation.
Go to primary source ↓8.5 Primary sources
FlowWAM: Optical Flow as a Unified Action Representation for World Action Models ↗
PDF · 12,324 extracted words
Source fingerprint
a4dea6542be553e1bda65617085861650dab686b84014d6f50473f6d432a0283