OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining
1. Paper overview
In one sentence: OpenWAM-α couples a video prior and dedicated action generator through joint denoising, improving embodied transfer while retaining pronounced weaknesses under visual disturbance. problemcompositionalphatransfermixturesimulation-protocolsingle-armlibero-plus
| At a glance | What to know |
|---|---|
| Research problem | Source description Video pretraining supplies abundant visual dynamics knowledge, while executable robot demonstrations are scarce. Coupled WAM implementations obscure whether transfer comes from the backbone, representation, action capacity, information flow or data mixture. OpenWAM exposes these choices to controlled comparison. problem |
| Core mechanism | Source description OpenWAM-Infra composes encoders, backbones and masks independently of training and serving. Architectures cover shared transformers, separate action/video streams and an added understanding stream. composition |
| A key reported result | Single-arm physical manipulation: 99/120 (82.5%). Task success rate. Six Franka-Research-3 tasks; 100 demonstrations and 20 trials per task after SFT. LingBot-VA 93/120 (77.5%); π0.5 66/120 (55.0%). A five-percentage-point aggregate lead over LingBot-VA, with lower ring-stacking success (12/20 versus 15/20). single-armsft |
| Reading caution | Source description Appendix A leaves post-training recipes and broader fusion mechanisms open, omits UMI-style pretraining data, and acknowledges viewpoint/noise sensitivity of the chosen encoder. limitations |
Core contributions
- Source description
OpenWAM-Infra composes encoders, backbones and masks independently of training and serving. Architectures cover shared transformers, separate action/video streams and an added understanding stream. composition
- Reader analysis
The study selects a capable video prior, compact latents, dedicated action capacity, world-to-action access and synchronized denoising; mixed pretraining motivates mutual visibility. These are sequential experimental defaults, not independently proven universal optima. study-protocolrepresentationcapacitymaskschedule-studytransferpretrained-mask
- Source description
The final model is evaluated after benchmark-specific adaptation across eight simulation benchmarks and physical single-arm, bimanual and dexterous platforms. simulation-protocolsingle-armbimanualdexterous
Figure 12. Two learned streams exchange features while retaining their own parameters and noise levels. Original paper, p. 17 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with panel (a), following the encoded video and action inputs upward through their respective generators. Each side has its own timestep conditioning and feed-forward blocks; the central attention block permits interaction. Language and proprioception enter both sides through cross-attention. Read the miniature mask by query rows and key columns: the observed-frame row stays isolated from noisy future/action tokens, while future and action rows communicate. Panel (b) distinguishes robot supervision from human video without robot actions. Panel (d) places heterogeneous embodiments into fixed action slots. Panel (c) illustrates independent training coordinates and a shared inference diagonal, but its sample density needs the caution below. alphamaskmixtureaction-spacealpha-samplingflowalpha-serving
What it supports. OpenWAM-α is a coupled dual-backbone policy: a 5B video DiT and a dedicated 1B ActionDiT communicate at all 30 layers. The world branch supplies features during action refinement. Physical control uses the resulting action chunk; decoding a displayable video is unnecessary in the serving path.
Where the evidence stops. Panel (c) clusters samples near high τ, but the text defines t=0 as noise and samples t=1−f₅(u), favoring low t. No τ-to-t mapping is given. Preserve this schematic ambiguity; use the written formula when specifying training.
2. Motivation
2.1 The problem and the proposed response
Video pretraining supplies abundant visual dynamics knowledge, while executable robot demonstrations are scarce. Coupled WAM implementations obscure whether transfer comes from the backbone, representation, action capacity, information flow or data mixture. OpenWAM exposes these choices to controlled comparison. problem
2.2 What this reading follows
A world–action model needs a usable connection between predicted change and executable control. OpenWAM studies that connection by separating the video backbone, latent representation, action capacity, attention mask, denoising schedule and pretraining mixture. Its final model, OpenWAM-α, combines a pretrained video transformer with a separate action transformer, then adapts the resulting checkpoint to simulation and physical tasks. Read the following evidence in three stages: how information crosses the streams, why pretraining changes transfer, and where the resulting policy succeeds or fails. The controlled 600-hour study and the final 6,369-hour model answer different questions; neither should be mistaken for zero-shot robot deployment. problemcompositionalphatransfermixturesimulation-protocolsingle-armlibero-plus
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is WAMs. View the current classification.
3.1 Evidence-based assessment
Insufficient evidence to decide
The catalog is entirely unassigned, so there is no substantive placement to confirm. Architecture evidence supports a coupled dual-backbone WAM with joint future/action prediction for α. It differs from the supported sequential IDM variant; end-to-end training alone does not justify a shared-backbone “One Model” label. compositionalphaalpha-sampling
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Separate capacity from the route through which information travels
A joint objective does not guarantee that action prediction can use future-world features. OpenWAM makes this distinction testable by separating the parameter layout from the attention mask. Table 1 compares shared-backbone, dual-backbone and tri-system architectures; the selected dual joint-attention design scores 92.36% versus 85.50% for the shared vanilla design, while tri-system reaches 92.60%. These comparisons change capacity and structure, so they do not isolate information flow by themselves. Table 2 then changes visibility within the selected design. Figure 8 clarifies that even isolated actions retain access to the observed frame, while the major improvement appears when they can read the future stream. Reader interpretation: the evidence favors dedicated action processing plus useful future features, rather than treating joint training or total parameter count alone as the mechanism. compositioncapacitymaskstudy-protocol
5.2 Follow the noise convention through training, refinement and execution
The source defines clean data at t=1 and Gaussian noise at t=0. During training, video and action receive independently sampled timesteps; their velocity targets are clean data minus sampled noise. The validity mask restricts action supervision to real embodiment coordinates, and human videos contribute no robot-action loss. At inference, both streams traverse the same warped ten-step grid. Each joint forward pass refines actions using the current video features while the observed latent stays clamped. This is different from the infrastructure’s IDM variant, which generates video and then infers actions in a second stage. It is also different from asynchronous serving: α’s reported evaluations wait for a new chunk when their buffer empties. Pixel video decoding is skipped, but video-stream computation still participates in action refinement. flowalpha-samplingalphaaction-spacemixturecompositionalpha-servingdeployment
5.3 Distinguish scene transfer, model scale and embodiment adaptation
The controlled transfer experiment asks whether embodied pretraining helps beyond the downstream clean-scene distribution. Its 600-hour mixtures and identical fine-tuning make the larger OOD gain informative, but do not imply that all kinds of shift become easy. Final α uses a different 6,369-hour mixture and benchmark-specific adaptation; its Clean2Random randomized success rises to 48.7%, while LIBERO-Plus still exposes major camera/noise weaknesses. Physical experiments answer another question: can the pretrained model adapt into executable policies? The dexterous platform was absent from pretraining, yet it receives task fine-tuning before testing. Reader interpretation: these results support useful initialization and transfer under the reported adaptation procedures. They do not establish zero-shot embodiment control, uniform OOD robustness, or a causal explanation of every advantage over differently trained competitors. transfermixturesftsimulation-resultslibero-plusdexterouslimitations
5.4 Training and inference
During training
One-stage pretraining uses 518.5M frames (6,369 hours), approximately 30% egocentric, 40% real-robot and 30% simulated-robot frames. Human videos supervise only the world stream because action/proprioception channels are masked. This differs from the 600-hour EgoDex/RoboCOIN study. mixturetransfer
Video DiT, ActionDiT and proprioception encoder are updated; VAE and text encoder remain frozen. Independent timesteps interpolate clean targets with Gaussian noise. Velocity losses have unit weights; bell-shaped timestep weighting is described without its exact formula. alphaflowalpha-sampling
Curation checks visual quality, signal integrity and alignment, retaining middle pauses. Table 9 reports 155,862 iterations, batch 3,072, AdamW at 10^-4, bf16/ZeRO-2 and 33-frame, 384×320 inputs. Appendix B specifies separate SFT schedules. curationtraining-configsft
During inference
Both modalities advance together through ten warped Euler steps, with observed latents re-pinned after each step. Control consumes actions without decoding predicted video pixels. All final benchmarks use synchronous serving: execution pauses for a new chunk when the buffer empties. alpha-samplingalpha-servingdeployment
The accelerated loop is reported at roughly 170 ms per chunk on RTX 5090. This is loop latency, not complete sensing-to-actuation latency or continuous control without pauses. alpha-serving
5.5 Implementation flow
- Encode the present
Frozen Wan2.2-VAE preserves the first frame as a clean anchor and groups later frames by four. Frozen umT5 encodes language; projected proprioception joins the context. alpha
- Bridge two parameter streams
Pretrained Wan2.2-TI2V-5B and a 1B ActionDiT interact in all 30 paired layers. Residual widths remain separate, while joint attention shares features. Clean observation rows cannot attend to noisy future or action tokens. alphamask
- Ground the action representation
Each arm has position, rotation, gripper and hand slots; twelve coordinates are reserved. Dataset maps and validity masks accommodate missing channels. Inverse mapping and denormalization connect predictions to physical commands. action-space
6. Experiments & results
OpenWAM turns world–action modeling into a modular design study, then instantiates OpenWAM-α: a video DiT coupled to a dedicated ActionDiT through mutual attention and joint denoising. Its strongest lesson is conditional: embodied pretraining improves scene transfer, but strong manipulation scores coexist with substantial visual-robustness failures. The evidence below separates controlled ablations, final-model benchmarks and physical execution.
6.1 Read the original evidence
Table 5. High task-fitting performance does not guarantee robustness to camera and noise changes. Original paper, p. 22 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Find OpenWAM-α in the bottom row and follow it across perturbation types before considering the average. Camera and Noise are particularly low at 33.8 and 39.8, while Light reaches 97.0; the failure is therefore uneven across disturbances. The model uses its LIBERO checkpoint without further LIBERO-Plus fine-tuning. Rows above include both VLA and WAM systems, so stronger WAM entries matter when assessing whether the weakness characterizes every world–action model. Bold and underline indicate the source’s best and second-best values. They do not denote statistical significance or evidence that the competing models were trained with matched data. libero-plussftlimitations
What it supports. OpenWAM-α averages 69.2%, below Qwen-RobotManip’s 89.0% and ImageWAM’s 83.1%. The table establishes a concrete visual-robustness limitation despite the model’s successes elsewhere. The authors connect it to limited single-arm pretraining coverage and pixel-video prediction, but these are explanations to test rather than isolated causal findings.
Where the evidence stops. Different methods vary in data, architecture and training. This leaderboard cannot by itself attribute the gap to pixel latents or establish that one model family generalizes better under all distribution shifts.
Table 6. Physical trials support an aggregate gain, with task-specific exceptions visible in the counts. Original paper, p. 25 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each cell gives a count and a percentage, so begin with the denominators. Every task has 20 evaluation trials after fine-tuning from 100 task-specific demonstrations. The six equally sized tasks make the aggregate success count directly interpretable. Drawer tasks require both placing the object and closing the upper drawer; stacking requires a stable stack, and hanging requires the object to remain on the shelf. Follow the OpenWAM-α row across tasks: both drawer tasks are 20/20, but Stack Ring is 12/20. The latter is below LingBot-VA’s 15/20, showing why the overall average should not replace task-level inspection. single-armsftalpha-serving
What it supports. OpenWAM-α records 99 successes out of 120 trials, or 82.5%, versus 93/120 (77.5%) for LingBot-VA and 66/120 (55.0%) for π0.5. This is evidence from executed physical actions after task adaptation. It supports a five-percentage-point aggregate lead over LingBot-VA within the reported setup.
Where the evidence stops. Only 20 trials support each task estimate, and confidence intervals are absent. The fixed task scenes and task-specific demonstrations limit claims about broad deployment generalization or zero-shot transfer to new robots.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Training-time visibility ablation RoboTwin2.0-Full: clean plus randomized training; mean over both evaluation conditions. | Action-sees-video 92.39; mutual 92.15. Success rate (%) | Isolated 87.41; video-sees-action 87.63. Future-world access for actions matters here; reverse visibility alone does little. No error bars establish significance of the 0.24-point difference. maskstudy-protocol |
| Inference denoising schedule ablation Mutual-mask RoboTwin controlled study; Figure 9 does not explicitly identify split aggregation. | Synchronized 93.0. Plotted success rate (%) | Leading schedules range from 88.5 to 92.3. Joint refinement beats tested leading schedules; this does not establish that generated visual content is unnecessary. schedule-study |
| Controlled embodied pretraining transfer 600-hour pretraining variants; identical clean-only RoboTwin fine-tuning, evaluated on clean ID and randomized OOD. | Joint ego+robot: ID 87.68; OOD 26.62. Success rate (%) | Scratch 87.00/14.50; robot-only 88.50/23.80; sequential 87.10/26.50. Joint pretraining adds 12.12 percentage points OOD and 0.68 ID; its OOD lead over sequential training is only 0.12 points. transfer |
| EBench mobile bimanual manipulation Final α, SFT with batch 256 for 100k steps; synchronous execution. | 49.4 SR; 64.7 Score. Overall SR (%) and benchmark Score | Qwen-RobotManip: 45.6 SR; 60.0 Score. Leads listed overall results, but Table Top SR is 30.0 versus Qwen’s 50.0. Score is not success rate. ebenchsftalpha-serving |
| RoboTwin2.0-Clean2Random final-model transfer Final pretrained model; clean-only SFT, clean/randomized evaluation. | 89.4 clean; 48.7 randomized; 69.0 average. Success rate (%) | Qwen-RobotManip: 84.7; 69.4; 77.1. Higher clean performance coexists with lower OOD success. These final-scale results differ from the 600-hour study. simulation-resultssimulation-protocolsft |
| LIBERO-Plus visual robustness LIBERO checkpoint reused without further fine-tuning. | 69.2 average; camera 33.8; noise 39.8. Success rate (%) | Qwen-RobotManip 89.0 average; ImageWAM 83.1. Camera/noise failures delimit transfer; cross-model differences do not isolate the proposed pixel-prediction mechanism. libero-plussft |
| Single-arm physical manipulation Six Franka-Research-3 tasks; 100 demonstrations and 20 trials per task after SFT. | 99/120 (82.5%). Task success rate | LingBot-VA 93/120 (77.5%); π0.5 66/120 (55.0%). A five-percentage-point aggregate lead over LingBot-VA, with lower ring-stacking success (12/20 versus 15/20). single-armsft |
| RoboDojo-Real bimanual manipulation 18 tasks across ARX X5, Piper and Piper X, following the official protocol after SFT. | 37.6 Score; 24.4 SR. Overall progress Score and SR (%) | π0.5: 22.9 Score; 12.8 SR. Leads the listed aggregate while many tasks still fail; partial progress is not completion. bimanualsft |
| Dexterous bottle-cap removal under OOD variations Wuji-hand/Tianji-arm after task SFT; OOD variations aggregated over 20 trials. | 18/20 (90.0) progress; 16/20 (80%) success. Progress Score and final SR | π0.5: 13/20 (65.0) progress; 6/20 (30%) success. Shows adaptation to a platform absent from pretraining, not zero-shot embodiment transfer. Stable grasp is required for final success. dexteroussft |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2. Allowing actions to read future-world features produces the major gain in this mask study. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare the Average column first, then check that the ordering broadly survives both Clean and Randomized columns. RoboTwin2.0-Full trains on both kinds of scenes, so these are in-distribution evaluation conditions. The rows change which future streams can read each other. Video Sees Action sends action information into video prediction; Action Sees Video gives actions access to future-world features. Mutual permits both directions. The source’s companion mask diagram still lets actions read the observed frame in the Isolated case. Thus the meaningful comparison concerns communication with the predicted future, rather than whether the policy can see its present observation. maskstudy-protocolpretrained-mask
What it supports. Action Sees Video reaches 92.39% average success, compared with 87.41% for Isolated and 87.63% for Video Sees Action. Mutual reaches 92.15%. Within this experiment, enabling world-to-action access explains the large performance separation; enabling only the reverse path does not produce a comparable improvement.
Where the evidence stops. This table precedes embodied pretraining and does not report uncertainty. The 0.24-percentage-point one-way advantage is small; Figure 11 revisits the comparison after pretraining and reverses its direction.
Figure 9. The tested policy benefits from keeping video and action denoising synchronized. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The horizontal coordinate is video progress and the vertical coordinate is action progress; both run from zero for noise to one for clean data. Above the diagonal, actions are cleaner than video; below it, video leads. The left plot uses the variance-shift family, with the number before each slash identifying its shift parameter. The right uses a linear offset that delays one stream. Numbers after the slash are the reported task scores, not coordinates or denoising steps. Training samples both noise levels independently under a mutual mask, so the comparison changes the inference trajectory without restricting training to a single diagonal. schedule-studydeploymentflow
What it supports. The synchronized trajectory is labeled 93.0, above every displayed alternative; leading schedules span 88.5–92.3. This supports synchronized joint refinement for the tested model. It does not support a requirement to finish a video plan before generating actions, nor establish that the world stream can simply be removed.
Where the evidence stops. Figure 9 does not explicitly identify the clean/randomized aggregation of its score or provide error bars. Denoising synchronization is separate from synchronous versus asynchronous action-buffer serving; the plot does not test those execution modes.
Figure 10. The controlled pretraining benefit is much larger under scene shift than on clean scenes. Original paper, p. 15 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the left panel as clean-scene evaluation after clean-only fine-tuning, and the right as evaluation in randomized scenes absent from that fine-tuning set. The four configurations share downstream optimization and inference. Pretraining variants receive 600 hours: robot-only uses all robot data, while sequential and joint mixtures use 350 hours of EgoDex and 250 hours of RoboCOIN. The numbers above bars are absolute success and gains from the scratch baseline. Notice the broken left axis between 5 and 85: apparent bar-height differences cannot be compared directly across panels. These experiments use a smaller, different mixture from final OpenWAM-α pretraining. transferstudy-protocolpretrained-maskmixture
What it supports. Joint pretraining raises OOD success from 14.50% to 26.62%, a gain of 12.12 percentage points, while ID moves from 87.00% to 87.68%. Sequential mixing nearly matches the OOD result at 26.50%. Mixed data support transfer here; the evidence for a decisive advantage of one-stage ordering is much weaker.
Where the evidence stops. No uncertainty is displayed. Figure 11’s separately matched mask comparison implies 26.42 for mutual OOD success, whereas this plot reports 26.62 for co-training. The source does not reconcile those values; retain each in its own comparison.
7. Analysis & limitations
7.1 What the evidence leaves open
Appendix A leaves post-training recipes and broader fusion mechanisms open, omits UMI-style pretraining data, and acknowledges viewpoint/noise sensitivity of the chosen encoder. limitations
The authors’ explanation that pixel-video prediction causes OOD failures remains a hypothesis: competitors differ in data and architecture. Strength is uneven; RoboCasa365 is 38.2 versus the listed leader’s 57.4. libero-pluslimitationssimulation-results
Figure 12(c) clusters samples near high τ, whereas the written convention and formula favor low t. No conversion is defined. Figure 11’s OOD baseline plus gain implies 26.42, while Figure 10 reports 26.62; these runs should not be silently equated. alpha-samplingflowpretrained-masktransfer
Small physical trial counts and absent uncertainty estimates limit strong ranking claims. Higher means do not establish reliable superiority across unseen settings. single-armdexterousmask
7.2 Questions for discussion
- Does mutual visibility retain its small pretrained advantage across seeds and budgets?
- Can compact semantic latents improve camera/noise robustness without sacrificing action precision?
- How much physical advantage persists under broader scene variation?
8. Reproducibility audit
8.1 Requirements and known gaps
Full pretraining requires the curated mixture, normalization/slot maps and the reported scale of 128 H200 GPUs for roughly seven days. Preserve checkpoint identity, encoder settings and action statistics through SFT and serving. mixturetraining-configaction-spaceflow
The PDF leaves exact bell-shaped weighting, numerical action-chunk/execution horizons, cache similarity threshold/skip limit and some cleaning cutoffs unspecified. Appendix C.3 defers detailed bimanual protocols externally; resolve these before claiming exact reproduction. alpha-samplingdeploymenttraining-configcurationbimanual
Proposed checks: repeat visibility comparisons with matched data and seeds before/after pretraining, and separate single-arm data diversity from representation robustness in a controlled LIBERO/Plus comparison. Neither check was run. maskpretrained-masktransferrepresentationlibero-plus
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test whether pretraining reliably changes the preferred mask
Reader-proposed experiment: compare action-sees-video and mutual masks both without embodied pretraining and after the same 350-hour EgoDex/250-hour RoboCOIN mixture. Hold the video/action backbones, optimization budget, clean-only RoboTwin fine-tuning and synchronized denoising fixed; use matched evaluation scenes and multiple independent training seeds. Report clean and randomized success separately with trial counts and uncertainty. A stable positive mutual-minus-one-way difference after pretraining, but not before, would support the claimed reversal. A difference fluctuating around zero would weaken it. First reconcile the Figure 10/11 OOD values rather than choosing whichever reference score is easier to match. maskpretrained-masktransferstudy-protocolalpha-sampling
Check 2: Separate data diversity from visual representation in robustness
Reader-proposed experiment: cross two visual encoders, Wan2.2-VAE and DINOv3 with a 48-D S-VAE, with two equal-frame single-arm mixtures differing in scene/camera diversity. Follow the representation study’s random video-backbone initialization and matched fourfold temporal compression so native pretrained-weight compatibility does not confound the encoder comparison. Keep action capacity, mask, optimization budget and LIBERO fine-tuning fixed; evaluate clean LIBERO and every LIBERO-Plus perturbation using the same seeds and trials. A semantic-encoder improvement concentrated in Camera/Noise at comparable clean success would support the representation hypothesis. Gains driven mainly by diverse data across both encoders would favor the coverage explanation; this would be a mechanism test, not an exact α reproduction. representationlibero-plusmixturesftstudy-protocollimitations
8.3 Reading coverage
Visual audit: Read the complete supplied 43-page text, including Appendices A–D and bibliography. Visually inspected the title/author page, Figures 1–12 and 15, Figures 17–21, and Tables 1–20 on the declared pages. Every final crop was inspected; all numerical, method, configuration and protocol evidence retained in this bundle is covered by these pages. Figure 12’s timestep-density ambiguity and the Figure 10/11 numerical mismatch are disclosed. Figures 13, 14 and 16 were read as extracted text/captions; their original page images were not inspected, and retained benchmark values were checked in original tables instead. Separate supplements, external documentation, code and continuous rollout videos were not inspected.
PDF pages inspected for this edition: 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 25, 26, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, abstract and contents (pp. 1–2)
- Sections 1–2: introduction and related work (pp. 3–5)
- Section 3: model, training, deployment and evaluation (pp. 5–10)
- Section 4: priors, architecture, information flow and controlled pretraining (pp. 11–16)
- Section 5: α architecture, data, simulation and physical evaluations (pp. 17–26)
- Section 6, acknowledgements and references (pp. 26–32)
- Appendices A–D in full: limitations, training, real-world protocols and simulation tables (pp. 33–43)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The retained PDF was visually inspected to resolve figure, equation and table layout; exact inspected pages are declared in the illustrated edition.
- Version scope: arXiv:2609.07398v1, 7 September 2026. Title and all authors match; the PDF uses a typographic World–Action dash where supplied observedTitle uses a hyphen. No other edition was supplied or compared.
- Code, weights, datasets, external benchmark documentation and rollout videos were not inspected. No experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title, author/affiliation block and arXiv margin
The title and all 24 authors agree with the supplied catalog, allowing typographic dash normalization. The margin identifies arXiv:2609.07398v1 [cs.RO], 7 September 2026. Seven institutions are explicitly credited.
Go to primary source ↓problemPDF p. 3, Introduction; p. 5, Section 3
The study separates inherited world priors, world/action interaction and transfer across domains; infrastructure decouples model composition, training, deployment and evaluation.
Go to primary source ↓compositionPDF pp. 5–7, Section 3.1, Figure 2
Frozen visual encoders, stream backbones and visibility masks compose single-, dual- and tri-system variants. Dual-system ActionDiT has separate parameters; IDM generates video first, then actions. Joint self-attention couples streams within bridge layers.
Go to primary source ↓maskPDF p. 6, Equation (1); p. 14, Figure 8 and Table 2
X←Y means X queries Y. Clean observation rows attend only to themselves; action chunks have bidirectional attention. Figure 8 retains observation access even in the isolated variant. Table 2 average success is 87.41, 87.63, 92.39 and 92.15 for isolated, video-sees-action, action-sees-video and mutual.
Go to primary source ↓flowPDF pp. 7–8, Section 3.2, interpolants, Equation (2) and checkpoint paragraph
At t=0 data are Gaussian noise and at t=1 clean. Video/action timesteps are independently sampled; predicted velocities target data minus noise. Validity masks restrict action loss; clean conditioning frames are excluded from video loss. Checkpoints retain model configuration and normalization statistics.
Go to primary source ↓deploymentPDF pp. 8–9, Section 3.3, Figure 3, Equation (3) and acceleration paragraphs
Serving mode and denoising schedule are independent. Synchronous serving pauses at buffer exhaustion; asynchronous serving prefetches. Denoising uses coupled Euler updates. Acceleration includes prompt caching, skipped pixel decoding, compilation and conditional reuse of both streams’ velocities.
Go to primary source ↓action-spacePDF p. 10, Section 3.4, Figure 5 and Equation (4)
The unified 80-D vector has two 34-D arm blocks, each with position 3, rotation 6, gripper 1 and hand 24, plus 12 reserved coordinates. Native normalized actions are scattered into fixed slots and gathered then denormalized for execution. Missing camera views are black-filled.
Go to primary source ↓study-protocolPDF p. 11, Section 4, Evaluation Protocol and Section 4.1.1, Figure 6
RoboTwin2.0-Full uses 2,500 clean and 25,000 randomized demonstrations; Clean2Random trains only on clean scenes. Backbone comparisons use dual joint attention. The 14B/5B backbones reach 93.79/92.39 average success, but architecture, data and objectives differ as well as parameter count.
Go to primary source ↓representationPDF pp. 12–13, Section 4.1.2 and Figure 7
Representation tests randomly initialize the Wan2.2-TI2V-5B architecture, use frozen visual encoders and match fourfold temporal compression, using frame averaging where needed. S-VAE compresses semantic features to 48 dimensions. DINOv3 rises from 76.42 to 90.18 average success with S-VAE, versus Wan2.2-VAE at 90.30.
Go to primary source ↓capacityPDF p. 13, Section 4.2.1 and Table 1
Average RoboTwin2.0-Full success is 85.50 for single-system vanilla, 84.63 for MoE, 92.36 for dual joint self-attention, 88.25 for end-to-end cross-attention, 91.85 for detached cross-attention, 87.95 for IDM and 92.60 for tri-system joint self-attention.
Go to primary source ↓schedule-studyPDF p. 14, Section 4.2.3 and Figure 9; p. 9, Equation (3)
With mutual visibility and independent training noise, the study compares synchronized, variance-shift and linear-offset schedules in both leading directions. Synchronization is labeled 93.0; alternatives span 88.5–92.3. Axes define zero as noise, with action-leading above the diagonal. The plot does not explicitly specify clean/randomized aggregation or uncertainty.
Go to primary source ↓transferPDF p. 15, Sections 4.3.1–4.3.2 and Figure 10
Pretraining variants share a 600-hour budget: 600 robot-only hours or 350 EgoDex plus 250 RoboCOIN hours, sequentially or jointly. Downstream fine-tuning is identical. Clean/OOD success: scratch 87.00/14.50, robot-only 88.50/23.80, sequential 87.10/26.50, joint 87.68/26.62.
Go to primary source ↓pretrained-maskPDF p. 16, Section 4.3.3, Figure 11 and Table 3
Mutual minus action-sees-video is −0.24 percentage points on scratch Full and +0.16 after pretraining; pretrained Clean2Random gains are +0.70 ID and +0.72 OOD. OOD one-way baseline is 25.70. These imply 26.42 for that matched mutual run, distinct from Figure 10’s 26.62; no reconciliation is supplied.
Go to primary source ↓alphaPDF pp. 17–18, Figure 12 and Sections 5.1.1–5.1.2
OpenWAM-α combines pretrained Wan2.2-TI2V-5B and a 1B ActionDiT, bridged in all 30 layers under mutual attention. Frozen Wan2.2-VAE and umT5 encode vision/language. Video/action residual widths are 3072/1024. Language and proprioception enter per-stream cross-attention. Clean first-frame rows cannot read noisy future/action tokens.
Go to primary source ↓alpha-samplingPDF p. 18, Sections 5.1.2–5.1.3; p. 9, Equation (3); p. 17, Figure 12(c)
Training uses unit stream-loss weights and an unspecified bell-shaped timestep weight, with t=1−fρ(u), ρ=5 and independent uniform u. Inference uses t^i=1−fρ(1−i/N), N=10, with observed latents re-pinned after each Euler step. Figure 12(c) instead clusters samples near high τ values; no τ-to-t conversion is defined.
Go to primary source ↓alpha-servingPDF p. 19, Section 5.1.3, Inference Mode and Inference Acceleration
All following simulation and physical benchmarks use synchronous serving. The accelerated ten-step loop is reported at roughly 170 ms per chunk on RTX 5090; VAE pixel decoding is omitted from control.
Go to primary source ↓mixturePDF p. 19, Section 5.2.1 and Table 4
Final pretraining uses 518.5M frames, 6,369 hours: authors’ egocentric data, AgiBotWorld-Beta, RoboCOIN, DROID and InternData-A1. Frame shares are 30.1%, 18.6%, 14.3%, 7.0% and 30.0%. Human data mask robot-action/proprioception channels and supervise only the world stream. Whole episodes are sampled under source budgets.
Go to primary source ↓curationPDF p. 20, Section 5.2.2
Curation removes visual defects, invalid egocentric language and hand-absent footage, checks robot state/action and video alignment, trims leading/trailing idle footage, retains middle pauses and deletes heavily defective episodes. Several signal thresholds are given, but not every visual-quality cutoff.
Go to primary source ↓training-configPDF pp. 33–34, Appendix B.1, Table 9
Pretraining is reported as approximately seven days on 128 H200 GPUs, AdamW at 10^-4, batch 3072, 155,862 iterations/one epoch, bf16 and DeepSpeed ZeRO-2. Input is 33 frames with video stride 4, window stride 1 and 384×320 resolution. Cosine scheduling has 5% warmup; ColorJitter is specified.
Go to primary source ↓sftPDF p. 34, Appendix B.2 and Table 10
Downstream models share the pretrained checkpoint and otherwise retain Table 9 settings. LIBERO uses batch 256 and 10 epochs; LIBERO-Plus reuses it without SFT. EBench uses batch 256 and 100k steps with ColorJitter. Clean2Random uses 5 epochs/10,740 steps with ColorJitter. Physical single-arm uses 10 epochs/9,860 steps; bimanual 30k steps; dexterous 5 epochs/10,925 steps.
Go to primary source ↓libero-plusPDF p. 22, Table 5, OpenWAM-α and comparison rows; pp. 22–23, Section 5.3.1
OpenWAM-α scores 69.2 average, camera 33.8 and noise 39.8, versus Qwen-RobotManip average 89.0 and ImageWAM 83.1. Authors attribute weaknesses to single-arm data coverage and pixel prediction; comparisons differ in architecture and data.
Go to primary source ↓simulation-protocolPDF p. 20, Section 5.3; p. 34, Appendix B.2
Eight simulation benchmarks span single-arm, bimanual, mobile single-arm, mobile bimanual and dexterous-hand tasks. Models are fine-tuned per benchmark; LIBERO-Plus reuses LIBERO. Full and Clean2Random are different RoboTwin training protocols.
Go to primary source ↓simulation-resultsPDF p. 39, Table 13; p. 40, Tables 14–15; p. 41, Tables 16–17; p. 42, Table 19; p. 43, Table 20, OpenWAM-α and comparison rows
Average success includes LIBERO 99.3, VLABench 58.9, RoboTwin-Full 93.60, RoboDojo 11.92, RoboCasa365 38.2 and RoboCasa-GR1 60.5. Clean2Random is 89.4 clean/48.7 randomized/69.0 average versus Qwen-RobotManip 84.7/69.4/77.1. RoboCasa365’s leader is Xiaomi-Robotics-1 at 57.4.
Go to primary source ↓ebenchPDF p. 42, Table 18, Overall and Table Top columns
OpenWAM-α leads listed Overall values at SR 49.4 and Score 64.7 versus Qwen-RobotManip 45.6/60.0; its Table Top SR is lower, 30.0 versus 50.0.
Go to primary source ↓single-armPDF p. 25, Figure 17 and Table 6; pp. 35–36, Appendix C.1, Figures 18–19 and evaluation paragraph
Franka-Research-3 uses six tasks with 100 task-specific demonstrations and 20 trials each. OpenWAM-α achieves 99/120 (82.5%), LingBot-VA 93/120 (77.5%) and π0.5 66/120 (55.0%). Ring stacking is 12/20 versus 15/20. Drawer success requires object placement and upper-drawer closure.
Go to primary source ↓bimanualPDF p. 26, Table 7, Overall Avg.; p. 39, Appendix C.3
RoboDojo-Real covers 18 tasks across ARX X5, Piper and Piper X. Overall Score/SR is 37.6/24.4 for OpenWAM-α versus 22.9/12.8 for π0.5. Detailed official task protocols are referenced externally rather than reproduced in full.
Go to primary source ↓dexterousPDF pp. 25–26, Section 5.4 and Table 8; pp. 37–39, Appendix C.2 and Figures 20–21
The Wuji-hand/Tianji-arm platform and its 9-D pose plus 21 hand dimensions were absent from pretraining; evaluation follows task SFT. Bottle-cap OOD Score is 18/20 (90.0), SR 16/20 (80%), versus π0.5 13/20 (65.0) and 6/20 (30%). Full success additionally requires stable grasp; progress counts removal alone. OOD aggregates task variations.
Go to primary source ↓limitationsPDF p. 33, Appendix A, items 1–4; p. 23, Section 5.3.2
Authors identify limited post-training investigation, restricted fusion mechanisms, omitted UMI-style data and visual-encoder robustness. VLA/WAM competitors differ in data, architecture and training, preventing an unconditional paradigm ranking.
Go to primary source ↓8.5 Primary sources
OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining ↗
PDF · 22,587 extracted words
Source fingerprint
09606aafdd27a377b669331c2d11ad87c11dbba31066689c8d48d9fd2f28151b