StageWAM: Joint-Embedding Stage Prediction for World-Action Models in Robot Manipulation
1. Paper overview
In one sentence: A frozen next-stage latent predictor improves a joint video-action policy through gated conditioning, but the experiments support the extra pathway more strongly than task-specific stage learning. two-futuresarchitecturebenchmarkreal-resultsdiagnosticsablations
| At a glance | What to know |
|---|---|
| Research problem | Author claim A short predicted video-action chunk can describe local motion while leaving the next task-relevant object relation implicit. The authors seek an internal progress target that guides manipulation without reconstructing a distant target image or prescribing transition duration. two-futures |
| Core mechanism | Source description The method combines automatically inferred demonstration boundaries, goal-conditioned representation prediction, and a gated video-stream interface. Stage guidance remains active during deployment, while target slices supply training supervision only. stage-pairsstage-learninggateinference |
| A key reported result | RoboTwin 2.0 manipulation success: 91.42 clean; 89.08 randomized; 90.25 overall. Task success rate (%). 50 tasks × clean/randomized settings × 100 closed-loop trials per pair. Motus: 88.66 clean, 87.02 randomized; derived overall 87.84, a 2.41-percentage-point gap. StageWAM improves five of seven clean categories and six randomized categories, but loses acquisition/lifting in both. Published baseline results are reused; this does not establish matched training compute. evaluationbenchmarkablations |
| Reading caution | Reader analysis Centered training windows overlap paired targets in 86.77% of clean and 86.34% of randomized pairs. Median target distances are 24/25 frames, and 81.72% of stage intervals are shorter than the 64-frame input. Thus stage prediction is temporally contextual supervision; strict distant extrapolation and semantic boundary accuracy are not established. diagnosticsstage-pairs |
Core contributions
- Source description
The method combines automatically inferred demonstration boundaries, goal-conditioned representation prediction, and a gated video-stream interface. Stage guidance remains active during deployment, while target slices supply training supervision only. stage-pairsstage-learninggateinference
Figure 2. Stage prediction supplies a condition to joint world-action generation. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the lower panels first. Panel (b) turns demonstration feature changes into boundary pairs. Panel (c) encodes current and next-stage clips with frozen V-JEPA2; instruction-conditioned prediction is compared with the target representation. Then follow panel (a): frozen Stage-JEPA guidance enters the green video stream, which interacts with orange action tokens and purple understanding tokens through tri-modal joint attention. The action head produces the chunk used for control. Cross-check the schematic addition against Eqs. (10)–(12): predicted tokens are mean-pooled, projected by an MLP, multiplied by a global scalar gate, and broadcast as one sample-specific vector. stage-pairsstage-learningarchitecturegateinference
What it supports. The architecture supplies two different kinds of future information: a representation of intended progress and local video-action predictions. Stage-JEPA is frozen while the WAM learns to use that condition. The guidance has an inference-time route to actions through shared attention; it is more than an auxiliary training target.
Where the evidence stops. The diagram omits pooling, projection and the scalar gate, and draws cosine alignment between token sets. The text and Eq. (9) define normalized pooled supervision. These schematic shortcuts should not be implemented as direct tokenwise addition or an unspecified tokenwise loss.
2. Motivation
2.1 The problem and the proposed response
A short predicted video-action chunk can describe local motion while leaving the next task-relevant object relation implicit. The authors seek an internal progress target that guides manipulation without reconstructing a distant target image or prescribing transition duration. two-futures
2.2 What this reading follows
A robot can predict plausible nearby motion yet still spend actions correcting an object without advancing the task. StageWAM addresses this by adding an internal target for the next stage of progress. One branch predicts that target in V-JEPA2 representation space; a Motus world-action model uses it while generating local video and actions. The paper reports gains across RoboTwin tasks and two physical manipulation tasks, with different metrics for each. Read the results alongside the supervision diagnostics: training clips often overlap their targets, and task-specific Stage-I training adds only a small gain over an initialized predictor that was never trained on those stage pairs. two-futuresarchitecturebenchmarkreal-resultsdiagnosticsablations
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Joint prediction |
| Quadrant | Q3 · Dual-system × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
The separate, frozen Stage-JEPA predictor feeding a joint video-action WAM supports Dual-system × Joint prediction. Actions are generated jointly with local video, not extracted by inverse dynamics. Latent/JEPA classification is directly supported; the memory/long-horizon label needs qualification because the measured inferred stages are often shorter than the encoder window. architectureinferencediagnostics
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 A next-stage target is constructed from a demonstration
The learning target begins with a heuristic, not a human stage label. Frozen V-JEPA2 describes centered clips from a demonstration. Changes in both pooled representations and individual token positions identify candidate transitions; smoothing, normalization and non-maximum suppression select boundaries. Every frame before the next boundary is paired with that boundary, producing dense supervision even though the boundary list is sparse. Stage-JEPA then learns to predict a normalized representation of the target clip from the current clip and instruction. This is not equivalent to predicting a distant image: the loss operates after representation pooling, and the centered clips can overlap heavily. The appendix explicitly frames this as temporally contextual supervision. At deployment, the input changes to an observed causal history, making context construction a central reproduction choice. stage-pairsstage-learningdiagnosticsinference
5.2 A global gate carries a sample-specific progress condition
After Stage I, the complete Stage-JEPA branch is frozen. Its predicted target tokens are pooled and projected to the width of the WAM's video tokens. A learned scalar gate multiplies this vector before it is added to every video token. Global sharing describes the gate strength; it does not mean all examples receive the same semantic vector. The vector still depends on the observed scene and instruction. Video tokens then interact with action and understanding tokens at each transformer layer, allowing the condition to influence both generated modalities. The WAM retains its video and action objectives while adding two gate penalties. The printed Eq. (14) has an ambiguous leading equality; the accompanying descriptions identify those terms as regularization, not a replacement for the native losses. architecturegatestage-learning
5.3 Separate useful conditioning from learned stage semantics
Reader analysis: three findings should be connected without treating them as the same claim. The main benchmark shows better average executed-task success. The no-JEPA ablation shows that removing the additional pathway lowers success substantially. But retaining the initialized predictor without Stage-I optimization nearly matches the full model, so the distinctive stage-pair objective has a much smaller measured contribution. The overlap diagnostics further limit interpreting that objective as learning wholly unseen future stages. Execution-length results add evidence about behavior, yet successful-only averages compare selected subsets and do not measure computation time. Finally, physical experiments use separately trained task policies and graded progress scores. Together these findings motivate the conditioning design while leaving the generality, causal role of stage semantics, and latency tradeoff open. benchmarkablationsdiagnosticsefficiencyreal-protocolreal-resultsboundaries
5.4 Training and inference
During training
Stage I uses facebook/vjepa2-vitl-fpc64-256, FP16 frozen encoding and FP32 learnable modules; AdamW runs 1,000 updates at learning rate 10⁻⁵, weight decay 0.01 and 100 warmup updates. Stage II uses BF16, four epochs, learning rate 10⁻⁶ and 200 warmup updates. Both use eight A800 80GB GPUs and effective batch 256; each stage is trained once with seed 0 and its final checkpoint is selected. training
Stage II retains unit-weight video/action objectives and gate penalties with weights 10⁻⁴ and 10⁻³ and target relative update 0.08. Eight video targets use raw-frame offsets 6,12,…,48 at 384×320; 16 action targets use offsets 3,6,…,48. A 48-frame span does not mean 48 predicted actions. gatetraining
During inference
Maintain a causal 64-observation buffer, repeating the earliest frame until full. Frozen Stage-JEPA predicts guidance from history and instruction; the WAM generates video and an action chunk. Execute the chunk, append resulting observations and replan. Neither target clips nor inferred demonstration boundaries are available online. inference
5.5 Implementation flow
- Infer transition targets
Sample candidate centers every eight frames, include the terminal frame, and subsample to at most 32 candidates. Frozen V-JEPA2 encodes centered clips. Combine smoothed, independently normalized pooled-feature and maximum token-change scores. NMS keeps at most five transitions, suppressing three retained-list positions around each selection. Add endpoints and pair every current frame with its next boundary. stage-pairs
- Learn the latent predictor
Encode both centered 64-frame slices with frozen V-JEPA2. Optimize its initialized predictor and instruction adapter against normalized pooled target representations. The appendix samples 512 context and 256 target positions from 8,192 tokens; predicted target tokens are pooled for conditioning. stage-learningtraining
- Condition joint world-action generation
Freeze Stage-JEPA. Project its pooled output with an MLP and add a gated, sample-specific vector to every video token before the WAM transformer layers. Motus couples video, action and understanding streams through joint attention; the video and action heads predict their respective flow fields. Stage guidance reaches actions through this coupling. architecturegate
6. Experiments & results
StageWAM predicts a latent next-stage target with Stage-JEPA, then conditions a Motus world-action model to generate local video and actions. It improves simulated manipulation success and reports shorter successful episodes. Its strongest ablation evidence concerns the extra conditioning pathway; the added benefit of task-specific stage training is small.
6.1 Read the original evidence
Table 1. Success improves overall, with clear exceptions by skill. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the two rightmost model groups and compare clean with clean, randomized with randomized. Every entry is a success percentage; task counts in parentheses specify each category's size. The categories partition 50 tasks, and their rows are unweighted means over tasks within that category. The all-task row therefore should not be reconstructed by simply averaging the seven category rows. The evaluation uses 100 closed-loop trials per task and setting. Baseline per-task values are taken from the Motus publication; the current paper states that StageWAM and Motus use the same closed-loop evaluation protocol. benchmarkevaluationtask-resultstraining
What it supports. StageWAM reaches 91.42% clean and 89.08% randomized success, compared with Motus at 88.66% and 87.02%. Handover improves substantially, from 82.00/68.00 to 96.50/93.50. Acquisition and lifting decline in both settings. These category contrasts support a heterogeneous benefit rather than uniform improvement across manipulation skills.
Where the evidence stops. Table 6 prints different clean aggregates for GO-1 and X-VLA: 37.80/72.80 instead of 37.86/72.88 here. The StageWAM and Motus means agree. Reused baseline results do not establish matched training budgets or uncertainty across training seeds.
Figure 4. Shorter successful episodes support an execution-step efficiency claim. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the legend first: hatched blue bars denote Motus and solid pink bars denote StageWAM. The vertical axis counts evaluation steps, so lower bars mean shorter episodes. The left pair includes successful episodes and failures that reach the evaluation horizon; the right pair includes only successful rollouts. The text says success triggers immediate termination, which makes termination rules essential to interpreting both pairs. Figure 3 on the same page provides one qualitative example of repeated corrections, but that example is not a population estimate. The aggregate claim rests on the four bar values shown here. efficiencyevaluationboundaries
What it supports. Across all rollouts, mean length falls from 156.28 to 132.80 steps, a reported 15.02% reduction. Restricting to successes gives 79.57 versus 74.82, or 5.97%. The smaller successful-only effect avoids conflating every shorter episode with a reduction in unnecessary motion, since higher success itself changes the all-rollout mixture.
Where the evidence stops. Successful subsets may differ between policies, and steps are not wall-clock latency. Page 5 says early stopping is disabled, while page 7 describes termination on success; the precise operational distinction is not explained. No error bars are supplied.
Table 4. Hardware evidence uses graded task progress and separate task models. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read these values in score units, not as success percentages. The neighboring Table 3 defines terminal-state scoring: cube grasping has levels 0, 30, 70 and 100; power-strip manipulation has levels 0, 30, 60 and 100. Page 11 specifies that safety failures receive zero, each task has 20 evaluation trials, and separate models are trained using 50 cube trajectories and 116 power-strip trajectories. The Average column gives equal weight to the two task means. Figures 5–7 show task stages and example physical rollouts, but do not provide the missing distribution of scores. real-protocolreal-resultsrobot-visuals
What it supports. StageWAM scores 45.00 on Cube_Grasp and 51.00 on Power_Strip, compared with π₀.₅ at 33.50 and 49.50. The overall mean is 48.00 versus 41.50. Most of this average advantage comes from the cube task; the power-strip difference is only 1.50 score points.
Where the evidence stops. No uncertainty or score distribution is reported. Cube terminal images show placement on a tray, whereas the top cube score requires stable holding until rollout end; the source does not assign these illustrated examples scores. Avoid treating their terminal captions as proof of maximum-score completion.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| RoboTwin 2.0 manipulation success 50 tasks × clean/randomized settings × 100 closed-loop trials per pair. | 91.42 clean; 89.08 randomized; 90.25 overall. Task success rate (%) | Motus: 88.66 clean, 87.02 randomized; derived overall 87.84, a 2.41-percentage-point gap. StageWAM improves five of seven clean categories and six randomized categories, but loses acquisition/lifting in both. Published baseline results are reused; this does not establish matched training compute. evaluationbenchmarkablations |
| Stage-conditioning ablations 50 RoboTwin tasks; clean/randomized aggregate. | 90.25 full model. Overall success rate (%) | 87.23 without JEPA; 89.89 current-frame-only guidance; 89.98 without Stage-I training; 89.88 without gate regularizers. Derived gains are 3.02 points over removing JEPA but only 0.27 over skipping Stage I. The no-JEPA ablation is distinct from the published Motus baseline. ablationsbenchmark |
| RoboTwin execution length Reported evaluation rollouts, separating all episodes from successful episodes. | 132.80 all rollouts; 74.82 successful only. Mean episode length (steps; lower is better) | Motus: 156.28 and 79.57; reported reductions 15.02% and 5.97%. All-rollout means mix success and horizon-limited failure. Successful-only means still compare potentially different successful subsets; step count does not measure wall-clock latency. efficiency |
| LIFT2 real-robot task progress Separate task models; 50 cube/116 power-strip training trajectories; 20 trials per task. | Cube_Grasp 45.00; Power_Strip 51.00; average 48.00. Mean terminal progress score (0–100) | π₀.₅: 33.50, 49.50 and 41.50. These are scored physical rollouts, not success percentages or evidence of one generalist hardware policy. Safety failures receive zero; uncertainty is not reported. real-protocolreal-results |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2. The conditioning pathway matters more than the incremental Stage-I optimization. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each row as a separate intervention, keeping the last row as the reference. Removing JEPA removes both its predictor and guidance. The current-frame-only row keeps the branch but restricts where the future signal enters; the prose calls this removal of future-latent injection. Skipping Stage-I training keeps the conditioning architecture and V-JEPA2 initialization. Removing gate regularizers leaves the bounded gate architecture intact and removes only the two auxiliary losses. Compare the Overall column to avoid selecting whichever environment favors a preferred explanation. Also distinguish the no-JEPA row from the published Motus row in Table 1. ablationsbenchmarktraining
What it supports. Removing JEPA costs 3.02 percentage points overall, whereas skipping Stage-I training costs 0.27. Restricting guidance to current-frame tokens costs 0.36, and removing gate regularizers costs 0.37. The table most strongly supports an additional conditioning pathway, with smaller reported gains from its particular supervision and integration choices.
Where the evidence stops. The appendix reports one seed-0 training run per stage, and this table supplies no uncertainty estimates. Small differences do not establish a stable ranking across retraining. The no-JEPA result also differs from the published Motus baseline and should not be substituted for it.
Table 5. Most stage-supervision pairs share source frames. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Treat this table as a description of training supervision, not a control-performance ablation. Episodes and Pairs count the diagnostic data after the training loader's validity checks. Mean Dist. and Median Dist. measure decoded-frame distance from a current frame to its next inferred boundary. Cross Boundary is the percentage of centered current clips containing frames outside their assigned interval. Slice Overlap is the percentage of current-target clip pairs sharing at least one decoded frame; it does not measure the proportion of frames shared within each pair. The last two columns are percentages, while the distance columns are frame counts. diagnosticsstage-pairsinference
What it supports. Median target distances are 24 frames in clean data and 25 in randomized data. Current-target slices overlap in 86.77% and 86.34% of pairs, respectively. This makes the source's own description—temporally contextual next-stage representation learning—more precise than interpreting every pair as prediction of a wholly unseen distant stage.
Where the evidence stops. These are offline centered-window diagnostics. Deployment uses a causal history and no target slices, so the table does not establish inference-time leakage. It does motivate testing the context shift and whether the inferred boundaries correspond to meaningful task progress.
7. Analysis & limitations
7.1 What the evidence leaves open
Centered training windows overlap paired targets in 86.77% of clean and 86.34% of randomized pairs. Median target distances are 24/25 frames, and 81.72% of stage intervals are shorter than the 64-frame input. Thus stage prediction is temporally contextual supervision; strict distant extrapolation and semantic boundary accuracy are not established. diagnosticsstage-pairs
Single-seed training and absent uncertainty limit interpretation of small ablation gaps. The authors identify contact-sensitive manipulation and inference latency as remaining challenges; no latency measurements support a faster-control claim. trainingablationsboundaries
Source discrepancies remain: Table 1 versus Table 6 gives GO-1 clean means 37.86 versus 37.80 and X-VLA 72.88 versus 72.80. Page 5's early-stopping wording differs from page 7's success termination. Eq. (14) is typeset as an equality after the total loss, although prose identifies its expression as auxiliary gate penalties. benchmarktask-resultsevaluationgate
The cube terminal illustrations show an object on a tray, while the highest cube score requires stable holding through rollout end. The source does not assign those illustrated rollouts scores; images cannot resolve their completion status. robot-visualsreal-protocol
7.2 Questions for discussion
- Does stage-specific supervision improve control consistently across training seeds beyond generic pretrained JEPA features?
- Would causal current-window training preserve the gains while reducing the training–deployment context mismatch?
8. Reproducibility audit
8.1 Requirements and known gaps
Reconstruction requires the named V-JEPA2 checkpoint, Motus/Wan2.2/Qwen3-VL components, trajectory manifests, validity filtering, and the appendix's exact sampling and optimizer schedules. Exact simulator split identifiers, data-validity rules, denoising-step configuration and measured runtime are not fully specified in this PDF. architecturetrainingdiagnosticsevaluation
A proposed minimal mechanism check would compare trained, untrained and shuffled stage conditions with identical WAM initialization and matched evaluation seeds. A separate causal-window training control would test whether the benefit survives removal of future-containing current slices. stage-learningablationsstage-pairsinference
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate stage learning from generic JEPA conditioning
Reader-proposed check, not performed: hold WAM initialization, demonstration split, training budget, token sampling and gate configuration fixed. Compare the trained Stage-JEPA condition, the initialized predictor without Stage I, and the trained condition shuffled between observations within the same task. Repeat matched training seeds and evaluate identical task–setting episode seeds. Report success and paired differences, plus successful-on-both-policy episode lengths. If trained guidance reliably beats untrained and shuffled guidance, this supports useful observation-specific stage prediction. If the small Stage-I gap vanishes across seeds or shuffled conditions retain the gain, the interpretation should shift toward generic representation conditioning rather than the learned next-stage target. ablationstraininggateevaluationefficiency
Check 2: Test the centered-training versus causal-deployment context shift
Reader-proposed check, not performed: keep inferred boundary centers, next-stage target clips, initialization and optimization budget fixed, but train one Stage-JEPA predictor with the paper's centered current clips and another with a trailing causal current window. Use the same causal inputs for both at deployment and train their downstream WAMs under matched settings. On held-out episodes, report representation alignment and closed-loop success stratified by next-boundary distance and whether a target lies within the centered current window. A benefit concentrated in overlapping short-distance training pairs that fails to transfer to causal control would weaken the extrapolation interpretation. Preserved gains under causal current-window training would support a more robust progress signal. stage-pairsstage-learningdiagnosticsinferencetraining
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version page; Figure 1; all formulation, method, training, gate and inference pages; Tables 1–6; Figures 2–7; appendix implementation, overlap diagnostics and hardware protocols. All six final original crops were individually inspected. Figure 2's simplified token arrows were cross-checked against pooled supervision and gated injection equations. All pages supporting retained numerical, method, evaluation and proposed-reproduction details are included. Bibliography pages 8–9 were read as supplied text, not images. Separate supplemental availability remains unverified; no code or experiments were inspected or executed.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 10, 11, 12, 13, 14. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and Introduction (pp. 1–2)
- Related Work (pp. 2–3)
- Task Formulation and Method (pp. 3–5)
- Experiment, ablations, trajectory analysis, efficiency and Conclusion (pp. 5–7)
- References (pp. 8–9)
- Appendix implementation, diagnostics, real-world evaluation and complete task results (pp. 10–14)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This acquisition limitation was addressed by visual inspection of all scientific-content pages and all final crops.
- Separate supplemental material availability has not been fully verified.
- The reviewed artifact is v3, stamped 14 Aug 2026. Title and nine-author identity match; earlier revisions were not supplied, so differences from v1/v2 cannot be established.
- All five supplied text chunks were read. References were read as text; bibliography pages 8–9 were not rendered. Code was not inspected and experiments were not reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title block and arXiv margin stamp
The exact catalog title and all nine authors match. The supplied artifact displays arXiv:2608.10780v3 [cs.RO], 14 Aug 2026; six affiliations, equal contribution by Xiao Liu and Yuguang Yang, and corresponding author Yan Wang are printed.
Go to primary source ↓two-futuresPDF pp. 1–3, Introduction; Figure 1 and caption; Task Formulation, Eqs. (1)–(4)
StageWAM distinguishes a local video-action future from a predicted latent task-progress target that conditions the WAM.
Go to primary source ↓stage-pairsPDF p. 3, Stage-pair construction, Eqs. (5)–(8); p. 10, Implementation Details / Stage-pair construction
Candidate centers use eight-frame spacing, terminal inclusion and a 32-candidate cap. Smoothed, normalized pooled/token feature changes drive NMS with radius three retained-list positions and at most five selections. Dense frames pair with the next boundary; centered slices use offsets −32 through 31 and endpoint clipping.
Go to primary source ↓stage-learningPDF pp. 3–4, Goal-conditioned stage prediction and Eq. (9); Figure 2(c)
Frozen V-JEPA2 independently encodes current and target slices. The initialized predictor and instruction adapter learn using cosine loss on normalized pooled prediction/target representations; the diagram depicts token-level outputs.
Go to primary source ↓architecturePDF p. 4, Figure 2(a–c), caption and Stage II: Training the Stage-Conditioned WAM
A frozen Stage-JEPA branch supplies video-stream guidance to a Motus WAM. Wan2.2 video, action, and frozen-Qwen3-VL understanding streams interact through joint attention at every transformer layer. Video and action heads predict flow fields from separately noised targets.
Go to primary source ↓gatePDF p. 5, Eqs. (10)–(14) and surrounding paragraphs; p. 6, w/o gate regularizers definition; p. 10, Gate and token sampling
Predicted tokens are pooled and MLP-projected before a sample-specific residual is broadcast to all video tokens. A global gate is 0.2 sigmoid(beta), initialized to 0.02. Eq. (13) adds video, action and regularization losses; the printed Eq. (14) starts with an equals sign but gives the two auxiliary gate terms described separately on p. 6.
Go to primary source ↓trainingPDF p. 5, Implementation Details; p. 10, Training details and Gate and token sampling
Both stages use eight A800 80GB GPUs, AdamW and effective batch 256. Stage I uses 1,000 updates and Stage II four epochs, each trained once with seed 0 and final-checkpoint selection. The appendix specifies precision, warmup, token subsampling and target-frame/action offsets.
Go to primary source ↓inferencePDF p. 5, Closed-Loop Inference, Eqs. (15)–(16); p. 11, Closed-loop inference
Deployment uses only an observed 64-frame history, padded with the earliest frame. Frozen Stage-JEPA predicts guidance before WAM prediction; executing the action chunk produces new observations for replanning. Ground-truth boundaries and target slices are unavailable.
Go to primary source ↓evaluationPDF p. 5, Experiment / Datasets and Baselines; p. 7, Execution Efficiency
Fifty RoboTwin tasks are evaluated in clean/randomized settings with 100 closed-loop episodes per pair, totaling 10,000 per checkpoint. Baselines come from Motus; the authors state the same StageWAM/Motus protocol. Page 5 says early stopping disabled, whereas p. 7 says successful episodes terminate immediately and failures reach the horizon.
Go to primary source ↓benchmarkPDF p. 6, Table 1, all rows and caption
StageWAM clean/randomized success is 91.42/89.08%, versus Motus 88.66/87.02%. Handover is 96.50/93.50 versus 82.00/68.00; acquisition/lifting is 92.60/92.00 versus 94.20/94.60. Seven categories partition 50 tasks, with unweighted task means.
Go to primary source ↓ablationsPDF p. 6, Table 2 and Ablation Study
Overall success is 87.23% without JEPA, 89.89% with guidance on current-frame tokens only, 89.98% without Stage-I training, 89.88% without gate regularizers, and 90.25% for StageWAM. The current-frame-only condition is described in prose as w/o future-latent injection.
Go to primary source ↓efficiencyPDF pp. 6–7, Trajectory analysis; p. 7, Figures 3–4 and Execution Efficiency
Figure 3 is one successful comparison with terminal-frame padding. Figure 4 reports Motus/StageWAM lengths of 156.28/132.80 steps across all rollouts and 79.57/74.82 on successful rollouts, corresponding to reported reductions of 15.02% and 5.97%.
Go to primary source ↓diagnosticsPDF p. 11, Table 5 and Stage-boundary and slice-overlap diagnostics
Diagnostics cover 27,244 episodes and 6,008,109 dense pairs. Median next-boundary distances are 24/25 frames for clean/randomized data; slice-pair overlap is 86.77/86.34%, and cross-boundary current slices are 84.67/82.97%. The median stage length is 34 frames; 81.72% of intervals are shorter than 64 frames.
Go to primary source ↓real-protocolPDF p. 10, Table 3; p. 11, Real-world evaluation / Scoring / Results
LIFT2 (ARX R5) evaluation uses 20 trials per task and separately trained models with 50 cube and 116 power-strip trajectories. Terminal-state progress scores range from 0 to 100; safety failures score zero. Cube scoring levels are 0/30/70/100; power-strip levels are 0/30/60/100.
Go to primary source ↓real-resultsPDF p. 10, Table 4, Cube_Grasp, Power_Strip and Average columns; p. 11, Results
Mean progress scores for StageWAM are 45.00, 51.00 and 48.00 overall; pi_0.5 scores are 33.50, 49.50 and 41.50. Only task means and an unweighted cross-task mean are reported.
Go to primary source ↓task-resultsPDF p. 13, Table 6, all 50 task rows, Average row and caption
Task-level results and StageWAM/Motus means are printed. GO-1 and X-VLA clean aggregate values are 37.80 and 72.80 here, versus 37.86 and 72.88 in Table 1; the Table 6 caption says baseline printed aggregates are preserved.
Go to primary source ↓robot-visualsPDF p. 12, Figures 5–6 and captions; p. 14, Figure 7 and caption; p. 10, Table 3
Real-robot still sequences depict cube manipulation and power-strip interaction. Cube terminal panels show the cube on the tray; Table 3 defines the top cube score as stable holding until rollout end. The stills do not establish which scoring state those example rollouts received.
Go to primary source ↓boundariesPDF p. 6, Main Results discussion; p. 7, Conclusion
The authors discuss contact, constrained motion and timing as challenges and propose lower-latency control and multiscale representations as future work.
Go to primary source ↓8.5 Primary sources
StageWAM: Joint-Embedding Stage Prediction for World-Action Models in Robot Manipulation ↗
PDF · 7,187 extracted words
Source fingerprint
2502bc81e7ad00c3c13544db03829670c1b5ddf4959eeb02fda4c3406438c4ac