JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling
1. Paper overview
In one sentence: Dense current–future embedding supervision improves action learning through a shared predictor, while deployment keeps the action pathway and omits explicit transition prediction. e03e04e07e08e11e14e17e20e21
| At a glance | What to know |
|---|---|
| Research problem | Author claim How can a policy learn useful transition structure without paying for future-video generation during control? The authors identify two weaknesses in latent alternatives: compressed targets can lose spatial detail, and separate predictive objectives may weakly influence the representations that generate actions. e02 |
| Core mechanism | |
| A key reported result | LIBERO-Plus scene-shift generalization: JEPA-WAM 79.2; π0.5 + JEPA objective 86.3. Average task success (%). Train jointly on four LIBERO suites; transfer without OOD fine-tuning. Without robot-policy pretraining: ResVLA 77.1. Pretrained π0.5: 84.5. Best listed averages within their respective pretraining groups. Gains are not uniform: π0.5 Camera falls from 69.4 to 66.0. e09e11 |
| Reading caution | Author claim Task-shared transition supervision may be less expressive when one observation supports substantially different futures under different instructions. Language-conditioned or multimodal targets are future work. e22 |
Core contributions
- Source description
The target also supervises pretrained π0.5 through auxiliary future tokens, while attention masking preserves its original action pathway. e07
Figure 3. A shared predictor learns transition structure and supplies a dedicated action readout. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the global and wrist images at the bottom center. The current encoder and visual projector feed Qwen. Follow the blue visual-position states upward through the MLP to the predicted joint target. On the left, the teacher jointly encodes current and future images; Equation (2) makes this target stop-gradient, despite the alignment line being drawn with two arrowheads. Next follow the orange action placeholders: their output states condition the expert through cross-attention. Proprioception, learnable future tokens and noisy actions enter the expert separately. The caption and deployment text identify the paired-image target branch and prediction head as training-only components. e03e04e05e08e14e24
What it supports. The architectural benefit is a shared learning signal with separate readouts. Transition loss shapes the predictor that also produces action-conditioning states, but the expert need not consume the entire predicted transition grid. Appendix A.1 specifies 64 action placeholders; removing them and using all hidden states reduces LIBERO-Plus performance to 73.1% from 79.2%.
Where the evidence stops. The diagram marks the attention stack ×8; Appendix A.1 calls the expert a 16-layer DiT-L. The counting convention is unresolved. Neither the alignment arrow nor the learned transition target implies deployment-time future simulation.
2. Motivation
2.1 The problem and the proposed response
How can a policy learn useful transition structure without paying for future-video generation during control? The authors identify two weaknesses in latent alternatives: compressed targets can lose spatial detail, and separate predictive objectives may weakly influence the representations that generate actions. e02
2.2 What this reading follows
JEPA-WAM asks a robot policy to learn what changes between two observations while it learns actions. A frozen video encoder supplies a structured training target: each camera retains a grid of patch features, and current and future images are encoded together. The policy predicts that target from the current view, but its action expert reads separate, task-aware hidden states. This distinction explains both the intended benefit and the deployment saving. The six visuals follow the standalone architecture, transfer to pretrained π0.5, generalization results, design ablations, a controlled representation probe and real robot evaluation. Throughout, policy success, feature predictability and partial-credit completion remain separate evidence. e03e04e07e08e11e14e17e20e21
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Supports the recorded classification
The distinct Qwen predictor and DiT action expert support a modular Dual-system reading, qualified by shared transition/action-conditioning representations inside Qwen. WAM and latent/JEPA labels fit the explicit embedding prediction objective. Other mechanisms and Outside quadrants fit auxiliary transition supervision without inference-time future generation or inverse-dynamics action extraction; joint training alone does not establish a single unified generator. e03e04e07e08
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Why encode the pair together?
The target is built from an observed transition, rather than from an action-conditioned imagined rollout. For each camera, the frozen V-JEPA encoder processes current and future images together. Its two-frame temporal tubelet produces one token per spatial location, allowing a target with the same camera and patch ordering as the current-image representation. The policy can therefore compare corresponding patch features directly. A future-only target supplies the resulting scene in isolation; explicit endpoint subtraction instead combines independently encoded features after encoding. Table 6 tests these constructions within policy training, while the frozen probes test what information their teacher features expose. The joint target performs better on policy success and temporal structure, but the displacement control shows that this advantage depends on what is being predicted. e03e15e16e17e18
Figure 4. Auxiliary target prediction can train a pretrained policy without supplying future features to its action expert. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the bottom path as the original policy: an image and task enter the pretrained VLM, which retains its action expert. The added future tokens create the upward auxiliary branch. Appendix A.4 specifies 64 tokens reshaped to an 8×8 grid, projected and bilinearly upsampled to match a 24×24 ViT-G target. The snowflake marks the teacher as frozen; the two-headed alignment line denotes a loss comparison, not teacher optimization. The sketch omits a crucial attention rule: future queries can read image and language context, while original action tokens cannot attend to those queries. That mask distinguishes auxiliary representation training from future-conditioned action generation. e07e08e11
What it supports. This transfer keeps π0.5's original perception and action routes while adding a training signal to its shared backbone. On LIBERO-Plus, the corresponding average rises from 84.5% to 86.3%. At inference the teacher and spatial prediction head disappear, although the masked future tokens remain in the prefix.
Where the evidence stops. This is a distinct instantiation: it uses a ViT-G teacher and upsampled query grid, unlike standalone JEPA-WAM's ViT-L patch-position prediction. The schematic does not display the attention mask; use Appendix A.4 to implement it.
5.2 How transition learning reaches the action expert
The two objectives meet inside the shared Qwen predictor. Visual-position hidden states are mapped back into V-JEPA space, and patch-wise cosine distance trains them against the detached joint target. Later action-placeholder states aggregate visual context and the instruction, then condition a separate DiT action expert. During robot learning, the base Qwen weights, visual projector and V-JEPA encoder are frozen; LoRA adapters let the shared predictor respond to both losses. This explains why action readout matters: the full-hidden ablation changes what the expert receives, even though predictive supervision remains. During deployment, only the current encoding, predictor and action expert run. Four Euler steps generate a chunk from noise; the source does not specify the execution prefix or when new feedback triggers replanning. e04e05e06e08e14
5.3 What transfers, and what the experiments establish
The π0.5 experiment transfers the supervision principle, not the complete standalone architecture. Auxiliary queries predict an upsampled teacher grid, while masks keep the original action tokens from reading those queries. The reader's interpretation is that this tests whether training can improve an existing action pathway without explicit future-feature conditioning. Its strongest listed LIBERO-Plus average supports that possibility, but the category regressions prevent a claim of universal robustness. RoboTwin similarly separates a substantial Clean gain from a very small Random gain for the pretrained model. Finally, the real-world bars concern actual robot rollouts but use partial-credit completion. Those outcomes complement the frozen feature probes; they do not turn probe accuracy into evidence of a deployed model predicting the correct future. e07e08e11e13e16e17e20e21
5.4 Training and inference
During training
Vision–language initialization trains the projector and full Qwen backbone on LLaVA v1.5 for two epochs, keeping V-JEPA frozen. Robot training then freezes the encoder, projector and base Qwen weights; it updates rank-32 LoRA adapters, the prediction head and action expert. e05e06
Main LIBERO training uses AdamW, 60K steps, global batch 128, BF16/FSDP and eight GPUs. Learning rate decays from 2×10⁻⁴ to 10⁻⁵ with 3% warmup; transition weight is 0.5. For π0.5, the auxiliary weight warms to 0.1 over 1K steps. e06e07
During inference
Current images and instruction produce C_t; the expert also uses proprioception and integrates Gaussian noise with four Euler steps. Joint-target encoding and the transition head are absent. The paper specifies action chunks but not how many actions execute before new feedback and replanning. e04e08
For π0.5, the teacher and spatial head are removed, but learned future tokens remain in the prefix, masked from action tokens. Predicted future features are never explicitly supplied to the action expert. e07e08
5.5 Implementation flow
- Encode each camera without pooling
Frozen V-JEPA 2.1 ViT-L/16 processes 384×384 images into 24×24 grids of 1024-dimensional features. Views are concatenated in fixed order. Jointly encoding current and future frames produces the same grid because the video tokenizer groups two frames into one temporal tubelet. e03e05
- Separate predictive positions from action readout
A visual projector feeds Qwen2.5-0.5B. Visual-position hidden states, Q_t^{wm}, predict the target through a token-wise MLP. Sixty-four appended action placeholders produce C_t, aggregating preceding visual and language context. These are distinct readouts of the shared predictor. e04e05
- Generate actions with a dedicated expert
The DiT action expert receives C_t, proprioception, noisy actions and 32 learnable future tokens. LIBERO uses 7-dimensional actions, horizon H=8 and target offset δ=31; RoboTwin uses 14-dimensional actions, H=50 and δ=50. Future frames are clipped at trajectory ends. e04e05e09e12
- Transfer supervision into π0.5
Sixty-four auxiliary tokens form an 8×8 grid of 2048-dimensional features. A projection to 1408 dimensions and bilinear upsampling to 24×24 match a frozen V-JEPA ViT-G target. These queries attend to image/language context; original action tokens cannot attend to them. e07
6. Experiments & results
JEPA-WAM couples dense current–future embedding prediction with robot action learning, then removes the transition head at deployment. Frozen V-JEPA features feed a Qwen predictor whose dedicated action representations condition a separate flow-matching expert. The experiments support improved scene-shift robustness, while exposing limits on language-dependent futures and the distinction between real-world completion scores and binary success.
6.1 Read the original evidence
Table 2. Scene-shift robustness improves on average, with different strengths across perturbations. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First choose the appropriate pretraining group. PT refers to large-scale robot-policy pretraining; its absence does not mean the vision and language components were trained from scratch. Params. gives the main backbone size in billions and excludes the action expert. Then read across Camera, Robot, Language, Light, Background, Noise and Layout before consulting Avg. All listed evaluations transfer from LIBERO demonstrations to LIBERO-Plus without OOD fine-tuning. The highlighted upper row evaluates standalone JEPA-WAM; the highlighted lower row evaluates transition supervision added to π0.5. Their averages answer different comparisons and should not be treated as an isolated scaling experiment. e06e09e10e11
What it supports. JEPA-WAM reaches 79.2%, exceeding the listed non-policy-pretrained ResVLA average of 77.1%. Adding the objective to π0.5 raises 84.5% to 86.3%, a 1.8 percentage-point improvement and the highest average in this table. The standalone model's particularly strong Camera result coexists with a Language score of 68.2%.
Where the evidence stops. Average improvement is not uniform robustness: π0.5's Camera score decreases from 69.4% to 66.0%. No seed uncertainty is reported, and group membership does not equalize architecture, pretrained data or total parameter count.
Figure 5. Real robot evaluation supports improved completion under altered scenes, with partial credit for subtasks. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the photographs to identify the evaluated domain: bread and fruit placement, three-block placement, stacking, and drawer manipulation. Background or initial-configuration changes create OOD conditions; tasks and instructions remain fixed. In each bar cluster, gray is π0, green JEPA-WAM, blue π0.5 and pink π0.5 with the JEPA objective. Compare colors within a task and then the outlined average bars. Although the plotted axis reads Success Rate, Appendix B.3 defines normalized completion scores, including partial credit for multi-step tasks. Table 15 provides unrounded task means; Tables 16–19 provide all ten rollout scores per task and setting. e20e21
What it supports. The rounded average bars are 52/60/78/90 for ID and 23/54/73/85 for OOD in legend order. Table 15 gives JEPA-WAM 59.82/54.18 and augmented π0.5 90.34/84.68. Standalone JEPA-WAM exceeds π0 but remains below π0.5; adding transition supervision improves the pretrained policy in both settings.
Where the evidence stops. These percentages are completion scores, not binary rollout success. Five tasks with ten rollouts per condition provide limited coverage, and the OOD changes do not test new task objectives or language instructions. The chart supplies no uncertainty intervals.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO-Plus scene-shift generalization Train jointly on four LIBERO suites; transfer without OOD fine-tuning. | JEPA-WAM 79.2; π0.5 + JEPA objective 86.3. Average task success (%) | Without robot-policy pretraining: ResVLA 77.1. Pretrained π0.5: 84.5. Best listed averages within their respective pretraining groups. Gains are not uniform: π0.5 Camera falls from 69.4 to 66.0. e09e11 |
| LIBERO in-distribution manipulation Corresponding test tasks after joint four-suite training. | JEPA-WAM 96.7; π0.5 + JEPA objective 97.8. Average task success (%) | Fast-WAM 97.6 without robot-policy pretraining; pretrained π0.5 96.9. Competitive ID performance; standalone JEPA-WAM is not the strongest listed ID model. e09e10 |
| RoboTwin 2.0 Clean-to-Random transfer Train on Clean demonstrations; evaluate the same 20 tasks in Clean/Random. | JEPA-WAM 79.9 / 36.9; π0.5 + JEPA objective 84.6 / 37.5. Mean task success (%), Clean / Random | DP3 73.9 / 8.3; pretrained π0.5 75.4 / 37.2. Large standalone advantage over DP3 under randomization; the pretrained Random improvement is only 0.3 percentage points. e12e13 |
| LIBERO-Plus mechanism ablations Controlled policy-training setup; category-wise evaluation. | Joint target 79.2. Average task success (%) | V-JEPA only 77.0; future-only 77.3; endpoint difference 70.9; iREPA alignment 74.7; Lower-16 76.5; full hidden 73.1. Joint targets and dedicated action readout help. The iREPA variant changes both convolutional mixing and normalization, so it does not isolate spatial correspondence alone. e14e15 |
| Fixed-future temporal-gap decoding RoboTwin Clean-20 frozen ridge probe; 30/10/10 episode-disjoint train/validation/test episodes per task. Future anchors are fixed across gaps 0, 10, 20, 30, 40 and 50. | Joint target 67.2 (95% CI 65.3–69.1). Six-way classification accuracy (%) | Endpoint difference 47.0 (45.0–49.1); future-only 16.7. Joint teacher features make temporal separation more accessible. This is a two-endpoint feature probe, not deployed-policy prediction accuracy. e16e17 |
| Unseen temporal-gap regression Same episode-disjoint RoboTwin probe splits; train numerical gap prediction on {0,20,40} and test on {10,30,50}. | Joint target 8.88 overall. Mean absolute error (frames), lower is better | Endpoint difference 13.32 overall. Temporal information transfers to unseen separations, including extrapolation at gap 50; joint encoding is worse at gap 10 (10.22 versus 8.09 frames). e16e17 |
| Residual trajectory decoding Frozen two-endpoint RoboTwin features predict the 49×14 intermediate-state residual after removing linear interpolation between states 50 steps apart; episode-disjoint probe splits. | Joint target 0.582; paired gain 0.097 (95% CI 0.082–0.112). Mean R², higher is better | Endpoint difference 0.485. Supports accessible within-interval structure, not universal superiority: the separate direct-displacement control favors differences, 0.740 versus joint 0.718 R². e16e18 |
| Real-world bimanual task completion AgileX Cobot Magic; five tasks, 100 demonstrations/task and 10 rollouts/task/setting; background or initial-configuration shifts without fine-tuning. | JEPA-WAM 59.82 / 54.18; π0.5 + JEPA objective 90.34 / 84.68. Mean normalized task-completion score (%), ID / OOD | π0 51.82 / 22.50; π0.5 77.52 / 72.50. Partial-credit scores, not binary success. JEPA-WAM exceeds π0 but trails π0.5; the auxiliary objective improves π0.5 in both settings. e20e21 |
| RoboTwin inference efficiency Recorded same-setting inference comparison; target branch disabled. | JEPA-WAM 85.00 ms / 11.76 Hz. Median latency (ms); corresponding frequency (Hz) | ABot-M0 125.23 / 7.99; π0.5 54.05 / 18.50; augmented π0.5 55.12 / 18.14. Faster than ABot-M0, slower than π0.5. Hardware and timing boundaries are unspecified; frequency is not an established low-level control rate. e23 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 4. The encoder, prediction target and action readout each affect generalization. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare rows a and b first: transition prediction is disabled in both, so the 73.2-to-77.0 change tests the visual representation choice. Compare b with the final row to locate the extra benefit of transition supervision on top of V-JEPA. Row c replaces the joint target with future-only features. Row d changes alignment using a per-view 3×3 convolution and spatial normalization. Row e supervises the sixteenth predictor layer rather than its final hidden states. Row f removes dedicated action placeholders and feeds the expert the full final hidden sequence. Each column is a perturbation category, so the average can conceal opposing category changes. e14e15
What it supports. The full model scores 79.2%, versus 77.0% for V-JEPA without transition supervision and 77.3% for future-only supervision. The full-hidden variant falls to 73.1%, consistent with a useful dedicated action readout. Appendix Table 6 adds a strong target control: explicit endpoint differencing reaches only 70.9%.
Where the evidence stops. The iREPA variant changes normalization as well as local mixing; its 74.7% cannot isolate patch correspondence alone. Policy ablations lack seed intervals, and claims about representation interference remain interpretations of these behavioral changes.
Figure 6. Holding the future image fixed tests whether embeddings expose the relation between endpoints. Original paper, p. 15 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read rows as true temporal gaps and columns as predicted gaps, in frames. Dark diagonal cells are correct predictions; the color bar shows row-normalized test percentages. Within each six-gap group, the future image is identical, so future-only features cannot distinguish the labels. Endpoint differencing sees both frames and provides a stronger comparator. Joint encoding produces a clearer diagonal, particularly away from the zero-gap class. These are frozen pooled teacher features evaluated with a linear probe, not the policy's predicted targets. Appendix C.2 splits episodes before fitting, tunes regularization on validation episodes and bootstraps at episode level; the figure reports 200 test episodes and 2,400 pairs. e03e08e16e17e18
What it supports. The joint target reaches 67.2% accuracy against 47.0% for endpoint differences and 16.7% for future-only features. Table 7 gives a 95% interval of 65.3–69.1% for the joint score. This supports more accessible temporal information in the teacher representation, providing a plausible complement to the target ablation.
Where the evidence stops. The probe receives both visual endpoints; the deployed policy receives no future frame. Better probe accuracy therefore does not establish accurate policy imagination. Joint encoding also loses to endpoint differences on the direct displacement probe in Table 10.
7. Analysis & limitations
7.1 What the evidence leaves open
Task-shared transition supervision may be less expressive when one observation supports substantially different futures under different instructions. Language-conditioned or multimodal targets are future work. e22
Spatial change correlates only moderately with RGB change (median 0.279 against matched static-current encoding); it does not establish precise motion localization. e19
Policy comparison and ablation tables do not report seed uncertainty. Real-world evidence covers five tasks with partial-credit scoring. The frozen-probe bootstrap intervals do not quantify uncertainty in robot success. e11e14e16e20e21
7.2 Questions for discussion
- Would language-conditioned transition targets help when identical scenes admit conflicting instructed actions?
- Does the joint-target advantage survive a matched two-frame static-target policy control?
8. Reproducibility audit
8.1 Requirements and known gaps
Preserve view/patch ordering, target offsets, end-frame clipping, frozen-module boundaries and π0.5 attention masks. Required pretrained assets include V-JEPA, Qwen and, for transfer, π0.5; initialization additionally requires LLaVA v1.5. e03e05e06e07
The supplied description omits GPU models, software versions, policy evaluation seeds/trial counts in simulation, detailed RoboTwin optimization settings and the action-execution/replanning schedule. Eight GPUs are specified for LIBERO, but elapsed training cost and inference hardware are not. e06e08e09e12e23
Figure 3 marks the action-attention stack ×8, whereas Appendix A.1 describes a 16-layer DiT-L. Their counting convention is unexplained; an implementation must resolve this ambiguity rather than silently choose a depth. e24
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate dynamic supervision from static video-mode alignment
Reader-proposed, not performed: start from identical V-JEPA/Qwen initialization and compare no transition loss, future-only targets, repeated-current two-frame targets, and true current–future targets on LIBERO. Keep camera/patch order, H=8, δ=31, boundary clipping, policy budget and predictive-arm loss weight fixed; evaluate matched seeds on all LIBERO-Plus categories without OOD fine-tuning. The repeated-current control adopts the matched video-mode logic of the spatial diagnostic, but applies it to policy training. Report per-seed and category results. If static supervision matches the joint target, the claim that transition content drives the added policy benefit would be weakened; a repeatable joint advantage would distinguish it from generic auxiliary alignment. e03e05e06e09e14e19
Check 2: Test whether π0.5 benefits through training or hidden future conditioning
Reader-proposed, not performed: fine-tune the same π0.5 initialization as baseline, with masked auxiliary tokens but zero transition weight, and with the published masked transition objective. Hold action training and evaluation fixed. At inference, fix observations and action noise, retain sequence positions, and perturb only auxiliary token contents. Verify that the teacher/head are absent and check whether action outputs change, including through indirect attention paths. Compare LIBERO-Plus categories and measured latency across variants. Supervision gains with invariant action outputs would support the intended training-mediated route; output sensitivity would expose unintended conditioning, while a zero-weight variant matching the supervised model would challenge attribution to the JEPA loss. e07e08e11e23
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version page, all seven figures, all nineteen tables, method equations, training/deployment details, benchmark protocols, all representation diagnostics, complete RoboTwin task tables and all real-world rollout tables on the declared pages. Read all six text chunks, including references on pages 9–10. Inspected every final crop at its extracted size; narrow Figure 4 and Table 4 were rendered at 400 DPI, other crops at 200 DPI. Figure 3's ×8 marker versus Appendix A.1's 16-layer description remains unresolved. Figure 4's omitted attention mask was checked against Appendix A.4. Figure 5's metric label is interpreted using Appendix B.3 and Table 15. No separate supplement or code was inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, authors, affiliations, arXiv v1 identifier/date and Abstract (p. 1)
- 1 Introduction (pp. 1–2)
- 2 Related Work (pp. 2–3)
- 3 Method, including 3.1–3.4 (pp. 3–5)
- 4 Experiments, including 4.1–4.4 (pp. 5–7)
- 5 Conclusion and 6 Limitations (p. 8)
- References (pp. 9–10)
- A Implementation Details, A.1–A.5 (pp. 11–12)
- B Experimental Details, B.1–B.3 (p. 13)
- C Additional Analysis, C.1–C.3, including all representation probes and spatial diagnostic (pp. 14–18)
- D Complete RoboTwin 2.0 Results (pp. 19–20)
- E Real-World Detailed Results, E.1–E.2 and all rollout records (pp. 21–22)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction-only image omission above was addressed by visually inspecting PDF pages 1–8 and 11–22, including every figure and table. References on pages 9–10 were read as text.
- Only the supplied arXiv:2608.09381v1, dated 10 August 2026, was reviewed; no later revision or inter-version comparison was supplied. Title and all nine authors match the catalog. The title page additionally lists XYZ Embodied AI, Beijing, China, which the catalog affiliation string omits.
- No linked code, project site or external material was inspected, and no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title block, affiliations and arXiv margin
Exact title and nine authors match the catalog; observed artifact is arXiv:2608.09381v1, 10 August 2026. Affiliation 2 is XYZ Embodied AI, Beijing, China.
Go to primary source ↓e02PDF pp. 1–2, Section 1 and Figure 2
Motivates dense latent transition targets and direct supervision of the action-conditioning backbone while avoiding deployment-time future generation.
Go to primary source ↓e03PDF p. 3, Section 3.1, Eqs. (1)–(2) and Figure 3
Current views are independently encoded and concatenated; two-frame joint targets are stop-gradient and preserve the current spatial grid through two-frame tubelets.
Go to primary source ↓e04PDF p. 4, Sections 3.2–3.3, Eqs. (3)–(7)
Shared Qwen predictor supplies visual-position target predictions and dedicated action readouts. Defines patch cosine loss, flow-matching action loss and joint objective; removes prediction head at deployment.
Go to primary source ↓e05PDF p. 11, Appendix A.1
Specifies ViT-L/16, 384×384 inputs, 24×24×1024 grids, projector and head dimensions, rank-32 LoRA with scaling 64/dropout 0.1, 64 action placeholders, 32 expert future tokens, horizons and clipped target offsets.
Go to primary source ↓e06PDF p. 11, Appendices A.2–A.3, Eqs. (10)–(12)
LLaVA v1.5 initialization trains projector/full Qwen for two epochs; subsequent policy training freezes base modules. Gives loss weight, Beta schedule and LIBERO optimizer, precision, eight-GPU batch and step settings.
Go to primary source ↓e07PDF pp. 4–5, Section 3.4 and Figure 4; p. 12, Appendix A.4
π0.5 uses 64 auxiliary future tokens, 8×8 reshape, 1408-dimensional projection, 24×24 upsampling and a frozen ViT-G teacher. Loss warms to 0.1 over 1K steps; action tokens cannot attend to new queries.
Go to primary source ↓e08PDF p. 4, Section 3.3; p. 12, Appendix A.5
Standalone inference omits target encoding/head and uses four Euler steps. π0.5 retains masked future tokens but removes teacher/spatial head. No execution-prefix or replanning interval is specified.
Go to primary source ↓e09PDF p. 13, Appendix B.1
Joint LIBERO training and direct LIBERO-Plus transfer; primary/wrist images, language, proprioception, 7-D actions, horizon 8 and offset 31. References standard evaluation without enumerating seeds or rollout counts.
Go to primary source ↓e10PDF p. 5, Table 1, Avg. column and caption
LIBERO averages: JEPA-WAM 96.7, Fast-WAM 97.6, π0.5 96.9 and augmented π0.5 97.8. Backbone parameter counts exclude the action expert.
Go to primary source ↓e11PDF p. 6, Table 2, pretraining groups, Camera and Avg. columns
LIBERO-Plus averages: ResVLA 77.1, JEPA-WAM 79.2, π0.5 84.5 and augmented π0.5 86.3. Camera for π0.5 falls 69.4 to 66.0; standalone Language is 68.2. No uncertainty is tabulated.
Go to primary source ↓e12PDF p. 13, Appendix B.2
RoboTwin trains on Clean demonstrations for 20 tasks, evaluated in Clean and Random. Three cameras, 14-D actions, horizon 50 and clean-trajectory x-prediction are specified.
Go to primary source ↓e13PDF p. 6, Table 3, AVG columns; pp. 19–20, Tables 13–14, Average rows
Clean/Random success: DP3 73.9/8.3, JEPA-WAM 79.9/36.9, π0.5 75.4/37.2 and augmented π0.5 84.6/37.5; complete per-task tables supplied.
Go to primary source ↓e14PDF p. 6, Table 4; p. 7, Section 4.3; p. 14, Appendix C.1
Ablation averages are 73.2 for DINO+SigLIP, 77.0 V-JEPA only, 77.3 future-only, 74.7 iREPA, 76.5 Lower-16, 73.1 full hidden and 79.2 full model. iREPA changes 3×3 convolution and spatial normalization; Lower-16 supervises layer 16; full hidden removes action placeholders.
Go to primary source ↓e15PDF p. 14, Appendix C.1, Eq. (13) and Table 6
Explicit endpoint feature difference scores 70.9 versus 77.3 future-only and 79.2 joint current–future on LIBERO-Plus.
Go to primary source ↓e16PDF p. 14, Appendix C.2, Eqs. (14)–(16), representation/probe setup
Frozen pooled representations and ridge probes use 1,000 RoboTwin Clean-20 episodes, per-task 30/10/10 episode splits, validation-only regularization and 1,000 paired episode bootstraps. Fixed-future gap classification uses six gaps and 12,000 pairs.
Go to primary source ↓e17PDF p. 15, Tables 7–8 and Figure 6
Fixed-future gap accuracy: joint 67.2% (65.3–69.1), difference 47.0% (45.0–49.1), future-only 16.7%. Figure gives 200 test episodes/2,400 pairs. Train-gap set {0,20,40}, test {10,30,50}: overall MAE 8.88 versus 13.32, with joint worse at gap 10 (10.22 versus 8.09).
Go to primary source ↓e18PDF p. 16, Appendix C.2, Eqs. (17)–(19), Tables 9–10
Visual endpoints predict a 49×14 residual trajectory after removing linear endpoint displacement. Joint/difference R² is 0.582/0.485; paired gain 0.097 (0.082–0.112). Direct endpoint-displacement R² reverses to 0.718/0.740; paired effect −0.022 (−0.036 to −0.009).
Go to primary source ↓e19PDF pp. 16–18, Spatial Diagnostic; p. 17, Eqs. (20)–(22), Table 11 and Figure 7
Matched two-frame static controls avoid image/video-mode confounding. On 1,000 transitions from 200 held-out episodes, median RGB-change correlation is 0.279 for static-current, 0.182 static-future and 0.190 static-endpoint-mean comparison; correspondence is moderate.
Go to primary source ↓e20PDF p. 13, Appendix B.3 and Table 5
Real platform has two 6-DoF arms and three cameras. Five tasks have 100 demonstrations each and 10 ID/OOD rollouts per task. OOD changes background/configuration without changing instruction or fine-tuning; scores include partial credit.
Go to primary source ↓e21PDF p. 7, Figure 5 and Section 4.4; p. 21, Table 15; pp. 21–22, Tables 16–19
Table 15 ID/OOD averages: π0 51.82/22.50, JEPA-WAM 59.82/54.18, π0.5 77.52/72.50, augmented π0.5 90.34/84.68. Figure 5 rounds averages and labels the axis Success Rate, while Appendix B.3 and Table 15 define completion scores.
Go to primary source ↓e22PDF p. 8, Section 6 Limitations
Task-shared, largely language-independent temporal targets may be less expressive for different instruction-dependent transitions from the same observation.
Go to primary source ↓e23PDF p. 18, Appendix C.3 and Table 12
Median ms/Hz: JEPA-WAM 85.00/11.76, ABot-M0 125.23/7.99, π0.5 54.05/18.50, augmented π0.5 55.12/18.14. Same RoboTwin setting is asserted, but hardware and timing boundaries are not given.
Go to primary source ↓e24PDF p. 3, Figure 3, action expert ×8 marker; p. 11, Appendix A.1, action generation paragraph
Figure depicts a self-/cross-attention stack marked ×8, whereas Appendix A.1 calls the expert a 16-layer DiT-L. No mapping between these counting conventions is supplied.
Go to primary source ↓8.5 Primary sources
JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling ↗
PDF · 9,332 extracted words
Source fingerprint
342e84d7920b38223a487c7cca0533f81b259fe85f29b3c3f3d8501acaea8d78