VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model
1. Paper overview
In one sentence: Predicting frozen video features helps a VLA learn representations that improve perturbation robustness, but the reported human-video benefit depends strongly on the evaluation setting. e03e04e06e11e12e13e14
| At a glance | What to know |
|---|---|
| Research problem | Reader analysis The authors argue that learning latent actions from pixel changes can encode camera motion, appearance and future-frame shortcuts instead of controllable transitions. They propose predicting semantic video features while withholding future observations from the VLM. Avoiding this input shortcut is an architectural property; learning disentangled physical dynamics is a stronger claim requiring empirical support. e02e04 |
| Core mechanism | |
| A key reported result | LIBERO-Plus perturbation robustness: 79.5 Average success rate (%). Seven perturbation dimensions, each covering the four original suites; fine-tuning excludes LIBERO-Plus augmentation. OpenVLA-OFT 69.6; without human videos 62.9. Using the paper’s printed Avg entries, the gains are 9.9 and 16.6 percentage points, respectively. OpenVLA-OFT’s seven displayed perturbation values average 71.4 under equal weighting, whereas its printed Avg is 69.6; the source supplies no weighting that resolves this discrepancy. The reported values are preserved, not independently reconstructed or officially corrected. VLA-JEPA leads five dimensions, but not Camera or Noise. e07e13 |
| Reading caution | Reader analysis Equation (4) asserts that deterministic target embeddings make a KL term vanish, without a complete probabilistic specification. Equation (5) then lacks a scalar distance. These are formulation gaps, not justification for inventing a corrected objective. e05 |
Core contributions
Figure 2. Human video supervises state prediction; labeled robot data add action supervision. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the upper panel. The left V-JEPA branch supplies the current state, the central VLM supplies latent actions, and both arrows enter the world model. Its predicted next state is aligned with the target from the right encoder. In the lower panel, the action query and action head add a second supervised output. Cross-check the generic “Video” labels against Equation (2): only initial observations enter the VLM, while state histories enter the teacher-forced world model. The frozen target and the within-step/full versus across-step/causal attention rule are specified in Section 3.2, rather than drawn as masks or stop-gradient markers here. e03e04e05e06e07e09
What it supports. The same latent-action pathway supports learning from videos without action labels and from robot demonstrations. Robot data retain the alignment objective while adding flow matching, so the contribution is a training connection between predicted states and executable actions. This diagram does not show a planner evaluating imagined action candidates.
Where the evidence stops. The panel titles simplify the schedule: Appendix A jointly pretrains on SSv2 and DROID before downstream fine-tuning. Future-state supervision should not be confused with future images entering the VLM; that restriction is explicit in Equation (2).
2. Motivation
2.1 The problem and the proposed response
The authors argue that learning latent actions from pixel changes can encode camera motion, appearance and future-frame shortcuts instead of controllable transitions. They propose predicting semantic video features while withholding future observations from the VLM. Avoiding this input shortcut is an architectural property; learning disentangled physical dynamics is a stronger claim requiring empirical support. e02e04
2.2 What this reading follows
VLA-JEPA asks what a robot policy should learn from human video when no robot actions are labeled. Its answer is to make latent tokens useful for predicting future semantic states, then let an action head learn how those tokens relate to robot trajectories. Read the architecture as a training system: the world predictor supplies supervision, while the deployment description uses a conditional action generator. The strongest evidence comes from LIBERO-Plus, where appearance and layout disturbances expose a larger advantage than ordinary LIBERO. Physical tests and the SimplerEnv ablation add essential qualifications: robustness to disturbance, transfer of skills and accurate instruction following remain different capabilities. e03e04e06e11e12e13e14
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Joint prediction |
| Quadrant | Q3 · Dual-system × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded Dual-system × Joint prediction classification is supported by separate VLM, latent world predictor and action-head modules coupled through latent conditioning and joint robot-data objectives. This is joint state/action learning with a distinct world-model branch, not one transformer emitting both modalities or inverse dynamics from observed future frames. The classification should not imply deployment-time world-model planning. e03e04e06e08
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Make latent actions predict state change without seeing the target
The key separation is between the VLM’s information and the world predictor’s information. Equation (2) generates every latent transition representation from initial images and the language instruction. Equations (1) and (3) instead let the separate predictor use encoded state histories to learn the next transition. During teacher forcing, those histories come from the training video. Causal attention blocks later time steps, while tokens within a time step interact bidirectionally. Consequently, “leakage-free” specifically excludes future observations from the VLM input and future time steps from the stated predictor mask; it does not mean all training branches see only the first image. Reader interpretation: this removes one shortcut, but does not prove that a latent variable uniquely represents controllable physics. The teacher-forced history may itself explain much of the next state. e02e04e05
5.2 Turn a predictive representation into a robot trajectory
The latent tokens are not robot commands. Robot demonstrations supply the correspondence between those representations and executable trajectories. The VLM appends action queries after latent queries, and its causal attention builds an action-conditioning representation from the instruction, initial views and latent context. The flow head then learns the velocity that moves an interpolated noisy trajectory toward the demonstrated action sequence. At deployment, integration starts from noise and produces actions conditioned on that representation. World-state alignment remains in the robot training objective, weighted by beta, so the policy can keep its predictive supervision while learning embodiment-specific control. The described inference does not route predicted future states into a search procedure. A reproduction must also resolve the printed alignment loss’s missing scalar distance rather than assuming an unstated implementation. e05e06e08
5.3 Ask which kind of generalization the evidence supports
A single success average would obscure the paper’s most useful distinction. Original LIBERO measures familiar simulation tasks and is already near saturation. LIBERO-Plus changes the evaluation conditions without adding its augmented demonstrations to fine-tuning; there, the human-video ablation has a much larger effect. SimplerEnv changes embodiment-specific visual conditions across a real-to-simulation gap, and its Google Robot result actually improves without human-video pretraining. Physical task OOD asks yet another question: can the system perform an instruction or placement absent from training? That result is much weaker than physical ID success or shuffled-layout robustness. Reader interpretation: the evidence favors more resilient use of learned skills over a general claim of acquiring new executable skills from video. Qualitative regrasping is suggestive, but lacks a matched causal ablation. e07e10e11e12e13e14e18
5.4 Training and inference
During training
All parameters except the world-state encoder are trainable during pretraining. The paper describes 220K Something-Something-v2 human videos and 76K DROID trajectories. Appendix A specifies joint pretraining for 50K steps, followed by 30K simulation or 20K real-world fine-tuning steps from the pretrained checkpoint. e07e09
LIBERO and LIBERO-Plus share approximately 2K original LIBERO demonstrations; no LIBERO-Plus augmentation is used. SimplerEnv adaptation uses Fractal and BridgeV2. Physical evaluation uses 100 demonstrations across three training tasks. Thus the abstract’s two-stage description includes joint human/robot pretraining in the detailed implementation. e01e07e09
Training uses eight NVIDIA A100 GPUs, global batch 256, cosine decay with linear warmup, and peak learning rates of 1e-5 for VLM/world model and 1e-4 for the action head. Human data optimize latent alignment; labeled robot data combine it with action flow matching. e05e06e07e09
During inference
The VLM supplies action-conditioning embeddings; the learned velocity field transforms Gaussian noise into an action trajectory. Table 6 lists four denoising timesteps, action dimension 7 and future action horizon 7. e06e08
No inference-time world-model search or candidate-rollout scoring is specified. The paper also leaves the exact observation refresh, action execution chunk and replanning cadence unspecified; repeated grasping is an observed behavior, not a documented feedback algorithm. e06e08e18
5.5 Implementation flow
- Construct latent states
A frozen V-JEPA2 encoder represents each view; concatenation produces the world state. The implementation uses two views, duplicates a sole available state representation, and selects two when more are available. e04e08e09
- Predict transitions from current observations
Qwen3-VL-2B receives initial images, instruction and learnable latent tokens. Their embeddings condition a separate autoregressive world predictor. Teacher forcing supplies encoded state histories: attention is bidirectional within each time step and causal across time. Withholding future images from the VLM does not mean the world model receives no teacher-forced states. e03e04e08
- Condition robot actions
Learnable action tokens follow latent tokens in the VLM sequence, letting causal attention incorporate the latent representations. Their output conditions a DiT-B flow-matching action head. The default configuration uses video horizon 8, three latent tokens per step, and 32 repetitions of the action token. e06e08
6. Experiments & results
VLA-JEPA trains a vision-language policy to produce latent actions that help a separate world model predict frozen video features, then learns continuous robot actions with flow matching. Its clearest gain is robustness to LIBERO-Plus perturbations; human-video benefits are weaker or reversed in other settings. The world model supplies training supervision, while the described deployment path generates actions without explicit future-state planning.
6.1 Read the original evidence
Table 1. Ordinary LIBERO performance is strong, with only a narrow lead over the nearest listed baseline. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each suite column before moving to Avg. VLA-JEPA is strongest on Object and LIBERO-10, whereas π0.5 leads Spatial and Goal. The indented final row removes human-video pretraining; compare it vertically with VLA-JEPA to see which suites move. The caption specifies 50 evaluation episodes per task and 500 per suite. Section 4.1 says the policy is fine-tuned on the original LIBERO demonstrations. Bold entries mark the reported best values, while underlining and shading provide additional emphasis. These markings describe the table ranking; they do not encode confidence intervals or control for different pretraining histories. e07e11
What it supports. VLA-JEPA reports 97.2% average success, compared with 97.1% for OpenVLA-OFT. Removing human videos gives 96.1%. The Object score stays at 99.6%, so the average benefit does not reflect improvement in every suite. The most informative comparison with human-video removal is the suite-level pattern, not just the bold average.
Where the evidence stops. The 0.1 percentage-point lead over OpenVLA-OFT has no reported uncertainty. Baseline rows are not a matched data-and-compute ablation, and high in-distribution success alone does not establish robustness to novel disturbances.
Table 3. The clearest advantage appears when the original LIBERO policy faces perturbations. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each column is a perturbation dimension, aggregated across the four original LIBERO suites. First compare the VLA-JEPA and indented no-human-video rows: every dimension improves with human videos. Then compare against OpenVLA-OFT to see that the aggregate advantage is uneven: Language and Layout improve markedly, while Noise is lower for VLA-JEPA. The benchmark is evaluated using a policy fine-tuned on original LIBERO demonstrations, without LIBERO-Plus augmented data. This matters because the table probes resistance to evaluation-time changes rather than adaptation on those same augmentations. Keep Camera, Robot and Noise separate; the common average can hide their different failure patterns. e07e11e13
What it supports. Using the paper’s printed Avg column, VLA-JEPA has 79.5%, versus 69.6% for OpenVLA-OFT and 62.9% without human videos: differences of 9.9 and 16.6 percentage points. The OpenVLA-OFT aggregation discrepancy described below remains unresolved. VLA-JEPA leads five of seven dimensions; the results support perturbation robustness in simulated execution, with the reported aggregate comparison kept distinct from a reproduced score.
Where the evidence stops. OpenVLA-OFT’s seven visible entries have an equal-weight mean of 71.4, but the source prints Avg 69.6 and provides no weighting that reconciles them. We preserve the original table and quote its reported average; this is not an official correction. VLA-JEPA is not best under Camera or Noise: π0-Fast reaches 65.1% Camera, and π0 reaches 79.0% Noise. No uncertainty is supplied, and this comparison does not isolate which pretraining choice causes the gain.
Figure 4. Physical deployment separates familiar-task robustness from success on unseen tasks. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the legend first: blue is π0, yellow is π0.5, and green is VLA-JEPA. The vertical axis is a success fraction from zero to one, unlike the percentage tables. The left group tests familiar tasks; the middle group changes the task; the right group shuffles object layouts for training-set tasks. Section 4.4 reports 100 training demonstrations and fine-tuning the baselines on the collected data. Appendix B specifies 10 independent trials per task and describes the unseen banana, peach and shelf tasks. Read the labels as rounded reported fractions, without inferring exact event counts from the two-decimal display. e14e18
What it supports. VLA-JEPA leads these comparisons on ID tasks at 0.70 and shuffled layouts at 0.47. Its unseen-task score is 0.17, below π0.5 at 0.20. The practical distinction is that keeping a learned skill working under rearrangement appears easier than transferring it to a new instruction, object or destination.
Where the evidence stops. No error bars are shown, and the trial counts are small. Appendix B reports failure of every model on the unseen top-shelf placement task. More stable trajectories and qualitative regrasping should not be interpreted as general safety or reliable unseen-task execution.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO manipulation Four suites; 50 evaluation episodes per task, 500 per suite; original LIBERO demonstration training. | 97.2 average; Spatial 96.2, Object 99.6, Goal 97.2, LIBERO-10 95.8. Task success rate (%) | OpenVLA-OFT 97.1 average; VLA-JEPA without human videos 96.1. Competitive in-distribution performance; the 0.1 percentage-point lead over OpenVLA-OFT is small and has no reported uncertainty. e07e11 |
| LIBERO-Plus perturbation robustness Seven perturbation dimensions, each covering the four original suites; fine-tuning excludes LIBERO-Plus augmentation. | 79.5 Average success rate (%) | OpenVLA-OFT 69.6; without human videos 62.9. Using the paper’s printed Avg entries, the gains are 9.9 and 16.6 percentage points, respectively. OpenVLA-OFT’s seven displayed perturbation values average 71.4 under equal weighting, whereas its printed Avg is 69.6; the source supplies no weighting that resolves this discrepancy. The reported values are preserved, not independently reconstructed or officially corrected. VLA-JEPA leads five dimensions, but not Camera or Noise. e07e13 |
| SimplerEnv visual matching Google Robot and WidowX simulation; Fractal/BridgeV2 adaptation. LAPA uses simulation expert demonstrations, unlike the other rows. | Google 65.2; WidowX 57.3. Reported average success rate (%) | Without human videos: Google 78.4, WidowX 57.3; RoboVLMs Google 51.7. Human-video pretraining reduces the Google average here. Section 4.3’s WidowX ranking conflicts with Table 2; table values are retained without repairing its baseline averages. e07e12 |
| Physical Franka manipulation 100 training demonstrations; ID, unseen-task and shuffled-layout tests; Appendix B specifies 10 independent trials per task. | ID 0.70; task OOD 0.17; object-layout OOD 0.47. Success fraction, rounded labels in Figure 4 | π0: 0.57/0.00/0.37; π0.5: 0.37/0.20/0.27, in the same order. Best of these three policies on ID and shuffled layouts; π0.5 leads unseen tasks. Small trial counts and absent error bars limit confidence. e14 |
| Future video horizon ablation Direct LIBERO fine-tuning; Section 4.5 Q3 says other hyperparameters are held fixed. | T=4: 94.8; T=8: 96.1; T=16: 95.5. Average success rate (%) | T=8 is 1.3 points above T=4; LIBERO-10 rises from 89.0 to 94.0. Longer prediction is not uniformly better. The token repetition rule makes exact capacity control unclear at T=16. e17e08 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 5. More human video improves all endpoint comparisons, with nonmonotonic intermediate behavior. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow one color at a time using the two-row legend. The horizontal axis labels the proportion of human video at four settings, and the vertical axis reports success rate for each LIBERO-Plus perturbation. Compare the first and last marker on each curve before examining intermediate points. All endpoint comparisons improve. However, the green Language curve and brown Robot curve drop between 0.3 and 0.7, while the orange Camera and cyan Noise curves rise throughout. Figure 5 therefore supports an overall scaling tendency across these settings, rather than the stronger claim that every perturbation improves at every increase in human-video proportion. e15
What it supports. The visual is consistent with human-video exposure helping resistance to disturbances, particularly for dimensions that start with lower success. It does not itself measure new action-skill acquisition. Section 4.5 interprets the gains as strengthening existing skills, an explanation that remains distinct from the measured success curves.
Where the evidence stops. The paper does not give a complete sampling or fixed-compute definition for these proportions, nor uncertainty bars. Intermediate values should not be reconstructed as exact measurements from the plot, and the dips should be preserved when describing scaling.
Table 4. A moderate video horizon performs better than either tested extreme. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read T as the future video prediction horizon. The remaining columns are the LIBERO suites; “10” denotes LIBERO-10. Section 4.5 Q3 describes direct LIBERO fine-tuning and says other hyperparameters remain fixed. Move down Avg to see the peak at T=8, then inspect individual suites: the longest horizon improves Goal but lowers Spatial. The authors interpret the middle setting as aligning better with the action horizon. Appendix A is needed to evaluate that interpretation because it specifies both the action-head horizon and the rule for repeating latent tokens. This table is a separate protocol from the headline pretrained-model comparison. e17e08
What it supports. Average success rises from 94.8% at T=4 to 96.1% at T=8, then falls to 95.5% at T=16. LIBERO-10 similarly favors the middle setting. These results support tuning predictive horizon for the downstream task; they do not establish that modeling a longer future always supplies more useful control information.
Where the evidence stops. Appendix A sets K=24/T repetitions, which gives a noninteger value at T=16. The source does not explain that allocation, so an exact constant-capacity reconstruction remains unresolved. The table also provides no uncertainty estimates.
7. Analysis & limitations
7.1 What the evidence leaves open
Equation (4) asserts that deterministic target embeddings make a KL term vanish, without a complete probabilistic specification. Equation (5) then lacks a scalar distance. These are formulation gaps, not justification for inventing a corrected objective. e05
Figure 5 shows endpoint improvements with more human video, but Language and Robot dip between intermediate proportions. Attention maps in Figure 6 are qualitative and cannot isolate leakage removal from backbone, data or optimization differences. e15e16
The authors report wrong-object grasps from weak instruction reasoning, despite comparatively stable trajectories. Regrasping is attributed to human videos, but Appendix B also reports complete failure on the unseen shelf-placement task. These observations do not establish general safety or new-task mastery. e14e18
7.2 Questions for discussion
- Does latent alignment improve control after equalizing robot exposure and total training compute?
- How much prediction depends on latent actions rather than teacher-forced visual histories?
8. Reproducibility audit
8.1 Requirements and known gaps
Use the specified Qwen3-VL-2B/V-JEPA2 components and separate predictors. Appendix A gives a 12-layer, eight-head world model and a 16-layer, 12-head action head. Inputs resize to 224×224 for the VLM and 256×256 for state encoding; end-effector deltas and axis-angle deltas are min–max normalized to [0,1], with binary gripper commands. e08e09
Resolve the alignment distance, beta, exact V-JEPA2 checkpoint, optimizer, warmup duration, frame sampling and evaluation seeds before claiming a faithful reproduction. K=24/T is specified, but integer token allocation for T=16 is unexplained. Fixed-budget human-video ablations and shuffled-latent controls are proposed in the illustrated edition. e05e06e08e09e15e17
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does state prediction actually require the latent action?
Reader-proposed check, not an executed experiment: after resolving the alignment distance and beta, evaluate a trained model on held-out clips with normal latent tokens and with tokens shuffled between episodes while keeping each encoded state history unchanged. Measure next-state prediction error and, separately, robot success when the action-conditioning pathway receives the shuffled latent context. Add a matched robot-training run with beta=0, holding initialization, demonstrations and update count fixed. If shuffling barely affects prediction and beta removal preserves control, the evidence for learning useful transition information through the alignment mechanism would weaken. e04e05e06e09
Check 2: Separate human-video coverage from training-budget effects
Reader-proposed check, not an executed experiment: repeat the human-video ablation using explicitly defined fractions of the available SSv2 set. Keep robot samples, robot update count, total optimizer steps, backbone initialization and downstream demonstrations fixed; for positive fractions, resample to equalize human-video update counts. Include a zero-human baseline with its update accounting disclosed. Evaluate original LIBERO, all seven LIBERO-Plus dimensions and SimplerEnv under fixed evaluation seeds, with multiple training seeds and confidence intervals. A persistent perturbation gain alongside flat LIBERO or lower Google Robot success would support the paper’s conditional robustness interpretation. e07e09e11e12e13e15
8.3 Reading coverage
Visual audit: Visually inspected the title and author block, both architecture figures, all seven figures and all six tables, equations and supporting method/evaluation text on pages 1–10, and the complete implementation and physical-experiment appendices on pages 16–18. Six final original crops were individually viewed, including higher-DPI renders for narrow visuals; labels, table headers and plot legends are retained. Figure 2 arrows were checked against Eqs. (1)–(9), the frozen-target description and causal-attention text. Generic video labels and schematic training-stage labels are qualified in its guide. Figure 5 intermediate declines, Table 2/prose disagreement and Table 4/token-allocation ambiguity are preserved. Reference-only pages 11–15 were read in the complete six-chunk text pass, but not visually inspected. Separate supplemental material was not supplied or verified; linked code and external videos were not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 16, 17, 18. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and title block (p. 1)
- 1 Introduction (pp. 1–3)
- 2 Related Works (p. 3)
- 3 Methodology, 3.1–3.3 (pp. 3–5)
- 4 Experiments, 4.1–4.5 (pp. 5–10)
- 5 Conclusion (p. 10)
- Acknowledgement and References (pp. 11–15)
- A Implementation Details, A.1–A.2 (pp. 16–17)
- B Real-world Experiments Details (pp. 17–18)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction limitation was addressed by visually inspecting PDF pages 1–10 and 16–18, including every figure and table; reference-only pages 11–15 were read as text.
- Identity: the title and all nine authors match the catalog. The supplied artifact displays arXiv:2602.10098v2, 14 February 2026, while its title block says Date: February 17, 2026. The catalog submission date is February 10, 2026. These dates are preserved; no earlier version was supplied for comparison (e01).
- Code, linked project materials and checkpoints were not inspected; no experiments were reproduced. Separate supplements were not supplied.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/author block, version margin and abstract
Title and nine-author list match the catalog. Margin: arXiv:2602.10098v2, 14 Feb 2026; title-block date: February 17, 2026. Abstract describes JEPA pretraining followed by action-head fine-tuning.
Go to primary source ↓e02PDF pp. 1–2, Section 1, four failure modes and Figure 1
Authors motivate latent prediction by appearance bias, nuisance motion, future-input shortcuts and complex training pipelines; Figure 1 sends the future observation only through the target branch.
Go to primary source ↓e03PDF pp. 3–4, Sections 3–3.1 and Figure 2/caption
VLM-generated latent actions feed a distinct latent world model. Robot training adds action queries, an action head and action prediction loss alongside latent alignment.
Go to primary source ↓e04PDF pp. 4–5, Section 3.2, Eqs. (1)–(3) and causal-attention paragraph
World states concatenate view encodings; VLM latent queries condition only on initial observations and language. The world model takes state and latent histories, with within-step bidirectional and across-step causal attention; training uses teacher forcing.
Go to primary source ↓e05PDF p. 5, Section 3.2, Eqs. (4)–(5) and surrounding text
The V-JEPA2 target is frozen with stop-gradient. The claimed ELBO reduction is followed by an alignment expression written as predicted state minus target, without a norm or scalar reduction.
Go to primary source ↓e06PDF p. 5, Section 3.3, Eqs. (6)–(9) and inference paragraph
Action tokens attend to latent tokens, initial visual tokens and language. Conditional flow matching learns a trajectory velocity field; inference integrates noise into actions. Robot loss is flow loss plus beta-weighted world loss.
Go to primary source ↓e07PDF pp. 6–7, Section 4.1
Reports 220K SSv2 videos, 76K DROID trajectories, approximately 2K LIBERO demonstrations without LIBERO-Plus augmentation, Fractal/BridgeV2 adaptation, 100 real-world demonstrations, and eight NVIDIA A100 GPUs.
Go to primary source ↓e08PDF p. 16, Appendix A.1, Tables 5–6
Qwen3-VL-2B, V-JEPA2 checkpoint and random predictor; K=24/T latent repetition. Table 5 specifies two views, T=8 and three action tokens per step; Table 6 and text specify DiT-B, 32 action-token repetitions, action dimension 7, horizon 7 and four denoising timesteps.
Go to primary source ↓e09PDF pp. 16–17, Appendix A.2
Documents input resolutions, control normalization, two-view handling, batch 32 per GPU/global 256, cosine learning-rate schedule and peak rates, joint SSv2/DROID pretraining for 50K steps and 30K/20K adaptation steps. Does not specify beta, optimizer, warmup duration, sampling schedule or evaluation seeds.
Go to primary source ↓e10PDF p. 7, Section 4.2, Benchmarks and Baselines
Separates LIBERO in-distribution simulation, SimplerEnv real-to-sim evaluation and LIBERO-Plus perturbed simulation, with seven perturbation dimensions.
Go to primary source ↓e11PDF p. 6, Table 1, caption and VLA-JEPA/OpenVLA-OFT/without-human rows
Caption specifies 50 episodes per task and 500 per suite. VLA-JEPA reports 96.2/99.6/97.2/95.8 and 97.2 average; OpenVLA-OFT 97.1 average; without human videos 96.1.
Go to primary source ↓e12PDF p. 8, Table 2/caption; PDF p. 7, Section 4.3 SimplerEnv
Visual matching table reports Google/WidowX averages 65.2/57.3 with human videos and 78.4/57.3 without. LAPA is marked as trained on simulation expert demonstrations. Prose calls villa-X the best WidowX model, while the table prints villa-X 40.8 and VLA-JEPA/LAPA 57.3.
Go to primary source ↓e13PDF p. 8, Table 3/caption, all columns
VLA-JEPA has Camera 63.3, Robot 67.1, Language 85.4, Light 95.6, Background 93.6, Noise 66.3, Layout 85.1, Avg 79.5; without-human average 62.9; OpenVLA-OFT average 69.6. Each perturbation covers four suites. OpenVLA-OFT’s seven displayed entries sum to 499.8 and have a simple mean of 71.4, unlike its printed Avg of 69.6. No aggregation weights resolving this discrepancy are supplied; 71.4 is a reader calculation, not an official corrected score.
Go to primary source ↓e14PDF pp. 8–9, Section 4.4 and Figure 4; PDF p. 17, Appendix B protocol
Franka Research 3/Robotiq setup uses 100 demonstrations. Figure 4 labels VLA-JEPA 0.70/0.17/0.47, π0 0.57/0.00/0.37 and π0.5 0.37/0.20/0.27 for ID/task OOD/layout OOD. Appendix B specifies 10 trials per task; Section 4.4 discusses wrong-object grasps and stability.
Go to primary source ↓e15PDF p. 9, Section 4.5 Q1 and Figure 5/legend/axes
Success is plotted by human-video proportion 0, 0.3, 0.7 and 1 for seven LIBERO-Plus perturbations. All curves improve from first to last point; Language and Robot decline from 0.3 to 0.7. No uncertainty or fixed-compute sampling definition is supplied.
Go to primary source ↓e16PDF pp. 9–10, Section 4.5 Q2 and Figure 6
Compares pretrained-only LAPA, UniVLA and VLA-JEPA attention maps for simulation, human and physical robot images; authors interpret operation-focused attention as evidence of improved latent learning.
Go to primary source ↓e17PDF p. 10, Table 4 and Section 4.5 Q3
Direct LIBERO fine-tuning compares T=4/8/16 with stated otherwise-fixed hyperparameters. Averages are 94.8/96.1/95.5; LIBERO-10 is 89.0/94.0/92.2. Authors favor a horizon near the action horizon.
Go to primary source ↓e18PDF pp. 17–18, Appendix B and Figure 7/caption
Describes banana, peach and shelf OOD tasks, failure of all models on top-shelf placement, and qualitative regrasping by VLA-JEPA. The attribution to human-video experience is an author interpretation without a matched regrasping ablation.
Go to primary source ↓8.5 Primary sources
VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model ↗
PDF · 9,969 extracted words
Source fingerprint
5deec841d5a7da63e5bab3593798e1741d44d354507586bec8a2f9b576f12677