JEPA Policy: Diffusion-Free Imitation Learning via Paired Action and Future Representation Prediction
1. Paper overview
In one sentence: A shared two-pass policy uses demonstrated future latents to improve action learning, with little measured inference overhead but additional training cost and task-dependent diagnostic reliability. e02e04e10e12e13e14e15e18e19
| At a glance | What to know |
|---|---|
| Research problem | Source description Action-only behavior cloning ignores the future observation already paired with each expert action sequence. The paper asks whether this additional supervision can improve manipulation within MIP’s two-pass inference budget, without fitting a distribution over actions or reconstructing future images. e02e03e07 |
| Core mechanism | |
| A key reported result | Nine-task simulated manipulation: 83.0 Mean success rate (%). LIBERO (50 demonstrations/task), robomimic PH (200/task), MimicGen D1 (1,000/task); three seeds, mean of each run’s best of 30 rollout evaluations. MIP 77.4; Dual-independent 76.3; Dual-cross 77.6; 100-step Diffusion Policy 75.1. Reported MIP gain is +5.6 percentage points. Coffee Preparation contributes 57.5→77.5 and Tool Hang 65.8→75.0; small gains near ceiling deserve caution. This measures executed simulated control. e08e09e10 |
| Reading caution |
Core contributions
Figure 2. One shared computation produces actions and a future latent; the realized future supplies supervision. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the solid current-observation path into the central two-step block, then the two output heads. The dashed lower path supplies the future target during training. The two encoder drawings share weights; they do not denote independently trained networks. The stop-gradient label beside the target prevents target-side updates, while current-observation gradients remain active. Figure 2 leaves proprioceptive fusion implicit: Section III-A defines conditioning c_t from current image features and low-dimensional state, whereas the future target is image-only. The caption and Eqs. (5)–(12) further specify H action tokens and one future token interacting in every unmasked self-attention layer. e03e04e05e08e14e15
What it supports. The architectural contribution is shared processing before the two heads. Future loss can alter the same intermediate parameters used to produce actions, and action tokens can attend to the future token. The picture alone establishes information flow; the topology and route-cut experiments provide the empirical test of its usefulness.
Where the evidence stops. The flame on the future encoder does not override the target stop-gradient: its parameters train through the shared current path. Policy-weight EMA at evaluation is also distinct from an EMA target encoder, which the main method does not use.
2. Motivation
2.1 The problem and the proposed response
Action-only behavior cloning ignores the future observation already paired with each expert action sequence. The paper asks whether this additional supervision can improve manipulation within MIP’s two-pass inference budget, without fitting a distribution over actions or reconstructing future images. e02e03e07
2.2 What this reading follows
An expert demonstration contains an action and the observation that followed it. JEPA Policy trains on both, asking one Transformer to predict an action chunk and its paired future visual representation. The useful question is where future supervision enters the action computation. Separate-branch controls, gradient measurements and route-cut experiments make that question testable. This reading follows the shared architecture into the simulation results, then examines checkpoint selection, causal controls and the smaller physical study. It also separates a promising by-product—future error that ranks failures within some tasks—from the additional calibration needed for operational monitoring. e02e04e10e12e13e14e15e18e19
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is WAMs. View the current classification.
3.1 Evidence-based assessment
Insufficient evidence to decide
The recorded snapshot is unassigned, so no existing quadrant can be confirmed or contradicted. Architecture supports a One Model, joint action–future-latent interpretation: one mutually visible stack generates both outputs. This is neither an inverse-dynamics action decoder nor a separate planning world model; the future objective trains the policy and enables a delayed diagnostic. e02e04e07e18
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Understand what stop-gradient does—and what action supervision adds
The future image is a training label only after the encoder converts it into a latent target. Stopping gradients at that target prevents the future loss from directly moving the target-side representation toward the prediction. It does not freeze the shared encoder: prediction still depends on current-image conditioning, through which gradients flow. Nor does stopping gradients alone rule out a constant representation. Action supervision supplies an additional demand that observations remain useful for predicting expert actions. The collapse audit tests this empirically, finding no complete collapse under the full objective but several low-rank exceptions. Removing action loss strongly contracts representations in the single Square control, while retrieval remains input-dependent. The defensible conclusion is a task-bounded anti-collapse effect, not a mathematical guarantee. e03e05e17
5.2 Trace training and deployment through the two passes
At deployment, start with zero action and future inputs. The shared predictor produces coarse values, then consumes those values in a second, noise-free refinement call. Training constructs a different second input: expert actions and stopped future targets, each perturbed by Gaussian noise scaled by 1−τ. Consequently, training does not backpropagate through a chained first prediction into the second pass. Both outputs are supervised at each pass, with squared errors scaled by the inverse squared interval lengths. Figure 3’s caption explicitly explains that its connecting arrows show inference, despite introducing the figure as a training illustration. The interval parameter τ is fixed at 0.9, not an environment timestamp. The paper also discloses simplified inputs without the original MIP multiplication by τ; a reproduction must retain this distinction. e04e05e07
5.3 Separate a better controller from a useful monitor
The success experiments ask whether executed action chunks complete tasks. The diagnostic audit asks a different question: whether the future latent predicted before execution disagrees with what the encoder observes afterward. For default simulation Future-4, the target is 0.2 seconds ahead, so the score describes an already-reached future. The audit aggregates five early decisions and uses failure labels to evaluate ranking. My interpretation is that a useful ordering can support monitoring only after its threshold and transfer behavior are tested separately. The raw cross-task AUROC near chance demonstrates why normalized prediction error cannot be treated as a common failure probability. Likewise, the physical success table is not a hardware validation of diagnostic thresholds; it evaluates the policies’ task completion. e08e18e19
5.4 Training and inference
During training
Both passes regress expert targets. Pass one receives zeros; pass two receives expert actions and stopped future latents plus Gaussian noise scaled by 1−τ. Training passes are decoupled. Squared residuals are scaled by 1/τ² and 1/(1−τ)², with τ=0.9. The simplified refinement inputs omit the original MIP target/output multiplication by τ. e07
The main recipe trains ResNet-18 and the policy from scratch for 300,000 steps, batch 256, learning rate 10⁻⁴. Current/future images share a random crop window. Evaluation uses policy-weight EMA at 0.995, distinct from a target encoder. Adaptive future weighting uses ρ=0.10 and clips λ_z to [10⁻⁶,0.1]. e06e08
During inference
Pass two refines pass-one predictions without noise. Simulation executes eight actions at 20 Hz before replanning: H=10 on robomimic/MimicGen and H=16 on LIBERO. Default k=4 predicts 0.2 seconds ahead within the 0.4-second executed chunk. No imagined rollout or action search is used. e02e07e08
After the corresponding future observation arrives, compare its encoding with the saved prediction using normalized MSE. This diagnoses preceding execution and may inform the next decision; it cannot prevent an action already executed. The control timing path skips the second future projection, while diagnostic use retains it. e18e12
5.5 Implementation flow
- Construct paired targets
Each demonstration supplies current observation, the next H actions, and an observation k control steps later. The shared image encoder pools and concatenates camera features. Current features fuse with proprioception into conditioning c_t; the future target excludes proprioception. The described configuration has a 1024-dimensional image target and 384-dimensional conditioning. e03
- Mix the two token types
H action tokens and one future token occupy an unmasked Transformer sequence. Every layer performs self-attention followed by cross-attention to observation and interval-endpoint memory. Separate output heads read the resulting action and future positions. Interval endpoints enter the memory, not the input tokens. e04
- Stop the target path only
The future target uses the live shared encoder with stop-gradient. Future loss still updates that encoder through the current-observation path and updates the shared Transformer. There is no separately frozen or EMA target encoder in the main method. e05e08
6. Experiments & results
JEPA Policy trains a shared Transformer to predict an action chunk and the visual representation observed later in the same demonstration. Two deterministic passes refine both outputs. Its strongest evidence is improved simulated control relative to action-only MIP, supported by topology controls; future error also offers a delayed, task-dependent rollout diagnostic.
6.1 Read the original evidence
Table II. The shared policy improves the task average, with especially large gains on Coffee Preparation and Tool Hang. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each large entry as a three-seed mean of best-checkpoint success, then use the smaller entries to inspect seeds 41, 42 and 43. The footer is essential: these are selected maxima from repeated evaluations, and the Transport topology comparison uses an aligned window. The mean row weights tasks equally. Compare JEPA first with action-only MIP to assess added future supervision, then with Dual-independent and Dual-cross to assess its placement. Dual-independent is a controlled ACT-JEPA-style topology proxy. Notice its winning Transport entry; the shared design is not the best arm on every task. e08e09e10e14
What it supports. JEPA averages 83.0% versus MIP’s 77.4%, a reported 5.6-point gain. Coffee Preparation rises from 57.5% to 77.5%, while Tool Hang rises from 65.8% to 75.0%. Coffee’s weak MIP seed contributes to the mean difference, although JEPA is higher for all three paired Coffee seeds.
Where the evidence stops. These scores are simulation outcomes under checkpoint selection. Diffusion Policy uses its trained 100-step DDPM setting and the better absolute/delta action parameterization; its action horizon is matched outside LIBERO. Near-ceiling differences are small relative to rollout resolution.
Table IV. Selection changes absolute success much more than the aggregate JEPA–MIP difference. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the first row, which reproduces the headline best-of-30 comparison. Move downward to protocols that use the same recorded runs without choosing the best checkpoint: average the final five evaluations, average all thirty, or take only the final checkpoint. The Baseline column is action-only MIP. Compare the absolute rates separately from the Gain column, and then inspect Improved, which counts tasks with positive three-seed mean differences. The footnote states that gains are computed before rounding, so subtracting the displayed last-row means need not reproduce its reported 5.1 exactly. e08e10e11
What it supports. The all-evaluation mean is 67.5% for JEPA and 62.8% for MIP, preserving a 4.7-point aggregate advantage. Last-only reporting also retains a positive aggregate gain, but just five of nine tasks improve. Thus, evidence for the average advantage is stronger than evidence for uniformly better final checkpoints.
Where the evidence stops. All four summaries reuse the same runs. They are sensitivity checks, not independent replications or validation-set checkpoint selection. This table does not eliminate finite-rollout noise, task dependence, or the need to specify a checkpoint rule before reproduction.
Table XI. Physical episodes preserve the pooled ordering, while individual tasks and checkpoints remain mixed. Original paper, p. 15 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each cell as successes over executed episodes, not a training-seed average. Every task has checkpoints at 60k, 100k and 140k steps, and the italic all rows pool those episodes. Three accidental repeat sessions produce denominators of twenty; their complete counts and identities remain in the footnote. The bottom row is episode-weighted, so it differs from an equal-task average. These tasks use the same 100 demonstrations per task across arms and a shared task-specific chunk budget for success. Keep Cabinet’s large separation alongside Plate Grape’s MIP advantage when interpreting the final ranking. e19e20
What it supports. Episode-weighted success is 66.9% for JEPA, 54.7% for MIP, 31.8% for DP-16 and 18.7% for DP-100. Cabinet reaches 60% versus MIP’s 30%; Plate Grape instead reports 47% versus 50%. The source treats the hardware advantage over MIP as directional rather than statistically established.
Where the evidence stops. The study uses one ARX-5, operator and laboratory; correlated checkpoints are not independent replications. Calls and action execution are synchronous. DP-16/DP-100 skip one/three expired positions, leaving eight/six executable actions, so the comparison includes each deployed controller’s timing behavior.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Nine-task simulated manipulation LIBERO (50 demonstrations/task), robomimic PH (200/task), MimicGen D1 (1,000/task); three seeds, mean of each run’s best of 30 rollout evaluations. | 83.0 Mean success rate (%) | MIP 77.4; Dual-independent 76.3; Dual-cross 77.6; 100-step Diffusion Policy 75.1. Reported MIP gain is +5.6 percentage points. Coffee Preparation contributes 57.5→77.5 and Tool Hang 65.8→75.0; small gains near ceiling deserve caution. This measures executed simulated control. e08e09e10 |
| Checkpoint-selection sensitivity Same nine-task, three-seed runs; average all 30 evaluations without checkpoint selection. | 67.5 Mean success rate (%) | MIP 62.8; reported gain +4.7 points. Absolute performance falls substantially, but aggregate advantage persists. Last-checkpoint reporting improves only five of nine tasks, versus nine under best-checkpoint selection. e11 |
| Future-to-action route ablation Eight tasks, seed 42; best-checkpoint protocol, aligned 10k–230k Transport window. | Full JEPA 82.8; Back-cut 73.1; Fwd-cut 78.4 Mean success rate (%) | Same-seed MIP 73.8. Removing future-loss gradients from the trunk removes aggregate gain; removing action attention to the future token retains part. Large offsetting task changes and one seed limit causal effect-size claims. e15 |
| Tool Hang model decision latency Seed 42, batch 1, EMA weights, PPU-ZW810E accelerator; eager synchronized timing. | 13.2 Mean latency (ms) | MIP 12.9; trained 100-step Diffusion Policy 439.5. The reported unrounded increment over MIP is 0.29 ms. DP uses a separate PyTorch 2.9 process versus 2.6 for JEPA/MIP. Shortened simulation diffusion chains are timing-only; these are model timings, not robot loop timings. e12 |
| Early rollout failure ranking 5,200 episodes, 24 frozen checkpoints, eight tasks; median Future-4 NMSE over first five valid policy decisions. | MugMug 0.754 [0.693,0.810]; Tool Hang 0.722 [0.676,0.766] Per-task AUROC | MokaMoka 0.554 [0.488,0.618]; cross-task raw pooling 0.501. Intervals condition on evaluated checkpoints. Ranking varies by task; thresholds need held-out outcome-labeled rollouts. Transport’s pooled positive ranking is unstable across checkpoints. e18 |
| Five-task physical manipulation ARX-5, same 100 demonstrations/task; 60k/100k/140k checkpoints; 630 episodes across all arms, including three accidental repeat sessions. | 66.9 Episode-weighted success rate (%) | MIP 54.7; DP-16 31.8; DP-100 18.7. The advantage over MIP is directional: JEPA leads 9 task–checkpoint cells, trails 4, ties 2. Cabinet favors JEPA, Plate Grape favors MIP. Correlated checkpoints are not independent task replications. e19e20 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table VI. Cutting gradient and attention routes produces different aggregate outcomes. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the footnote to define the interventions before comparing numbers. Back-cut keeps action-token attention to the future token but detaches future loss from the shared trunk. Fwd-cut retains future-loss training of shared parameters but removes the future token from the action tokens’ attention. The full policy keeps both routes. All columns use seed 42, so compare them within this table rather than with Table II’s three-seed averages. After the mean row, inspect Tool Hang and Coffee Preparation: their large, opposing responses explain why an aggregate return to baseline is not uniform behavior. e04e05e15
What it supports. Full JEPA reaches 82.8%, compared with 73.8% for MIP. Back-cut reaches 73.1%, while Fwd-cut retains 78.4%. This pattern supports a role for future-loss gradients reaching the shared representation, with an additional contribution from forward interaction, within the tested single-seed setting.
Where the evidence stops. Back-cut falls to 17.5% on Tool Hang but rises to 65.0% on Coffee Preparation. Such offsets and one-seed coverage prevent a stable numerical decomposition of the gain. Gradient-norm ratios in Figure 8 are not optimizer-update shares or evidence of stable gradient alignment.
Table X. Future error ranks failures within several tasks, with strongly task-dependent reliability. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each episode is scored by median normalized future-prediction error over its first five policy decisions, requiring valid future targets for all five. Read AUROC as failure–success ordering: a larger value means failures more often receive higher error. The confidence intervals bootstrap episodes conditional on the evaluated checkpoints. The failure-rate column supplies context for PR-AUC, whose interpretation depends on outcome prevalence. MugMug has the strongest reported AUROC; MokaMoka’s interval includes chance. Section IV-J then tests checkpoint stability and calibration, which cannot be inferred from this pooled per-task table alone. e05e07e08e18
What it supports. MugMug reaches AUROC 0.754 and Tool Hang 0.722, while MokaMoka reaches 0.554 with a chance-crossing interval. Pooling raw scores across tasks instead gives AUROC 0.501. The signal is therefore useful evidence for task-conditioned ranking, not evidence for a universal numeric alarm threshold.
Where the evidence stops. The target arrives after the action’s predicted future, so this is a delayed diagnostic. Transport’s positive pooled ranking is unstable across its checkpoints. Held-out outcome-labeled rollouts are required to calibrate thresholds; these conditional intervals do not establish deployment safety or calibrated failure probabilities.
7. Analysis & limitations
7.1 What the evidence leaves open
Deterministic paired regression can still average ambiguous futures; multimodal coverage is untested. The simulation DP horizon differs on LIBERO, and matched few-step generators were not compared. The hardware study uses one robot, operator and laboratory. e07e09e20
The 84-checkpoint collapse audit found no complete collapse, but encoder/predictor low-rank exceptions occurred. Removing action loss caused strong contraction on one task/seed, without complete collapse. This does not prove universal anti-collapse guarantees. e17
Fixed-camera-only gains were not reproduced; two LIBERO cells were invalidated by a data bug. SIGReg at weight 0.09 reduced performance on all five tested tasks, but no weight sweep establishes a general failure of regularization. e16
7.2 Questions for discussion
- Would future/action pairing retain its benefit when repeated observations precede deliberately incompatible actions?
- Would multi-seed route-cut results preserve the aggregate ordering without the exceptional Coffee Preparation baseline?
- Can task-calibrated diagnostic thresholds transfer to unseen checkpoints and physical deployment?
8. Reproducibility audit
8.1 Requirements and known gaps
Preserve demonstration pairing, temporal-consistent crops, target normalization, gradient stops, both training losses, action horizons and checkpoint rules. Obtain the custom four-camera Transport data. Transformer depth/head configuration, exact action normalization and hardware task-specific chunk budgets are not enumerated in the supplied PDF. e03e04e08e09e19
One Tool Hang JEPA run reports 22.6 training hours plus 4.8 evaluation hours on one accelerator; MIP totals 23.3 hours. DP evaluation/total duration is unreported. The paper links code but says deployment materials will be released with training code; availability cannot be inferred from the link. e13e21
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Repeat the route cuts across seeds and checkpoint rules
Reader-proposed check: train MIP, full JEPA, Back-cut and Fwd-cut on the same eight tasks with matched data order and seeds 41, 42 and 43. Preserve the shared recipe and Transport window, and verify the intended zero-gradient or attention cut without changing parameter capacity. Report task-wise differences under best-checkpoint and selection-free summaries, including uncertainty over seeds. A falsifying observation would be Back-cut retaining the full model’s gain consistently, or the reported ordering disappearing outside seed 42. This directly tests whether the single-seed result survives Coffee Preparation and Tool Hang variability. e08e11e15
Check 2: Test diagnostic calibration on held-out checkpoints
Reader-proposed check: compute the exact first-five-decision Future-4 score, then choose per-task thresholds using outcome-labeled calibration rollouts from two checkpoints and evaluate on the held-out third, rotating the holdout. Compare learned predictions with the persistence baseline and with one pooled raw-score threshold. Report AUROC, balanced accuracy, false-alarm rate and valid-target exclusions separately by task/checkpoint. Useful ranking accompanied by chance-level held-out threshold performance would refute operational calibration, even if retrospective AUROC remains positive. In a later hardware check, preserve target-time alignment and measure intervention delay rather than assuming immediate warning. e08e18e19e20
8.3 Reading coverage
Visual audit: All 17 supplied PDF pages were rendered and visually inspected, including the title/byline/version, all 12 figures, all 11 tables, equations, training details, hardware/software timing qualifications, limitations and references. All six final original crops were separately viewed at their extracted resolution with their headers and relevant footnotes intact. Figure 2’s omitted proprioceptive fusion was checked against p. 4 and its caption; the Figure 3 inference arrows were checked against Eqs. (17)–(24) and the caption’s explicit training/inference distinction. The supporting pages also cover the uncropped collapse, camera, SIGReg, latency and training-cost evidence. No appendix is present in this PDF. External supplements, code and videos remain uninspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and I. Introduction (pp. 1–3)
- II. Related Work, A–E (pp. 3–4)
- III. Method, A–F, including Eqs. (1)–(26) (pp. 4–7)
- IV. Experiments, A–K: setup, baselines, main results, topology and route controls, hyperparameters, observations, negative results, collapse, diagnostics, physical robot (pp. 7–16)
- V. Discussion and limitations (pp. 16–17)
- VI. Conclusion and complete References (p. 17)
Outside the original text pass
- Identity/version: the inspected title page identifies arXiv:2609.09630v1 [cs.RO], 9 September 2026. Its exact title and all eight authors match the supplied catalog; no revision or edition substitution was made.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- The preceding acquisition limitation was addressed by visually inspecting all 17 PDF pages and all six final original crops.
- Separate supplemental material availability has not been fully verified.
- No separate supplement was supplied. The 17-page PDF contains no appendix.
- Code, project pages, deployment videos and external references were not opened; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/byline, affiliation and arXiv margin
Exact title matches the manifest; authors are Jie Xu, Kangjin Yu, Ziyi Jin, Junjie Gao, Liqing Chen, Yixian Li, Shuai Tian and Zhongpu Xia. Affiliation: Anyverse Dynamics. The margin states arXiv:2609.09630v1 [cs.RO], 9 Sep 2026.
Go to primary source ↓e02PDF pp. 1–3, Abstract, I. Introduction and II-B–D
The method adds paired latent supervision within a two-pass policy trained from scratch. It neither reconstructs images nor learns a separate dynamics model for planning. Few-step generators are discussed but not benchmarked.
Go to primary source ↓e03PDF p. 4, III-A, Eqs. (1)–(4) and image/fusion paragraph
Demonstration tuples pair current observations, next H expert actions and observations k control steps ahead. Future features are image-only pooled per-camera features; current conditioning also fuses proprioception. The example dimensions are 1024 and 384.
Go to primary source ↓e04PDF pp. 4–5, III-B, Eqs. (5)–(12), Figure 2 and caption
H action positions and one future position share unmasked self-attention, then cross-attend to observation/time memory. Heads split the outputs. Figure 2 omits current proprioceptive fusion, explicitly acknowledged in its caption.
Go to primary source ↓e05PDF p. 5, III-C, Eqs. (13)–(15)
Action MSE and target-RMS-normalized latent MSE form the objective. Stop-gradient applies to future targets; shared encoder gradients still pass through current conditioning. Normalization uses the target RMS floor 10^-6 and latent dimension.
Go to primary source ↓e06PDF p. 6, III-C, Eq. (16) and following paragraph
Adaptive ratio weighting sums the interval-scaled losses across both passes, uses one stopped minibatch weight, epsilon 10^-8, action weight 1, rho 0.10 and clipping [10^-6,0.1]. Rho is neither an exact constraint nor total-loss share.
Go to primary source ↓e07PDF pp. 6–7, III-D–E, Eqs. (17)–(24), Figure 3 and caption
Training regresses both targets from zeros and from separately noised expert targets; tau=0.9. Inference feeds coarse predictions to refinement. Figure 3 depicts inference despite the training-oriented caption opening. Simplified inputs omit original MIP tau scaling; deterministic averaging remains possible.
Go to primary source ↓e08PDF pp. 7–8, IV-A Setup
Data come from nine tasks in three named suites, with 50/200/1000 demonstrations per task by suite. Training uses 300k steps, batch 256, learning rate 10^-4, seeds 41/42/43 and policy EMA 0.995. Evaluation every 10k selects the best of 30. ResNet-18 learns from scratch; current/future crops align. Control is 20 Hz, eight executed actions per chunk; horizons are H=10/16 and k=4 by default. Transport has custom four-camera data.
Go to primary source ↓e09PDF p. 8, IV-B Compared Methods and Table I
MIP removes future tokens/loss. DP is a 100-step DDPM Transformer evaluated with the better absolute/delta action parameterization; its H=10 mismatches LIBERO H=16. Dual-independent is an ACT-JEPA-style topology proxy, not a faithful reproduction; Dual-cross adds inter-branch attention. DINOv2 is supported as an alternative, not the main frozen backbone.
Go to primary source ↓e10PDF p. 9, Table II, complete rows and footnote; IV-C
Best-checkpoint task mean is 83.0 for JEPA, 77.4 MIP, 76.3 Dual-independent, 77.6 Dual-cross, 75.1 DP. Coffee is 77.5 versus MIP 57.5, including MIP seed 42 at 30.0; Tool Hang is 75.0 versus 65.8. Dual-independent wins Transport 99.2 versus 98.3. Table retains all seed values and notes Transport’s aligned window.
Go to primary source ↓e11PDF pp. 9–10, IV-C checkpoint discussion; p. 10, Table IV and footnote
Best/last-five/all-thirty/last-only JEPA means are 83.0/68.4/67.5/67.5; MIP means 77.4/63.1/62.8/62.5. Reported gains are 5.6/5.3/4.7/5.1 points, computed before rounding; improved-task counts are 9/8/8/5 of nine.
Go to primary source ↓e12PDF p. 10, Table III, footnote, Figure 6 and Inference cost
Tool Hang seed-42 batch-one PPU-ZW810E timings are JEPA 13.2 ms, MIP 12.9 ms and 100-step DP 439.5 ms. JEPA/MIP use interleaved PyTorch 2.6; DP uses separate PyTorch 2.9. JEPA's control second pass skips future projection; the full diagnostic path is separately reported at 12.8 ms. Shortened chains are timing-only.
Go to primary source ↓e13PDF pp. 10–11, Training and evaluation cost; p. 11, Table V and footnote
A single-accelerator Tool Hang run takes JEPA 22.6 hours training and 4.8 evaluation, total 27.4, versus MIP total 23.3. DP logs optimization time but not evaluation/total duration. Different codebases limit controlled training-cost comparison.
Go to primary source ↓e14PDF p. 8, IV-B architecture counts; pp. 9 and 11, Table II and IV-D
The controlled separate and cross-branch designs have 48.38M/53.73M parameters versus shared JEPA 43.96M, yet their mean success stays near MIP. This supports the shared-stack design within this recipe; it does not establish a benchmark result for full ACT-JEPA.
Go to primary source ↓e15PDF pp. 11–12, IV-E; p. 12, Figure 8, Table VI and footnote
Gradient-norm measurements concern gradients, not AdamW update shares or stable alignment. Seed-42 route-cut means across eight tasks are MIP 73.8, Back-cut 73.1, Fwd-cut 78.4, JEPA 82.8. Back-cut detaches future loss from trunk; Fwd-cut removes future-token visibility to action tokens. Task offsets are large and no single-seed significance claim is made.
Go to primary source ↓e16PDF pp. 12–13, IV-F–H, Tables VII–IX and Figure 9
Future-ratio/horizon sweeps cover five tasks. Camera ablations use seed 42; Transport is omitted and fixed-only future LIBERO cells are invalid after a data bug. Fixed-only gains are not reproduced. SIGReg weight 0.09 harms five tasks without a weight sweep; state-input results are task-dependent.
Go to primary source ↓e17PDF p. 8, lambda_a=0 configuration; pp. 13–14, IV-I and Figure 10
The collapse audit covers seven tasks, 84 checkpoints and fixed 4096-sample manifests. TE32 measures energy outside the top 32 centered spectral directions; boundary is 0.05. There are named task/representation exceptions. On Square seed 42, removing action supervision reduces encoder TE32 from 0.170 to 0.0014; retrieval remains input-dependent, so collapse is not complete.
Go to primary source ↓e18PDF p. 7, III-F, Eqs. (25)–(26); pp. 14–15, IV-J, Table X and Figure 11
Future-4 error is available after its target observation. Audit scores median NMSE of first five valid decisions across 5200 episodes and 24 checkpoints. MugMug AUROC is 0.754 [0.693,0.810], Tool Hang 0.722 [0.676,0.766], MokaMoka 0.554 [0.488,0.618]. Bootstrap intervals are conditional on checkpoints. Raw cross-task AUROC is 0.501; persistence is near chance and Transport is seed-unstable. Threshold calibration requires outcome-labeled held-out rollouts.
Go to primary source ↓e19PDF p. 15, IV-K, Table XI and complete footnote; p. 16, Figure 12 caption
ARX-5 uses three 128x128 RGB views and one RTX 3090; driver/actions run at 30/10 Hz with synchronous calls. JEPA/MIP execute eight actions; DP-16/100 skip one/three expired positions, leaving eight/six. All arms use 100 demos/task, three named checkpoints and task-specific chunk-budget success. 630 episodes include three accidental repeats. Pooled success is 66.9/54.7/31.8/18.7 for JEPA/MIP/DP-16/DP-100. Cabinet all-checkpoint success is 60/30/0/0; Plate Grape is 47/50/10/10 in that method order.
Go to primary source ↓e20PDF pp. 15–17, IV-K comparison, V. Discussion/Limitations and VI. Conclusion
JEPA versus MIP has 9 wins, 4 losses and 2 ties across correlated hardware task-checkpoint cells; the stated sign-test p=0.27 is not significant. Hardware scope is one robot/operator/lab. Generalization, matched few-step comparisons, held-out monitoring calibration and multimodal futures remain open.
Go to primary source ↓e21PDF p. 1, Code/Project links; p. 15, IV-K Reproducibility and release
The paper prints code/project links and states that deployment stack, launchers, safety checks and manifests are prepared for public release with training code. A printed link or release intention does not verify current availability.
Go to primary source ↓8.5 Primary sources
JEPA Policy: Diffusion-Free Imitation Learning via Paired Action and Future Representation Prediction ↗
PDF · 14,264 extracted words
Source fingerprint
b14df0a9ec300b4b5aacbe4b51a1fc1fe48139e047d03aa2669a311c56b74158