Awesome-WAMs
GitHub
PAPER REPORTENAll readings ↗

LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments

English reading report: Method, equations, original figures, experiments and reproducibility.

Authors: Pei Liu; Nan Zheng; Lang Zhang; Daojie PENG; Yanan Zhang; Feilong Kong; Mingyue Feng; Jiachao Liu; Yaonong Wang; Qifeng Chen; Jun Ma

Affiliations: The Hong Kong University of Science and Technology (Guangzhou); The Hong Kong University of Science and Technology; Leapmotor; Southeast University

Source: 2607.23969 ↗ · Catalog record

Reading: 109 / 558 · 6 original figures & tables · ~20 min ·

1. Paper overview

In one sentence: Future semantic guidance trains the action policy, while a static observation cache removes future rollout at deployment; the exact training recipe remains inconsistent in the supplied v2. e-overviewe-isaee-maske-cachee-robustnesse-objective-conflicte-robotwin-conflict

At a glanceWhat to know
Research problem
Author claim

The authors argue that reconstructing textures, lighting and backgrounds can distract robot policies from physical state transitions. Their proposed solution is to predict in a pretrained semantic space and transfer that training signal into an action policy. This motivates the design; it does not establish that all pixel-generating policies waste capacity or that Gaussian isotropy is universally necessary for diffusion. e-probleme-isae

Core mechanism
Source description

ISAE reconstructs predictive features while regularizing their latent distribution; camera identity is introduced downstream, allowing shared encoding across views. e-isaee-view

A key reported resultLIBERO-Plus zero-shot robustness: Average 73.1; Camera 33.8; Robot 75.1; Background 90.1; Noise 54.7.

Success rate (%). Transfer to systematic perturbations after expert-trajectory training; use the paper's stated 40-task/2,000-episode protocol.

VLA-JEPA 62.9, JEPA-VLA 25.6, ImageWAM 83.1 average. Strong improvement over listed latent baselines, with substantial camera/noise weakness and a remaining gap to the best listed pixel WAM. e-evaluatione-robustness

Reading caution
Source description

Foundation-model representation quality limits fine contact physics and high-frequency tactile interactions; longer-horizon planning remains future work. e-limits

Core contributions

  • Source description

    ISAE reconstructs predictive features while regularizing their latent distribution; camera identity is introduced downstream, allowing shared encoding across views. e-isaee-view

  • Source description

    Separate semantic and action transformers interact asymmetrically during training. Deployment retains semantic conditioning but bypasses future dynamics rollout. e-maske-cache

Figure 6. Training reads future-oriented semantics; deployment reuses current-observation semantics. Original paper, p. 12 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read the left panel from its two noise inputs. Semantic tokens supply keys and values (K, V); action tokens supply queries (Q). This direction agrees with the explicit mask on p. 10: semantic queries cannot read action keys, while action queries can read both streams. The diagram's Cross-Attn box is a simplified depiction of the detailed joint masked self-attention. On the right, follow the current observation through frozen V-JEPA and ISAE into the semantic cache. That cache conditions action denoising and is refreshed after the controller executes a prefix of the action chunk. The left training target and right cache share the label z0, but refer to future-target and current-observation latents respectively. e-pipeline-figuree-maske-cachee-stagese-objective-conflict

What it supports. The useful computational separation is between learning from future semantic targets and producing future semantic rollouts during execution. Current-observation encoding remains necessary. The diagram therefore supports bypassing the Anchor DiT's iterative future prediction, not eliminating all semantic computation or proving that the policy plans through an imagined future at test time.

Where the evidence stops. The training-only box also contains ISAE, although the right panel retains ISAE and Stage III freezes it. Its JEPA-loss label does not resolve the incompatible objectives on pp. 11–13. Read this figure as information flow, with the explicit mask and stage text supplying the precise boundaries.

2. Motivation

2.1 The problem and the proposed response

Author claim

The authors argue that reconstructing textures, lighting and backgrounds can distract robot policies from physical state transitions. Their proposed solution is to predict in a pretrained semantic space and transfer that training signal into an action policy. This motivates the design; it does not establish that all pixel-generating policies waste capacity or that Gaussian isotropy is universally necessary for diffusion. e-probleme-isae

2.2 What this reading follows

Imagine teaching a robot to anticipate where a manipulated object will go without asking it to render the object's texture. LeapBot-WA uses V-JEPA features as the prediction space, reshapes them with a semantic autoencoder, and lets an action transformer read a separate dynamics transformer during training. At deployment, that changing semantic stream is replaced by a cache of the current observation. The results favor this design among the listed latent WAMs, particularly under several visual perturbations. Reading the tables alongside the appendix, however, reveals conflicting objectives and result summaries that constrain claims about reproducibility and the cause of the gains. e-overviewe-isaee-maske-cachee-robustnesse-objective-conflicte-robotwin-conflict

3. Research context

We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.

Catalog dimensionRecorded classification
Major categoryWAMs
ArchitectureDual-system
Prediction paradigmOther mechanisms
QuadrantOutside quadrants

3.1 Evidence-based assessment

Supports the recorded classification

Reader analysis

Distinct Anchor and Action DiTs support Dual-system even though they are jointly trained. Other mechanisms / Outside quadrants fits training-time semantic guidance with a static observation cache at deployment: future/action denoising occurs during learning, but execution is neither joint future rollout nor an explicit inverse-dynamics pipeline. The latent/JEPA category is well supported; efficiency is conditional on the incomplete timing protocol. e-maske-cachee-efficiency

This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.

4. Problem formulation

4.1 Inputs and outputs

InputsOutputs
  • Multi-view visual observations
  • Language instruction and current proprioceptive state
  • Training only: demonstrated action sequences and future semantic targets
  • A continuous action chunk for receding-horizon execution
  • Training only: semantic predictions/velocity estimates, with conflicting loss definitions disclosed below

4.2 Equations and their role

LISAE=Lrec+βLKL+λisoLiso\mathcal{L}_{\mathrm{ISAE}}=\mathcal{L}_{\mathrm{rec}}+\beta\mathcal{L}_{\mathrm{KL}}+\lambda_{\mathrm{iso}}\mathcal{L}_{\mathrm{iso}}
Reconstruction combines squared feature error and weighted cosine distance; KL pulls the posterior toward a standard Gaussian; SIGReg penalizes departures from isotropy across random projections. Beta and lambda_iso weight the regularizers. The paper does not supply all numerical weights. e-isaee-reproduction-gaps
M=[0Ns×Ns−∞ 1Ns×Na0Na×Ns0Na×Na]M=\begin{bmatrix}0_{N_s\times N_s}&-\infty\,\mathbf{1}_{N_s\times N_a}\\0_{N_a\times N_s}&0_{N_a\times N_a}\end{bmatrix}
For concatenated semantic then action tokens, rows are queries and columns are keys. N_s and N_a count semantic and action tokens. Adding M to attention logits blocks semantic queries from action keys while preserving the reverse direction. e-mask
zt=(1−t)z+tϵz,at=(1−t)a+tϵaz_t=(1-t)z+t\epsilon_z,\qquad a_t=(1-t)a+t\epsilon_a
Here z and a are clean semantic and action sequences; epsilon_z and epsilon_a are independent standard Gaussian noise; t is the shared diffusion timestep. The detailed flow formulation targets velocities epsilon_z−z and epsilon_a−a. Later appendix losses conflict with this account. e-flowe-objective-conflict

5. Method in detail

5.1 Understand what the semantic bottleneck reconstructs

Source description

The central representation is a sequence of predictive features, not a sequence of generated images. V-JEPA produces spatial tokens for each camera frame; ISAE reduces their channel width while learning to recover the original features. Its reconstruction objective therefore preserves information in the pretrained feature space rather than image appearance directly. KL and SIGReg provide separate distributional pressures, and a cosine term complements squared reconstruction error. The training stages matter: robot-domain LoRA adaptation precedes ISAE fitting, and the adapted encoder is frozen during that fitting. For the final world/action stage, ISAE is frozen too. Camera identity enters through downstream learned residuals, preserving a shared semantic encoder across views. These design details explain what is transferred and what is optimized; they do not establish that the resulting representation perfectly isolates physical dynamics. e-isaee-architecturee-viewe-stages

Figure 2. ISAE preserves predictive features while regularizing their latent geometry. Original paper, p. 4 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Start with the snowflake on V-JEPA: its features are fixed during this autoencoder stage. The flame-marked semantic encoder compresses those features, and the decoder reconstructs feature tokens rather than pixels. The dashed upper path denotes the reconstruction comparison, not an extra visual decoder. The lower contours illustrate the posterior q(z|F) and the standard Gaussian prior. KL and isotropy regularization act together on this distribution. Cross-check this schematic with Eqs. (4)–(7): reconstruction includes squared error and cosine distance, while SIGReg probes random one-dimensional projections. The widths marked 1664 and 96 describe feature channels; the bottleneck does not imply one token per image. e-isaee-architecturee-componentse-reproduction-gaps

What it supports. The design separates retaining feature information from regularizing its distribution. That is the intended bridge between structured JEPA representations and semantic diffusion. The full component ablation improves average and LIBERO-10 success, but the figure itself is a conceptual diagram of the optimization target, not a measured visualization of the learned latent distribution.

Where the evidence stops. The bidirectional contour arrow denotes regularization, not an invertible mapping or proof of exact Gaussianity. Numerical reconstruction, cosine, KL-maximum and isotropy weights are incompletely specified. The source's broader claim that this geometry is required for stable diffusion remains an author rationale.

5.2 Follow the permitted direction of information

Reader analysis

Place semantic tokens before action tokens in the attention matrix and read rows as queries. The negative-infinity upper-right block forbids semantic queries from seeing action keys. The lower-left block remains open, allowing the action branch to read semantic guidance. Consequently the Anchor DiT predicts intent-conditioned futures from language and state, rather than the outcome of an arbitrary proposed low-level command. This distinction limits the paper's inverse-controller analogy: the deployed policy does not first imagine a candidate future and then solve a separate inverse-dynamics problem. It denoises actions while reading a cache of the present. As a reader interpretation, the mechanism aims to preserve benefits of predictive training in a cheaper execution pathway. Table 9 is suggestive evidence for that transfer, but contradictory descriptions of its predictive loss prevent identifying one definitive training implementation. e-maske-flowe-cachee-future-ablatione-objective-conflict

5.3 Separate performance, mechanism and reproducibility

Reader analysis

The three benchmark tables answer different questions. RoboTwin evaluates a policy trained on a mixed clean/randomized demonstration set; LIBERO measures standard suite performance; LIBERO-Plus probes zero-shot transfer under perturbations. Their percentages should not be pooled. Within LIBERO-Plus, the favorable Robot and Background columns coexist with weak Camera and Noise results, so average robustness is not invariance. The component ablation and future-prediction ablation then probe different interventions, and neither documents how its lower final score connects to the main LIBERO checkpoint. Linear probes and selected execution frames add supporting observations, not proof of causal physical understanding. Finally, Figure 4 reports latency without enough matched timing details, while the UR5 experiment lacks trial counts. Together these boundaries make the architectural proposal worth testing, but leave exact reproduction and broad efficiency claims open. e-evaluatione-robotwine-liberoe-robustnesse-componentse-future-ablatione-probinge-efficiencye-physicale-qualitative

5.4 Training and inference

During training

Source description

The three-stage account first adapts V-JEPA through LoRA and masked prediction, then freezes the adapted encoder to train ISAE, then freezes both encoders for world/action training. It describes five-source robot data, 7:3 external/wrist sampling, two-second windows and separate 7-/14-DoF action heads. The shorter implementation section covers only the last two stages. e-stagese-optimization

Source description

Table 7 specifies 200k ISAE and 400k policy steps, AdamW at 1e-4, batch 256, bf16 and noise shift 5.0; the main text reports 24 H200 GPUs. The semantic DiT inherits Wan2.2-TI2V-5B weights. These pretrained components qualify claims of avoiding video or robot pretraining. e-architecturee-optimizatione-stagese-robotwin

Reader analysis

The training objective is unresolved: Table 7 uses unit action/semantic velocity losses and zero extra future loss; p. 12 instead gives action/JEPA weights 1/0.1 with SmoothL1; p. 13 gives 0.1/1; Table 9 uses future weight 0.2. These are not interchangeable recipes. e-objective-conflict

During inference

Reader analysis

Encode the current observation once and reuse its semantic keys/values throughout action denoising. Execute the first R actions of the predicted horizon H, then observe again. Future semantic generation is removed, but visual encoding, cache formation and action denoising remain; 'zero-overhead' is therefore an author characterization of the removed dynamics branch. e-cachee-pipeline-figure

5.5 Implementation flow

  1. Encode and compress

    V-JEPA 2.1 converts each resized 384×384 frame into 576 tokens of width 1664. The shared ISAE samples a Gaussian latent of width 96 per token and reconstructs features, without pixel decoding. e-architecturee-isae

  2. Restore view identity

    Add a learned camera residual after projecting normalized noisy latents. Semantic tokens use spatiotemporal 3D RoPE; action tokens use temporal 1D RoPE. e-viewe-mask

  3. Transfer future semantics into action learning

    The intent-conditioned Anchor DiT cannot read target action tokens. The Action DiT reads both semantic and action tokens, while both branches attend to language/proprioception. This is auxiliary predictive guidance; the source's inverse-controller analogy does not specify a separate executed inverse-dynamics planner. e-maske-flowe-cache

6. Experiments & results

LeapBot-WA trains a manipulation policy with future semantic guidance, then executes actions using a cache of the current observation. Its V-JEPA features pass through an isotropy-regularized semantic autoencoder before asymmetric world/action training. Reported simulation performance is competitive, especially against the listed latent WAMs, but the supplied v2 contains unresolved objective, pretraining and numerical inconsistencies. The architectural idea is clearer than the exact reproducible recipe.

6.1 Read the original evidence

Table 1. RoboTwin's strongest result is the randomized WAM comparison. Original paper, p. 5 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read the Clean and Randomized columns separately before comparing their averages. The paper trains on 27.5K demonstrations spanning both conditions, so this experiment differs from the zero-shot LIBERO-Plus transfer table. LeapBot-WA's randomized score is the highest among the WAM rows shown, but Qwen-RobotManip is higher in the VLA group. The Embodied PT. column records the authors' pretraining designation, not a guarantee that every pretrained component or robot-data exposure is matched. Table 8 states 100 evaluation trials per task and condition for the 50-task benchmark. Its subset of task rows cannot independently reconstruct the full benchmark average. e-robotwine-evaluatione-robotwin-conflicte-stages

What it supports. LeapBot-WA reports 91.76% average success, with 92.48% under randomization. Its overall average is close to Fast-WAM's 91.85% and below Lingbot-VA's 92.24%. Thus the table supports competitive simulation performance and a favorable randomized WAM comparison, while giving a narrower conclusion than an unrestricted state-of-the-art claim.

Where the evidence stops. Table 8 repeats LeapBot-WA's values but changes baseline averages; its prose gives conflicting 92.64/89.80 scores. Robot-corpus adaptation also complicates the no-pretraining flag. The PDF does not establish a common data, compute and uncertainty-controlled comparison.

Table 2. LeapBot-WA leads the listed latent WAMs on average LIBERO success. Original paper, p. 6 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Use the four suite columns to see where the average comes from. LeapBot-WA reaches 96.2 in Spatial, 99.6 in Object, 97.6 in Goal and 95.6 in LIBERO-10. These columns represent different task families, rather than repeated measurements of the same task. Compare first within the latent-space group, then look upward at the pixel-space and VLA groups. The stated evaluation aggregates expert-trained policies over 40 tasks and 2,000 episodes. Bold type is a group-oriented emphasis and should not replace direct numerical comparison. In particular, a leading latent average does not imply the highest score in every suite or across the whole table. e-liberoe-evaluatione-components

What it supports. The table's 97.3% average exceeds JEPA-VLA's 96.4%, VLA-JEPA's 96.1% and PALM's 94.5%. That places LeapBot-WA first among the listed latent WAMs. Lingbot-VA and Cosmos-Policy both report 98.5%, so the result is competitive with, but below, the leading displayed pixel-space averages.

Where the evidence stops. The surrounding prose says 97.1%, whereas the table says 97.3%; this edition retains the table value. The 92.0% ablation configuration is not linked clearly to this main result, and no uncertainty is supplied to assess small differences.

Table 3. Robustness gains depend strongly on the kind of perturbation. Original paper, p. 7 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Follow each column as a separate stress test. Robot changes and background changes favor LeapBot-WA, with 75.1 and 90.1 success respectively. Camera changes are harder: its 33.8 is below VLA-JEPA's 40.3 and far below ImageWAM's 80.8. Noise also remains a weakness at 54.7. Read actual values rather than treating every bold entry as a global maximum: the caption's claim about best WAM values does not agree consistently with the highlighting. Finally compare the Avg. column within each group and across groups. This evaluation is described as zero-shot transfer under perturbations, unlike RoboTwin training on a clean/randomized mixture. e-robustnesse-evaluatione-physicale-qualitative

What it supports. The 73.1% average improves on VLA-JEPA's 62.9% by 10.2 percentage points, a reader-computed difference. It remains below ImageWAM's 83.1% average. The most defensible finding is stronger robustness than the listed latent baselines across many conditions, with camera and noise failures still visible in the same table.

Where the evidence stops. These are simulated success rates, not measured real-robot robustness or a proof that features are visually invariant. Perturbation-specific uncertainty is absent, and the PDF's protocol summary does not give enough detail to reproduce every environmental variation.

6.2 Results and evaluation conditions

Task & protocolReported resultComparison & interpretation
RoboTwin 2.0 manipulation

50 simulated tasks; 27.5K mixed demonstrations; Table 8 states 100 trials per task/condition.

Table 1: Clean 91.04; Randomized 92.48; average 91.76.

Success rate (%)

Average: Fast-WAM 91.85; Lingbot-VA 92.24; Qwen-RobotManip 93.85.

Best randomized result among listed WAMs, not best overall average. Table 8 agrees for LeapBot-WA but changes baselines; its prose gives conflicting 92.64/89.80 values. e-evaluatione-robotwine-robotwin-conflict

LIBERO four-suite manipulation

40 tasks; expert-trajectory training; stated aggregation over 2,000 episodes.

Table 2: Spatial 96.2; Object 99.6; Goal 97.6; LIBERO-10 95.6; average 97.3.

Success rate (%)

JEPA-VLA 96.4; VLA-JEPA 96.1; PALM 94.5; Lingbot-VA and Cosmos-Policy 98.5 average.

Leads the listed latent WAMs. The accompanying 97.1% prose is inconsistent; retain the table's 97.3%. No uncertainty is reported. e-evaluatione-libero

LIBERO-Plus zero-shot robustness

Transfer to systematic perturbations after expert-trajectory training; use the paper's stated 40-task/2,000-episode protocol.

Average 73.1; Camera 33.8; Robot 75.1; Background 90.1; Noise 54.7.

Success rate (%)

VLA-JEPA 62.9, JEPA-VLA 25.6, ImageWAM 83.1 average.

Strong improvement over listed latent baselines, with substantial camera/noise weakness and a remaining gap to the best listed pixel WAM. e-evaluatione-robustness

ISAE and robot-domain adaptation ablation

Table 4 LIBERO ablation; relationship to the main Table 2 configuration is unspecified.

Baseline 89.4/74.2; Robot-Mix 91.2/81.0; ISAE without SIGReg 91.4/80.2; full 92.0/85.0.

Average / LIBERO-10 success (%)

Adding SIGReg yields +0.6 average and +4.8 LIBERO-10 percentage points, but Goal falls 92.4→89.2.

Supports a task-dependent benefit, not universal improvement or proof of isotropic geometry; no seed variation is given. e-components

Future prediction ablation

Table 9 LIBERO comparison with semantic context held present.

Static context 90.7/78.4; future weight 0.2 gives 92.0/85.0.

Average / LIBERO-10 success (%)

Without semantic context: 47.7/1.8.

Future supervision adds a reported 1.3 average and 6.6 long-horizon percentage points. Loss-definition conflicts limit exact reproduction. e-future-ablatione-objective-conflict

Hardware efficiency

Figure 4; matched inference hardware, horizon and denoising settings are unspecified.

2.9B parameters; 546 ms.

Reported model size and inference latency

Fast-WAM 6.0B/486 ms; Cosmos-Policy 2.1B/1413 ms.

Smaller than Fast-WAM but slower in this chart. The plot supports neither universally fastest execution nor a fully controlled efficiency comparison. e-efficiency

Representation linear probing

Base JEPA versus mixed-domain adaptation; probe split/training details unspecified.

Action 0.0437→0.0417; dynamics 0.00496→0.00308; proprioception 0.00460→0.00329.

MSE, lower is better

Reported relative reductions: 4.6%, 37.9%, 28.5%.

Features improve on the reported probes; this does not independently establish causal physical understanding. e-probing

6.3 Ablations and diagnostic examples

Read component removals and qualitative examples within their stated evaluation conditions.

Table 4. The complete latent pipeline helps LIBERO-10 while exposing a Goal-suite tradeoff. Original paper, p. 7 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read the switches on the left before the scores on the right. The first transition adds Robot-Mix JEPA. The next introduces the ISAE without SIGReg; the last adds SIGReg. Treat the two ISAE column labels as the source's configuration switches, not evidence of two independently deployed autoencoders. Follow LIBERO-10 down the rows: 74.2, 81.0, 80.2 and 85.0. Then check Goal: the last step falls from 92.4 to 89.2. The overall average can hide that regression. Table 9 separately tests semantic context and future prediction, so its intervention should not be conflated with this regularization ablation. e-componentse-future-ablatione-liberoe-objective-conflict

What it supports. Adding SIGReg to the displayed ISAE configuration raises average success by 0.6 percentage points and LIBERO-10 by 4.8 points, while Goal drops by 3.2 points. These reader-computed differences support a selective downstream benefit. They do not directly measure isotropy, exclude optimization confounds or explain the higher main-table LIBERO result.

Where the evidence stops. The paper supplies no seed variability or configuration bridge to Table 2's 97.3% average. Table 9 also uses a future-loss setting that conflicts with other appendix definitions. Reproduction needs a fixed recipe before assigning the gains to latent geometry alone.

7. Analysis & limitations

7.1 What the evidence leaves open

Source description

Foundation-model representation quality limits fine contact physics and high-frequency tactile interactions; longer-horizon planning remains future work. e-limits

Reader analysis

UR5 execution is shown qualitatively, without trial denominators or controlled success rates. Selected simulation images likewise cannot verify claimed attention shifts, invariance or absence of memorization. e-physicale-qualitative

Reader analysis

The 'no embodied pretraining' table label is difficult to reconcile with the robot-corpus adaptation and cross-embodiment training account. No pixel reconstruction loss is used in the described policy training, but Wan video-model initialization remains. Dataset overlap and the origin of each reported checkpoint are unresolved. e-robotwine-stagese-architecture

Reader analysis

Appendix Table 8 omits many task rows and contradicts its prose, including claims of perfect task success. Ablations peak below the main LIBERO score without a documented configuration bridge. These reporting gaps prevent pooling results into one consistent experimental record. e-robotwin-conflicte-componentse-libero

7.2 Questions for discussion

  1. Which objective and training-data mixture produced each reported checkpoint? [e-objective-conflict, e-stages]
  2. Does SIGReg improve physical prediction or primarily optimization stability when reconstruction quality is controlled? [e-isae, e-components]

8. Reproducibility audit

8.1 Requirements and known gaps

Source description

Required ingredients include V-JEPA 2.1, adapted LoRA weights, ISAE, transferred Wan weights, robot demonstrations, frozen T5 conditioning and embodiment-specific action heads. Table 7 supplies optimizer and duration details, but the loss/checkpoint association must first be resolved. e-architecturee-stagese-optimizatione-objective-conflict

Reader analysis

Missing essentials include a complete data manifest and split/overlap accounting, LoRA rank/duration, ISAE loss weights, action normalization, denoising count, numeric H/R, cache projections and software versions. A meaningful first check would hold data and checkpoint initialization fixed while toggling future supervision and SIGReg separately. e-reproduction-gapse-cachee-componentse-future-ablation

8.2 Proposed reproduction checks

The following checks are proposals motivated by the paper. They have not been run as part of this reading.

Check 1: Test whether future semantics add information beyond the current cache

Reader-proposed; not run. Fix the LIBERO split, adapted encoder, ISAE, Action DiT initialization, training steps and inference cache. Compare static-context training, true future-target guidance, and future targets shuffled across matched task/timestep groups. Use the explicit velocity-loss formulation on p. 10 as a declared experimental choice; document that it does not resolve the competing appendix recipes. Keep action-loss scaling fixed and report extra training compute. Evaluate identical episodes over multiple seeds, especially LIBERO-10, and time cache construction separately from action denoising. If true future targets do not reliably beat static and shuffled controls, the claim that predictive temporal information explains the gain is weakened. e-flowe-maske-cachee-future-ablatione-objective-conflicte-evaluation

Check 2: Disentangle latent isotropy from reconstruction quality

Reader-proposed; not run. With the same frozen robot-adapted V-JEPA features, train two ISAE variants differing only in SIGReg. Match latent width, reconstruction and KL weights, data, initialization protocol and optimization budget; disclose chosen values where the PDF omits them. Measure held-out feature reconstruction error, cosine agreement and covariance spectra before fitting otherwise identical policies. Evaluate per-suite LIBERO success with paired seeds, tracking both LIBERO-10 improvement and Goal regression. A reproducible success gain accompanying reduced anisotropy would support the proposed link. If geometry improves but control does not, or gains disappear after matching reconstruction quality, isotropy alone is an insufficient explanation. e-isaee-componentse-architecturee-optimizatione-reproduction-gaps

8.3 Reading coverage

Visual audit: All 18 supplied PDF pages were rendered and visually inspected, including the title/version block, Figures 1–10, Tables 1–9, equations, appendix implementation settings and qualitative pages. All six final original crops were separately inspected. Figure 3 and the p. 10 mask were checked for query/key direction; Figure 2's frozen/trainable markers and regularization arrows were checked against the equations. Figure 6's simplified attention, training-only ISAE label and incompatible loss descriptions are disclosed. Table 2/prose and Table 8/prose conflicts and Table 3 highlighting inconsistencies are preserved. Pages 6, 7 and 10–15 supply numerical, hardware, objective and reproduction context even where not cropped. No separate supplement, external video, code or earlier revision was inspected.

PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18. Appendix coverage: reviewed.

Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.

Text reading scope & known omissions
  • PDF pp. 1–2: Abstract, Introduction, Related Work
  • PDF pp. 2–5: all Methodology subsections and Eqs. (1)–(14)
  • PDF pp. 5–8: Experimental Setup, Simulation Results, Real-World Results, Ablation Studies, Conclusion and Limitations
  • PDF pp. 8–9: References
  • PDF p. 10: Detailed Formulation of Asymmetric MoT Denoising
  • PDF p. 11: Implementation Details
  • PDF pp. 12–13: Training and Inference; three-stage Training Paradigm
  • PDF pp. 14–15: Experiment Results, component ablations and qualitative discussion
  • PDF pp. 16–18: qualitative Figures 8–10 and captions

Outside the original text pass

  • Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
  • Separate supplemental material availability has not been fully verified.
  • The extraction-only visual limitation was addressed by inspecting every PDF page and all ten figures and nine tables; six final original crops were also inspected.
  • Identity/version scope: the title page identifies arXiv:2607.23969v2, 30 Jul 2026. Title and eleven authors match the catalog; the catalog affiliation string omits the verified Leapmotor affiliation. Earlier revisions and a revision history were not supplied, so changes between versions cannot be established.
  • No separate supplement was supplied. Linked code, checkpoints and datasets were not inspected; no experiment was reproduced.

The visual audit above records the subsequent illustrated pass.

8.4 Traceable evidence

e-identityPDF p. 1, title, author/affiliation block and arXiv marginInspect

Title matches the supplied catalog; all eleven authors agree. The margin reads arXiv:2607.23969v2 [cs.RO], 30 Jul 2026. Affiliation 3 is Leapmotor, alongside the two HKUST institutions and Southeast University.

Go to primary source ↓
e-problemPDF pp. 1–2, Abstract, Introduction and Related WorkInspect

The authors motivate predictive semantic alignment by arguing that visual reconstruction spends capacity on appearance and couples world generation to execution.

Go to primary source ↓
e-overviewPDF p. 3, Figure 1 and Framework Overview, Eq. (1)Inspect

Multi-view observations pass through V-JEPA and ISAE; language and proprioception form shared context. Anchor and Action DiTs exchange semantic guidance during training.

Go to primary source ↓
e-isaePDF pp. 3–4, Diffusion-Friendly Semantic Autoencoding, Figure 2, Eqs. (2)–(7)Inspect

A Gaussian posterior compresses JEPA tokens; reconstruction combines squared error and cosine distance. KL regularization and SIGReg encourage Gaussian, isotropic latents. Figure 2 distinguishes frozen JEPA from trainable semantic encoder/decoder.

Go to primary source ↓
e-viewPDF p. 4, View-Aware Residual Conditioning, Eq. (8)Inspect

A learned camera embedding is added after normalization and projection of noisy semantic tokens at each denoising step, separately from positional encoding.

Go to primary source ↓
e-maskPDF pp. 4–5, Figure 3, Intent-Conditioned Dynamics Modeling, Eqs. (10)–(11); p. 10, Eqs. (18)–(27)Inspect

The semantic-query/action-key mask block is negative infinity. Action queries read semantic and action keys/values; semantic queries read only semantic keys/values. Both streams attend to language/proprioception context. Semantic and action tokens use 3D and 1D RoPE respectively.

Go to primary source ↓
e-flowPDF pp. 4–5, Eqs. (9), (12)–(13); p. 10, Eqs. (29)–(34)Inspect

Independent Gaussian noise corrupts actions and semantic latents at a shared timestep. The detailed formulation predicts velocities targeting noise minus clean data, with an optional normalized SmoothL1 future-token head.

Go to primary source ↓
e-cachePDF p. 5, Eq. (14); p. 10, Eqs. (35)–(37); p. 12, Inference, Eq. (45)Inspect

Current-observation semantic tokens are cached across action denoising steps; future semantic rollout is bypassed. Receding-horizon control executes the first R actions before observing again; numerical H and R are not supplied.

Go to primary source ↓
e-architecturePDF p. 11, Model Architecture, Table 6, Eqs. (38)–(40)Inspect

384×384 frames produce 576 tokens of width 1664, compressed to width 96. ISAE encoder/decoder each have three blocks and 16 heads. Both DiTs have 30 layers, width 1664 and 24 heads. The semantic branch is initialized from Wan2.2-TI2V-5B with interpolation and scaling.

Go to primary source ↓
e-optimizationPDF p. 6, Implementation Details; p. 11, Table 7 and Training Objectives and Optimization, Eqs. (41)–(42)Inspect

The main text specifies 24 NVIDIA H200 GPUs and global batch 256. ISAE/policy training takes 200k/400k steps with AdamW, learning rate 1e-4, weight decay 0.05, beta values 0.9/0.95, 5k warmup, cosine decay, bf16 and clipping 1.0. Table 7 sets action/semantic weights to 1/1 and the extra future weight to zero; schedulers use shift 5.0 and SIGReg uses 1024 projections.

Go to primary source ↓
e-stagesPDF pp. 12–13, Training Paradigm, Stage I–III and Figure 7Inspect

LoRA and a masked predictor adapt a frozen V-JEPA 2.1 ViT-G/16 backbone on a described five-source robot corpus, using third-person:wrist sampling 7:3. Adapted encoder and ISAE are frozen for final training. Stage II uses two-second windows and up to three views. Stage III describes frozen T5 conditioning, source-homogeneous batches and separate 7-DoF/14-DoF action heads.

Go to primary source ↓
e-objective-conflictPDF p. 11, Eqs. (41)–(42), Table 7; p. 12, Figure 6 and Eq. (44); p. 13, Eq. (46); p. 15, Table 9Inspect

The implementation section gives two unit-weight velocity losses and zero extra future loss. Page 12 instead describes SmoothL1 between predicted semantic velocity and target V-JEPA features, with action/JEPA weights 1/0.1. Page 13 describes a latent future-token SmoothL1 loss with weights 0.1/1. Table 9 labels future prediction weight 0.2. No reconciliation is supplied.

Go to primary source ↓
e-pipeline-figurePDF p. 12, Figure 6 and Training and Inference; p. 10, Eqs. (23)–(25); p. 13, Stage IIIInspect

Figure 6 shows semantic K/V and action Q on the left and a frozen V-JEPA/ISAE cache on the right. Its training-only box contains an ISAE label although ISAE remains at inference and is frozen in Stage III. The figure uses a Cross-Attn abstraction; the detailed formulation uses joint masked self-attention that also includes action keys/values.

Go to primary source ↓
e-evaluationPDF p. 5, Experimental Setup; p. 14, Table 8 captionInspect

RoboTwin 2.0 has 50 tasks and uses 27.5K mixed clean/randomized demonstrations. LIBERO and LIBERO-plus are described as 40 tasks and 2,000 evaluation episodes with zero-shot perturbation transfer. Table 8 states 100 trials per RoboTwin task and condition.

Go to primary source ↓
e-robotwinPDF p. 5, Table 1, LeapBot-WA, Fast-WAM, Lingbot-VA and Qwen-RobotManip rowsInspect

LeapBot-WA reports 91.04% clean, 92.48% randomized and 91.76% average. Fast-WAM reports 91.85% average; Lingbot-VA 92.24%; Qwen-RobotManip 93.85%. LeapBot-WA has the best randomized score among the listed WAMs and is marked without embodied pretraining.

Go to primary source ↓
e-robotwin-conflictPDF p. 14, Table 8, Average/Open Laptop/Pick Dual Bottles rows and Evaluation on RoboTwin Benchmark prose; p. 5, Table 1Inspect

Table 8 repeats LeapBot-WA's 91.04/92.48 values but prose says 92.64/89.80. Prose claims perfect Open Laptop and Pick Dual Bottles success, while table values are 99/97 and 97/98. Baseline averages differ from Table 1, and an ellipsis replaces many task rows.

Go to primary source ↓
e-liberoPDF p. 6, Table 2 and Simulation Results paragraphInspect

LeapBot-WA reports Spatial/Object/Goal/LIBERO-10 success 96.2/99.6/97.6/95.6 and average 97.3%. The prose instead says 97.1%. JEPA-VLA averages 96.4%, VLA-JEPA 96.1%, PALM 94.5%, and Lingbot-VA/Cosmos-Policy 98.5%.

Go to primary source ↓
e-robustnessPDF p. 7, Table 3, all columnsInspect

LIBERO-Plus averages are LeapBot-WA 73.1%, VLA-JEPA 62.9%, JEPA-VLA 25.6%, Cosmos-Policy 82.2% and ImageWAM 83.1%. LeapBot-WA scores Camera 33.8, Robot 75.1, Language 87.8, Light 93.4, Background 90.1, Noise 54.7 and Layout 76.7. Some bold entries are below other displayed values.

Go to primary source ↓
e-componentsPDF p. 7, Table 4 and Performance Analysis of Architectural ComponentsInspect

Across baseline, Robot-Mix JEPA, ISAE without SIGReg and full ISAE, averages are 89.4/91.2/91.4/92.0 and LIBERO-10 scores 74.2/81.0/80.2/85.0. Adding SIGReg changes Goal from 92.4 to 89.2. No seed variation or link to the Table 2 configuration is supplied.

Go to primary source ↓
e-future-ablationPDF pp. 14–15, Ablation Analysis of World-Action Components and Table 9Inspect

No semantic context averages 47.7%; static semantic context with zero future weight averages 90.7%; future prediction weight 0.2 averages 92.0%. The latter two LIBERO-10 values are 78.4 and 85.0. The exact relationship to the other loss descriptions is unresolved.

Go to primary source ↓
e-efficiencyPDF p. 7, Figure 4 and Analysis of Hardware EfficiencyInspect

Figure 4 labels LeapBot-WA 2.9B parameters/546 ms, Fast-WAM 6.0B/486 ms, Cosmos-Policy 2.1B/1413 ms, Lingbot-VA 5.5B/4482 ms, and Motus 8.0B/3231 ms. The figure and discussion do not define a matched inference hardware, denoising-step or action-horizon protocol.

Go to primary source ↓
e-probingPDF p. 7, Table 5 and Cross-Domain Transfer and Representation QualityInspect

Base versus mixed-domain JEPA action MSE is 0.0437/0.0417, dynamics MSE 0.00496/0.00308 and proprioception MSE 0.00460/0.00329. Relative reductions are reported as 4.6%, 37.9% and 28.5%; probe split and fitting details are not specified here.

Go to primary source ↓
e-physicalPDF pp. 5–6, Real-World Deployment, Real-World Results and Figure 5Inspect

UR5 pick-and-place examples involve fruits, vegetables and containers under changed textures and lighting. The reported real-world evidence is qualitative; no trial count, success-rate table or matched physical baseline is provided.

Go to primary source ↓
e-limitsPDF pp. 7–8, Conclusion and LimitationsInspect

The authors identify dependence on foundation representations and difficulty with fine contact physics and high-frequency tactile interactions. Multimodal anchors and long-horizon planning are future work.

Go to primary source ↓
e-qualitativePDF p. 15, Qualitative Results and Visualizations; pp. 16–18, Figures 8–10Inspect

Figures show paired external/wrist LIBERO views, perturbed LIBERO-Plus scenes, and five RoboTwin execution sequences. These selected images do not measure attention shifts, prove invariance or rule out memorization.

Go to primary source ↓
e-reproduction-gapsPDF pp. 3–4, Eqs. (4)–(7); pp. 11–13, Implementation Details and Training ParadigmInspect

The PDF supplies ISAE dimensions, optimizer details and some view sampling choices, but not numerical cosine/isotropy/KL-maximum weights, a complete five-source data manifest, LoRA rank or adaptation duration, action normalization, exact cache projection implementation, software versions, or a full evaluation configuration.

Go to primary source ↓

8.5 Primary sources

Scroll across the image to inspect details. Press Esc to close.