Decoupling Intention from Trajectory: A Representational Deduction Framework for World Action Models
1. Paper overview
In one sentence: PILOT uses training-time future-feature prediction to shape protected motion tokens for faster action generation, but its reported gains come with unresolved implementation and evaluation inconsistencies. e02e03e04e05e07e10e11e12e16
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors argue that reconstructing a future image does not explicitly teach how an interaction changes the scene. An action decoder must then infer both high-level transition structure and detailed motor commands from entangled visual features. PILOT introduces a supervised motion-semantic bottleneck to separate these responsibilities. e02 |
| Core mechanism | |
| A key reported result | LIBERO manipulation: 97.9 average; Spatial 97.1, Object 99.2, Goal 98.1, Long 97.2. Success rate (%). Four suites, 40 tasks; appendix specifies 20 episodes/task and a 500-step limit. Motus 97.7 average; reported gap 0.2 percentage points. A small reported advantage without uncertainty does not establish statistical superiority. Results use the best evaluated checkpoint. e08 |
| Reading caution | Reader analysis Qualitative t-SNE/PCA plots suggest task structure, but cannot establish causal identification or background invariance. Section 4.5 swaps Figure 6/7 references; captions identify PCA as Figure 6 and t-SNE as Figure 7. e14 |
Core contributions
Figure 3. Motion queries connect action generation to feature-space transition supervision. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the upper-left world-model branch and follow its context arrow into the Action-Perceiver. State and learnable queries enter beside noisy actions. In panel (b), the upper attention matrix excludes action columns from state/query updates; the lower matrix lets actions read the context and non-action tokens. Time-conditioned AdaLN appears only on the action stream. Read this using Eqs. (3)–(4), which agree with the full block equations on page 15. The graphic labels an auxiliary path “generalization pipeline,” whereas Section 3.5 calls it generation. At the right, distinguish current features used by the CDE from future features used as its loss target: Eq. (10) conditions on current features, motion and state; Eq. (11) supplies future supervision. e03e04e05e07e22
What it supports. The mechanism gives motion queries their own predictive task while protecting them from the action decoder’s changing noise. Actions can use the learned motion context without waiting for an image to be generated. The action stream also reads visual context directly, so motion tokens are not its exclusive information channel.
Where the evidence stops. Page 19’s independent-group attention prose conflicts with the displayed asymmetric access. The CDE future-frame arrow and temporal sketches are schematic training illustrations: Eqs. (10)–(11) do not specify a test-time future-input pathway or a deployed multi-step CDE rollout.
2. Motivation
2.1 The problem and the proposed response
The authors argue that reconstructing a future image does not explicitly teach how an interaction changes the scene. An action decoder must then infer both high-level transition structure and detailed motor commands from entangled visual features. PILOT introduces a supervised motion-semantic bottleneck to separate these responsibilities. e02
2.2 What this reading follows
A robot policy must infer what an interaction should accomplish and choose the motor commands that accomplish it. PILOT assigns part of that burden to learnable motion queries: they read the present scene and instruction, then receive supervision for predicting a future scene representation. The same queries help an action decoder refine a trajectory. During deployment, the future-image and dynamics-supervision branches are bypassed. This reading follows that information flow through the original architecture and result tables, separating executed manipulation from latent prediction. It also preserves the source’s conflicting settings and aggregates, which matter when assessing the reported advantages or planning a reproduction. e02e03e04e05e07e10e11e12e16
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | IDM |
| Quadrant | Q4 · Dual-system × IDM |
3.1 Evidence-based assessment
Conflicts with the recorded classification
Separate world-model/action branches support Dual-system architecture, and RD supports the latent/JEPA category. The recorded IDM quadrant is misleading for deployment: motion tokens arise from present inputs, CDE predicts forward features only during training, and no future-pair inverse-dynamics policy is specified. Joint optimization alone does not imply One Model architecture or joint future/action inference. e03e05e07e21
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Protect motion tokens from the action sampler
Consider one observation and instruction while the action sampler changes its Gaussian starting point. If the motion queries can attend to noisy actions, their representation can change for reasons unrelated to the scene. PILOT’s main equations prevent that route: state/query updates read only themselves and visual context, while action updates may also read the state/query group. Plain layer normalization on non-action tokens removes the explicit flow-time modulation used by actions. The reader’s deduction is that motion semantics should remain stable across noise draws, an intended property also checked in the training description. This is not an exclusive bottleneck, because actions retain direct access to visual context. The appendix’s independent-group description must therefore be resolved against the equations before implementing this mechanism. e04e07e22
5.2 Use the future as a teacher, then remove the teacher
The motion queries are initially learnable slots, not named physical concepts. Representational Deduction gives them a task: help a CDE predict frozen VJEPA2-AC features of the demonstrated future observation from current features and robot state. The future representation is the target of a Smooth-L1 loss; gradients flow through the trainable predictor and projector into the queries. A second auxiliary objective retains future-image generation in the video branch, including a described option to emphasize changed regions. The two teachers supervise different outputs, yet both influence a policy that operates from present inputs. At test time, neither the CDE prediction nor an explicitly generated image is required. The remaining policy performs action flow matching; it does not run a CDE rollout to search over candidate actions. e03e05e06e07
5.3 Distinguish useful supervision from a causal explanation
The experiments ask several different questions. Standard LIBERO measures task completion; LIBERO-Plus changes the observation or task conditions; Agibot-G1 tests executed physical manipulation; the ablation changes training components. Together, the printed results are consistent with useful extra supervision, especially when RD+CDE and decoupled attention are added to Motion-CoT. They do not prove that the latent tokens identify physical causes. Qualitative projections show organization but supply no intervention-based test, and the cumulative ablation does not separate RD from CDE. Latency is another question: bypassing image generation can save time even if the representation’s causal interpretation remains uncertain. Finally, contradictory protocols and averages prevent treating every printed improvement as a verified ranking. A reproduction should resolve these reporting issues and test the mechanism with controlled alternatives. e08e09e11e12e13e14e15
5.4 Training and inference
During training
The main formulation fine-tunes world model/CDE, trains action branch/projectors from scratch, and freezes the VJEPA2-AC encoder. Algorithms also freeze text encoder and VAE; later prose inconsistently calls all backbone components trainable. e05e06e07e16
Training uses 224×224 observations and future offset 16. LIBERO/RoboCasa report 100,000/300,000 steps on 32 H20 GPUs, effective batches 512/256, and approximately 90/200 hours. These are reported costs, not measurements from this reading. e17
During inference
Compute c once, initialize an action chunk from Gaussian noise and integrate predicted velocities with Euler updates. The action branch recomputes its tokens during updates. CDE and future-image generation are bypassed. The appendix describes executing a prefix before replanning, but horizon and denoising-step specifications conflict. e07e16
5.5 Implementation flow
- Encode the present
A Wan2.2 understanding pass encodes current image and instruction into context c. The generation pathway shares transformer weights but has a different training role; ground-truth future frames are not policy inputs in the method equations. e03e07
- Form motion semantics
The Action-Perceiver combines state token S, learnable queries Q and noised action tokens. Query outputs define m. Main equations let state/query tokens read c and each other, while actions also read state/query tokens. Only actions receive flow-time modulation. e04e22
- Supervise the transition
Frozen VJEPA2-AC encodes current features and future targets. A pretrained, trainable CDE receives current patches plus projected m and state, predicts future patches, and sends the representation loss gradient into queries. This is a learned feature transition, not a physics simulator. e05
- Learn trajectories
The action head regresses velocity from Gaussian noise toward demonstration actions. A parallel future-latent loss trains the video branch; optional change weighting emphasizes locations whose latent features change. e04e06
6. Experiments & results
PILOT trains motion-semantic query tokens to predict future VJEPA2-AC representations, then uses those tokens to guide action flow matching from current observations. Future-image generation and the Causal Dynamics Engine supervise training but are bypassed during deployment. Reported manipulation gains are promising, although conflicting implementation details, evaluation protocols and table arithmetic limit reproducibility.
6.1 Read the original evidence
Table 2. Robustness gains vary substantially across perturbation types. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read down a column to compare methods under one perturbation, then across the PILOT row to see the shape of its robustness. Cam., Rob., Lang., Light, Back., Noise and Layout identify camera, robot, language, lighting, background, noise and layout changes. These are zero-shot tests from standard LIBERO according to the caption, not separate fine-tuning results. The strongest reported average comparator is PokeVLA, in the latent-action group. Inspect its row alongside PILOT rather than interpreting the bottom-row average alone. The size column records reported parameter counts; this table does not hold pretraining data or training compute constant. e09
What it supports. PILOT’s printed average is 81.0%, versus 79.3% for PokeVLA, a 1.7-percentage-point difference. Robot improves from 46.1% to 70.0% and Layout from 77.2% to 82.0%. However, PokeVLA remains stronger on Camera, Language, Light and Noise, making the generalization result selective rather than universal.
Where the evidence stops. The table supplies no uncertainty, dedicated per-perturbation rollout counts or precise averaging rule. Preserve the averages as printed. Better transfer success is compatible with useful motion supervision, but does not by itself establish that learned features are invariant to appearance.
Table 10. The detailed task rows reveal both strong transfer cases and unresolved aggregate reporting. Original paper, p. 24 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the far-right PILOT column within each group before moving to the printed averages. PnP Close concerns placing objects into destinations such as cabinets or drawers; the remaining groups organize novel-object tasks by source surface. The abbreviation key identifies CbBox as cardboard box, TBasket as tiered basket and TShelf as tiered shelf. The retained note states 50 rollouts, ego-centric view, 29-DoF actions and 1000 demonstrations per task; daggers denote internet-scale pretraining. These notes matter because Section 7.13 gives conflicting evaluation settings. Compare particular task rows directly when possible, and treat the bottom-row aggregates as author-reported values requiring an arithmetic audit. e10e11
What it supports. The printed overall comparison is PILOT 62.6% versus FastWAM 56.7%. PILOT reaches a 74.8% Tray-group average, yet Tray-to-tiered-shelf is only 22%, below FastWAM’s 36%. Reader arithmetic over the displayed task cells yields approximately 62.7% and 54.8% overall, respectively; these calculations expose a discrepancy rather than replace the source’s reported results.
Where the evidence stops. Section 7.13 instead specifies 20 episodes, third-person inputs and 32-dimensional actions. Category names also differ across prose and tables. PILOT does not lead every group: LDA exceeds it on Cuttingboard and PhysBrain on Plate. Aggregate rankings need clarification before reuse.
Table 4. Physical deployment is evaluated separately from simulation and reduced-data training. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the eight Standard Tasks columns: they report object-specific robot successes, with 50 rollouts per standard task stated in Section 4.3. The three Generalization columns then change lighting plus camera angle, desk appearance, or object color/appearance. These are distribution shifts, not extra stages in a long-horizon task. The far-right column changes the training data budget to 10% of demonstrations; it should not be combined with the transfer columns as one protocol. Compare PILOT with Fast-WAM across these blocks, keeping the reported averages separate from the individual task cells. The original table, not the nearby prose, gives pencil-case success as 56%. e12e18
What it supports. The printed averages favor PILOT: 83.1% standard, 68.3% transfer and 62.4% few-shot, compared with Fast-WAM’s 73.3%, 50.0% and 40.8%. These are physical robot results, not judgments of generated videos. The reduced-data result is particularly relevant to demonstration cost, but it does not establish a universal sample-efficiency curve.
Where the evidence stops. PILOT’s displayed cells average to 83.0% standard and 68.0% transfer, not the printed 83.1% and 68.3%. The printed full-to-few-shot drop is 20.7 percentage points, about 24.9% relative. Transfer trial counts, few-shot sampling details and uncertainty are not separately specified.
Table 6. Bypassing explicit image generation reduces the reported deployment latency. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the Size column first: the two strategies have the same reported 5.4B size. The intended difference is how the shared video backbone is used. Predict-then-act invokes iterative generation before actions; the understanding pathway extracts context with a single transformer pass. The action decoder still performs its own integration, so a single understanding pass does not mean a single network call for the entire policy. The table then reports both success and wall-clock latency. Section 7.3 discusses fifty video denoising steps, whereas Section 7.10 and Table 7 specify twenty; do not assume one fully resolved timing configuration from these descriptions. e07e15e16e20
What it supports. The decoupled row reports 111.60 ms at 62.6% success, against 1329.95 ms at 58.4% for predict-then-act. Calculated from those latency cells, the speedup is approximately 11.9× and the reduction 91.6%. This is evidence for a cheaper reported inference pathway, not a measurement of deployment timing performed for this report.
Where the evidence stops. The printed 8.26 Hz does not equal the reciprocal latency, about 8.96 Hz. Figure 1’s 819/105 ms and Section 7.14’s 6.9 ms/action, 145 Hz raw and 58 Hz effective claims are not reconciled to this table; do not combine them.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO manipulation Four suites, 40 tasks; appendix specifies 20 episodes/task and a 500-step limit. | 97.9 average; Spatial 97.1, Object 99.2, Goal 98.1, Long 97.2. Success rate (%) | Motus 97.7 average; reported gap 0.2 percentage points. A small reported advantage without uncertainty does not establish statistical superiority. Results use the best evaluated checkpoint. e08 |
| LIBERO-Plus zero-shot robustness Transfer from standard LIBERO to seven perturbation types; dedicated rollout count unspecified. | 81.0 reported average; Robot 70.0 and Layout 82.0. Success rate (%) | PokeVLA 79.3 reported average, Robot 46.1 and Layout 77.2. The reported average gain is 1.7 percentage points; improvements are uneven and do not prove appearance invariance. e09 |
| RoboCasa-GR1 tabletop manipulation 24 tasks. Table 10 specifies 50 rollouts, ego-centric view, 29-DoF actions and 1000 demonstrations/task; Section 7.13 conflicts. | 62.6 reported overall; Tray group 74.8. Success rate (%) | FastWAM 56.7 reported overall. Preserve these as printed aggregates: equal-weight arithmetic from Table 10 gives about 62.7 for PILOT and 54.8 for FastWAM. Aggregate comparison and protocol require clarification. Tray-to-tiered-shelf remains 22%. e10e11 |
| Agibot-G1 desk manipulation and transfer Eight standard tasks with 50 rollouts/task; three transfer settings; separate fine-tuning with 10% of demonstrations. | 83.1 standard, 68.3 transfer and 62.4 few-shot averages, as reported. Success rate (%) | Fast-WAM reports 73.3, 50.0 and 40.8 respectively. Reader arithmetic gives 83.0 and 68.0 from PILOT’s standard/transfer cells. The printed 83.1→62.4 decrease is 20.7 percentage points, approximately 24.9% relative, not the stated 20.7% relative. e12 |
| Component ablation Table 5 cumulative configurations on LIBERO/RoboCasa. | Motion-CoT alone 94.5/56.7; with RD+CDE 96.1/59.8; adding decoupling 96.9/61.3; full 97.9/62.6. Success rate (%) | Bare baseline 91.3/51.2; future-only 93.6/54.8. Both supervision pathways help in this sequence. RD and CDE are added together; this is not a factorial isolation of every mechanism. e13 |
| RoboCasa pipeline latency Table 6, equal reported 5.4B sizes; Figure 4 identifies H20 timing hardware. | Decoupled: 111.60 ms and 62.6%. Latency (ms) and success rate (%) | Predict-then-act: 1329.95 ms and 58.4%. Reader calculation: about 11.9× faster, or 91.6% lower latency. Printed 8.26 Hz is not the reciprocal of 111.60 ms (about 8.96 Hz). Timing scope is not reconciled with other latency/frequency claims. e15e20 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 5. Representation supervision and asymmetric attention add gains within the reported ablation sequence. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Treat the first three rows as alternative starting points, not a purely cumulative ladder: the third row removes future prediction while introducing Motion-CoT. Read the remaining rows sequentially. RD and CDE turn on together, then Decoup. adds the asymmetric attention mechanism, and finally Fut. Pred restores the auxiliary image objective. The last two columns are success percentages on different benchmarks; they are not two measurements of the same task. The comparison from row four to row five most directly addresses the attention design, while the comparison from row five to row six tests whether future-image supervision still adds value once motion supervision is present. e04e06e13
What it supports. Adding RD+CDE to Motion-CoT raises LIBERO/RoboCasa from 94.5/56.7 to 96.1/59.8. Decoupling then adds 0.8/1.5 percentage points, and restoring future prediction reaches 97.9/62.6. This supports complementary training signals in the tested sequence, despite omitting future-image generation during normal inference.
Where the evidence stops. RD and CDE are not separated experimentally, and this is not a full factorial design. No variance or confidence intervals are shown. Success improvements do not isolate physical causality, and the paper’s conflicting attention descriptions complicate exact reproduction.
7. Analysis & limitations
7.1 What the evidence leaves open
Qualitative t-SNE/PCA plots suggest task structure, but cannot establish causal identification or background invariance. Section 4.5 swaps Figure 6/7 references; captions identify PCA as Figure 6 and t-SNE as Figure 7. e14
Protocol conflicts, unreconciled averages and absent uncertainty weaken exact ranking claims. Best-checkpoint selection is reported without a separate validation-selection split. Real-world transfer trial counts and few-shot sampling/seeds are not specified separately. e08e09e10e11e12e18
Hierarchical Motion-CoT for multi-stage planning remains future work. There is no dedicated limitations section or quantitative evidence for every architectural portability claim in the abstract. e01e19
7.2 Questions for discussion
- Does RD improve held-out actions because its target contains transition information, or because it supplies an additional representation loss?
- Which configuration reconciles the attention, horizon, timing and evaluation contradictions?
8. Reproducibility audit
8.1 Requirements and known gaps
Required ingredients include Wan2.2-TI2V-5B, frozen VJEPA2-AC features, pretrained predictor, demonstrations and the described StarVLA-based integration. Real data require three camera views and LeRobot preprocessing; Table 9’s shared-versus-per-task trajectory wording needs clarification. e03e16e18
Resolve four versus twenty action steps, sixteen action tokens versus longer horizons, and twenty-four versus four CDE layers. Clarify main-text 1024-dimensional queries against appendix 768 internal/1024 output dimensions, and contradictory attention/freezing descriptions. e04e16
Resolve shifted-Beta versus Logit-Normal action sampling and twenty versus fifty video steps. Equation (31) can yield negative values and is not a Logit-Normal transform. RoboCasa warmup/rates disagree between prose/Table 8. Change-mask percentile/maximum weight, exact software versions and evaluation seeds remain unspecified. e06e17e20
A proposed minimal check holds observations fixed, varies action noise/time and measures motion-query invariance under asymmetric attention. Then compare true future-feature supervision with shuffled targets at matched data/compute. These are proposals, not reproduced experiments. e04e05e07e13
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does asymmetric attention actually make motion queries noise-invariant?
Reader-proposed check: fix current image, instruction and state, disable stochastic layers, and evaluate repeated Gaussian action samples across flow times. Compare the Eq. (3)–(4) asymmetric mask with a symmetric-attention control at identical weights. Measure maximum query-output differences and sensitivity to action inputs, using the paper’s 1e-5 agreement tolerance as a reference. The protected queries should remain stable while action outputs can change. Dependence on action noise/time in the protected path would falsify the intended isolation property. Report separately how the contradictory independent-group appendix description is implemented. e04e07e22
Check 2: Does correctly paired future supervision improve held-out actions?
Reader-proposed comparison: train matched Motion-CoT models with true paired future-feature targets, future targets shuffled across demonstrations, and RD disabled. Keep architecture, decoupled attention, future-image loss, data budget, optimization and seeds fixed; measure target loss as well as LIBERO and zero-shot LIBERO-Plus success with uncertainty. Resolve the source’s configuration conflicts before fixing this protocol. If shuffled targets match true targets in transfer success, the interpretation that transition-specific supervision drives the gains is weakened. A reproducible advantage from correctly paired targets would support that mechanism without proving causal identification. e05e06e09e13e16e17e20
8.3 Reading coverage
Visual audit: Original title/author page, Figures 1–12, Tables 1–10, method equations, Algorithms 1–4 and appendix configuration/evaluation pages were rendered and visually inspected. Every final crop was separately viewed. Figure 3’s token access was checked against Eqs. (3)–(4), (18)–(19) and the conflicting page-19 prose. Figure 6/7 references, Table 4/10 arithmetic and Table 6 timing were checked against the printed pages. Table 10 retains its abbreviation and evaluation notes because they define the columns and expose the protocol discrepancy. References on pages 9–11 were read in text only. No separate supplements, linked code or external artifacts were inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, Abstract and Sections 1–2, pp. 1–3
- Sections 3.1–3.5: problem, Motion-CoT, CDE, objectives and inference, pp. 3–5
- Sections 4.1–4.5: implementation, simulation, real robots, ablations and representations, pp. 5–8
- Section 5: Conclusion, p. 8; References, pp. 9–11
- Section 6: Extended Related Work, pp. 12–13
- Sections 7.1–7.11: algorithms, architectures, losses and flow matching, pp. 13–19
- Sections 7.12–7.15: training, evaluation, efficiency and full results, pp. 19–24
Outside the original text pass
- Identity/version note: the supplied PDF is arXiv:2608.06994v1, 7 August 2026. Title and all nine authors match the catalog. Five affiliation labels extend the catalog’s single Joy Future Academy, JD entry. No other revision or edition was supplied or compared.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- The extraction limitation was addressed by inspecting original PDF pages 1–8 and 12–24, including all figures and tables. References on pages 9–11 were read in text, not visually inspected.
- Separate supplemental material availability has not been fully verified.
- All nine supplied text chunks were read individually. No linked project, code, checkpoints, dataset artifacts or separate supplements were inspected; no experiments were reproduced.
- Conflicting source descriptions remain unresolved and are identified below; no undocumented implementation is selected as the correct one.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/author/affiliation block and arXiv margin stamp
The exact catalog title and all nine catalog authors appear. Affiliations are NJU, HKUST, CUHK-SZ, THU and Joy Future Academy, JD. The artifact identifies arXiv:2608.06994v1, 7 August 2026.
Go to primary source ↓e02PDF pp. 1–2, Abstract and Section 1; p. 4, Section 3.2
The authors attribute weak control representations to entanglement between visual reconstruction, transition semantics and trajectory refinement, and propose PILOT with Representational Deduction and Motion-CoT.
Go to primary source ↓e03PDF p. 3, Figure 3(a–c), caption and Section 3; p. 4, Eqs. (2)–(6); p. 5, Inference
The architecture separates a Wan2.2 world-model branch, Action-Perceiver and training-time CDE. Current observations/instructions produce context; future observations provide supervision. Figure 3 does not explicitly mark frozen targets or distinguish every training arrow from deployment.
Go to primary source ↓e04PDF p. 4, Section 3.3, Eqs. (3)–(7); p. 15, Section 7.5, Eqs. (15)–(21); p. 19, Separation attention
Main and full-block equations allow non-action tokens to attend to context and themselves, and actions additionally to attend to non-action tokens. Only actions receive time-conditioned AdaLN. Page 19 instead describes independent within-group self-attention.
Go to primary source ↓e05PDF pp. 4–5, Section 3.4, Eqs. (8)–(11); p. 16, Section 7.6, Eqs. (22)–(26)
Frozen VJEPA2-AC encodes current/future features. A trainable predictor initialized from VJEPA2-AC predicts future patches from current patches, projected motion tokens and state. Smooth-L1 gradients reach queries through the CDE/projector. Original predictor action encoders are bypassed and frozen.
Go to primary source ↓e06PDF p. 5, Section 3.5, Eqs. (12)–(13); pp. 16–17, Section 7.7, Eqs. (27)–(29)
Joint training combines action regression, future-latent regression and RD. Future positions are masked; optional change weighting uses a batch percentile threshold and a larger weight for changed locations. Values for that percentile and maximum weight are not supplied.
Go to primary source ↓e07PDF p. 14, Sections 7.1–7.3; p. 17, Algorithms 1–3; p. 18, Algorithm 4
Action inference computes context once and iterates Euler updates from Gaussian noise without CDE/video generation. Training repeats independent action noise and checks query agreement at absolute tolerance 1e-5. Understanding/VJEPA encoders use bfloat16; action/CDE/projectors use float32. Algorithms mark text encoder and VAE frozen.
Go to primary source ↓e08PDF p. 6, Table 1; pp. 20–21, Sections 7.12–7.13, LIBERO and Evaluation protocol
PILOT reports Spatial/Object/Goal/Long success 97.1/99.2/98.1/97.2 and 97.9 average versus Motus 97.7. Appendix specifies 40 tasks, 20 episodes/task and 500-step limit, selecting the highest-scoring checkpoint. The table gives no uncertainty.
Go to primary source ↓e09PDF p. 6, Table 2 and caption, PILOT/PokeVLA rows
Zero-shot LIBERO-Plus reports PILOT/PokeVLA averages 81.0/79.3, Robot 70.0/46.1 and Layout 82.0/77.2. PILOT trails PokeVLA on Camera, Language, Light and Noise. Dedicated rollout count and aggregation formula are not supplied.
Go to primary source ↓e10PDF p. 7, Table 3; p. 24, Table 10, task rows and aggregates
PILOT/FastWAM overall averages are printed as 62.6/56.7. PILOT Tray is 74.8, but NvTR to TShelf is 22 versus FastWAM 36. Cuttingboard is 61.2 versus LDA 64.2; Plate is 60.5 versus PhysBrain 62.5. Printed overall means are not reproduced by equal weighting of all task entries.
Go to primary source ↓e11PDF pp. 20–21, Section 7.13, RoboCasa-GR1 and Evaluation protocol; p. 24, Table 10 caption/headings
Prose specifies 20 episodes, third-person view and 32-dimensional actions. Table 10 specifies 50 rollouts, ego-centric view, 29-DoF actions and 1000 demonstrations/task. Prose categories differ from the table groups organized by pick-and-place source/destination.
Go to primary source ↓e12PDF p. 7, Section 4.3; p. 8, Table 4; pp. 21–22, Real-World benchmark and Evaluation settings
Eight Agibot-G1 standard tasks use 50 rollouts each. Three transfer settings and 10%-data training are also evaluated. PILOT standard/transfer/few-shot means are printed as 83.1/68.3/62.4 versus Fast-WAM 73.3/50.0/40.8. PILOT standard cells are 90,100,70,94,56,88,72,94; transfer cells are 72,68,64. Prose instead says pencil case 55 and calls the 20.7-point few-shot decrease relative.
Go to primary source ↓e13PDF pp. 7–8, Section 4.4 and Table 5, all configurations
Baseline, future-only, Motion-CoT-only, addition of RD+CDE, addition of decoupling, and full-model rows give LIBERO/RoboCasa 91.3/51.2, 93.6/54.8, 94.5/56.7, 96.1/59.8, 96.9/61.3 and 97.9/62.6. RD/CDE switch on together; no factorial separation or uncertainty is shown.
Go to primary source ↓e14PDF p. 2, Figure 2; p. 7, Figures 6–7 and captions; p. 8, Section 4.5
Figure 6 is PCA pseudocoloring of CDE predictions; Figure 7 is t-SNE of Motion-CoT embeddings. Section 4.5 reverses these references. The authors interpret qualitative clustering/spatial structure as decoupled semantics.
Go to primary source ↓e15PDF p. 14, Table 6 and Section 7.3; p. 5, Figure 4; p. 23, Section 7.14; p. 1, Figure 1(c)
Table 6 gives equal-size 5.4B strategies: predict-then-act 58.4%, 1329.95 ms, 0.75 Hz; decoupled 62.6%, 111.60 ms, 8.26 Hz. Figure 4 names H20 hardware. Figure 1 instead shows 819/105 ms; Section 7.14 separately reports 6.9 ms/action, 145 Hz raw and 58 Hz effective control.
Go to primary source ↓e16PDF pp. 5–6, Section 4.1; pp. 16–19, Section 7.6, Algorithms 1–2, Section 7.10 and Table 7; p. 21, Table 8; p. 23, Section 7.14
Main text uses 64 queries, d=1024 and four action steps; Table 7 gives internal d=768, output 1024 and twenty steps. Algorithms/Table 7 use sixteen action tokens, while Table 8 lists horizons 50/100/30. CDE is 24 layers in Section 7.6/Table 7 but four in Section 7.14. All-trainable backbone prose conflicts with frozen-encoder algorithms.
Go to primary source ↓e17PDF pp. 19–21, Section 7.12 and Table 8
Training uses 224×224 inputs, future offset 16, AdamW, 32 NVIDIA H20 GPUs and DeepSpeed ZeRO Stage 2. LIBERO/RoboCasa effective batches are 512/256, training lengths 100000/300000 steps, approximately 90/200 hours. RoboCasa warmup is 500 in prose versus 3000 in Table 8; learning rates are uniformly 1e-5 in prose but 5e-5 for action/predictor/projectors in the table.
Go to primary source ↓e18PDF pp. 21–23, Real-World benchmark, Evaluation settings, Ablation configurations and Table 9; pp. 13–16 and 22, Figures 8–12
Real data use head/two wrist cameras, teleoperation, LeRobot format and min-max state/action normalization. Table 9 gives 7,135,187 frames and 3,965 trajectories per task, but its caption says all tasks share the same 3,965 trajectories from 32 sessions. Full/few-shot and RD-only configurations are described without a separate RD-only real-world result table.
Go to primary source ↓e19PDF p. 8, Section 5, Conclusion
Hierarchical Motion-CoT for multi-stage long-horizon planning is future work. There is no dedicated limitations section.
Go to primary source ↓e20PDF p. 17, Section 7.8, Eqs. (30)–(31); p. 19, Section 7.11, Flow time sampling; p. 18, Section 7.10; p. 14, Section 7.3
Action sampling is shifted Beta on p. 17 and shifted Logit-Normal on p. 19. The printed world-time equation uses a signed rational transform, not a sigmoid. Video generation uses fifty steps in timing prose but twenty in Section 7.10/Table 7.
Go to primary source ↓e21PDF p. 4, Eq. (1) and Section 3.3; p. 5, Sections 3.4–3.5; p. 12, Section 6.2
The policy factorizes through a latent motion variable derived from present inputs. A separate pretrained world-model backbone supplies an action head; future-state regression is auxiliary. Deployment does not specify inverse dynamics taking a current/future-state pair.
Go to primary source ↓e22PDF p. 3, Figure 3(b); p. 4, Eqs. (3)–(4); p. 15, Eqs. (18)–(19); p. 19, Separation attention
Figure 3(b) shows asymmetric token access and time/AdaLN only for actions, consistent with the equations. The later independent-groups prose would remove action access to state/query tokens if implemented literally.
Go to primary source ↓8.5 Primary sources
Decoupling Intention from Trajectory: A Representational Deduction Framework for World Action Models ↗
PDF · 14,164 extracted words
Source fingerprint
72b17529204e41601b853a6256d8e610247a8d077c8c261a7e4a319447f7318b