SLIM-0.5B: Learning Action-Grounded Predictive Latents for Robot Manipulation
1. Paper overview
In one sentence: SLIM uses bidirectional latent prediction to train a compact action policy, retaining predictive structure at deployment while avoiding explicit future generation. e-probleme-maskede-policye-liberoe-real-protocole-progresse-efficiency
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors argue that large semantic backbones and pixel-level prediction can spend capacity on information unnecessary for continuous control. They instead seek representations that explain an observed transition through actions and predict its latent outcome from actions, providing dynamics structure without pixel decoding in the control loop. e-problem |
| Core mechanism | Source description Masked action reconstruction and future-latent prediction train the same MoT subsequently used for control. The latent target space evolves with the visual encoder instead of coming from a fixed external teacher. e-maskede-implementation |
| A key reported result | Zero-shot LIBERO-Plus robustness: 77.45% Overall success rate. Original-LIBERO training; unchanged checkpoint on 10,030 perturbation cases. OpenVLA-OFT 69.6%; Fast-WAM 50.0%; VLA-JEPA 79.5%. Strong relative to several larger baselines, but below VLA-JEPA overall. Robot-state robustness is only 36.90%; embodied pretraining differs. e-liberoe-benchmark-protocol |
| Reading caution | Source description One compact scale is tested; scaling with capacity, pretraining volume and embodiment diversity remains unestablished. e-limitations |
Core contributions
- Source description
Masked action reconstruction and future-latent prediction train the same MoT subsequently used for control. The latent target space evolves with the visual encoder instead of coming from a fixed external teacher. e-maskede-implementation
- Source description
The 0.47B count is 472.14M trainable parameters: 86.58M visual encoder, 377.96M interaction trunk and 7.60M interfaces/heads. Frozen T5-small and the Stage-1-only EMA encoder are excluded; this is not an all-components memory count. e-parameters
Figure 2. Two complementary prediction tasks make observation latents sensitive to actions and their consequences. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the left branch upward: online encoders supply current and clean future latents, while the action input is a flow-noised chunk. The predictor outputs action velocity, not a discrete action label. In the middle branch, learned embeddings M_z occupy future positions and clean actions provide transition information. The predicted future latent is compared with the EMA target; the dashed target path explicitly marks stop-gradient. On the right, rows are queries and columns are keys. Dark cells allow attention. IDM observation queries cannot read noisy-action keys; FDM future-slot queries can read clean actions. These directions agree with the conditioning variables in Equations (5)–(7). e-inputse-maskede-policy
What it supports. The paired objectives constrain the same trajectory from opposite directions: an action must explain an observed change, and the resulting latent must be predictable from the action. This supplies a concrete mechanism for action grounding without reconstructing future image pixels. Both branches belong to Stage 1.
Where the evidence stops. The gray future-image placeholder represents missing latent information, not a pixel reconstruction target. Stage 2 uses learned future slots without clean future inputs or explicit future-latent loss; this training diagram is not the deployed inference graph.
2. Motivation
2.1 The problem and the proposed response
The authors argue that large semantic backbones and pixel-level prediction can spend capacity on information unnecessary for continuous control. They instead seek representations that explain an observed transition through actions and predict its latent outcome from actions, providing dynamics structure without pixel decoding in the control loop. e-problem
2.2 What this reading follows
A robot policy needs to connect what it sees with what its movements will change. SLIM puts that relationship inside a compact observation–action transformer. During its first training stage, one branch reconstructs actions from observed transitions and another predicts the latent consequence of clean actions. A second stage turns the same backbone into a language-conditioned flow policy that can act without seeing the future. The useful reading question is whether this training structure improves executed control at a lower inference cost. The results support that direction, but require care about partial-credit physical scores, unequal baseline pretraining and different native action horizons. e-probleme-maskede-policye-liberoe-real-protocole-progresse-efficiency
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | VLA |
| Architecture | Dual-system |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Conflicts with the recorded classification
VLA and the future-representation/flow-matching subcategories are supported. Dual-system conflicts with the depicted shared MoT: observation/action streams interact inside one control backbone, while EMA only supplies training targets. Architecture evidence supports an integrated control model, not a separate world-model planner feeding a policy. Keeping prediction paradigm/quadrant not applicable is reasonable for this deployed VLA; Stage-1 inverse/forward objectives do not establish joint future–action generation at inference. e-backbonee-maskede-policye-parameters
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 1. Learn which actions explain a change, then which change an action predicts
Begin with a demonstrated transition: current observation, an H-step action chunk and the observation after that chunk. IDM receives both observation latents, but replaces the clean action input with the flow interpolation between Gaussian noise and the demonstrated action. Its target is the velocity A_t minus epsilon, so “action masking” here means conditional flow reconstruction. FDM reverses the available information: clean actions are visible, while learned embeddings replace future latent positions. Its prediction is compared with a detached EMA-encoder target using normalized L1 distance. The attention masks prevent inappropriate access to noisy actions while permitting future slots to use clean actions. Both tasks reuse the same interaction backbone and observation-latent space; their weighted combination is intended to make those latents useful for understanding transitions. e-inputse-maskede-backbone
Figure 3. One interaction backbone connects observation tokens, proprioception and continuous actions. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the bottom. The vision encoder produces Z_t, and q_t supplies proprioceptive state on the observation side. The action encoder produces continuous action tokens on the other side. Each stream forms its own queries, keys and values, but both enter the wide joint-attention block. The output then separates into per-stream processing and language cross-attention. The side arrows identify language ell and flow time tau as conditioning signals rather than extra image patches. The repeated blocks correspond to the 16-layer trunk described in Appendix A.3. Section 3.2 also specifies learned future-slot positions that this compact backbone drawing does not display. e-backbonee-inputse-maskede-policye-parameters
What it supports. The architecture supports an integrated control backbone: two interacting token streams share an attention operation. This is stronger evidence than merely knowing that two losses are trained together. It explains why the recorded Dual-system classification deserves review even though the future-representation and flow-matching VLA subcategories fit.
Where the evidence stops. The action label A_t is schematic. Section 3.2 makes its contents objective-dependent: clean actions for FDM and noised actions for velocity prediction. Do not infer a separate world-model rollout or planner from the two streams.
5.2 2. Remove future observations without discarding the trained interaction structure
The future observation that makes IDM possible is unavailable when a robot must choose its next action. Stage 2 addresses this mismatch explicitly: clean future latents disappear from the inputs and there is no future-latent supervision. Learned future slots remain within the observation stream, and action tokens attend to their hidden states through joint attention. The final objective supervises only the action velocity field conditioned on current vision, proprioception and language. At inference, observation and language context stay fixed while four flow-sampling steps update the noisy action chunk. DINOv2 has been fine-tuned for the policy; T5-small remains frozen with cached language embeddings. The EMA target encoder is removed. Thus, the paper’s implicit predictive structure does not require a separate future-latent prediction to be generated and then passed to a planner. e-policye-backbonee-implementatione-parameters
5.3 3. Judge the representation through control, collapse diagnostics and cost
Reader interpretation: three distinct tests are needed to assess this design. First, downstream control must improve, which motivates the Stage-1 and EMA ablations. Second, the latent objective must remain meaningful: Table 3 shows that removing EMA lowers MSE while reducing effective rank and worsening control, so a smaller loss cannot by itself establish better dynamics. Third, deployment should benefit from the compact design; Table 4 supports lower per-call cost under native configurations. These tests answer different questions. LIBERO-Plus success measures robustness in simulation, whereas physical progress can include a grasp without task completion. The latency experiment excludes parts of the robot loop and compares unequal horizons. Together the results support a useful compact policy, while leaving causal attribution of each architectural choice and equal-budget efficiency open. e-ablatione-collapsee-liberoe-progresse-efficiency
5.4 Training and inference
During training
Stage 1 runs three epochs with IDM:FDM weights 0.125:1 and EMA momentum 0.999; Stage 2 runs 40 simulation epochs or 20 real-world epochs. DINOv2 is fine-tuned and T5 frozen. Simulation uses eight H100 80GB GPUs in BF16. e-implementation
LIBERO Stage 1 includes LIBERO-90 plus four target suites; Stage 2 uses only those suites. Both CALVIN stages use language-annotated ABC. Physical training mixes 150 demonstrations per task. “No additional embodied pretraining” therefore does not mean no predictive training or no pretrained encoders. e-benchmark-protocole-real-protocole-liberoe-implementation
During inference
Starting from Gaussian action noise, four flow-sampling steps update actions while observation and language context stay fixed. Simulation uses seven-dimensional actions, with horizons eight for LIBERO and twelve for CALVIN. Neither future observations nor EMA are required. The text does not specify the numerical integrator or how much of each chunk executes before replanning. e-maskede-backbonee-policye-implementatione-parameters
5.5 Implementation flow
- Encode control context
DINOv2-B/14 maps images to Z_t tokens; proprioception becomes a prepended state token. T5-small provides language conditioning. A three-layer action encoder combines continuous actions with the flow timestep. Tokens enter a width-768 interaction space. e-inputse-backbonee-implementation
- Exchange information
Sixteen MoT layers use observation/action streams with separate queries, keys and values, shared joint attention and per-stream language cross-attention. Future positions contain clean future latents for IDM or learned embeddings M_z for FDM and policy training. e-backbonee-parameters
- Constrain both directions
IDM predicts noisy-action velocity from current and clean future latents. FDM predicts future latents from current latents and clean actions. Figure 2 blocks observation queries from noisy-action keys in IDM; FDM future-slot queries may read clean actions. EMA targets are detached. e-masked
- Train deployable actions
Stage 2 removes clean future inputs and explicit future supervision. Action flow matching uses current context and learned future slots, whose hidden states remain accessible to action tokens. e-policy
6. Experiments & results
SLIM learns manipulation representations by connecting actions and observation changes in both directions. A shared observation–action transformer learns predictive latents before flow-matching policy training; deployment retains learned future slots without requiring future images. The 0.47B trainable policy combines strong benchmark performance with low measured inference cost, subject to differing baseline pretraining and native sampling configurations.
6.1 Read the original evidence
Table 1. Nominal success and perturbation robustness tell different stories. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Separate the LIBERO column from the seven perturbation columns and their rightmost Overall summary. LIBERO-Plus evaluates the unchanged original-LIBERO checkpoint; it is not an additional training suite. P.T. means additional embodied policy/world pretraining and explicitly excludes generic backbone initialization. Size reports the paper’s model counts; SLIM’s 0.47B excludes frozen T5-small. Read across the SLIM row before ranking its overall score: the robot-state column is much weaker than its lighting or sensor-noise columns. Finally, compare the rightmost values against both strong VLA baselines and VLA-JEPA. The blue emphasis is formatting, not a confidence interval or statistical test. e-liberoe-benchmark-protocole-parameters
What it supports. SLIM obtains 77.45% overall, above OpenVLA-OFT at 69.6% and Fast-WAM at 50.0%, but below VLA-JEPA at 79.5%. Its 36.90% robot-state result shows that robustness is uneven. High original-LIBERO success therefore does not imply uniformly reliable behavior under the controlled shifts.
Where the evidence stops. Appendix A.4 adds a crucial training detail: Stage 1 mixes LIBERO-90 with the target suites. Baseline pretraining differs, and the table supplies no uncertainty estimates, so these scores do not isolate architecture under equal data and compute.
Figure 6. Physical progress improves in three settings, while background shifts remain a weakness. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each setting group averages five physical tasks; the final shaded group averages the four settings. The blue bars are SLIM, light gray is pi_0.5 and dark gray is Fast-WAM. Read the vertical axis as progress score, not binary task success. Appendix A.6 assigns each trial 0, 0.5 or 1 and multiplies the mean by 100. For example, grasping the carrot earns partial credit, while putting it in the bowl earns full credit. Each task-condition pair has ten trials. All policies use the same demonstration mixture and task-condition configurations, allowing a more direct physical comparison than a visual-generation example would. e-real-protocole-real-resultse-progress
What it supports. SLIM’s aggregate progress is 67.8, compared with 56.8 and 40.0. It leads under nominal, distractor and lighting conditions, but background progress is 49 versus pi_0.5’s 54. Appendix Figure 11 further shows SLIM background scores ranging from 10 for blocks to 80 for whiteboard wiping.
Where the evidence stops. These averages combine partial and complete trials across heterogeneous tasks. They cannot recover binary completion rates, and ten trials per task-condition with no displayed error bars do not establish precise population differences.
Table 4. The compact policy reduces measured inference cost under each model’s native configuration. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each column as a different resource measurement. Latency is mean model-inference time; peak VRAM is incremental device memory, not total installed memory or parameter count; FLOPs are per action chunk. Appendix A.5 fixes one H100 80GB, batch one, BF16 and PyTorch eager. Latency pools 600 synchronized calls across three fresh processes after warmup. The native action horizons are eight for SLIM, ten for pi_0.5 and 32 for Fast-WAM, with four, ten and ten sampling steps respectively. SLIM and Fast-WAM use cached language embeddings, while pi_0.5 follows its native policy path. These details define exactly what the comparison measures. e-efficiency
What it supports. SLIM reports 60.6 ms, 4.26 GiB and 490.73 GFLOPs per chunk. The corresponding latency is 193.1 ms for pi_0.5 and 360.6 ms for Fast-WAM. This supports a lower-cost replanning call in the measured configurations, with SLIM’s pooled p95 latency reported as 61.6 ms.
Where the evidence stops. The benchmark excludes loading, simulator stepping and RPC. Different horizons, sampling counts and language paths prevent interpreting the ratios as equal-horizon cost per executed action. Incremental VRAM also excludes the established model/input/language-cache setup.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Zero-shot LIBERO-Plus robustness Original-LIBERO training; unchanged checkpoint on 10,030 perturbation cases. | 77.45% Overall success rate | OpenVLA-OFT 69.6%; Fast-WAM 50.0%; VLA-JEPA 79.5%. Strong relative to several larger baselines, but below VLA-JEPA overall. Robot-state robustness is only 36.90%; embodied pretraining differs. e-liberoe-benchmark-protocol |
| Original LIBERO manipulation Four target suites; 50 rollouts per task, 2,000 overall. | 97.5% Overall success rate | OpenVLA-OFT 97.1%; Fast-WAM 97.6%. Near the strongest baseline; these small point-estimate differences do not establish a meaningful advantage. e-liberoe-benchmark-protocol |
| CALVIN ABC to D composition ABC training; 1,000 held-out-D instruction chains. | 4.556 Average sequence length, maximum 5 | FLOWER 4.53. SLIM also reports 80.2% five-instruction success. Average length must not be read as a percentage. e-calvine-benchmark-protocol |
| Physical multi-task manipulation Five tasks, four settings, 10 trials per task-condition; shared 750-demonstration mixture. | 67.8 overall; nominal 86, distractor 63, lighting 73, background 49. Mean progress score, 0–100 | pi_0.5 averages 56.8; Fast-WAM 40.0. Background: 54 and 2 respectively. Partial milestones receive 0.5 credit. This is executed physical progress, not binary completion. e-real-protocole-real-resultse-progress |
| EMA target ablation Full recipe versus no EMA on LIBERO-Plus and CALVIN. | EMA: 77.45%; 4.556. Success rate; average sequence length | No EMA: 66.82%; 4.382. Effective rank falls from 61.28 to 13.95 without EMA despite lower MSE (0.166 versus 0.245). Prediction error alone can favor degeneration. e-ablatione-collapse |
| Controlled policy inference One H100 80GB; batch 1, BF16, PyTorch eager; native horizons/sampling schedules. | 60.6 ms; 4.26 GiB; 490.73. Mean latency; incremental peak VRAM; GFLOPs/chunk | pi_0.5: 193.1 ms/7.94 GiB/4714.59; Fast-WAM: 360.6 ms/13.63 GiB/2090.07. Per-call model measurements exclude simulator/RPC overhead and do not establish equal-horizon cost or end-to-end robot throughput. e-efficiency |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 7. Predictive training, balanced objectives and a stable target each matter in the reported ablations. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Track each color against its own vertical axis: blue squares use LIBERO-Plus success on the left, while green triangles use CALVIN average sequence length on the right. Their slopes are not commensurate effect sizes. Panel (a) asks whether adding Stage 1 helps downstream control. Panel (b) varies the relative IDM loss weight while FDM stays at one; the reported default 0.125:1 is the strongest joint setting shown. Panel (c) removes or retains the EMA target. Use the accompanying Section 4.3 text for exact EMA values, because most points in the plot have no numerical labels. Table 3 on the next page supplies complementary latent-diversity diagnostics. e-ablatione-collapsee-benchmark-protocol
What it supports. EMA improves LIBERO-Plus from 66.82% to 77.45% and CALVIN length from 4.382 to 4.556. The diagnostic table explains why lower prediction error alone is misleading: no EMA has lower MSE, 0.166 versus 0.245, but much lower effective rank, 13.95 versus 61.28.
Where the evidence stops. No uncertainty bars or seed variation are shown. Adding Stage 1 changes training exposure as well as objectives; the supplied comparison does not establish equal total compute or data exposure. A controlled repetition is needed to isolate the mechanism.
7. Analysis & limitations
7.1 What the evidence leaves open
One compact scale is tested; scaling with capacity, pretraining volume and embodiment diversity remains unestablished. e-limitations
Main success tables and ablation plots give no uncertainty estimates. Physical conditions have ten trials per task, and background shifts expose substantial task variation. Small margins and aggregate progress require caution. e-liberoe-calvine-ablatione-real-protocole-real-results
Layer-15 action-to-patch attention, max-aggregated over action tokens, qualitatively compares two selected sequences. Highlighted regions are not established as causal explanations of successful control. e-attention
7.2 Questions for discussion
- Does Stage 1 still help after matching trajectory exposure and optimizer steps to direct policy training? (e-ablation; e-benchmark-protocol)
- Does the latency advantage persist at equal executed horizon and comparable policy quality? (e-efficiency)
8. Reproducibility audit
8.1 Requirements and known gaps
Preserve dataset mixtures, global batches 128/256 for LIBERO/CALVIN, two 224×224 views and 1st/99th-percentile action normalization. AdamW uses betas 0.9/0.95, epsilon 1e-8, decay 0.01, clipping 1.0 and no accumulation. Action/vision rates are 1e-4/1e-5, remaining-parameter base rate 2.5e-5, and cosine minimum 1e-6. e-implementatione-benchmark-protocol
Exact integrator, chunk execution schedule, software versions, random seeds and real-robot hardware model/control frequency are unspecified in the supplied implementation and physical-evaluation descriptions. These details and the documented milestone scoring are needed for an exact deployment reproduction. e-implementatione-efficiencye-real-protocole-progress
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Match training exposure before attributing robustness to predictive learning
Reader-proposed check, not a performed experiment: compare the full two-stage recipe with direct flow-policy training that receives the same total optimizer steps and the same LIBERO-90/target-suite trajectory exposure. Add an otherwise matched no-EMA arm to examine target stability. Keep backbone, initialization, normalization, batch and evaluation cases fixed, and repeat across multiple seeds. Report original LIBERO, all seven Plus perturbation groups and aggregate success with uncertainty. Track effective rank and token cosine alongside each arm’s latent error. If the Stage-1 advantage disappears under matched exposure, extra training/data is a viable explanation; persistent control gains together with preserved diversity would strengthen the representation-learning account. e-ablatione-collapsee-benchmark-protocole-implementation
Check 2: Separate native replanning speed from equal-horizon control efficiency
Reader-proposed check, not a performed experiment: first repeat the native H100/BF16/eager benchmark with the documented warmups, synchronized calls and separate memory measurement. Then train or configure comparable policies for a common action horizon, using a shared language-caching convention and sampling-step sweeps. Report latency, incremental and total memory, and closed-loop success at every setting; changing horizon or sampling may change quality. Include a second timing measurement that adds the deployment observation/communication loop. If SLIM’s advantage survives at a comparable success level and common execution horizon, the efficiency claim becomes stronger. If it narrows substantially, native chunk length, sampling or caching explains part of the reported gap. e-efficiencye-implementatione-libero
8.3 Reading coverage
Visual audit: The title/author/version block, Figures 1–11 and Tables 1–6 were visually inspected on the declared original PDF pages. The pass includes the method equations and attention masks, all reported results, parameter accounting, optimizer/data settings, inference hardware/protocol, and physical scoring definitions used in this edition. All six final original crops were separately viewed at their final dimensions. Figure 2 arrows, mask orientation and stop-gradient path were checked against Equations (4)–(8); Figure 3 was checked against Section 3.2, including objective-specific action inputs and future slots omitted from that schematic. Pages 12–14, containing references and the brief Appendix A.1 introduction, were read in the complete text chunks but are not declared as visually inspected. Separate supplements and linked code were not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 15, 16, 17, 18. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract (p. 1)
- 1 Introduction (p. 2)
- 2 Related Work (p. 3)
- 3 Method, including 3.1–3.3 (pp. 3–6)
- 4 Experiments, including 4.1–4.4 (pp. 6–10)
- 5 Conclusion (p. 11)
- References (pp. 11–14)
- Appendix A.1–A.6 (pp. 14–18)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- This extraction limitation was addressed by inspecting the original PDF pages and final crops. Pages 12–14, containing references and the brief Appendix A.1 introduction, were read as text but not visually inspected.
- Separate supplemental material availability has not been fully verified.
- Identity/version: the inspected title page establishes arXiv:2608.09771v1, 10 August 2026. Title and all ten authors match the catalog; no revision/edition difference was observed. No other version was supplied or compared.
- All five supplied text chunks were read. Code, project links and separate supplements were not inspected; experiments were not reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author/affiliation block and arXiv margin stamp
Title and all ten authors match the catalog. The artifact identifies arXiv:2608.09771v1, 10 Aug 2026. Jingkai Wang and Zihan Tang contributed equally; four institutions are listed.
Go to primary source ↓e-problemPDF p. 2, Section 1, motivation and contributions
The authors motivate compact representations of action-induced changes without decoding pixels or retaining a large language-centric backbone for every control step.
Go to primary source ↓e-inputsPDF pp. 3–4, Section 3.1, Eqs. (1)–(3)
Language y is encoded as ell; observations o_t become latent tokens Z_t, q_t denotes proprioception, and A_t spans H continuous actions. Online and EMA encoders define future latents and targets.
Go to primary source ↓e-backbonePDF p. 4, Figure 3 and Section 3.2; p. 5, opening two paragraphs
Observation/action streams use separate queries, keys and values with shared joint attention, then per-stream language cross-attention and feed-forward layers. Hidden width is 768; state is one token. Future positions hold clean latents or learned embeddings according to the objective.
Go to primary source ↓e-maskedPDF p. 4, Figure 2(a)–(c); p. 5, Section 3.3, Eqs. (4)–(8)
IDM predicts flow velocity of noised actions from current/future latents. FDM predicts masked future latents using clean actions and a detached EMA target with normalized L1 loss. Dark mask cells allow attention, light cells block it. IDM observation queries cannot read noisy-action keys; FDM future slots can read clean actions.
Go to primary source ↓e-policyPDF pp. 5–6, Stage-2 flow-matching policy training, Eqs. (9)–(10)
Stage 2 uses only action flow matching with current observations, learned future slots, proprioception and language. Future latents are neither supplied nor explicitly supervised; actions attend to observation-side hidden states.
Go to primary source ↓e-liberoPDF p. 7, Table 1, SLIM/OpenVLA-OFT/Fast-WAM/VLA-JEPA rows and P.T. definition; p. 15, Table 5
SLIM reports 97.5% original LIBERO and 77.45% zero-shot LIBERO-Plus. Plus overall is 69.6% for OpenVLA-OFT, 50.0% for Fast-WAM and 79.5% for VLA-JEPA. SLIM robot-state robustness is 36.90%. P.T. excludes generic backbone initialization. Table 5 lists SLIM Long/Spatial/Object/Goal as 94.4/99.4/99.4/96.8%.
Go to primary source ↓e-calvinPDF p. 8, Table 2, SLIM/FLOWER rows; p. 6, Section 4.1 benchmark description
On CALVIN ABC to held-out D, SLIM has average sequence length 4.556 and five-instruction success 80.2%; FLOWER has 4.53 and 77.8%. Chains contain up to five language instructions.
Go to primary source ↓e-real-protocolPDF p. 8, Section 4.2, Data and training and Evaluation protocol; Figure 5
Five tasks each contribute 150 demonstrations, mixed into a 750-demonstration dataset used by each policy. Each task-condition pair has 10 trials, under nominal, background, lighting and distractor conditions.
Go to primary source ↓e-real-resultsPDF p. 9, Figure 6, all groups; p. 18, Figure 11, task-wise panels
SLIM nominal/distractor/lighting/background progress is 86/63/73/49, averaging 67.8; pi_0.5 averages 56.8 and Fast-WAM 40.0. Background averages are 49/54/2. SLIM background progress is 10 for blocks and 80 for wiping the whiteboard.
Go to primary source ↓e-progressPDF p. 17, Progress-score definitions and Figures 9–10
Trial scores are 0, 0.5 or 1, averaged then multiplied by 100. Partial milestones are two plates stacked, carrot/toast/eraser grasped, or the first block stacking action. Full credit requires completing the task. Perturbation configurations are shared across policies.
Go to primary source ↓e-ablationPDF pp. 8–9, Section 4.3 and Figure 7(a)–(c)
Stage 1 improves both benchmarks; IDM:FDM 0.125:1 is the strongest joint setting shown. EMA raises LIBERO-Plus from 66.82% to 77.45% and CALVIN length from 4.382 to 4.556. Figure 7 has separate benchmark axes and no uncertainty bars.
Go to primary source ↓e-collapsePDF p. 9, paragraph beginning Table 3; p. 10, Table 3, both rows
Across the last 20 Stage-1 probes, EMA/no-EMA effective rank is 61.28/13.95, token cosine 0.071/0.352, and future-latent MSE 0.245/0.166. Top-1 energy is 0.097/0.395 and top-5 is 0.362/0.674; the authors interpret low no-EMA error as degeneration.
Go to primary source ↓e-attentionPDF p. 10, Figure 8 and action-grounding analysis; p. 15, Appendix A.2
Two representative diagnostic sequences compare pretrained and direct policies. Maps use layer-15 action-to-observation attention, max aggregation over action tokens and projection onto patches. The paper calls these qualitative diagnostics, not causal attributions.
Go to primary source ↓e-efficiencyPDF p. 10, latency paragraph; p. 11, Table 4; p. 16, Appendix A.5
On one H100 80GB, BF16/PyTorch eager, SLIM/pi_0.5/Fast-WAM latency is 60.6/193.1/360.6 ms, incremental VRAM 4.26/7.94/13.63 GiB and GFLOPs/chunk 490.73/4714.59/2090.07. Horizons are 8/10/32 and sampling steps 4/10/10. Three processes each use 20 warmups and 200 synchronized calls. SLIM p95 is 61.6 ms. Loading, simulator stepping and RPC are excluded; language caching differs.
Go to primary source ↓e-limitationsPDF p. 11, Section 5, final paragraph
A single compact model scale is evaluated; effects of capacity, pretraining scale and embodiment diversity remain unestablished.
Go to primary source ↓e-parametersPDF p. 15, Appendix A.3; p. 16, Table 6
The 472.14M trainable policy comprises DINOv2-B/14 at 86.58M, a 16-layer MoT trunk at 377.96M, and interfaces/heads at 7.60M. Frozen T5-small is excluded; the 86.58M EMA target has no gradients and is removed for Stage 2.
Go to primary source ↓e-implementationPDF pp. 15–16, Appendix A.4, Inputs and model configuration and Optimization
Two 224×224 workspace/wrist views feed a fine-tuned DINOv2 encoder; T5-small is frozen/cached. Actions use dataset 1st/99th-percentile normalization. Simulation actions have seven dimensions, horizons 8/12 for LIBERO/CALVIN, and four flow steps. Training uses eight H100 80GB GPUs, BF16, AdamW, three Stage-1 epochs, loss weights 0.125 and 1, EMA 0.999, then 40 Stage-2 epochs (20 real-world). Optimizer rates, clipping and warmups are specified.
Go to primary source ↓e-benchmark-protocolPDF p. 16, Appendix A.4, Benchmark-specific settings
LIBERO Stage 1 mixes LIBERO-90 and four target suites; Stage 2 uses only the target suites, with global batch 128. Evaluation uses 50 rollouts per task, 2,000 overall, then 10,030 Plus cases without adaptation. CALVIN uses language-annotated ABC, batch 256, and 1,000 five-instruction chains in D.
Go to primary source ↓8.5 Primary sources
SLIM-0.5B: Learning Action-Grounded Predictive Latents for Robot Manipulation ↗
PDF · 8,368 extracted words
Source fingerprint
fb875bbdadce5867f41853945170ee1ee5013731afc0a2ec4d672390fde56a73