LDA-1B: Scaling Latent Dynamics Action Model via Universal Embodied Data Ingestion
1. Paper overview
In one sentence: LDA-1B turns mixed-quality embodied recordings into policy and latent-dynamics supervision through one task-conditioned generator, gaining data efficiency while leaving deployment details and some causal attributions unresolved. e02e03e04e07e09e10e11e13
| At a glance | What to know |
|---|---|
| Research problem | Source description Behavior cloning mainly rewards expert action imitation, leaving suboptimal trajectories and actionless videos underused. LDA treats those recordings as evidence about transitions and visual futures, while reserving policy supervision for reliable demonstrations. Its central question is whether differentiated objectives make heterogeneous data useful at scale. e02e06 |
| Core mechanism | |
| A key reported result | Mixed-quality fine-tuning on pen placement and lid removal: LDA: pen 70→80; lid 50→60. Reported success (%). Pen: 63 high versus 63 high + 37 low trajectories; lid: 66 high versus 66 high + 34 low. π0.5: pen 60→40; lid 50→40. Both LDA gains are 10 percentage points. Dedicated trial counts, scoring definitions and uncertainty are absent. e13 |
| Reading caution | Source description Fixed DINO features and predominantly egocentric views may limit transfer to other perspectives and sensory modalities. e16 |
Core contributions
Figure 2. One generator changes prediction tasks while preserving cross-modal interaction. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the four small panels at lower left. Their conditioning variables distinguish policy, forward dynamics, inverse dynamics and visual planning: o denotes the current observation, o′ the future, a the action chunk and ℓ the instruction. Follow the middle arrows upward from modality-specific projections to predicted actions and future DINO features. A register substitutes for a modality omitted by the selected task. In the enlarged block, the two streams meet in shared self-attention while keeping separate projections and feed-forward networks. Task/time embeddings enter AdaLN; VLM tokens enter cross-attention. This routing is supported by the caption and Section III-D, beyond the simplified conditioning overview. e02e03e04
What it supports. The architecture provides direct evidence that action and world prediction interact within one MM-DiT. Separate experts preserve modality-specific processing without requiring separate world-model and policy generators. Task selection changes available information and active losses, so the diagram should not be read as requiring both outputs on every policy invocation.
Where the evidence stops. The bottom visual input is labeled Current DINO + noise, although Sections III-C–D describe noised future features and the output says DINO Future. This discrepancy remains in the faithful crop. The guide follows the text; the figure also shows VLM cross-attention despite the prose's broad AdaLN-conditioning statement.
2. Motivation
2.1 The problem and the proposed response
Behavior cloning mainly rewards expert action imitation, leaving suboptimal trajectories and actionless videos underused. LDA treats those recordings as evidence about transitions and visual futures, while reserving policy supervision for reliable demonstrations. Its central question is whether differentiated objectives make heterogeneous data useful at scale. e02e06
2.2 What this reading follows
An imperfect demonstration can still reveal what an action does to the world. LDA-1B builds on that distinction by assigning different learning tasks to expert trajectories, noisier interactions and videos without actions. A multimodal diffusion transformer exchanges information between action tokens and DINO visual features, with task embeddings selecting what to predict. Read the architecture first, then separate the simulation ablations from physical-robot comparisons and the mixed-quality experiment. The results are promising, but their interpretation depends on details hidden behind the phrase success rate: some tasks award partial credit, and the main real-world baselines receive less fine-tuning data. e02e03e04e07e09e10e11e13
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Pending verification |
| Quadrant | Pending verification |
3.1 Evidence-based assessment
Insufficient evidence to decide
WAM and latent prediction are supported. Figure 2 shows one joint action/future generator: modality experts exchange tokens inside MM-DiT. Its separate VLM conditioning encoder could justify a broader two-stage description, so the recorded Dual-system label depends on the catalog's system boundary and remains ambiguous. Prediction supports direct policy, forward/inverse dynamics and forecasting, rather than exclusively inverse dynamics or obligatorily simultaneous action–future generation. Pending catalog fields are preserved. e02e04
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Separate action optimality from transition information
Start with a demonstration that pauses or retries. Copying every action can teach an inefficient policy, yet the observation sequence may still record a useful response to an action. This is the rationale behind LDA's quality-dependent supervision. The source sends high-quality trajectories to all objectives, lower-quality trajectories to dynamics and forecasting, and actionless video to forecasting. Learned task embeddings select the objective; modality registers handle missing inputs. As reader analysis, the important distinction is between an action being undesirable and its consequences being observable. The approach cannot automatically make corrupted action labels physically accurate, and its preprocessing still removes unreliable hand observations. Successful reproduction therefore needs the quality criteria and task-routing details as well as the nominal number of hours. e02e06e17
5.2 Locate the shared model, then distinguish training from control
The visual prediction target is a spatial DINO representation, while Qwen3-VL supplies observation and instruction conditioning. Within MM-DiT, action and visual tokens communicate through shared self-attention but retain modality-specific processing. Appendix A adds a short observation/action history and specifies length-16 action chunks; Section III-C samples actions more frequently than visual targets. Training updates the denoiser around frozen encoders, with the VLM unfrozen during fine-tuning. At inference, task selection changes which conditional distribution is invoked. Policy mode does not require a future observation as input. My interpretation is that dynamics supervision can influence action prediction through learned shared computation without proving an explicit online planning loop. The supplied paper does not specify rollout scoring, search, denoising steps or the chunk-execution feedback schedule. e02e03e04e05e17
5.3 Read performance as a sequence of increasingly specific tests
First use the simulation table to ask which design changes accompany improvements: DINO representation, MM-DiT and capacity each have reported comparisons, with the representation change producing the largest gap. Next inspect the physical-task bars using the appendix protocols, because object-completion and partial-credit scores answer different questions from binary success. Then examine Table IV horizontally: adding the same lower-quality portion helps LDA and hurts π0.5 within the respective systems. Finally, use the scaling plot to ask whether the trend persists in offline action error as heterogeneous data grows. As reader analysis, this sequence builds evidence for useful co-training but does not prove that the learned representation contains causal physical rules. Unequal fine-tuning sets, changed data composition, missing uncertainty and qualitative attention diagnostics leave alternative explanations open. e07e08e09e10e11e13e14e15
5.4 Training and inference
During training
Equation (1) uses flow matching, with action/observation losses activated by task. The DDPM description belongs to the UWM preliminary. Pretraining freezes Qwen3-VL-4B-Instruct and DINOv3-ViT-s, updating MM-DiT and action encoders/decoders; fine-tuning unfreezes the VLM. e02e05
Reported pretraining uses 400k iterations, 48 H800 GPUs and 4,608 GPU-hours. Table V specifies 16 layers, width 1536, 32 heads, AdamW at 1e-4 with cosine decay to 5e-7, and batches 32×48 for pretraining and 12×8 for fine-tuning. e05
During inference
Select a task embedding and its conditioning inputs. Policy mode predicts actions from observation/language without requiring a generated future; inverse dynamics additionally conditions on future observations. Forward dynamics predicts future features conditioned on actions, while forecasting omits actions. e02
The paper demonstrates robot execution but does not specify rollout scoring, search or model-predictive control at deployment. Joint dynamics training therefore does not establish that each executed action is selected through imagined futures; denoising steps and feedback cadence remain unspecified. e02e09e17
5.5 Implementation flow
- Align heterogeneous trajectories
EI-30k converts robot and human data to LeRobot 2.1, aligns end-effector frames and separates camera motion from hand motion. Actions encode delta wrist pose with gripper width or wrist-relative finger configuration. The model samples actions at 10 Hz and visual states at 3 Hz. e03e06
- Route supervision by quality
High-quality demonstrations support all four tasks; lower-quality trajectories train dynamics and forecasting; actionless videos train forecasting. Four learned task embeddings choose the objective. Action and visual register tokens stand in for a modality absent from that task, rather than supplying fabricated future observations. e02
- Exchange information between modality experts
Qwen3-VL supplies observation/language conditioning; DINOv3 supplies spatial latent targets. Modality-specific projections and FFNs surround shared action–visual self-attention. Figure 2 shows task/time conditioning through AdaLN and VLM cross-attention. Its label Current DINO + noise conflicts with the text's future-feature target; this account follows Sections III-C–D. e03e04e05
6. Experiments & results
LDA-1B learns policies and latent dynamics from heterogeneous embodied data by selecting supervision according to data quality. A shared multimodal diffusion transformer predicts actions or DINO future features under four task conditions. Simulation and physical manipulation results favor the system, but unequal fine-tuning data, partial-credit metrics and incomplete deployment details limit causal and reproducibility claims.
6.1 Read the original evidence
Table II. Simulation results separate data, representation, backbone and capacity comparisons. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the rightmost column as average success across 24 RoboCasa-GR1 tasks, each fine-tuned with 1,000 demonstrations and evaluated with 51 trials. The top three rows establish policy baselines; GR00T-EI10k is the reproduced baseline pretrained on the high-quality subset. Then compare the VAE-based UWM rows with the DINO-based LDA rows, checking the MM-DiT column before attributing a difference to representation. The last three rows probe the standard DiT backbone, smaller model and full model. Section V-A describes matched data and optimization for these ablations; that statement does not make every pretraining pipeline in the entire table identical. e07e08
What it supports. LDA-1B reports 55.4%, versus 51.3% for GR00T-EI10k, a 4.1 percentage-point difference. The full model exceeds LDA(DiT) by 6.5 points and LDA-0.5B by 4.7 points. The VAE-to-DINO comparison, 20.0 to 55.4, is the largest reported ablation gap and motivates examining representation choice before adding capacity.
Where the evidence stops. The small UWM baseline is 14.2 here but 14.3 in Appendix Table VI; both values remain unreconciled. The appendix also shows task-level exceptions to LDA's average advantage. No uncertainty estimates accompany this table.
Figure 6. Gripper performance improves across tasks whose score definitions differ. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the legend to compare GR00T-N1.6, π0.5 and LDA within each task, rather than treating the four categories as a single benchmark score. The source reports 100 collected trajectories per task, with main baselines trained on the expert subset and LDA on the mixed-quality set. Read Appendix Table VII alongside the bars: handover is binary placement success, wiping uses a 0–5 completeness score, and sweeping and rubbish removal measure proportions of objects transferred. Most tasks have ten evaluation trials with a 200-second timeout; hammer grasp-and-knock has 60. Even Water Flower checks a pouring posture, not a measured volume of water delivered. e09e10
What it supports. Handover reaches 90% for LDA, compared with 70% for π0.5 and 50% for GR00T. Rubbish removal reaches 35 versus zero for both baselines. The latter is evidence of increased object completion in a multistage task; it does not establish that 35% of trials completed the entire procedure.
Where the evidence stops. The plot's common Success Rate axis hides heterogeneous scoring. Different fine-tuning data and small evaluation sets also limit architectural attribution. These physical-robot outcomes should remain distinct from generated-feature quality and from binary episode-level success.
Figure 7. Dexterous gains include partial-credit tool-use and flipping scores. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The dashed divider separates the first three Unitree/BrainCo tasks from the two Galbot/Sharpa tasks. Compare methods within each group using the bottom legend. Appendix Table VIII provides the essential scoring key: bottle placement and MacBook opening use 20 trials, while nail extraction, bread placement and bread flipping use ten. Pull Nail gives credit for locating the nail, single-claw removal and full removal. Flip Bread assigns full credit to a first-attempt flip and half credit to a second-attempt flip. Those rules explain why a tall bar can represent better progress or fewer retries without directly stating the fraction of perfectly completed episodes. e09e11
What it supports. LDA's Pull Nail score is 80, versus 0 for π0.5 and 40 for GR00T; Flip Bread is 90 versus 10 for both baselines. These large task-specific gaps support improved dexterous performance under the reported adaptation protocol. They do not independently isolate the contribution of human-data pretraining from architecture or fine-tuning data.
Where the evidence stops. The source caption says LDA consistently outperforms, but the Open MacBook bars show a tie with π0.5 at 100. Preserve that exception. Partial-credit scores and unequal main fine-tuning sets further prevent a simple interpretation as complete-success counts or an isolated pretraining effect.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| RoboCasa-GR1 average manipulation success 24 simulated tasks; 1,000 fine-tuning demonstrations and 51 evaluation trials per task. | 55.4 Average task success (%) | GR00T-EI10k 51.3; GR00T-N1.6 47.6; StarVLA 47.8. A 4.1 percentage-point gain over the stronger reproduced GR00T baseline. Pretraining data/objectives differ; Table VI does not show universal task-wise superiority. e07 |
| RoboCasa representation and architecture ablations Section V-A describes matched training data and optimization for ablations. | LDA-1B 55.4; LDA(DiT) 48.9; LDA-0.5B 50.7. Average task success (%) | VAE-based UWM(MM-DiT) 20.0; UWM-1B 19.3. DINO substitution accompanies the largest reported gain; removing MM-DiT costs 6.5 percentage points. This does not independently prove causal-physics reasoning. e08 |
| Galbot gripper handover and rubbish removal Ten trials per task, 200-second timeout; LDA uses mixed-quality fine-tuning data, main baselines expert-only data. | Handover 90.0; Clean the rubbish 35.0. Handover binary success; rubbish paper-ball completion (%) | π0.5: 70.0 and 0.0; GR00T-N1.6: 50.0 and 0.0. The 35.0 value measures the proportion of balls dumped, not the percentage of entirely successful episodes. e09e10 |
| Dexterous nail extraction and bread flipping Unitree/BrainCo Pull Nail and Galbot/Sharpa Flip Bread; ten trials each; mixed-quality versus expert-only fine-tuning. | Pull Nail 80; Flip Bread 90. Task-specific partial-credit score, displayed as success (%) | π0.5: 0 and 10; GR00T-N1.6: 40 and 10. Localization/removal and first/second-attempt flipping earn different credits. These bars cannot be converted to counts of complete successes. e09e11 |
| Mixed-quality fine-tuning on pen placement and lid removal Pen: 63 high versus 63 high + 37 low trajectories; lid: 66 high versus 66 high + 34 low. | LDA: pen 70→80; lid 50→60. Reported success (%) | π0.5: pen 60→40; lid 50→40. Both LDA gains are 10 percentage points. Dedicated trial counts, scoring definitions and uncertainty are absent. e13 |
| Pick-and-place visual and spatial generalization Novel objects, changed backgrounds and OOD starting positions, separately from the main in-domain tasks. | 60.0 / 60.0 / 40.0 Reported success (%) | GR00T 40/40/20; π0.5 26.7/20/6.7. These controlled perturbations favor LDA; trial counts and uncertainty are not supplied. e12 |
| Held-out Agibot World action prediction scaling Full co-training at 30k training hours on an unseen held-out subset. | 6.6×10^-3, using Figure 10's axis multiplier. Action-prediction L1 error; lower is better | Full co-training ends below the policy-plus-forecasting curve; policy-only worsens when lower-quality data are added. An action-prediction proxy, not measured execution success. Data quantity and composition change together. e14 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table IV. Adding lower-quality trajectories helps LDA on both reported adaptation tasks. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each task's pair of columns horizontally within a method. For pen placement, 63 high-quality demonstrations become a 100-trajectory set after adding 37 lower-quality demonstrations. For lid removal, the corresponding split is 66 plus 34. The numbers in parentheses are changes from the high-quality-only condition; interpret them as percentage points. Then compare the direction of change between methods. LDA gains in both tasks, while π0.5 declines. This is a more focused test of the ingestion argument than comparing unrelated pretrained policies on a single data setting, although it still compares complete systems with different learning objectives. e02e13e17
What it supports. LDA improves by ten percentage points on both tasks, from 70 to 80 and from 50 to 60. π0.5 drops by twenty and ten points. The evidence supports the usefulness of these particular additional trajectories for LDA; it does not establish that arbitrary noisy data will help or that expert-data requirements fall by a measured percentage.
Where the evidence stops. The paper supplies neither dedicated trial counts and success rubrics for these two tasks nor uncertainty or a fixed-update data-budget control. Tables VII–VIII describe other tasks and cannot silently fill those gaps.
Figure 10. Scaling improves a held-out action-prediction proxy as data composition changes. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the upper panel's legend. Gray squares represent policy only; brown triangles add visual forecasting; green diamonds add forward and inverse dynamics; blue circles show full co-training. Follow the annotated progression from a high-quality subset to more high-quality data, then lower-quality interactions, then actionless video. The vertical axis is action-prediction L1 error on held-out Agibot World data, with a ×10⁻³ multiplier. In the lower panel, color selects UWM or LDA and circle size selects 0.1B, 0.5B or 1B capacity. Circle size encodes parameters, not uncertainty. Read corresponding capacities and data settings together. e14e17
What it supports. The full co-training curve reaches 6.6×10⁻³ at the 30k-hour endpoint; the caption's 6.6 must retain the axis multiplier. Policy-only performance worsens after lower-quality data are added, while the full model improves. The figure supports an interaction between supervision choice and available data, rather than a claim that more imitation data always helps.
Where the evidence stops. This is an offline action-error proxy, not robot success or a fitted scaling law. Hours and data composition change together. The supplied description omits the held-out subset size, detailed split construction and error normalization, limiting exact replication and leakage assessment.
7. Analysis & limitations
7.1 What the evidence leaves open
Fixed DINO features and predominantly egocentric views may limit transfer to other perspectives and sensory modalities. e16
Main real-robot comparisons combine architecture, pretraining and unequal fine-tuning data. Small trial sets, heterogeneous scoring and absent uncertainty restrict significance claims. PCA features, attention differences and gradient alignment are diagnostics, not proof of causal physical understanding. e09e10e11e13e15
Internal discrepancies remain: Figure 1 says 1.6B, Table I lists 1B trainable parameters excluding frozen modules, and Appendix A names a 4B VLM. EI-30k category summaries differ from Table IX row totals. UWM averages are 14.2 in Table II and 14.3 in Table VI. e01e06e07
7.2 Questions for discussion
- Does matched-budget quality routing improve execution beyond simply increasing usable training data?
- How well does held-out action L1 error predict task-specific physical scores?
- Would dynamics prediction remain useful after larger viewpoint changes or additional sensing?
8. Reproducibility audit
8.1 Requirements and known gaps
A faithful attempt needs the specified pretrained encoders, EI-30k quality labels/transforms, Table V configuration, RoboCasa splits and task-specific robot scoring. Exact alignment offsets, quality cutoffs, mixture sampling, fine-tuning duration and deployment solver/cadence remain unresolved. e05e06e07e10e11e17
Reader-proposed priorities are a fixed-budget noisy-data routing comparison and a matched DINO/VAE × shared/separate-backbone ablation, measuring both action error and execution. These would test whether the proposed mechanisms survive tighter controls. e02e08e13e14
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Hold the update budget fixed while changing noisy-data routing
Reader-proposed, not run: reproduce the pen-placement split of 63 high-quality and 37 lower-quality trajectories. From one LDA checkpoint, compare expert-only fine-tuning with resampling, the mixed set routed by quality, and the same mixed set allowed to train the policy objective. Hold optimizer updates, high-quality example exposure, augmentation and evaluation conditions fixed; report seeds, per-trial binary completion and confidence intervals under a declared success rubric. The routing hypothesis predicts that adding lower-quality dynamics supervision improves completion without the degradation caused by imitating its actions. If the gain disappears after update-budget matching, increased training exposure is an alternative explanation for Table IV. e02e05e13e17
Check 2: Factor representation choice from cross-modal parameter sharing
Reader-proposed, not run: compare DINO and VAE targets crossed with either shared MM-DiT attention or separated action/visual attention, holding the VLM, data mixture, task probabilities, trainable parameter budget and optimization schedule as closely matched as possible. Use identical RoboCasa-GR1 demonstrations and evaluation trials, and report executed success alongside action-prediction L1 error on a fixed documented held-out set. Declare unavoidable token-count or compute differences. A persistent DINO advantage would support the representation explanation; a gain from shared attention within each representation would support cross-modal transfer. Improved offline error without improved execution would weaken its use as the main scaling proxy. e04e07e08e14e17
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version page, Figures 1–17 and Tables I–IX on the declared PDF pages, plus method, training and appendix text supporting retained claims. All six final original crops were individually viewed with axes, legends and table headers retained. Figure 2 was checked against Sections III-B–D and Equation (1); its Current DINO input-label discrepancy is disclosed. Appendix Tables VII–VIII supply score semantics for the physical-task charts, and Figure 10's axis multiplier is preserved. Pages 10–12 contain references and were read as text. No separate supplements, code or external datasets were inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 13, 14, 15, 16, 17, 18, 19, 20. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and I. Introduction (pp. 1–2)
- II. Related Work (pp. 2–3)
- III. Latent Dynamics Action Model, A–E (pp. 3–4)
- IV. Embodied Interaction Dataset (pp. 4–5)
- V. Experiments, A–D (pp. 5–9)
- VI. Conclusion, Limitations, and Future Directions; Acknowledgments (p. 9)
- References (pp. 10–12)
- Appendix A. Details of Model (p. 13)
- Appendix B. Detailed Results on the Simulation Benchmark (pp. 13–15)
- Appendix C. Details Regarding Real-world Experiment (pp. 13–17)
- Appendix D. Details of EI-30k (pp. 17–19)
- Appendix E. Details of Other Experiments (pp. 19–20)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction's image omission was addressed by inspecting PDF pages 1–9 and 13–20, including all numbered figures and tables. References on pages 10–12 were read as text.
- Version scope: the inspected artifact is arXiv:2602.12215v2, 3 June 2026. Title and all authors match the catalog. The catalog submission date is 12 February 2026; that earlier edition was not supplied, so scientific differences between editions cannot be established.
- No external code, datasets, project pages or separate supplements were inspected, and no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, author/affiliation block, arXiv margin and Figure 1; p. 2, Table I; p. 13, Appendix A
The catalog title and all 23 authors appear in arXiv:2602.12215v2, dated 3 June 2026. Affiliations are Peking University, Galbot, CASIA, BAAI, Tsinghua University, Sun Yat-sen University and NVIDIA. Figure 1 says 1.6B parameters, Table I lists 1B trainable parameters excluding frozen components, and Appendix A specifies Qwen3-VL-4B-Instruct. These counts are not fully reconciled.
Go to primary source ↓e02PDF p. 3, Sections III-A–B and Figure 2 task panels; p. 4, Section III-B, Equation (1) and following paragraph
One model supports policy, forward dynamics, inverse dynamics and visual forecasting. High-quality demonstrations support all objectives, lower-quality trajectories support dynamics/forecasting, and actionless videos support forecasting. Four task embeddings and two modality registers select inputs/losses. Equation (1) uses flow matching; the preceding UWM preliminary describes DDPM.
Go to primary source ↓e03PDF p. 4, Section III-C; p. 13, Appendix A and Table V
Visual targets are DINO features. Actions use delta wrist poses and gripper width or wrist-relative finger keypoints. Visual/action sampling is 3/10 Hz. Appendix A adds two history timesteps of DINO observations and actions; Table V specifies 224×224 images, 14×14×384 latents and action chunks of length 16.
Go to primary source ↓e04PDF p. 3, Figure 2 and caption; p. 4, Section III-D
MM-DiT retains modality-specific QKV projections, FFNs and output heads while sharing self-attention over concatenated action/visual tokens. Figure 2 routes task/time embeddings to AdaLN and VLM tokens to cross-attention. Its visual input says Current DINO + noise, whereas the text describes noised future features and the output says DINO Future.
Go to primary source ↓e05PDF p. 4, Section III-E; p. 13, Appendix A and Table V
Pretraining freezes the VLM and DINO, updating MM-DiT and action encoding/decoding; fine-tuning unfreezes the VLM. Reported training uses 400k iterations, 48 NVIDIA H800 GPUs and 4,608 GPU-hours. Appendix A specifies Qwen3-VL-4B-Instruct, DINOv3-ViT-s, width 1536, 16 layers, 32 heads, AdamW at 1e-4, weight decay 1e-5, betas [0.9,0.95], epsilon 1e-8, cosine scheduling to 5e-7, and batches 32×48 and 12×8.
Go to primary source ↓e06PDF pp. 4–5, Section IV and Figures 3–4; pp. 17–19, Appendix D, especially p. 18 Table IX
EI-30k combines real/simulated robots and human recordings with/without actions. LeRobot 2.1 standardization resamples to 10 Hz; dataset-specific rigid offsets align end-effectors, camera extrinsics separate camera motion, MANO keypoints are wrist-relative, and unreliable hand frames are removed. Quality labels preserve usable suboptimal trajectories. Section IV category totals differ from corresponding Table IX row sums; both describe more than 30k hours.
Go to primary source ↓e07PDF p. 5, Section V-A and Table II; p. 13, Appendix B-A; p. 15, Table VI
RoboCasa-GR1 uses 24 tasks, 1,000 fine-tuning demonstrations per task and 51 evaluation trials per task. Table II reports LDA-1B 55.4%, GR00T-EI10k 51.3%, GR00T-N1.6 47.6% and StarVLA 47.8%. GR00T-EI10k uses a high-quality pretraining subset. Table VI confirms these averages but LDA does not lead on every task; the UWM average is 14.3 rather than Table II's 14.2.
Go to primary source ↓e08PDF p. 5, Table II, representation/backbone/size rows; p. 6, Section V-A ablation continuation
Success is 19.3% for UWM-1B, 20.0% for UWM(MM-DiT) with VAE features, 48.9% for LDA(DiT), 50.7% for LDA-0.5B and 55.4% for LDA-1B. The authors describe identical training data and optimization for these ablations.
Go to primary source ↓e09PDF pp. 6–7, Section V-B setup and fine-tuning protocol; pp. 13–14, Appendix C-A–B; p. 17, Figure 14
Physical experiments use Galbot G1 with grippers or 22-DoF Sharpa hands, and Unitree G1 with 10-DoF BrainCo hands, with head-camera RGB as visual input. The paper describes 100 teleoperated trajectories per task, approximately 50–80% expert. Main baselines use filtered expert data, while LDA uses all trajectories. Galbot G1 is excluded from pretraining. Appendix C states a 200-second timeout and task-specific scoring.
Go to primary source ↓e10PDF p. 6, Figure 6; p. 16, Table VII
Figure 6 gives LDA/π0.5/GR00T values of 90/70/50 for handover, 65/53/35 for sweeping and 35/0/0 for rubbish removal. Table VII defines sweep/rubbish scores as proportions of nails/paper balls collected or dumped; wiping is scored 0–5. Water Flower requires a pouring posture with the spout above the pot. Hammer grasp-and-knock uses 60 trials; the other gripper tasks use ten, with a 200-second timeout.
Go to primary source ↓e11PDF p. 7, Figure 7; p. 16, Table VIII; p. 19, Figure 16
LDA/π0.5/GR00T scores are 80/0/40 for Pull Nail and 90/10/10 for Flip Bread. Both use ten trials: nail extraction credits 0.25 for localization, 0.5 for single-claw removal and 1.0 for full claw removal; bread flipping credits 1.0 on the first attempt and 0.5 on the second. Bottle and MacBook tasks use 20 trials each. The first three tasks use Unitree/BrainCo; the last two use Galbot/SharpaWave.
Go to primary source ↓e12PDF p. 7, Table III and Generalization Ability paragraph; p. 8, Figure 8
For pick-and-place perturbations, LDA reports 60.0% on new objects, 60.0% on backgrounds and 40.0% on OOD positions; GR00T reports 40/40/20 and π0.5 reports 26.7/20/6.7. Figure 8 illustrates the perturbations. Trial counts and uncertainty for these separate conditions are not specified.
Go to primary source ↓e13PDF p. 7, Table IV and Data-Efficient Fine-tuning paragraph
Pen placement compares 63 high-quality trajectories with 63 high plus 37 low: LDA rises 70→80, while π0.5 falls 60→40. Lid removal compares 66 high with 66 high plus 34 low: LDA rises 50→60, while π0.5 falls 50→40. Values are reported success percentages. Dedicated trial counts, success rubrics and uncertainty for these two tasks are not supplied.
Go to primary source ↓e14PDF p. 7, Section V-C evaluation setup; p. 8, Figure 10, both panels and caption; pp. 8–9, Section V-C
Scaling uses action-prediction L1 error on an unseen held-out Agibot World subset. Figure 10's axes specify ×10^-3; the full model reaches the caption's 6.6 at 30k hours, meaning 6.6×10^-3 in plotted units. Curves compare policy-only, policy plus forecasting, policy plus forward/inverse dynamics and full co-training; circle sizes distinguish 0.1B, 0.5B and 1B. Data scale also changes composition.
Go to primary source ↓e15PDF p. 8, Figure 9; p. 9, Section V-D and Figures 11–12; p. 19, Figure 17 and Appendix E-A; p. 20, Appendix E-B
PCA visualizations compare predicted and observed DINO features. Attention diagnostics use the absolute difference between active-action and No-Op maps. Gradient similarities are shown at 10k and 400k iterations. The authors interpret these qualitative/optimization diagnostics as evidence of dynamics understanding; no intervention-based causal-physics test is reported.
Go to primary source ↓e16PDF p. 9, Section VI
The authors identify fixed DINO features and predominantly egocentric viewpoints as limitations for perspective and multimodal generalization, proposing joint representation/dynamics learning, richer sensing and automatic data-role selection.
Go to primary source ↓e17PDF pp. 3–4, Sections III-B–E; p. 13, Appendix A/Table V; pp. 17–18, Appendix D-A
The supplied specification includes model dimensions, optimizer settings and an alignment/cleaning outline but omits the denoising solver/step count, number of actions executed before feedback, complete task/data sampling probabilities, fine-tuning iteration count, exact quality cutoffs and dataset-specific alignment transforms.
Go to primary source ↓8.5 Primary sources
LDA-1B: Scaling Latent Dynamics Action Model via Universal Embodied Data Ingestion ↗
PDF · 11,835 extracted words
Source fingerprint
7e3619baa7129f509e21c842b4bc5aca08808bbb30b85ca9b240a779ebf2264b