AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems
1. Paper overview
In one sentence: AgiBot World couples standardized robot demonstrations with latent-action pretraining, improving selected manipulation scores while leaving the effects of data scale, quality and embodiment alignment partly entangled. e-probleme-datasete-pipelinee-lame-plannere-actione-protocole-policy
| At a glance | What to know |
|---|---|
| Research problem | Source description Robot demonstrations differ in hardware, viewpoints and quality, making scale alone an unreliable route to general manipulation. The paper seeks richer bimanual, long-horizon experience through standardized acquisition and a policy that can also learn from videos without robot action labels. e-probleme-lam |
| Core mechanism | Source description The reported release contains 1,001,552 trajectories over 2976.4 hours: 217 tasks, 87 skills and 106 scenes across domestic, retail, industrial, restaurant and office domains. More than 100 homogeneous robots collect data involving over 3,000 objects. e-dataset |
| A key reported result | Five-task real-world policy comparison: GO-1 0.78 Mean normalized task-completion score, with fractional partial credit. AgiBot World beta pretraining; usually task-specific fine-tuning. Figure 5 averages 30 trials per task: 10 seen and 20 varied/distracted. RDT-1B 0.46; π0 0.58; GO-1 without latent planner 0.66. The planner gain is 0.12 score units; the RDT difference is 0.32 score units. Neither is automatically a relative percentage improvement or binary success-rate gain. These comparisons retain the author-printed averages. For π0, the five displayed task scores (0.83, 0.60, 0.41, 0.47, 0.47) have a diagnostic equal-task mean of 0.556, approximately 0.56, rather than the printed 0.58. Section V-C states thirty trials per task but does not explain a different aggregation; the discrepancy remains unresolved. e-protocole-finetunee-policy |
| Reading caution | Reader analysis All evaluation is physical; the authors describe simulation as under development. Five policy-comparison tasks and three transfer tasks cover only a small subset of 217 dataset tasks, so broad dexterity and generalization claims remain bounded by those protocols. e-probleme-datasete-protocole-transfer |
Core contributions
- Source description
The reported release contains 1,001,552 trajectories over 2976.4 hours: 217 tasks, 87 skills and 106 scenes across domestic, retail, industrial, restaurant and office domains. More than 100 homogeneous robots collect data involving over 3,000 objects. e-dataset
- Source description
The platform supplies task/substep annotations, calibration and recovery labels, while GO-1 combines latent-action pretraining with an inference-time planner and diffusion controller. These are distinct dataset and model contributions. e-tablee-pipelinee-lame-plannere-action
Figure 2. Quality control changes future collection as well as filtering recorded episodes. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin on the edge side: teleoperator training leads to data collection, local validity verification and upload. Follow the lower arrow into cloud processing and the quality-check diamond. A failed quality check leads to failure-reason categorization and discard; a successful task reaches the failure-recovery decision. Its Yes branch leads to Annotation, while its No branch joins the delivery path. Finally, trace the dashed blue feedback arrow from model training back toward collection. Section III-B gives a concrete example of that loop: deployment pauses exposed excessive idle time, leading to revised collection instructions and removal of idle frames. e-pipeline
What it supports. Recovery is not treated as the same thing as an unusable episode. The paper retains recoverable mistakes with reasons and timestamps, approximately 1% of the dataset, while rejecting episodes that fail its standards. This supplies supervision about deviations and recovery, although the paper does not separately measure the value of those labels.
Where the evidence stops. The No branch bypasses the Annotation box, but Section III-B describes episode language annotation generally. The drawing therefore leaves ordinary-episode annotation routing unclear. Exact acceptance rules and idle-frame thresholds are also absent; the flowchart is not a complete preprocessing specification.
2. Motivation
2.1 The problem and the proposed response
Robot demonstrations differ in hardware, viewpoints and quality, making scale alone an unreliable route to general manipulation. The paper seeks richer bimanual, long-horizon experience through standardized acquisition and a policy that can also learn from videos without robot action labels. e-probleme-lam
2.2 What this reading follows
A robot dataset can grow without becoming easier to learn from. AgiBot World addresses that problem by collecting demonstrations with a common robot platform, checking them locally and centrally, and feeding policy failures back into collection rules. GO-1 adds a second bridge: it learns discrete changes from video, predicts those changes from current observations and language, and conditions a diffusion controller on them. Read the following visuals as three separate evidence threads: what the collection contains, how GO-1 uses latent actions, and what the physical evaluations measure. The reported improvements are normalized completion-score differences on selected tasks, with partial credit and limited uncertainty reporting. e-probleme-datasete-pipelinee-lame-plannere-actione-protocole-policy
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Datasets |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The dataset/platform contribution supports the recorded Datasets category and non-applicable architecture/quadrant fields. Cross-robot collection here uses homogeneous G1 units; cross-embodiment videos inform GO-1 pretraining, not demonstrated cross-embodiment coverage of the collected dataset. GO-1 itself has separate LAM, VLM, planner and action-expert roles; joint training does not establish a single joint future/action world model. e-datasete-tablee-lame-plannere-action
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Collection policy becomes part of the learning system
Start with a teleoperator demonstrating a task, not with a neural-network loss. AgiBot World first checks whether the task and its collection standard are feasible. Local checks remove incomplete recordings; cloud review judges adherence to the standard and supplies language annotations. The distinction between an unrecoverable failure and an episode containing a recoverable mistake is consequential: the latter can retain failure reasons and timestamps. Training then exposes weaknesses in the demonstrations themselves. The paper’s idle-pause example links observed robot behavior back to data collection and preprocessing. Reader interpretation: this makes the effective training distribution a product of the feedback loop, not merely the number of recorded trajectories. The Wipe Table comparison motivates that interpretation, but its unequal data sizes prevent a clean estimate of quality alone. e-pipelinee-quality
Figure 4. Video-derived latent targets become an intermediate representation for robot control. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the upper row as representation training. The LAM encoder receives the current and future frames, and the decoder uses the current frame and discrete tokens to reconstruct the future. Then move to the lower row: InternViT and the tokenizer provide multiview-image and instruction inputs to the VLM; the planner learns the latent targets; the action expert generates continuous controls. The paper fixes the latent-token count at four. The image-to-latent-to-action stripe summarizes the hierarchy, while Section IV-C explicitly states the deployment order: predict latent tokens first, then condition denoising on them. Future images are training supervision, not deployment observations. e-lame-plannere-action
What it supports. The architecture gives action-free human video a training role without treating latent tokens as motor commands. The action expert is still required to produce continuous robot controls. During inference the planner remains part of GO-1, unlike a representation objective discarded after pretraining; the no-planner comparison tests the complete resulting variant.
Where the evidence stops. Figure 4 labels controls through a_{H-1}; Section IV-C writes a_t through a_{t+H} with H=30. Exact chunk length is unresolved. Section IV’s introductory conditioning sentence also reads in reverse; the explicit IV-C inference description supports latent prediction before action denoising.
5.2 Learn a target from the future, then predict it from the present
The LAM can use a future frame during training because its job is to encode an observed transition. Its inverse encoder compresses the current/future pair into discrete tokens, and its forward decoder checks those tokens through future-image reconstruction conditioned on the current frame. GO-1’s latent planner faces a different information constraint: it receives current head and wrist images plus language, and learns to predict targets obtained from head-view frame pairs. Those predicted tokens then condition an action expert that uses proprioception and iterative denoising to generate controls. This separation explains how videos without motor labels can participate in representation learning while robot control still requires an action-generating module. Section IV-C is the clearest account of deployment order; the future image and reconstruction branch are not stated to run as an inference-time search procedure. e-lame-plannere-action
5.3 Keep the unit of evidence fixed when reading gains
Figure 5 asks about policy variants after AgiBot beta pretraining and usually task-specific fine-tuning. Its 0.78 average summarizes five tasks with partial credit; comparing it with 0.66 isolates the reported no-planner variant, subject to unspecified parameter and compute matching. Figure 6 instead changes the pretraining source for RDT and evaluates three tasks in separate seen and unseen settings. Figure 7 changes either GO-1’s data scale on seen tasks or RDT’s Wipe Table filtering. Reader interpretation: these experiments form complementary evidence, but their averages cannot be pooled into a single generalization score. A convincing next step would retain their individual scoring protocols, disclose split identities and uncertainty, and introduce controls that separate latent information from extra model capacity and data quality from sample count. e-protocole-finetunee-policye-transfere-scalinge-quality
5.4 Training and inference
During training
Stage 1 learns the LAM from heterogeneous video data, including Ego4D. Stage 2 trains latent prediction using LAM pseudo-labels. Stage 3 jointly trains the planner and diffusion action expert. Masked language modeling predicts discrete tokens; diffusion models continuous actions. e-lame-plannere-action
Unless otherwise specified, pretrained GO-1 is fine-tuned on task-specific demonstrations: learning rate 2×10⁻⁵, batch size 768 and 30,000 steps. Exact mixture weights, pretraining budgets and frozen-module choices are absent. e-finetunee-lame-planner
During inference
Current multiview images and the instruction produce latent tokens; these condition action denoising with proprioception. The future frame supplies training supervision, not an available deployment input. The paper does not specify denoising iterations or how many controls execute before observations are refreshed. e-plannere-action
5.5 Implementation flow
- Collect compatible demonstrations
G1 combines dual 7-DoF arms, mobile chassis, adjustable waist and modular grippers or 6-DoF hands. Eight cameras and proprioception record at 30 Hz. VR supports end-effector control with predefined hand gestures; motion capture enables individual finger and body motion. e-hardware
- Make quality a feedback loop
Validate task feasibility and collection standards, check local completeness, then upload for quality review and language annotation. Recoverable mistakes retain failure reasons and timestamps; these comprise about 1% of data. Policy deployment exposed idle pauses, prompting protocol revisions and idle-frame removal. e-pipeline
- Compress observed change
The LAM inverse encoder sees current and future frames. Its spatial-temporal transformer produces four VQ-VAE-quantized tokens; a spatial forward decoder reconstructs the future frame from those tokens and the current frame. This trains a motion representation without requiring physical action labels. e-lam
- Predict a latent plan
InternVL2.5-2B and InternViT encode language and multiview observations. A 24-layer planner receives layer-wise VLM conditioning with bidirectional attention. Head-view frame pairs provide latent targets, although prediction uses head and both wrist views. e-planner
- Decode executable controls
The action expert receives hierarchical conditioning and proprioception, then iteratively denoises continuous controls. The latent planner and action expert have different objectives despite sharing an architectural framework. The described deployment predicts latent tokens and controls, with no video rollout search specified. e-action
6. Experiments & results
AgiBot World Colosseo combines a million-trajectory manipulation dataset, standardized collection and the GO-1 policy. GO-1 learns discrete motion representations from videos, predicts those representations from images and language, and uses them to condition continuous robot controls. Real-world experiments support dataset-transfer and policy gains on selected tasks; they do not establish general competence across the entire collection.
The paper’s only table compares dataset properties; policy performance and ablations are reported as plots in Figures 5–7. This edition therefore includes Table I as the original quantitative table and preserves the policy results in their original chart form. No separate tactile-sensor or recovery-label ablation is supplied. e-tablee-policye-transfere-quality
6.1 Read the original evidence
Table I. The collection’s distinguishing feature is its combination of scale, hardware and annotation. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across the bold AgiBot World row before comparing individual columns. It reports more than one million trajectories, 87 skills and 106 scenes; the later columns describe capabilities and collection choices rather than measured policy performance. The dual-arm entry and checks for detailed annotation, calibration, dexterous hands, recovery and human review explain what kinds of supervision the dataset aims to supply. Compare the gray OXE aggregation row and the DROID scene count separately. These counts describe different collection designs, and neither trajectory count nor scene count is a common measure of downstream difficulty or generalization. e-tablee-dataset
What it supports. The table supports a large, consistently instrumented bimanual collection with several annotation and recovery features. It does not show AgiBot World leading every size column: OXE is listed at 1.4M trajectories, while DROID has 564 scenes versus AgiBot World’s 106. The practical contribution is the combination of properties.
Where the evidence stops. The caption’s unqualified largest-trajectory claim conflicts with the OXE row. Checks indicate reported dataset features, not controlled evidence of their benefit. The paper’s fleet is homogeneous, so many robot units do not establish broad embodiment diversity.
Figure 5(b). The latent planner improves the reported average, with gains that vary by task. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the legend to keep four policies separate: pink RDT-1B, yellow π0, turquoise GO-1 without the latent planner and blue full GO-1. Each task combines thirty physical trials, split into ten seen and twenty varied or distracted trials. The bars report normalized completion scores, including fractional partial credit. First compare turquoise and blue to examine the planner variant; then compare the average group at the right. Pour Water rises from 0.42 without the planner to 0.67 with it, whereas Fold Shorts rises from 0.60 to 0.66. These differences show why the average alone cannot describe every manipulation behavior. e-protocole-finetunee-policy
What it supports. Full GO-1 averages 0.78, versus 0.66 without the planner, 0.58 for π0 and 0.46 for RDT-1B. Thus the reported planner gain is 0.12 score units and the RDT gap is 0.32. The evidence concerns physical execution on the five plotted tasks under the stated evaluation mixtures.
Where the evidence stops. The chart has no error bars and the paper does not give detailed partial-credit rules or a parameter-matched planner control. The five quantitative task groups also differ from the six setup tasks listed in Section V-A.1. Do not interpret 0.78 as a binary success fraction. The printed π0 Average is 0.58, while its five labeled task scores average to 0.556 (approximately 0.56) under equal task weighting. The paper states thirty trials per task without explaining another aggregation. Retain the original bars and aggregate; their discrepancy is unresolved.
Figure 6. Pretraining source changes RDT’s performance in both seen and varied scenes. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. This comparison fixes the named policy family to RDT and changes its pretraining data. Read the upper panel for in-distribution evaluation and the lower panel for out-of-distribution evaluation. Both show Restock Bag, Table Bussing and Wipe Table, followed by their average. Yellow denotes OXE, turquoise the smaller alpha dataset and blue beta. The shared 0–1 scales help compare the two settings, but the panels remain separate protocols. Inspect task-level reversals before interpreting average improvement: for seen Table Bussing, alpha scores 0.65 and beta 0.60, even though beta has the higher overall mean in both panels. e-protocole-transfere-finetunee-revision
What it supports. Beta’s average is 0.77 in distribution and 0.67 out of distribution, compared with OXE’s 0.47 and 0.38. These are absolute score gains of 0.30 and 0.29. Alpha also exceeds OXE on both averages, supporting the usefulness of this collection beyond a simple more-hours explanation.
Where the evidence stops. Embodiment alignment, task composition, quality and data volume change together. The source also describes alpha as roughly 14% of beta but lists 92,214 trajectories, around 10%, in its appendix. Exact manifests are needed to reproduce the pretraining comparison.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Five-task real-world policy comparison AgiBot World beta pretraining; usually task-specific fine-tuning. Figure 5 averages 30 trials per task: 10 seen and 20 varied/distracted. | GO-1 0.78 Mean normalized task-completion score, with fractional partial credit | RDT-1B 0.46; π0 0.58; GO-1 without latent planner 0.66. The planner gain is 0.12 score units; the RDT difference is 0.32 score units. Neither is automatically a relative percentage improvement or binary success-rate gain. These comparisons retain the author-printed averages. For π0, the five displayed task scores (0.83, 0.60, 0.41, 0.47, 0.47) have a diagnostic equal-task mean of 0.556, approximately 0.56, rather than the printed 0.58. Section V-C states thirty trials per task but does not explain a different aggregation; the discrepancy remains unresolved. e-protocole-finetunee-policy |
| RDT dataset transfer on Restock Bag, Table Bussing and Wipe Table OXE versus AgiBot alpha/beta pretraining; seen setups and two unseen scenarios per task, ten rollouts per scenario and method. | Beta 0.77 / 0.67; alpha 0.68 / 0.56 Average normalized completion score, in distribution / out of distribution | OXE 0.47 / 0.38; beta gains 0.30 / 0.29 score units. This tests pretraining source using RDT. It does not isolate dataset size from embodiment match, content or quality; beta also trails alpha on seen Table Bussing (0.60 versus 0.65). e-protocole-transfer |
| GO-1 data scaling on four seen tasks Out-of-the-box evaluation after pretraining on 10% alpha, full alpha or beta; about 9.2k–1M trajectories. | r = 0.97 Reported Pearson correlation for the plotted scaling relationship | Three dataset-size settings; author-fitted power-law curve. A positive trend over these settings supports scaling locally; three points, changing dataset composition and absent uncertainty do not establish a universal law or unseen-task scaling. e-scaling |
| Wipe Table data-quality ablation RDT fine-tuning; 528 verified and 482 unverified trajectories described in Section V-E. | Verified only 0.59 Normalized completion score | All 0.41, a 0.18 difference. The plot compares All with Verified only, not explicitly unverified-only with verified-only. Filtering changes both composition and size, so this is not a matched-size quality control. e-quality |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 7(a–b). Data volume and filtering provide different, limited tests of the scaling argument. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Treat these panels as separate experiments. On the left, the horizontal axis is training-trajectory count and the vertical axis is labeled Performance. The three markers correspond to ten percent of alpha, full alpha and beta, evaluated using out-of-the-box GO-1 on four seen tasks. The legend reports Pearson r=0.97. On the right, the axis is Completion Score and the model is RDT fine-tuned for Wipe Table. Preserve the labels All and Verified only: Section V-E describes 528 verified and 482 unverified trajectories, but the plotted baseline is All, not explicitly unverified-only. Do not merge the two panels’ scales or protocols. e-scalinge-quality
What it supports. The left panel supports increasing measured performance across the three supplied dataset sizes. The right reports a 0.18 completion-score advantage for verified-only data, from 0.41 to 0.59. Together they motivate investigating both volume and filtering, but neither result alone identifies a general scaling law or a size-independent quality effect.
Where the evidence stops. Three size settings cannot establish reliable extrapolation, and their dataset composition changes. The filtering comparison changes sample count as well as content. Its pooled baseline is suggested by All, but the prose leaves that construction insufficiently explicit; matched-size controls and uncertainty are missing.
7. Analysis & limitations
7.1 What the evidence leaves open
All evaluation is physical; the authors describe simulation as under development. Five policy-comparison tasks and three transfer tasks cover only a small subset of 217 dataset tasks, so broad dexterity and generalization claims remain bounded by those protocols. e-probleme-datasete-protocole-transfer
Preserve source inconsistencies: alpha is roughly 14% of beta in implementation details but 92,214 trajectories, around 10%, in the appendix. Table I’s largest-dataset caption conflicts with its 1.4M OXE row. Figure 4 and action-chunk text disagree on endpoint indexing. e-finetunee-revisione-tablee-action
Reported scores lack error bars, seed variation and detailed partial-credit rubrics. The quality ablation is not size-matched. Dataset coverage and selected physical demonstrations do not quantify tactile sensing, failure recovery or cross-embodiment transfer benefits individually. e-protocole-qualitye-tablee-hardware
7.2 Questions for discussion
- Would matched-size verified and mixed-quality data retain the Wipe Table advantage?
- Does a parameter- and compute-matched planner control preserve GO-1’s gain when predicted latent tokens are shuffled?
- How much of the OXE comparison reflects embodiment alignment rather than transferable manipulation knowledge?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the exact alpha/beta manifests, task demonstrations, G1 observation/action interfaces, LAM targets, InternVL2.5-2B and matched real-world setups. The paper states CC BY-NC-SA 4.0 for the dataset; its release statements alone do not verify accessible artifacts. e-datasete-hardwaree-lame-plannere-protocole-revision
Missing details include numeric codebook size, full loss weights, pretraining sampling/schedules, compute and software configuration, frozen modules, denoising schedule, action encoding, idle-frame thresholds and train/evaluation instance separation. The appendix supplies credits and release history rather than these implementation details. e-lame-actione-finetunee-pipelinee-protocole-revision
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does predicted latent information drive the Pour Water gain?
Reader-proposed, not performed: train full GO-1 and a parameter- and compute-matched control on the same beta manifest and task demonstrations. Keep VLM inputs, action expert, proprioception, optimization and denoising budget fixed; replace planner conditioning in the control with a learned constant of the same shape. At evaluation, also shuffle predicted latent-token sequences across full-model episodes. Use identical Pour Water seen and position-variation setups, thirty trials per condition as in the paper, repeated across independent seeds. Report the original completion score with a disclosed rubric, binary completion separately, and confidence intervals. If intact predicted tokens help while constant and shuffled conditioning do not, the result supports useful latent information. If all conditions perform similarly, extra capacity or training effects remain a stronger explanation than the claimed planning mechanism. e-plannere-actione-protocole-policye-finetune
Check 2: Does verification help after matching the training-set size?
Reader-proposed, not performed: resolve the Figure 7 All manifest first, using the source’s 528 verified and 482 unverified Wipe Table episodes. Construct verified-only, unverified-only and balanced mixed subsets with the same total count, no greater than 482, and multiple random selections. Hold RDT initialization, frame sampling, optimizer steps, augmentations and evaluation scenes fixed. Retain a separate full-pool condition to compare with the published All bar. Evaluate all conditions on the same untouched rollouts with explicit fractional scoring, and report variation across training seeds and subsets. A persistent verified-only advantage over size-matched controls supports a quality effect. An advantage confined to one subset or absent after matching would weaken that explanation and motivate checking task difficulty and selection bias. e-pipelinee-protocole-quality
8.3 Reading coverage
Visual audit: All nine PDF pages were rendered and visually inspected, including title/byline/version and affiliations (p. 1), motivation and stated limitation (p. 2), Table I and dataset/hardware description (p. 3), Figure 2 and collection protocol (p. 4), Figure 3 and LAM formulation (p. 5), Figure 4 and planner/action/evaluation definitions (p. 6), all result charts and training/scaling details (p. 7), quality-ablation text and conclusion/references (p. 8), and contributor credits/release history (p. 9). Every final crop was separately viewed at its native rendered resolution. Figure 2’s annotation bypass and Figure 4’s chunk endpoint discrepancy were cross-checked against their text and disclosed. The appendix contains no additional experiment/configuration appendix. Separate supplements and linked artifacts were outside the supplied material.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and I. Introduction, including stated limitation (pp. 1–2)
- II. Related Work (pp. 2–3)
- III. AgiBot World: Platform and Data, A–C (pp. 3–5)
- IV. AgiBot World: Model, A–C (pp. 5–6)
- V. Experiment and Analysis, A–E (pp. 6–8)
- VI. Conclusion and References (p. 8)
- Appendix: Acknowledgement, Contributions and Change Log (p. 9)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Identity and edition: the exact title and identifier match. The inspected artifact is arXiv:2503.06669v4, dated 4 August 2025. Its appendix records a July 2025 IROS camera-ready update adding π0 comparisons; earlier editions were not supplied or compared. See e-identity and e-revision.
- The PDF byline is Team AgiBot-World, rather than the catalog’s expanded AgiBot-World-Contributors/individual-name string. Appendix contributors substantially overlap the catalog, but Ziyu Xiong appears in post-training credits and several catalog names appear in acknowledgements. The PDF does not provide an equivalent expanded formal author list; metadata preserves the verified collective byline.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This limitation was addressed by visually inspecting all nine PDF pages and the six final crops.
- Separate supplemental material availability has not been fully verified.
- Only the supplied PDF was reviewed. Linked code, checkpoints, dataset files, project pages and external papers were not inspected; access, installation and experiments were not reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, byline, affiliation footnote and arXiv margin; p. 9, Contributions and Change Log
The exact catalog title appears with collective byline Team AgiBot-World and arXiv:2503.06669v4, 4 Aug 2025. The footnote names The University of Hong Kong, AgiBot Inc., Shanghai Innovation Institute and Shanghai AI Lab. The appendix lists contributor roles and records a July 2025 IROS camera-ready update adding π0 comparisons.
Go to primary source ↓e-problemPDF pp. 1–2, Abstract and Section I, motivation, contributions and Limitation
The platform addresses fragmented robot data, limited task diversity and inconsistent collection quality. It combines standardized collection, a dataset and GO-1. All evaluations are real-world; a corresponding simulation environment is under development.
Go to primary source ↓e-datasetPDF p. 3, Section III opening; pp. 4–5, Section III-C and Figure 3
The latest dataset contains 1,001,552 trajectories, 2976.4 hours, 217 tasks, 87 skills and 106 scenes, collected with more than 100 homogeneous robots. It covers five domains and over 3,000 objects in a facility exceeding 4,000 square meters. Figure 3 describes long tasks and at least 100 trajectories per skill. Section III states a CC BY-NC-SA 4.0 dataset license and links release plans with code and checkpoints.
Go to primary source ↓e-tablePDF p. 3, Table I, column headers, AgiBot World, OXE, DROID and RoboMIND rows; table caption
AgiBot World is listed as 1M+ trajectories, 87 skills and 106 scenes, with dual arms, detailed annotation, calibration, dexterous hands, recovery data and human-in-the-loop collection. OXE is listed at 1.4M trajectories and DROID at 564 scenes. The caption nevertheless calls AgiBot World the largest by trajectory count; an unqualified largest-overall claim is not supported by its own OXE row.
Go to primary source ↓e-hardwarePDF pp. 3–4, Section III-A, Hardware
AgiBot G1 has dual 7-DoF arms, mobile chassis, adjustable waist and modular grippers or 6-DoF dexterous hands. The eight-camera setup includes front, wrist and rear views; observations and proprioceptive states are recorded at 30 Hz. VR uses inverse kinematics and predefined hand gestures; motion capture maps human joints, including fingers, to robot posture.
Go to primary source ↓e-pipelinePDF p. 4, Figure 2 and Section III-B, Data Collection, Failure recovery and Human-in-the-loop
Task feasibility and collection standards precede local validity checks, upload, cloud quality review and annotation. Figure 2 routes successful episodes through a recovery decision: its Yes branch leads to Annotation and its No branch bypasses that box toward delivery, while Section III-B generally describes episode language annotations. Failed quality checks lead to reason categorization and discard. Recoverable mistakes are retained with reasons and timestamps, approximately 1% of the dataset. Deployment feedback led to removing idle frames; the exact idle-frame threshold is not specified.
Go to primary source ↓e-lamPDF p. 5, Section IV overview and IV-A; p. 6, Figure 4, Stage 1 and caption
Stage 1 trains an inverse-dynamics encoder on current/future images and a forward decoder that reconstructs the future image from the current image and latent tokens. Four tokens are quantized using a VQ-VAE objective; codebook size is denoted |C| without a numerical setting. Figure 4 names Ego4D human videos among heterogeneous pretraining sources.
Go to primary source ↓e-plannerPDF pp. 5–6, Sections IV overview and IV-B; p. 6, Figure 4, Stage 2
Stage 2 uses LAM-derived pseudo-labels for latent-action prediction. InternVL2.5-2B supplies the VLM backbone; InternViT encodes multiview images. A 24-layer latent planner receives layer-wise VLM conditioning with full bidirectional attention. Head-view frame pairs supply targets while head and both wrist images plus the instruction condition predictions.
Go to primary source ↓e-actionPDF p. 6, Section IV-C and Figure 4, Stage 3; p. 5, Section IV overview
Stage 3 jointly trains the planner and a diffusion action expert. The planner uses masked language modeling; the expert iteratively denoises continuous controls using hierarchical conditioning and proprioception. Inference predicts latent tokens before denoising controls. The text writes A_t=[a_t,...,a_{t+H}] with H=30, whereas Figure 4 displays a_0,...,a_{H-1}. The Section IV introductory conditioning sentence is awkwardly reversed relative to the explicit inference description.
Go to primary source ↓e-protocolPDF p. 6, Section V-A.1, Evaluation Tasks and Scoring rubrics; p. 7, Section V-C and Figure 5
Each rollout receives 1.0 for complete success and fractional partial credit. There are ten rollouts per task, scenario and method. Figure 5 compares five task scores, each over thirty trials: ten seen and twenty under variations or distractions. Section V-A enumerates six tasks and mentions Handover Bottle as a capability example, but Figure 5 quantitative bars cover Restock Bag, Table Bussing, Pour Water, Restock Beverage and Fold Shorts. Per-task fractional scoring rules and uncertainty estimates are not supplied.
Go to primary source ↓e-finetunePDF p. 7, Section V-A.2, Implementation Details
GO-1 is usually fine-tuned on high-quality task-specific demonstrations after three-stage pretraining. Fine-tuning uses learning rate 2e-5, batch size 768 and 30,000 steps. Alpha is described as roughly 14% of beta here; detailed pretraining schedules, compute hardware, software versions and frozen-module settings are not provided.
Go to primary source ↓e-policyPDF p. 7, Figure 5(b), all five task groups and Average; Section V-C
Figure 5 reports average normalized scores of 0.46 for RDT-1B, 0.58 for π0, 0.66 for GO-1 without its latent planner and 0.78 for GO-1. Pour Water scores are 0.13, 0.41, 0.42 and 0.67 respectively; Fold Shorts scores are 0.42, 0.47, 0.60 and 0.66. The caption states that the policies are pretrained on AgiBot World beta. Section V-C reports a 0.12 average planner gain. The π0 task bars are 0.83, 0.60, 0.41, 0.47 and 0.47: their diagnostic equal-task mean is 0.556, which does not reproduce the printed 0.58 Average. With thirty trials per task stated in Section V-C, no alternative weighting is specified. The original aggregate is retained as reported; this unresolved discrepancy is not a replacement experimental result.
Go to primary source ↓e-transferPDF p. 7, Figure 6(a–b), three tasks and Average; Section V-B
The dataset comparison uses RDT on Restock Bag, Table Bussing and Wipe Table. OXE/alpha/beta average scores are 0.47/0.68/0.77 in distribution and 0.38/0.56/0.67 out of distribution. In-distribution Table Bussing is 0.20/0.65/0.60, so beta does not dominate alpha in every cell. Section V-B compares 236 hours of alpha to approximately 2000 hours of OXE.
Go to primary source ↓e-scalingPDF p. 7, Figure 7(a) and Section V-D
The scaling experiment evaluates out-of-the-box GO-1 on four seen pretraining tasks using 10% alpha, full alpha and beta, spanning approximately 9.2k to 1M trajectories. Three points are plotted with a power-law fit and Pearson r=0.97. Figure 7(a) labels its vertical axis Performance; it is not the 0–1 axis used for the adjacent quality comparison.
Go to primary source ↓e-qualityPDF p. 7, Figure 7(b); p. 8, Section V-E
The Wipe Table RDT ablation reports completion scores 0.41 for All and 0.59 for Verified only. Section V-E describes 528 verified and 482 unverified trajectories. The plot labels support a pooled-data versus verified-only reading, while the prose says both verified and unverified data; no matched-size control or uncertainty is reported.
Go to primary source ↓e-revisionPDF p. 9, Change Log; p. 7, Section V-A.2; p. 3, Section III
The January 2025 alpha release is listed as 92,214 trajectories, around 10% of the dataset. March records full data and the initial technical report; July records IROS camera-ready comparisons with π0. The 92,214 count and roughly 10% appendix description do not match the roughly 14% alpha description in Section V-A.2.
Go to primary source ↓8.5 Primary sources
AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems ↗
PDF · 6,807 extracted words
Source fingerprint
200a3029108f26c3408b827f4d45d16dc19d7468864fb5a46458172756c5e737