Open X-Embodiment: Robotic Learning Datasets and RT-X Models
1. Paper overview
In one sentence: Pooling robot demonstrations can transfer skills across embodiments through direct action policies, but the gain depends on model configuration and task distribution. e02e03e05e06e07e11e12e16
| At a glance | What to know |
|---|---|
| Research problem | Source description Robot datasets are individually narrow in environments, objects, tasks, and hardware. The paper asks whether their union supports positive transfer: can one policy improve a particular robot by training on other robots’ experience? Its second goal is infrastructure for that research. The experiments concern language-conditioned manipulation and executed robot actions, rather than predicting future images or learning a simulator. e02e05e08 |
| Core mechanism | |
| A key reported result | Cross-embodiment emergent skills on Google Robot: RT-2-X: 75.8%. Task success rate (%). Objects and skills occur in WidowX Bridge data but are absent from the Google Robot’s original RT-2 data; Table II rows (1)–(2), both 55B, web-pretrained and co-trained, without history. RT-2: 27.3%. The improvement is 48.5 percentage points, approximately 2.78 times the baseline (reader arithmetic). It supports skill transfer to a training embodiment, not zero-shot control of an unseen robot. e11e15e16 |
| Reading caution |
Core contributions
Figure 2. Dataset diversity depends on whether one counts datasets, scenes, trajectories, or language labels. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the top row as three different denominators. Panel (a) counts datasets per embodiment; panel (b) partitions scenes; panel (c) partitions trajectories. A robot with many contributed datasets need not dominate the number of episodes. The broken vertical axis in (a) is essential for interpreting Franka’s tall bar. The lower row switches from hardware to language-derived content: (d) counts common skills and also has a broken axis, while (e) groups objects into shapes, containers, furniture, food, appliances, and utensils. Section III-B explains that PaLM extracts these objects and behaviors from existing instructions. These panels characterize the assembled repository, rather than only the experimental training mixture. e03e04e07
What it supports. The repository offers substantial variety, but its coverage is uneven: Franka dominates dataset and scene counts, while the trajectory pie has a different balance. Picking and related manipulation verbs dominate the skill plot. This distribution motivates asking which source data transfers useful behavior, rather than treating every additional trajectory as interchangeable.
Where the evidence stops. These are annotation and collection statistics, not success rates. The 22-embodiment repository is broader than the nine-manipulator policy mixture. No balanced sampling or equal language-annotation quality should be inferred from this figure.
2. Motivation
2.1 The problem and the proposed response
Robot datasets are individually narrow in environments, objects, tasks, and hardware. The paper asks whether their union supports positive transfer: can one policy improve a particular robot by training on other robots’ experience? Its second goal is infrastructure for that research. The experiments concern language-conditioned manipulation and executed robot actions, rather than predicting future images or learning a simulator. e02e05e08
2.2 What this reading follows
Open X-Embodiment makes a practical proposal: combine existing robot datasets, standardize enough of their interfaces to train shared policies, and measure whether one robot benefits from another’s experience. The full repository and the policy-training mixture have different scopes, so this reading first separates data coverage from experimental coverage. It then follows images and instructions into RT-1-X and RT-2-X, examines physical robot success rates, and tests the interpretation against the ablations. The central finding is strong transfer on selected skills missing from the target robot’s own data. Broader visual generalization changes little, and entirely new robots remain outside the evaluation. e02e03e05e06e07e11e12e16
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Datasets |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The dataset classification is supported by consolidated robot trajectories spanning embodiments, tasks, and language instructions. The reannotation portion of the recorded subcategory is not specifically established beyond annotation-based analysis. Keeping architecture, prediction paradigm, and quadrant “Not applicable” is appropriate for the dataset record: RT-X baselines predict actions directly, with no joint future/action world-model mechanism. Their shared policy training does not establish a One Model WAM classification. e03e04e05e06
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
5. Method in detail
5.1 Standardize the training interface before claiming shared behavior
The repository first solves a data-handling problem. RLDS lets heterogeneous episodes share a storage format while preserving their original modalities and action records. Training RT-X requires an additional, narrower decision: select one camera, resize its images, and convert actions to an end-effector representation. Dataset-specific normalization makes discretization practical, but it does not align robot coordinate frames or command semantics. Thus, “the same action” in token space can mean different physical motion after decoding. Reader interpretation: shared training must exploit observation and instruction context while tolerating this remaining heterogeneity. The result should be understood as empirical evidence that coarse alignment can work in the tested manipulation settings. It is not evidence that the repository has already solved arbitrary embodiment incompatibility. e03e05e06e16
Figure 3. RT-X shares an action-prediction interface while retaining robot-specific command meanings. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow each pathway from left to right. In the upper branch, image history and the instruction enter the FiLM-conditioned EfficientNet and Transformer used by RT-1-X. In the lower branch, an image enters ViT while the instruction enters the language-model pathway; action text is then de-tokenized for RT-2-X. Both paths end in discrete actions. The panels at the right show why this shared output format is only a coarse alignment: illustrated controllers differ in rate, gripper interpretation, and whether movement means velocity or position delta. Section IV-A confirms that coordinate frames and these original control conventions are retained, despite per-dataset normalization. e05e06e07
What it supports. The shared policy predicts a command from visual and language context, and that command must be decoded using the target robot’s convention. Cross-embodiment training therefore succeeds despite substantial interface heterogeneity. RT-2-X additionally brings a vision-language pretrained backbone into the same direct-control problem.
Where the evidence stops. The graphic does not establish a universal physical meaning for action tokens or a future-state model. Section IV-B specifies eight dimensions, including termination, but its illustrative action string contains seven integers; complete serialization requires clarification.
5.2 Separate the source of learned knowledge from the runtime control loop
RT-1-X and RT-2-X have similar input/output roles but different sources of representation capacity. RT-1 combines pretrained visual and language features inside a robotics-oriented Transformer policy. RT-2-X starts from a large vision-language model and represents actions as text, allowing robotics training to share the language-token objective. The training distinction matters: RT-1-X uses the robotics mixture, whereas the main RT-2-X recipe also retains original VLM data in an approximately equal split. At deployment, both pathways still terminate in robot commands, interpreted according to the current platform. RT-1 runs locally and RT-2 is queried remotely at the robot’s required rate. Reader interpretation: internet pretraining and multi-robot co-training explain where useful representations may come from; neither changes this paper into a predictive world-model planner. e05e06e07
5.3 Use the two evaluation columns to narrow the transfer claim
The strongest result appears when the test asks Google Robot to perform skills found in Bridge’s WidowX demonstrations but absent from its own original dataset. RT-2-X improves this emergent-skills score from RT-2’s 27.3% to 75.8%. Removing Bridge lowers it to 42.8%, making the source of additional experience relevant to the explanation. However, the other Table II column asks a different question: generalization to unseen objects, backgrounds, and environments. There the same main comparison is 62% versus 61%, approximately unchanged. Reader interpretation: cross-robot data expands this robot’s tested behavioral repertoire more clearly than it improves an already strong visual-generalization baseline. The Bridge ablation strengthens that explanation, while its unmatched data removal leaves semantic content, volume, and sampling effects incompletely separated. e11e12e15
5.4 Training and inference
During training
Both policies minimize categorical cross-entropy over their output vocabulary. RT-1-X trains on robotics data; RT-2-X co-fine-tunes on approximately equal amounts of original VLM data and robotics data. The experimental mixture spans nine manipulators, not the repository’s 22 embodiments. Section V’s loose reference to the “full” dataset is interpreted using the explicit Section IV-C mixture definition. e07e08
Figure 2’s object and behavior statistics are extracted from existing language annotations using PaLM. This is an annotation-based diversity analysis; the paper does not describe a comprehensive trajectory reannotation pipeline. e04
During inference
Images and the instruction feed the action predictor; output tokens are decoded and de-normalized according to the robot’s convention. Policies run at the required 3–10 Hz, with RT-1 local and RT-2 queried from a cloud service. The described loop directly predicts control actions; it contains no future-state rollout, inverse-dynamics action extraction, or inference-time search. e05e06e07
5.5 Implementation flow
- Consolidate records without erasing embodiment differences
RLDS stores trajectories in serialized tfrecord files and accommodates heterogeneous cameras, depth, point clouds, and actions. The policy experiments use a narrower interface: one camera and seven end-effector quantities. Storage compatibility does not imply every modality is consumed by RT-X. e03e05
- Normalize and tokenize actions
Normalize actions separately for each dataset, then discretize into 256 bins along each of eight dimensions: termination plus x, y, z, roll, pitch, yaw, and gripper. Coordinate frames remain unaligned; commands can represent absolute positions, deltas, or velocities. The same vector can cause different motions on different robots. e05e06
- Encode observations and predict action tokens
RT-1 is described as a 35M-parameter model using 15 images, ImageNet-pretrained EfficientNet, USE language embeddings, FiLM conditioning, 81 vision-language tokens, and a decoder-only Transformer. RT-2-X uses the PaLI-X recipe: ViT plus UL2, pretrained primarily on WebLI, with discretized actions represented as text tokens. Table II distinguishes 55B and 5B configurations and their histories. e06e14
6. Experiments & results
Open X-Embodiment pools heterogeneous robot experience into a common dataset format and tests whether existing action policies benefit from it. The repository spans 22 embodiments, while RT-X experiments use a nine-manipulator mixture. The strongest transfer result concerns skills available in another robot’s demonstrations: RT-2-X reaches 75.8% success on the Google Robot, versus RT-2’s 27.3%. Gains depend on evaluation setting; generalization to unseen robots is untested.
6.1 Read the original evidence
Figure 4. Shared training improves all five domains relative to RT-1, but does not beat every domain-specific method. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. For each domain, read the hatched bar as the method contributed by that dataset’s authors, the gray bar as RT-1 trained on that dataset alone, and the rainbow bar as RT-1-X trained on the robotics mixture. The legend distinguishes several different Original Method architectures; they are not one common baseline. Compare gray and rainbow first because those policies share an architecture. Then compare against the specialized method. Finally inspect the mean group at the right, whose bars summarize the domains. The logos and robot drawings identify evaluation sites and embodiments; the measured quantity is robot task success, not model training accuracy. e08e09
What it supports. The displayed means are 63% for RT-1-X, 44% for RT-1, and 41% for Original Method. Cable Routing rises from 18% with RT-1 to 56% with RT-1-X. Yet Autolab UR5 reaches 45%, below its original method’s 53%, demonstrating that positive transfer against one baseline does not guarantee beating a specialized policy.
Where the evidence stops. The caption’s “50% higher” is an approximate relative summary, not a percentage-point difference or an exact ratio against both mean bars. No error bars or per-domain trial counts are given here; the mean is not identified as pooled trial success.
Table I. Large-data domains reveal the limits of the smaller shared policy and qualify the larger model’s gains. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start above the numeric rows: the two Bridge columns use WidowX robots at different laboratories, while the last column evaluates six RT-1-paper skills on Google Robot. Read comparisons vertically within one column; the three settings are not interchangeable replicates. LCBC is the Original Method for Bridge, and the dash in the Google column denotes an absent entry, not zero success. The RT-1 row is the domain-specific training baseline, RT-1-X uses the shared mixture, and RT-2-X is the larger 55B vision-language policy. Boldface draws attention to strong results, but the actual percentages determine which model wins or ties. e06e08e10
What it supports. Relative to RT-1, RT-2-X improves Bridge at IRIS from 40% to 50%, ties Bridge at RAIL at 30%, and scores 91% versus 92% on Google Robot. RT-1-X falls below RT-1 in all three columns. The table supports a capacity-related concern for the smaller shared policy, with gains that depend on the setting.
Where the evidence stops. The caption incorrectly generalizes RT-1-X as worse than Original Method: both Bridge scores exceed LCBC’s 13%. The prose also overstates RT-2-X superiority over RT-1. Moving between model families changes architecture and pretraining as well as parameter count.
Figure 5. What transfers is tested through concrete distinctions in instructed movement and destination. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with panel (a), where colored direction words correspond to arrows indicating alternative absolute motions for the same scene. Panel (b) changes the reference frame: the instruction specifies a destination between different pairs of objects, so the target depends on their relative arrangement. In panel (c), focus on the preposition rather than the object name. Placing an apple on cloth differs from moving it near the cloth; similarly, moving an orange into a pot differs from moving it near the pot. The arrows illustrate these instruction-dependent alternatives. The caption identifies these as Bridge skills evaluated on Google Robot, linking the examples to Table II’s emergent-skills column. e11e15e16
What it supports. The transfer evaluation asks for more than recognizing a familiar object: instructions distinguish spatial relations and action destinations. Together with the 75.8% aggregate success and Bridge-removal diagnostic, these examples make the cross-embodiment claim concrete. Skills seen with WidowX are tested through Google Robot execution under a separate embodiment.
Where the evidence stops. The arrows are explanatory annotations, not measured trajectories or per-example success statistics. The target robot already contributes training data. These selected scenes and aggregate results do not establish unrestricted compositional language understanding or transfer to an unseen robot.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| In-distribution manipulation on five small-data domains Kitchen Manipulation, Cable Routing, NYU Door Opening, Autolab UR5, and Task-Agnostic Play; real-robot protocols inherited from the contributing studies. | Figure 4 labels the means RT-1-X 63%, RT-1 44%, Original Method 41%. Task success rate (%) | RT-1-X beats Original Method in four of five domains; Autolab UR5 is 45% versus 53% (RT-1: 25%). Positive transfer is broad but not universal. The caption’s “50% higher” is an approximate relative summary, not a 50-percentage-point gain or an exact ratio for both displayed means. e08e09 |
| In-distribution manipulation on large-data domains Table I columns: Bridge at IRIS/Stanford; Bridge at RAIL/UCB; six RT-1-paper skills at Google Robotic Lab. | RT-2-X (55B): 50%, 30%, 91%; RT-1-X: 27%, 27%, 73%. Task success rate (%) | RT-1: 40%, 30%, 92%; LCBC: 13%, 13%, with no Original Method value for Google Robot. RT-2-X improves over RT-1-X throughout, but versus RT-1 it wins, ties, and slightly trails across the three columns. Architecture and pretraining also change; this is not a pure parameter-count ablation. e06e10 |
| Cross-embodiment emergent skills on Google Robot Objects and skills occur in WidowX Bridge data but are absent from the Google Robot’s original RT-2 data; Table II rows (1)–(2), both 55B, web-pretrained and co-trained, without history. | RT-2-X: 75.8%. Task success rate (%) | RT-2: 27.3%. The improvement is 48.5 percentage points, approximately 2.78 times the baseline (reader arithmetic). It supports skill transfer to a training embodiment, not zero-shot control of an unseen robot. e11e15e16 |
| Generalization to unseen objects, backgrounds, and environments Google Robot evaluation following the RT-2 protocol; Table II rows (1)–(2). | RT-2-X: 61%. Task success rate (%) | RT-2: 62%. Performance is approximately maintained; the emergent-skill gain should not be generalized to this separate evaluation. e12 |
| Bridge removal diagnostic Table II rows (2)–(3): 55B RT-2-X, no history, web-pretrained and co-trained; robotics mixture with versus without Bridge. | Without Bridge: 42.8%. Emergent-skills success rate (%) | With Bridge: 75.8%. A 33.0-percentage-point decrease (reader arithmetic) supports a contribution from Bridge. Removing a dataset also changes training-data composition and volume; the table does not establish a matched-volume causal control. e11 |
| Image-history ablation Table II rows (4)–(5): 5B RT-2-X with the same listed data, web pretraining, and web co-training; two images versus no history. | Two-image history: 44.4% / 52%. Emergent-skills / generalization success rates (%) | No history: 14.5% / 30%. History helps both reported evaluations in this configuration. Trial denominators and confidence intervals are not supplied in the table. e13 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table II. Transfer, temporal context, and pretraining affect different evaluations in different ways. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the two rightmost columns separately: emergent skills test behavior from other robots’ datasets on Google Robot, whereas generalization tests unseen objects, backgrounds, and environments. Rows (1) and (2) compare single-embodiment RT-2 with shared RT-2-X at 55B. Rows (2) and (3) remove Bridge while retaining the other listed settings. Rows (4) and (5) isolate the listed history setting at 5B. For initialization, rows (6) and (7) both omit web co-training; comparing them avoids the extra co-training change in the paper’s cited rows (4)/(6) comparison. Always scan every configuration column before attributing a change to one factor. e11e12e13e14
What it supports. RT-2-X reaches 75.8% emergent-skill success versus RT-2’s 27.3%, while generalization is 61% versus 62%. Removing Bridge reduces emergent success to 42.8%. At 5B, two-image history gives 44.4% emergent success versus 14.5% without history. These patterns support transfer and temporal context without implying a uniform gain on all tasks.
Where the evidence stops. Rows (2)/(4) change both size and history. Bridge removal changes data composition and volume, with no matched-volume control documented. The source reports no per-row uncertainty, and its fine-tuning/co-fine-tuning comparison is not a statistical equivalence test.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors explicitly leave generalization to new robots and substantially different sensing/actuation modalities untested. Repository breadth therefore exceeds the scope established by policy evaluation. e03e16
Table I’s caption says RT-1-X is worse than Original Method, although both Bridge values are 27% versus LCBC’s 13%. Section V-A also overstates RT-2-X superiority over RT-1. The report preserves the table entries rather than these blanket claims. e10
Table II’s 55B-versus-5B comparison changes history as well as size. Its stated pretraining comparison, rows (4)/(6), also changes web co-training; rows (6)/(7) offer a cleaner initialization contrast. None of these tables gives per-condition trial counts or uncertainty. e13e14
Figure 2 shows strong embodiment and pick/place-family imbalance. Language-derived skill/object counts measure annotation content, not balanced coverage or independently verified competence. e04
7.2 Questions for discussion
- Does Bridge transfer survive matching data volume and task frequency?
- How much of the history benefit requires temporal order?
- Which embodiment differences prevent positive transfer?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs the exact nine-manipulator mixture, camera selection, action normalization and robot-specific decoder, model checkpoint, history configuration, and original evaluation protocols. The PDF does not supply mixture sampling weights, optimizer schedule, training steps, compute hardware, freezing policy, or detailed trial allocation; linked resources cannot fill those gaps within this reading. e05e06e07e08
The paper describes openly shared data and selected checkpoints, but gives no explicit per-dataset license inventory, unified train/test split, or leakage audit. It specifies eight action dimensions while its illustrative RT-2 action string contains seven integers, leaving complete serialization ambiguous in this summary. These details must be resolved before implementing faithful data access and decoding. e03e06
Proposed checks: repeat Bridge removal with matched training exposure, and compare two ordered images against a duplicated-current-image control at fixed model size. These would test semantic transfer and temporal information, respectively; neither check was run for this report. e11e13
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does Bridge-specific transfer survive a training-exposure control?
Reader-proposed check, not performed: fix the 55B checkpoint, no-history configuration, optimizer schedule, Google Robot exposure, and total updates. Compare the full robotics mixture with a Bridge-removed mixture that resamples remaining data to match robotics token exposure; add a matched-size non-Bridge-data control where feasible. Keep web-data exposure and evaluation tasks identical. Record per-task successes, denominators, and uncertainty for both Table II evaluations. A persistent advantage concentrated on Bridge-associated skills would support content-specific transfer. If a matched-size alternative closes the gap, the original removal result would be less specific to Bridge’s skill content. e07e11e12
Check 2: Does two-image history help because it carries temporal information?
Reader-proposed check, not performed: at fixed 5B initialization and training mixture, compare ordered two-image history with two copies of the current image, using identical token count and compute. Include a reversed-order history control and the original no-history configuration. Reuse the same tasks, robot rates, trial allocation, and success definition, and report confidence intervals. Better performance for ordered history than duplicated images would support useful temporal information; sensitivity to reversal would further implicate temporal order. If duplication matches the gain, extra visual processing or repeated observation may explain part of the reported history effect. e06e07e13
8.3 Reading coverage
Visual audit: Actually viewed the title/contributor/version page and overview (p. 1), abstract/introduction and affiliations (p. 2), dataset analysis (p. 3), architecture and action/training definitions (p. 4), physical-robot results and training/inference continuation (p. 5), ablations and transfer examples (p. 6), and the limitations continuation (p. 7). All six final crops were viewed, including the revised Table II crop. Figure 3 pathways and controller labels were checked against Sections IV-A–C; Figure 5 colored instructions and arrows were checked against its caption and Section V-B. Table I and Figure 4 prose/value discrepancies are explicitly discussed. All retained method, numeric, evaluation, and proposed-check source facts are supported on these pages. Reference-only pp. 8–12 were read in the supplied text chunks but not visually inspected. No appendix or separate supplement was supplied; external code and resources were not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, contributor roster, and abstract (PDF pp. 1–2)
- I. Introduction (p. 2)
- II. Related Work (pp. 2–3)
- III. The Open X-Embodiment Repository, A–B (pp. 3–4)
- IV. RT-X Design, A–C (pp. 4–5)
- V. Experimental Results, A–C (pp. 5–6)
- VI. Discussion, Future Work, and Open Problems (pp. 6–7)
- References [1]–[134] (pp. 7–12)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Identity/version: the title and Open X-Embodiment Collaboration author credit match the catalog. The inspected title page identifies arXiv:2310.08864v9, 14 May 2025. The catalog cites ICRA 2024; the venue edition and earlier revisions were not supplied, so their content and author-list differences cannot be reconstructed. This report covers the supplied v9 artifact. Metadata uses its printed collective author credit; the individual contributor roster was also read.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This acquisition limitation was addressed by inspecting PDF pages 1–7 and every final crop.
- Separate supplemental material availability has not been fully verified.
- All six supplied text chunks were read individually, including the complete reference list. No appendix is present in this PDF. Reference-only pages 8–12 were read as text, not rendered; cited external works were not opened.
- Code, dataset files, checkpoints, project website, and separately linked implementation details were not inspected. No experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, collective author credit, contributor roster, and arXiv margin stamp
The observed title matches the supplied title; Open X-Embodiment Collaboration is credited above the individual roster. The stamp identifies arXiv:2310.08864v9 [cs.RO], 14 May 2025.
Go to primary source ↓e02PDF p. 2, abstract continuation and Section I, final three paragraphs
The abstract reports 22 robots, 21 institutions, 527 skills and 160266 tasks. The introduction distinguishes positive-transfer evaluation from repository construction and describes training existing RT-1/RT-2 models on nine manipulators.
Go to primary source ↓e03PDF p. 3, Section III and III-A
The repository pools 60 datasets from 34 labs, with over one million real-robot trajectories across 22 embodiments, stored in RLDS serialized tfrecord format. Heterogeneous sensing/action records and selected pretrained checkpoints are described; detailed dataset breakdowns are referred to the website.
Go to primary source ↓e04PDF p. 3, Figure 2(a–e), caption and Section III-B; p. 4, III-B continuation
Dataset, scene, and trajectory distributions differ: Franka dominates dataset/scene coverage; xArm and Google Robot supply many trajectories. Skills concentrate in the pick/place family. PaLM extracts objects and behaviors from language annotations. Panels (a) and (d) use broken vertical axes.
Go to primary source ↓e05PDF p. 4, Section IV-A and Figure 3
Policies select one camera, resize images, normalize seven-dimensional end-effector actions per dataset, and retain heterogeneous coordinate frames and absolute/relative/velocity conventions. Figure 3 illustrates robot-specific action interpretations and control rates.
Go to primary source ↓e06PDF p. 4, Section IV-B and Figure 3/caption
Eight output dimensions including termination are discretized into 256 bins. RT-1 uses a 35M network, 15-image history, EfficientNet, USE, FiLM, 81 tokens and a decoder-only Transformer. RT-2 uses action text tokens with PaLI-X/ViT/UL2 and primarily WebLI pretraining. The example action string shows seven integers despite the eight-dimension description.
Go to primary source ↓e07PDF pp. 4–5, Section IV-C
Categorical cross-entropy trains the policies. The named robotics mixture spans nine manipulators; RT-1-X uses robotics alone, RT-2-X approximately equal VLM and robotics data. The source explicitly distinguishes this mixture from the later 22-embodiment repository. Inference is 3–10 Hz, local for RT-1 and cloud-served for RT-2.
Go to primary source ↓e08PDF p. 5, Section V introduction and V-A evaluation setup
The paper reports 3600 total trials across six robots, separates small- and large-data domains, and uses the contributing studies’ evaluations. Baselines are each domain’s Original Method and an RT-1 trained on that domain alone. Section V-A points to the Section IV-C mixture despite also saying full X-embodiment dataset.
Go to primary source ↓e09PDF p. 5, Figure 4, labeled bars and caption; Section V-A small-scale paragraph
Original Method/RT-1/RT-1-X bars are 43/48/63 for Kitchen Manipulation, 24/18/56 for Cable Routing, 53/65/80 for NYU Door Opening, 53/25/45 for Autolab UR5, and 33/68/72 for Task-Agnostic Play. Displayed means are 41/44/63. The caption summarizes an approximately 50% mean improvement.
Go to primary source ↓e10PDF p. 5, Table I, all three columns, caption, and Section V-A large-scale paragraph
Bridge IRIS / Bridge RAIL / Google Robot: RT-1 40/30/92%, RT-1-X 27/27/73%, RT-2-X 55B 50/30/91%; LCBC 13/13% and no Google entry. These entries qualify the caption’s blanket Original Method comparison and the prose’s blanket RT-2-X superiority claim.
Go to primary source ↓e11PDF p. 6, Table II rows (1)–(3), Emergent Skills Evaluation column; Section V-B emergent-skills and Bridge-removal paragraphs
For 55B models without history, RT-2 scores 27.3%, RT-2-X 75.8%, and RT-2-X without Bridge 42.8%. The tasks involve objects/skills present in Bridge but absent from the original Google Robot data.
Go to primary source ↓e12PDF p. 6, Table II rows (1)–(3), RT-2 Generalization Evaluation column; Section V-B first evaluation paragraph
The separate unseen-object/background/environment evaluation gives RT-2 62%, RT-2-X 61%, and RT-2-X without Bridge 54%.
Go to primary source ↓e13PDF p. 6, Table II rows (4)–(5) and Section V-C
5B models with two-image history versus none score 44.4/14.5% on emergent skills and 52/30% on generalization, with other listed settings held constant. No per-row trial counts or uncertainty are provided.
Go to primary source ↓e14PDF p. 6, Table II rows (2), (4), (6), (7) and Section V-C
Row (2) is 55B with no history; row (4) is 5B with history two. For 5B/history-two, scratch/no-web-co-training gives 0% emergent and 1% generalization; web-pretrained/no-web-co-training gives 48.7/47%, versus 44.4/52% with web co-training. The authors attribute benefits to capacity, pretraining and history and describe co-fine-tuning versus fine-tuning as similar.
Go to primary source ↓e15PDF p. 6, Figure 5(a–c), caption and Section V-B
Examples distinguish absolute motion, object-relative motion, and preposition-conditioned behavior. Colored words and arrows contrast destinations such as on versus near. The caption identifies Bridge skills evaluated on Google Robot; images themselves provide no per-example success rates.
Go to primary source ↓e16PDF pp. 6–7, Section VI, final paragraph
The experiments do not consider substantially different sensing/actuation modalities and do not study generalization to new robots. Conditions governing positive transfer remain a future-work topic.
Go to primary source ↓8.5 Primary sources
Open X-Embodiment: Robotic Learning Datasets and RT-X Models ↗
PDF · 10,077 extracted words
Source fingerprint
13f16aff5afdee583dc25b3c570e9bb0cf6baefb59b539928d919588cd8d9246