RoboSynChallenge: Mastering Real-World Dexterity via Generalizing Synthesized Manipulation Skills
1. Paper overview
In one sentence: RoboSynChallenge couples expandable simulated demonstrations with physical manipulation tests, but its initial baselines leave the value of co-training and several aggregate results unresolved. e-pipelinee-datae-hardwaree-taskse-resultse-aggregate
| At a glance | What to know |
|---|---|
| Research problem | Source description Real robot demonstrations are costly and hardware-specific, while simulated training introduces differences in dynamics, kinematics and sensing. The benchmark asks how effectively synthetic experience can support manipulation in unseen physical conditions. Its proposed contribution is standardized real-world evaluation across task complexity and environmental shifts. e-pipelinee-benchmark |
| Core mechanism | Source description A generative benchmark connects simulation, optional real-data co-training and physical evaluation on ten manipulation tasks spanning entry, mid and high difficulty. e-pipelinee-tasks |
| A key reported result | Click Bell: Motus (real): 14/20, 420.50 steps, 70.11 s. Success count; action steps; reported Real Time (s). Real-world deployment; real-only training rows in Table 2; 20 trials per model. The table does not identify the precise ID/OOD configuration mixture. π0.5 (real): 6/20; π0 (real): 5/20. Motus (sim), under a different training-data condition: 13/20. Motus has the strongest displayed bell success count. This is a narrow physical task result, without reported uncertainty or a controlled architecture-only comparison. e-resultse-criteriae-evaluation |
| Reading caution | Reader analysis Simulation-only versus real-only training changes data source and potentially data quantity: collection specifies 1,000 versus 60 samples per task, without documenting the actual per-baseline subset or exposure. The results do not isolate equal-budget data quality, co-training gains, streaming, randomization, or recovery-module effects. e-datae-real-collectione-resultse-streaming |
Core contributions
- Source description
A generative benchmark connects simulation, optional real-data co-training and physical evaluation on ten manipulation tasks spanning entry, mid and high difficulty. e-pipelinee-tasks
- Source description
The dataset description includes interaction trajectories, RGB-D streams, proprioception, physics metadata, and annotations for task semantics, contacts and success. Per task, collection specifies 1,000 simulated trials and 60 real samples. e-datae-real-collection
- Author claim
The authors argue that synthetic data has quality comparable to real data because several simulation-trained policies match or outperform real-trained counterparts. This is their interpretation of the initial baselines. e-author-conclusion
Figure 1. Expandable synthetic experience and static real demonstrations meet before physical evaluation. Original paper, p. 2 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the orange loop: synthesis feeds filtering, retained manipulation trials feed policy learning, and the learned policy is tested in simulation. The red cross marks a return toward synthesis; the green check marks the path into the synthetic-data store. The blue teleoperation branch supplies the real-data store, and both sources connect to co-training. Finally, the green panel separates candidate model families from learning algorithms and routes them to real deployment. Read these as stages of the benchmark pipeline. The caption describes the real dataset as static; the snowflake beside that store does not establish a frozen neural backbone. e-pipelinee-datae-baselinese-results
What it supports. The benchmark can expand training experience while retaining real demonstrations as correspondence to the deployment domain. Its final output is an evaluated physical policy. However, the initial table compares simulation-only with real-only training, so the combined-data route shown here remains an intended capability without a reported controlled gain.
Where the evidence stops. The diagram does not specify optimization losses, mixture ratios or an inference-time planning algorithm. Its flame and snowflake icons should not be interpreted as a complete parameter-freezing schedule for the baseline models.
2. Motivation
2.1 The problem and the proposed response
Real robot demonstrations are costly and hardware-specific, while simulated training introduces differences in dynamics, kinematics and sensing. The benchmark asks how effectively synthetic experience can support manipulation in unseen physical conditions. Its proposed contribution is standardized real-world evaluation across task complexity and environmental shifts. e-pipelinee-benchmark
2.2 What this reading follows
A simulator can supply many robot trajectories, yet their usefulness depends on what transfers to a physical task. RoboSynChallenge organizes that question around a dual-arm platform, ten manipulation tasks, and environmental changes such as lighting, distractors and object placement. Its pipeline combines generated experience with a smaller teleoperated dataset, while its initial results compare policies trained separately on simulated or real data. Read this as a benchmark design with preliminary execution evidence. The figures explain how data and tests connect; the results reveal task-specific strengths, an unsolved assembly task, and aggregate values that require reconciliation with the displayed trial counts. e-pipelinee-datae-hardwaree-taskse-resultse-aggregate
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Benchmarks & simulators |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded Benchmarks & simulators category matches a task, data-generation and real-robot evaluation resource. Architecture, prediction paradigm and quadrant are appropriately Not applicable: WAM is one baseline family, not evidence that RoboSynChallenge introduces a unified future/action predictor or an inverse-dynamics controller. e-benchmarke-pipelinee-baselines
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
5. Method in detail
5.1 Separate the generator's feedback loop from the deployed controller
The central mechanism is a data pipeline. EmbodiChain creates simulator-ready assets, optimizes scene layouts for plausibility and reachability, and expands trajectories through diverse feasible interactions. Its recovery module reacts to simulated execution failures and turns corrective motions into additional supervision. Online Data Streaming then supplies changing experiences and visual augmentation to learning. Figure 1 places this expandable source beside static teleoperated demonstrations before policy training and physical deployment. This distinction prevents a common interpretive error: recovery during data generation does not prove that the deployed model runs the same recovery planner. The paper describes multiple baseline families rather than specifying one common latent state, action decoder or inference procedure. Its detailed contribution should therefore be understood at the benchmark and data-generation level. e-generatione-streaminge-pipelinee-baselinese-hardware
Figure 3. Paired scenes make the benchmark's transfer question concrete. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Within each task column, compare the upper real-world view with the lower simulated view. The first strip contains four green entry-level tasks and the blue hand-over task; the second continues the blue mid-level group and then shows three red high-level tasks. The color indicates the paper's difficulty category, not a measured success value. Observe how the task identity is retained even when object appearance and rendering differ. The figure's Handle Basket corresponds to Basket Pick-and-Place in the result and criterion tables, while the abbreviated hand-over label refers to the pen-transfer-and-placement task described in Table 5. e-taskse-criteriae-evaluation
What it supports. The benchmark connects corresponding simulated and physical task settings, allowing success on a real robot to test transferred behavior. Its progression runs from short routines to coordinated sequences and precise laboratory-style operations. This visual establishes the task inventory and intended correspondence, rather than the fidelity of contact dynamics.
Where the evidence stops. These are static scene examples, not execution sequences or a quantitative domain-gap measurement. Visual resemblance alone cannot establish equivalent dynamics, successful manipulation, or coverage of the held-out test conditions.
5.2 Turn environmental variation into a controlled transfer test
The real collection protocol starts from five conditions, each crossed with four positions and three orientations, giving sixty samples per task. Simulation supplies one thousand trials per task with additional randomization. Evaluation then changes background, lighting, object instance, distractor count or spatial position while holding the other factors fixed. This design can help localize a transfer failure: for example, a loss under lighting changes has a different implication from a loss under new object placements. That is a reader interpretation of the protocol's purpose, not a reported factor-specific result. The initial table aggregates twenty trials per task without explaining their allocation to the proposed variations. Nor does it document matched training exposure. Controlled transfer claims therefore require the split manifest and per-condition outcomes. e-datae-real-collectione-streaminge-evaluatione-results
Table 4. The augmentation specification mixes usable numerical bounds with unresolved configuration fields. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the Feature column to distinguish appearance changes from geometry and sensing changes. Light, object material and background rows vary colors; object location, orientation and size alter scene geometry. Table height has a numerical range described as plus or minus four centimeters. Camera intrinsics use focal-length range symbols, whereas extrinsic position and rotation rows provide numerical vectors. The robot rows perturb the initial end-effector position and joint configuration. The table's \(p_{\mathrm{texture}}\) denotes the probability of a random table texture, but no probability value is given. Read this as an inventory of augmentation controls rather than a fully executable sampling specification. e-streaminge-generatione-results
What it supports. Randomization extends beyond superficial texture changes to camera placement, object geometry and robot initialization. Appendix A.2 places visual augmentation inside a streaming learning pipeline. This provides concrete factors to reproduce or isolate, but the table contains configuration information rather than measured contributions to real-world success.
Where the evidence stops. Several bounds remain symbolic, including focal lengths, light intensity and joint limits. The source does not fully specify sampling distributions or all parameter units. The listed perturbations have no accompanying component-ablation results.
5.3 Read the measured outcomes before accepting the average
Start with a task whose success predicate is concrete. On Click Bell, Motus trained on real demonstrations succeeds fourteen times in twenty trials; on Table Rearrangement, simulation-trained π0.5 leads its simulation-trained counterparts. Assembly yields no successes for any displayed configuration. These observations support task-dependent strengths and a clear remaining difficulty. Next audit the aggregation. The simulation-trained π0.5 row sums to sixty-seven successes over two hundred trials, implying 33.50%, while its printed mean is 38.50%. Other SR and step/time means also require clarification. Finally, separate the authors' claim of comparable synthetic-data quality from what is controlled: the table changes the training domain without demonstrating matched data exposure or co-training gains. The useful conclusion is preliminary physical transfer evidence, with unresolved aggregate and protocol details. e-resultse-aggregatee-criteriae-datae-baselinese-author-conclusion
Table 5. Task execution steps and the condition counted as success must be read separately. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read horizontally from the task name through the prescribed steps to the terminal success condition. For water pouring, success includes both pouring into the cup and returning the objects to the table. For assembly, the two silicone tubes must be joined. Now compare the drawer row: its step list ends with closing the drawer, yet its success cell only mentions placing the item inside. The bell row similarly lists an arm return that is absent from its success condition. These differences matter when interpreting the success counts; a completed terminal predicate need not demonstrate every procedural step in the neighboring column. e-criteriae-metricse-results
What it supports. The benchmark ties success to observable physical outcomes within an action budget, giving the result table more meaning than image plausibility alone. At the same time, its own step/criterion differences leave room for different scoring implementations. Reproduction should make that choice explicit before comparing policies.
Where the evidence stops. No numerical placement tolerance, required liquid volume or detailed adjudication procedure is supplied here. Do not infer that every listed step is enforced by the success metric, or that a single task success verifies general dexterity.
5.4 Training and inference
During training
The proposed baseline suite covers ACT action chunks, Diffusion Policy denoising, π0 and π0.5 VLA policies, and Motus latent world-action modeling. Only π0, π0.5 and Motus receive numerical results. The paper gives family descriptions rather than a complete benchmark-specific optimizer, loss, checkpoint, freezing, batch-size or training-compute recipe. e-baselinese-results
Section 1.2 says training contains only successful trajectories; Appendix A.2 describes relabeled recovery motions following failures. Whether the released training set keeps failed prefixes, successful recoveries, or both is unresolved. No formal benchmark objective or equation is specified. e-datae-generatione-streaming
During inference
Deployment uses visual observations and proprioceptive feedback to produce physical robot actions. The synthesis and recovery pipeline supplies training experience; the paper does not establish that every deployed baseline performs online world-model planning or executes the generator's recovery algorithm. e-hardwaree-generatione-baselines
Success rate follows Table 5's criteria. Action steps can include corrections or exhausted failure budgets; Table 2 caps them at 1,000. Section 1.4 specifies an NVIDIA A800 evaluation GPU and calls the timing metric inference time, whereas Table 2 labels it Real Time in seconds. Its precise aggregation and per-command meaning are unspecified. e-metricse-resultse-criteria
5.5 Implementation flow
- Generate environments and trajectories
EmbodiChain first synthesizes assets with generative models and multi-objective optimization, then uses gradient-based layout synthesis for realism and reachability. Reachability-aware sampling diversifies interactions; a recovery module detects failures, replans corrective motions, and relabels them as supervision. e-generation
- Expand visual and physical coverage
Online Data Streaming perturbs lighting, textures and sensor parameters during learning, with asynchronous shared-memory exchange through lock-free circular buffers. Table 4 lists object, table, camera, robot and distractor randomization; some fields have numerical bounds, while others remain symbolic. e-streaming
- Anchor simulation to real demonstrations
Five real collection conditions cross four positions and three orientations. Table 3 varies illumination, white/blue/yellow backgrounds and a condition with 2–3 distractors. Figure 1 combines expandable synthetic data with static teleoperated data, but Table 2 evaluates separate simulation-only and real-only training. e-real-collectione-pipelinee-results
- Evaluate controlled physical shifts
Two six-degree-of-freedom AgileX Piper arms execute tasks. The protocol varies one factor at a time: background, lighting, seen/unseen object instances, 2/4/8 distractors, or unseen positions in a 3-by-3 grid. Small position shifts provide repeated trials; development testing is in-distribution and competition testing is held-out out-of-distribution. e-hardwaree-evaluation
6. Experiments & results
RoboSynChallenge proposes a benchmark for learning manipulation policies from synthesized trajectories and evaluating their transfer on physical dual-arm robots. Its contribution is a data-generation and evaluation protocol: EmbodiChain expands simulated experience, a smaller teleoperated dataset supplies real correspondence, and task success is checked through robot execution. Initial simulation-only and real-only baselines show task-dependent transfer, but inconsistent table aggregates and incomplete training specifications limit conclusions about data quality and generalization.
This benchmark proposal has a pipeline diagram and one quantitative baseline table, but no controlled co-training, streaming, recovery-module or domain-randomization ablation. Table 4 specifies perturbations without reporting their effects, and Appendix B only interprets Table 2. Consequently, this edition uses configuration and scoring tables to explain reproducibility instead of presenting a nonexistent component-ablation visual. No factor-specific robustness plot is supplied. e-pipelinee-resultse-streaminge-author-conclusione-evaluation
6.1 Read the original evidence
Table 2. Task-level results are informative, while several printed averages need reconciliation. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each model's sim and real rows as training-data conditions; both are evaluated through real-world deployment. Each task's SR cell is a success count out of twenty, followed by action steps and a Time column. The table caption calls this Real Time in seconds, with a maximum of one thousand action steps; Section 1.4 instead names inference time, so do not treat these cells as measured per-call latency. Compare models within the same training-data condition before comparing sim with real. Inspect all three table blocks: easy-task strengths can coexist with high-level failures, and the printed Task Average block is not internally consistent with several displayed rows. e-resultse-aggregatee-metricse-evaluatione-criteriae-author-conclusion
What it supports. Motus (real) achieves 14/20 on Click Bell, compared with 6/20 for π0.5 (real). On Table Rearrangement, π0.5 (sim) reaches 12/20 versus Motus (sim)'s 4/20. Every Item Assembly row is 0/20. These outcomes support a task-dependent comparison, without establishing uniformly superior architecture or data quality.
Where the evidence stops. Printed π0 (real), π0.5 (sim), and π0.5 (real) SR means are 22.50%, 38.50%, and 33.00%; their task counts imply 20.00%, 33.50%, and 30.00%. Some step/time means also disagree. No weighting rule, uncertainty, or exact test-condition allocation resolves these discrepancies.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Click Bell Real-world deployment; real-only training rows in Table 2; 20 trials per model. The table does not identify the precise ID/OOD configuration mixture. | Motus (real): 14/20, 420.50 steps, 70.11 s. Success count; action steps; reported Real Time (s) | π0.5 (real): 6/20; π0 (real): 5/20. Motus (sim), under a different training-data condition: 13/20. Motus has the strongest displayed bell success count. This is a narrow physical task result, without reported uncertainty or a controlled architecture-only comparison. e-resultse-criteriae-evaluation |
| Table Rearrangement Real-world deployment; simulation-only training; 20 trials per model. | π0.5 (sim): 12/20. Success count | Motus (sim): 4/20; π0 (sim): 7/20. The ordering reverses relative to Click Bell, showing that the stronger bell policy is not uniformly better across tasks. e-resultse-criteria |
| Item Assembly Real-world deployment; all six simulation-only or real-only rows; 20 trials per row. | Every row: 0/20 and 1,000.00 steps. Success count and action steps | No evaluated π0, π0.5 or Motus training condition succeeds. Joining the two silicone tubes remains unsolved in this evaluation; the shared step ceiling indicates exhausted action budgets. e-resultse-criteriae-metrics |
| Ten-task average success as printed Table 2 Task Average column; simulation-only versus real-only training, evaluated on real robots. | π0: 22.00 / 22.50; π0.5: 38.50 / 33.00; Motus: 31.50 / 27.50 (sim / real). Reported average success rate (%) | Reader sums of the ten displayed x/20 counts instead give π0: 22.00 / 20.00; π0.5: 33.50 / 30.00; Motus: 31.50 / 27.50. These are printed values, not validated aggregate estimates. Three SR aggregates disagree with the task counts; no alternative weighting is explained. The apparent aggregate improvements must remain provisional. e-resultse-aggregate |
6.3 Ablations and diagnostic examples
No ablation or diagnostic visual is included in this edition. The source-specific visual limitations are recorded above.
7. Analysis & limitations
7.1 What the evidence leaves open
Simulation-only versus real-only training changes data source and potentially data quantity: collection specifies 1,000 versus 60 samples per task, without documenting the actual per-baseline subset or exposure. The results do not isolate equal-budget data quality, co-training gains, streaming, randomization, or recovery-module effects. e-datae-real-collectione-resultse-streaming
Table 2 supplies neither uncertainty nor a factor-by-factor breakdown connecting its 20 trials to the proposed OOD protocol. Several step/time aggregates also disagree with simple task means; for example Motus (real) reports 721.35 average steps, while its ten entries average 771.295. Raw trial logs are needed to resolve aggregation. e-aggregatee-evaluation
Task instructions and scoring conditions are not always equivalent. Drawer Open and Place lists closing the drawer as its final step, but its success cell only requires placing an item inside. Placement tolerances, liquid volumes and detailed adjudication rules are not numerically specified. e-criteria
7.2 Questions for discussion
- Does synthetic augmentation help once real-data exposure and total training updates are controlled?
- How would model rankings change if completing every listed task step were required?
- Which aggregate values can be reconstructed from the underlying trial records?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the dual-Piper platform, calibrated cameras and controllable lighting/backgrounds, the real collection conditions, simulated task assets and randomization settings, and the declared A800 evaluation environment. Training hardware, software versions, simulator configuration, learning schedules and several randomization bounds remain unspecified. e-hardwaree-real-collectione-streaminge-baselinese-metrics
The paper names CC BY-NC-SA 4.0 for the dataset and plans participant-accessible releases with metadata and generation code. Release wording is mixed: Section 1.2 says the collection pipeline is released, while Sections 1.5–1.6 describe future kit, dataset and website availability. e-datae-release
Proposed checks: compare real-only training with equal-update co-training on identical physical test configurations; independently recompute every aggregate from trial records and score listed task steps separately from terminal success. These would test augmentation value and metric reliability, respectively. e-pipelinee-datae-evaluatione-aggregatee-criteria
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does synthetic augmentation help under a fixed training budget?
Reader-proposed experiment, not performed: use Click Bell and Table Rearrangement with the same π0.5 initialization, real demonstration set, optimizer, batch size and total update count. Compare a real-only replay control against a declared 1:1 real/synthetic mixture drawn from the supplied simulation pool; record actual sample exposures and repeat across training seeds. Evaluate both on identical held-out backgrounds and positions using the same physical setup, action budget and success rules. Report paired per-condition counts and uncertainty. If the mixture's benefit disappears under the fixed update budget or occurs only on development conditions, the claim that synthetic augmentation improves held-out transfer is weakened. The source provides neither this controlled comparison nor its outcome. e-datae-real-collectione-pipelinee-evaluatione-baselinese-resultse-criteria
Check 2: Can the aggregate scores survive a trial-level scoring audit?
Reader-proposed check, not performed: obtain the original baseline trial records if released, reconstruct every success numerator and denominator, and recompute unweighted task means alongside any organizer-declared alternative weighting. Compare these directly with Table 2, particularly π0.5 (sim) SR and Motus (real) action steps. On the same drawer and bell recordings, have independent raters score both the written success predicate and completion of every listed step; hold the trials and action budget fixed. Also distinguish model-command latency from whole-trial elapsed time in the records. Unexplained aggregate mismatches or changed success counts under the two rubrics would falsify the assumption that the published metrics uniquely determine a reproducible ranking. e-aggregatee-resultse-criteriae-metricse-release
8.3 Reading coverage
Visual audit: All 12 PDF pages were rendered and actually viewed. The pass covered the title/authors/version, Figure 1 pipeline and branch directions, Figures 2 and 4 hardware, Figure 3 task pairs, Tables 1–5, collection/training/evaluation descriptions, appendix generation and randomization details, and reference/release pages. Each of the five final original crops was separately viewed, including every row of the baseline and success-criterion tables. All pages supporting retained methodological, numerical and reproduction claims are included. Only the supplied primary PDF was inspected; linked implementations, datasets and separate supplements remain outside this reading.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- PDF pp. 1–3: title, abstract, competition background, impact and Section 1.1 Novelty
- PDF pp. 3–4: Section 1.2 Data
- PDF pp. 4–6: Sections 1.3 Tasks and application scenarios and 1.4 Metrics
- PDF pp. 6–8: Sections 1.5 Baselines, code, and material provided and 1.6 Website, tutorial, and documentation
- PDF pp. 8–10: References
- PDF pp. 10–11: Appendix A, including A.1 and A.2
- PDF p. 12: Table 5 and Appendix B
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Identity: the inspected title page identifies arXiv:2608.12416v1 [cs.RO], 12 August 2026, and labels the work a preprint. The exact title and all 18 authors agree with the catalog after name-order normalization. The PDF includes affiliation 11, DexForce, which the catalog affiliation string omits. No other revision or venue edition was supplied or compared; the catalog venue label is not independently established by this PDF.
- All four supplied text chunks were individually read without sampling. All 12 PDF pages and Figures 1–4 and Tables 1–5 were visually inspected; five original crops accompany the edition. The extraction's missing figure images were addressed by this visual pass.
- Linked code, datasets, documentation, cited papers and separate supplements were not inspected. No experiments were run or reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author/affiliation block and arXiv margin
The title matches the supplied primary title; all 18 catalog authors appear in order. The margin states arXiv:2608.12416v1 [cs.RO], 12 Aug 2026. Affiliation 11 is DexForce; the footer says Preprint.
Go to primary source ↓e-pipelinePDF p. 2, background, Figure 1 and caption, Impact
The orange generation/validation loop supplies synthetic data; a blue teleoperation branch supplies static real data for co-training; the green branch deploys policies on real robots. The stated goal is evaluating synthetic data's contribution when real data is scarce.
Go to primary source ↓e-benchmarkPDF p. 3, Section 1.1 and Table 1, RoboSynChallenge row
The paper presents a competition with generative data and physical evaluation. Its table characterizes RoboSynChallenge as bimanual, generative, trained in simulated and real environments, and evaluated in realistic environments.
Go to primary source ↓e-dataPDF pp. 3–4, Section 1.2 Data, collection protocols, leakage and license paragraphs
The dataset contains RGB-D, proprioception, scene metadata and task/contact/success annotations. The simulation protocol generates 1,000 trials per task. Training is described as successful trajectories; held-out real conditions address overlap. The stated dataset license is CC BY-NC-SA 4.0.
Go to primary source ↓e-real-collectionPDF p. 4, real collection protocol; PDF p. 10, Appendix A.1 and Table 3
Five conditions crossed with four positions and three orientations yield 60 real samples per task. Conditions are white/fixed/no distractors; white/enhanced/no distractors; white/fixed/2–3 distractors; blue/fixed; yellow/fixed.
Go to primary source ↓e-hardwarePDF p. 4, Section 1.3 Hardware and Manipulation tasks; PDF p. 5, Figure 2 and caption; PDF p. 11, Figure 4 and caption
The platform has two six-DoF AgileX Piper arms and uses visual observations with proprioception. Figures show cameras, controllable lighting and replaceable table backgrounds. The caption describes three identically configured backup workstations.
Go to primary source ↓e-tasksPDF p. 5, Section 1.3 task levels; PDF p. 6, Figure 3 and caption; PDF p. 12, Table 5
Ten tasks span four entry-level, three mid-level and three high-level tasks. Figure 3 pairs real and simulated views. The basket task is labeled Handle Basket in the figure and Basket Pick-and-Place in Table 5.
Go to primary source ↓e-evaluationPDF p. 5, final Section 1.3 paragraph and Section 1.4 Evaluation Protocol; PDF p. 6, opening continuation
The protocol distinguishes ID development from OOD held-out evaluation, varies one factor at a time, and uses three backgrounds, lighting variation, seen/unseen objects, 2/4/8 distractors and a 3-by-3 position grid with small random shifts. No allocation of Table 2's trials across these conditions is provided.
Go to primary source ↓e-metricsPDF p. 6, Section 1.4 Evaluation Metrics; PDF p. 7, Table 2 caption
Metrics are success percentage, inference time and action steps. All models are said to run on the same NVIDIA A800 machine. Steps may reflect corrections or exhausted budgets. The table caption calls time Real Time (s) and specifies a maximum of 1,000 action steps.
Go to primary source ↓e-baselinesPDF pp. 6–7, Section 1.5 Baselines
ACT, Diffusion Policy, π0, π0.5 and Motus are described as baseline families. Table 2 is introduced as initial results for simulation-only and real-only VLA/WAM training. Standardized loaders, training scripts and evaluation wrappers are promised; no full training recipe or controlled component ablation is supplied.
Go to primary source ↓e-resultsPDF p. 7, Table 2, all task blocks and six model/data rows
The table reports 20-trial success counts, steps and time for π0, π0.5 and Motus using sim or real training. Click Bell Motus real is 14/20, 420.50, 70.11; π0.5 real is 6/20 and π0 real 5/20. Table Rearrangement sim rows give π0.5 12/20, Motus 4/20 and π0 7/20. Item Assembly is 0/20 and 1,000 steps in all rows.
Go to primary source ↓e-aggregatePDF p. 7, Table 2, Task Average columns and corresponding ten task columns
Printed SR averages (sim/real) are π0 22.00/22.50%, π0.5 38.50/33.00%, Motus 31.50/27.50%. Their displayed success numerators total 44/40, 67/60 and 63/55 over 200 trials. Motus real's ten step entries are 420.50, 492.30, 744.95, 900.05, 608.55, 546.60, 1000, 1000, 1000, 1000; its printed mean is 721.35. No alternative aggregation rule is explained.
Go to primary source ↓e-releasePDF p. 4, Section 1.2 release statements; PDF pp. 7–8, release protocol and Section 1.6
Section 1.2 calls the collection pipeline released, but Section 1.5 plans the starting kit and full datasets, and Section 1.6 describes future website/tutorial availability. Links do not establish their current contents.
Go to primary source ↓e-generationPDF p. 10, Appendix A.2 stage 1; PDF p. 11, Appendix A.2 stage 2
EmbodiChain describes generative assets, multi-objective optimization, gradient-based scene layouts, reachability-aware trajectory sampling, and failure-triggered replanning whose corrective motions are relabeled as recovery supervision.
Go to primary source ↓e-streamingPDF p. 11, Appendix A.2 stage 3 and Table 4
Online Data Streaming uses visual augmentation and asynchronous shared-memory circular buffers. Table 4 gives table-height variation ±4 cm, numerical camera position/rotation ranges, and symbolic focal-length ranges, texture probability and other unspecified bounds. It supplies a configuration inventory rather than ablation measurements.
Go to primary source ↓e-criteriaPDF p. 12, Table 5, Steps and Success Criteria columns
Criteria include bell pressing, pen placement into a brush pot, pouring with both items returned to the table, tube joining, pipette insertion/button pressing and test-tube placement into a rack. Drawer closing and bell-arm return appear in steps but are absent from the respective success cells. Numerical precision thresholds are not supplied.
Go to primary source ↓e-author-conclusionPDF p. 12, Appendix B Evaluation Results of baselines
The authors interpret the simulation-only and real-only baseline comparison as evidence that synthetic data is comparably high quality for real-world deployment. Appendix B adds no new result table or controlled co-training ablation.
Go to primary source ↓8.5 Primary sources
RoboSynChallenge: Mastering Real-World Dexterity via Generalizing Synthesized Manipulation Skills ↗
PDF · 5,951 extracted words
Source fingerprint
cf8aff2898c58e90d31971fb259ba370cb14a20820f0a3e3a1abf70205b89a5b