MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations
1. Paper overview
In one sentence: Object-relative replay can turn a few human demonstrations into useful policy-training data, but collection yield and downstream learning quality respond differently to noise, interpolation and scene coverage. E02E03E04E05E11E13E15E17
| At a glance | What to know |
|---|---|
| Research problem | Source description Repeated teleoperation often demonstrates similar manipulation behavior in different contexts. MimicGen asks whether those motions can be reused to expand data coverage while reducing human collection effort. The system trades additional environment interaction and explicit task/pose assumptions for fewer human demonstrations. E02E03E04 |
| Core mechanism | Source description The main scale breakdown is 50,000 simulated demonstrations from 175 source demonstrations: 36,000 across nominal/reset variants, 12,000 for robot transfer, and 2,000 for object transfer. The headline also includes two real tasks; inconsistent real-data counts prevent a single reconciled total. E01E12E16 |
| A key reported result | Square manipulation (image BC-RNN): D0 90.7 ± 1.9; D1 73.3 ± 3.4; D2 49.3 ± 2.5. Task success (%). Simulation; 10 D0 sources versus 1,000 generated demonstrations per distribution; three training seeds, best evaluated checkpoint per seed. Source-only D0: 11.3 ± 0.9. Generation supports learning across wider resets, but each variant has its own generated training data; this is not zero-shot D0-policy transfer. E09E10E11E24 |
| Reading caution | Source description Known fixed subtask order, one reference object per subtask, and pose estimates constrain applicability. Tests concern quasi-static rigid objects with aligned category frames; changed object dynamics, deformables and multi-arm tasks are untested. Mobile base motions are copied and do not overlap arm motion. E19 |
Core contributions
- Source description
The main scale breakdown is 50,000 simulated demonstrations from 175 source demonstrations: 36,000 across nominal/reset variants, 12,000 for robot transfer, and 2,000 for object transfer. The headline also includes two real tasks; inconsistent real-data counts prevent a single reconciled total. E01E12E16
Figure 2. Reuse object-relative motion, then execute it to collect a new trajectory. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the left grid: rows are source demonstrations and columns are successive subtasks. Blue borders mark selected reference segments. On the right, follow the green arrows from Current Observation to reference selection, down to Transform segment, left to Interpolate to start, then left to Execute segment and back to observation. The red dotted path is adapted to the yellow mug; the green bridge connects the robot to that adapted path. Figure 2 permits segments from different demonstrations, but Appendix N.3 makes this optional: many tasks retain one source demonstration throughout an episode to avoid incompatible grasp and placement strategies. E03E04E05E07E10
What it supports. The system acquires fresh environment interaction while reusing demonstrated motion structure. Object poses are required at subtask boundaries for generation; downstream image policies can act without those estimates. Successful full executions become training trajectories, so the loop itself should not be read as the final learned policy architecture.
Where the evidence stops. The arrows agree with the described sequence, but the printed transform on pages 4 and 34 contradicts the frame-preservation equality in Appendix M. The report explains that equality and labels its algebraic consequence as reader analysis; the implemented convention is unverified.
2. Motivation
2.1 The problem and the proposed response
Repeated teleoperation often demonstrates similar manipulation behavior in different contexts. MimicGen asks whether those motions can be reused to expand data coverage while reducing human collection effort. The system trades additional environment interaction and explicit task/pose assumptions for fewer human demonstrations. E02E03E04
2.2 What this reading follows
MimicGen starts with a practical observation: moving a familiar object in a new place often reuses much of an existing manipulation strategy. Its system splits demonstrations into object-centric segments, adapts their controller targets to a new scene, executes the resulting motions, and keeps successful trials. A separate imitation learner then turns those trials into a reactive policy. Read the figures as evidence about this two-stage process. The broad simulation gains are compelling, but transfer requires newly generated training data, and real-world policy success is more limited. The appendix also exposes a printed transform inconsistency and several useful tests of what makes generated data learnable. E02E03E04E05E11E13E15E17
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Datasets |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded Datasets category and synthetic-data, robot-demonstration and cross-robot subcategories fit the contribution. Architecture, prediction paradigm and quadrant are Not applicable: geometric trajectory reuse plus separate BC policies does not constitute a world-action model. This is outside that architectural taxonomy, rather than an unverified One Model assignment. E03E04E13E14E10
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Attach each manipulation stage to its own reference object
Consider Square: grasping is defined relative to the nut, while insertion is defined relative to the peg. Moving the peg should therefore alter the insertion segment without forcing the grasp segment to use the peg as its reference. MimicGen first parses demonstrations into this known sequence using task metrics. At generation time, it obtains the relevant object's pose at the start of each stage, selects source motion, and preserves the controller-target relationship to that reference frame. Figure 2 illustrates this repeated process. Source selection matters because a grasp can determine which placement motion is compatible with it; per-subtask mixing is optional. The paper's intended frame invariant is clear, but its printed final transform reverses the required mapping. A reproduction must explicitly resolve that discrepancy before trusting any generated trajectory. E03E04E05E06E07E24
5.2 Turn imperfect replay into an offline learning dataset
A failed generation attempt is discarded; a successful one becomes a state–action trajectory for a separate learner. This means the generator can be imperfect while still producing useful supervision, provided it collects enough suitable successes. Table N.1 makes the distinction concrete: removing action noise increases Threading D0 collection yield but sharply lowers the image policy's success. Reader interpretation: the accepted noisy trajectories supply useful variation that raw yield alone cannot measure. The noisy-replay control also improves over source-only learning, yet remains below full generation across new configurations. After collection, BC-RNN learns demonstrated action likelihood from observations; there is no future-image objective in this pipeline. Its observation feedback enables a different deployment behavior from following a predetermined reference path, although the experiments do not isolate feedback as the sole cause of improvement. E02E03E04E08E10E11
5.3 Audit generalization and collection cost together
The D1 and D2 scores answer whether newly generated data can support policies on broader reset distributions. They do not show that the D0 policy transfers without retraining. Likewise, robot transfer reuses Panda demonstrations to create new per-arm datasets before training the respective policies. Reader interpretation: MimicGen primarily transfers a data-collection strategy. Its limits become clearer when success is paired with collection cost and coverage. Pose perturbations can preserve substantial downstream success while reducing generation yield to a few percent. Success filtering can also leave holes in the initial-state distribution, as Appendix R's occupancy analysis shows. Finally, the physical tasks use different checkpoint selection and have much lower BC-RNN performance than simulation; Appendix H's Diffusion Policy result demonstrates that the choice of learner remains consequential even after data generation. E09E10E13E15E18E20E22
5.4 Training and inference
During training
Typically 10 D0 demonstrations produce 1,000 successful trajectories per task variant; Mobile Kitchen uses 25 sources. The simulation environments use robosuite/MuJoCo and Factory/Isaac Gym. Separate BC-RNN policies learn from each generated dataset. E09
Image policies receive front and wrist RGB images plus end-effector poses and gripper positions: 84×84 in simulation and 120×160 on real tasks, with pixel shifts up to 10% per dimension. Low-dimensional policies instead include ground-truth object poses. Most BC-RNN settings are inherited from robomimic; low-dimensional learning rate is increased to 10^{-3}. E10
During inference
The learned image policy responds to observations without the generator's object-pose requirement. Geometric replay is the collection mechanism; the trained policy predicts actions for closed-loop execution. The paper does not introduce a learned future-state or video model, joint future/action predictor, or inverse-dynamics controller. E02E03E04E10
Simulation results select the maximum checkpoint success per training seed, using 50 rollouts per checkpoint and three seeds. Real BC-RNN results use the last checkpoint and 50 episodes. These are different evaluation protocols, and the printed ± statistic is not explicitly named. E10E11
5.5 Implementation flow
- Parse demonstrations
Split each source trajectory into contiguous object-centric subtasks using task-specific end metrics. For Square, finger–nut contact ends grasping and task success ends insertion. Manual annotation is discussed as an alternative; the experiments use automatic metrics. E03E06
- Choose compatible segments
Select a reference randomly or from nearby object poses. Selection may be per episode or per subtask; mixing incompatible grasps and placements motivates keeping a single source episode. Square uses a random choice among the three nearest sources, held across subtasks. E07
- Transform and connect
Represent actions as controller target poses and preserve their relationship to the reference object's starting frame. Connect the current end effector to the transformed start using linear positional and spherical rotational interpolation, optionally holding the endpoint. The printed final transform is inconsistent with its stated invariant; see the equation note. E04E05E07
- Execute and filter
Convert each target back to a delta-pose action using the current end-effector pose, retain the source gripper command, and execute with Gaussian action noise excluding the gripper. After all subtasks, store only successful attempts. Data-generation rate is successes divided by attempts, not the downstream policy score. E04E08
6. Experiments & results
MimicGen expands a small human demonstration set by transforming object-relative motion segments, executing them in new scenes, and retaining successful trajectories for behavioral cloning. Its strongest evidence concerns useful manipulation data across reset distributions, objects and robot arms. Generation yield, learned-policy success and real-world performance are distinct outcomes (E03–E04, E11, E13–E15).
6.1 Read the original evidence
Figure 4, left table. Generated demonstrations improve nominal-task learning and support broader training distributions. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each row horizontally. Source is an image BC-RNN trained on the small human set; D0, D1 and D2 are policies trained separately on generated datasets for those reset variants. Each generated dataset contains 1,000 successful trajectories. Most source sets contain 10 demonstrations, with 25 for Mobile Kitchen. D0 is nominal, D1 broadens object placements, and D2 adds a harder reset where available. Dashes denote absent entries. The paper evaluates simulation checkpoints with 50 rollouts and selects the maximum success for each of three training seeds. The printed ± values are retained without assigning an unstated uncertainty convention. E09E10E11E24
What it supports. Square rises from 11.3 ± 0.9% on source data to 90.7 ± 1.9% on generated D0 data. Its broader D1 and D2 results are 73.3 ± 3.4% and 49.3 ± 2.5%. The drop across variants makes the increased difficulty visible even when generation greatly improves nominal learning.
Where the evidence stops. These columns do not test a single D0-trained policy on unseen resets: each generated variant supplies training data. Success measures robot task execution in simulation, and maximum-checkpoint scores need not equal final-checkpoint performance.
Table F.2. Human motion transfers across robot arms through newly generated datasets. Original paper, p. 22 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Keep the source robot and the policy's robot separate. The ten source demonstrations for each task were collected with Panda, but MimicGen executes adapted motions to create a separate dataset for each arm. Columns then report the success of image policies trained on those respective datasets, using three seeds. Compare arms within one row before comparing reset distributions. Square D0 is relatively strong across all four arms; moving to Square D1 lowers every column. The un-cropped Table F.1 on the same inspected page reports collection success and should be consulted when estimating how many attempts each arm needs. E13E04E10
What it supports. Square D0 policies reach 90.7 ± 1.9% on Panda, 86.0 ± 1.6% on Sawyer, 80.0 ± 4.3% on IIWA and 84.7 ± 0.9% on UR5e. The D1 row shows that transfer does not remove task difficulty: IIWA falls to 48.0 ± 3.3%.
Where the evidence stops. This is simulated cross-robot data generation followed by robot-specific training. It does not demonstrate one shared policy transferring directly between physical robots, and similar policy success can conceal substantially different collection yields.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Square manipulation (image BC-RNN) Simulation; 10 D0 sources versus 1,000 generated demonstrations per distribution; three training seeds, best evaluated checkpoint per seed. | D0 90.7 ± 1.9; D1 73.3 ± 3.4; D2 49.3 ± 2.5. Task success (%) | Source-only D0: 11.3 ± 0.9. Generation supports learning across wider resets, but each variant has its own generated training data; this is not zero-shot D0-policy transfer. E09E10E11E24 |
| Threading and Three Piece Assembly (image BC-RNN) Same simulation protocol as Figure 4; source demonstrations collected on D0. | Threading 98.0 ± 1.6; Three Piece Assembly 82.0 ± 1.6. D0 task success (%) | Source-only: 19.3 ± 3.4 and 1.3 ± 0.9, respectively. Successful generated data substantially improves these manipulation policies; this does not establish arbitrary task generalization. E09E10E11 |
| Cross-robot Square D0 Panda source demonstrations generate separate 1,000-trajectory datasets for each simulated arm; image BC-RNN, three seeds. | Panda 90.7 ± 1.9; Sawyer 86.0 ± 1.6; IIWA 80.0 ± 4.3; UR5e 84.7 ± 0.9. Policy success (%) | Generation rates differ: 73.7%, 55.8%, 37.7%, 64.7%, respectively. The transferable element is demonstration geometry; policies are trained for each arm. E13E10 |
| Action-noise ablation on Threading D0 Simulation; Table N.1 compares normal generation, no action noise, and noisy replay of original source initial states. | Normal: 51.0 generation, 98.0 ± 1.6 policy; no noise: 84.5 generation, 59.3 ± 6.8 policy. Generation rate and image-policy success (%) | Replay with noise: 53.8 generation, 74.0 ± 3.3 policy. Higher collection yield does not imply better training data. Noisy replay alone does not recover full MimicGen performance. E08E10 |
| Longer interpolation segments Simulation image-policy comparison using 5 interpolation steps versus 25 interpolation plus 25 fixed-pose steps. | Stack D1: 99.3 to 68.7; Pick Place D0: 50.7 to 11.3. Policy success (%) | Figure H.1 compares the short and long generation settings. Longer bridges can hurt imitation. This supports a possible contributor to the real-world gap, not a controlled explanation of the entire gap. E17E07E10 |
| Real-world Stack and Coffee Section 6.4: D1 generated data, image BC-RNN, last checkpoint, 50 evaluations; 200 Stack and 100 Coffee successes stated in the main text. | Stack 36%; Coffee 14%. Task success (%) | Source-only policies score 0% on their narrower D0 tasks. Appendix H reports Stack Diffusion Policy at 76% over 50 evaluations. Physical execution is demonstrated, but source-policy resets differ and dataset counts conflict across passages. The Diffusion Policy result has no reported seed spread. E15E16E18E10 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table N.1. Noise can reduce collection yield while improving the resulting policy. Original paper, p. 36 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the three row blocks separately. DGR is the success percentage of generation attempts; SR, image and SR, low-dim describe policies trained afterward using different observation spaces. Normal includes Gaussian action noise with scale 0.05, excluding the gripper. No noise removes that perturbation. Replay w/ noise resets the environment to initial states from the original source demonstrations and collects successful noisy replays, so it lacks MimicGen's new scene configurations. The replay column is populated only for D0. Follow a task from the top block to its corresponding policy block to see whether a change that helps collection also helps learning. E08E09E10E20
What it supports. For Threading D0, removing noise raises generation success from 51.0% to 84.5%, yet image-policy success drops from 98.0 ± 1.6% to 59.3 ± 6.8%. Noisy replay reaches 74.0 ± 3.3% policy success. Together these comparisons support contributions from both action perturbation and collecting across new configurations.
Where the evidence stops. The ablation does not establish that more noise is always better. Success filtering changes which trajectories survive, and equal successful dataset size does not imply equal collection effort or identical scene coverage.
Figure H.1. Connecting demonstrations is a learning-design choice as well as a control choice. Original paper, p. 24 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the legend before interpreting the bars: blue is the default five-step bridge; orange is the longer setting associated with the real-world collection procedure. Appendix H specifies the latter as 25 interpolated poses followed by 25 steps holding the target, not 50 interpolated poses. The vertical axis is trained image-policy success, not collision frequency or data-generation success. Compare the two bars within each task. Stack, Square, Three Piece Assembly, Threading and Pick Place decline, while Coffee shows no analogous drop. The adjacent paragraph supplies exact values for Stack D1 and Pick Place D0; the chart itself has no error bars. E17E07E15E10
What it supports. The supporting text reports Stack D1 falling from 99.3% to 68.7% and Pick Place D0 from 50.7% to 11.3%. The authors suggest that slow bridge motions are difficult to infer from observations. This is evidence that faithful motion execution and easy-to-learn demonstrations can require different design choices.
Where the evidence stops. These are simulation experiments motivated by real-robot behavior. They do not isolate every cause of the real-world gap: real datasets are smaller, perception differs, and the comparison changes both interpolation duration and endpoint holding.
Table U.1. Pose errors can make collection expensive even when surviving data still trains useful policies. Original paper, p. 45 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across each DGR row to assess collection sensitivity, then find the matching SR row to assess the trained policy. The columns inject uniform pose noise into both source-demonstration object poses and new-scene object poses during generation. Level 1 uses 5 mm and 5 degrees; Level 2 uses 10 mm and 10 degrees. These perturbations affect the object-relative transformation before data collection, rather than adding observation corruption at policy deployment. Compare tasks as well as noise levels: Square tolerates the intervention differently from Coffee and Threading. The repeated DGR/SR pairs are essential to understanding the authors' robustness claim. E22E04E10
What it supports. Coffee D1 generation declines from 63.5% without noise to 4.3% at Level 2, while policy success declines from 90.7 ± 2.5% to 77.3 ± 4.1%. Thus useful downstream behavior can survive a severe loss of generation efficiency; describing the result simply as pose robustness hides that cost.
Where the evidence stops. The tested errors are bounded uniform perturbations, not a full model of perception failures, correlated bias or object misidentification. Success-filtered data and nonzero policy scores do not establish acceptable collection cost or general robustness.
7. Analysis & limitations
7.1 What the evidence leaves open
Known fixed subtask order, one reference object per subtask, and pose estimates constrain applicability. Tests concern quasi-static rigid objects with aligned category frames; changed object dynamics, deformables and multi-arm tasks are untested. Mobile base motions are copied and do not overlap arm motion. E19
Success-only filtering permits biased scene coverage and undesirable collisions. Appendix R estimates support with coarse occupancy bins; Three Piece Assembly D1 covers 43.5% of those bins. This diagnostic neither measures uniformity nor proves complete behavioral coverage. E20
Pose robustness is conditional: Coffee D1 generation falls from 63.5% to 4.3% under 10 mm/10° pose noise, while policy success falls from 90.7 ± 2.5% to 77.3 ± 4.1%. Maintaining useful policies can therefore require many more collection attempts. E22
Most datasets use one generation seed. Appendix T tests additional generation seeds only on Stack Three and Square with low-dimensional policies. Best-checkpoint reporting and unnamed ± terms limit uncertainty interpretation. E23E10
Source inconsistencies remain unresolved: the transform's final line contradicts its derivation; real Stack dataset counts differ across passages; and Section 6.1 gives Mobile Kitchen's low-dimensional source baseline as 2.7%, whereas Table Q.1 gives 6.7 ± 3.8%. E05E16E26
7.2 Questions for discussion
- When does an additional human demonstration add a new strategy rather than duplicate a motion that can already be retargeted?
- Can generation be optimized for downstream learning and scene coverage instead of raw acceptance rate?
- Which interpolation design preserves safe execution while remaining easy for a reactive policy to imitate?
8. Reproducibility audit
8.1 Requirements and known gaps
A reproduction needs task assets/reset samplers, source demonstrations, subtask termination and success checks, canonical frames, and controller conventions. Default simulation generation uses σ=0.05 and 5 interpolation/0 hold steps; real tasks use σ=0.02 and 25/25. Resolve the transform inconsistency before testing. E03E06E07E08E05
Appendix O reports a Volta V100, 8 CPUs and 32 GB RAM per simulation machine, sometimes sharing 2–4 runs; real experiments use an RTX 3090 machine. Exact software versions, full BC-RNN configurations, total compute and detailed Diffusion Policy training settings are not specified here. E10E18E25
Reader-proposed checks: first test frame-invariant retargeting against the printed formula; then repeat the noise-versus-replay ablation while controlling successful dataset size and measuring reset coverage. The illustrated edition specifies controls and falsifiable outcomes. E05E08E20
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test the transform invariant before running a manipulation benchmark
Reader-proposed check, not performed: construct source object poses and controller targets with a known relative transform, then introduce independent translations and noncommuting rotations of the new object. Compare the final formula printed in Section 4.2/Appendix M with the formula obtained by left-multiplying Appendix M's frame equality. Measure the residual difference between source and transformed object-relative controller poses. Include an unchanged-object control, which both candidates can pass, plus changed-pose cases that distinguish them. The invariant-preserving candidate should have numerical-zero residual; a systematic nonzero residual would falsify the claim that a candidate preserves the demonstrated relative path. Record coordinate conventions and test a short simulated segment only after the algebraic check passes. This resolves a reproduction prerequisite without claiming what the uninspected implementation does. E05E03E04
Check 2: Separate action-noise benefits from success-filtered scene coverage
Reader-proposed check, not performed: repeat Threading D0 with the same ten source demonstrations under normal MimicGen, no-noise MimicGen and noisy replay of source initial states. Collect 1,000 successful trajectories per condition across multiple generation seeds, logging every attempt and its initial state. Train image BC-RNN with matched settings and evaluate both final and best checkpoints on a fixed independent reset set. Report collection yield, policy success, occupied pose bins and source-demonstration frequencies. Then compare normal and no-noise subsets matched for occupied reset bins and sample count. If the noise advantage persists after matching coverage, scene selection alone cannot explain it; if it disappears, that weakens a direct perturbation-benefit interpretation. Keep replay's deliberately restricted initial-state support visible rather than treating it as an equivalent distribution. E08E09E10E20E23
8.3 Reading coverage
Visual audit: Visually inspected the title/author page; method, assumptions and transform pages; Figure 4 and its two comparison plots; real-robot Figure 5; transfer figures and tables; subtask/task illustrations; interpolation, action-noise, selection, pose-error and seed diagnostics; bias discussion; and Appendix O's training, evaluation and hardware details. All six final original crops were separately viewed and preserve complete relevant headers, legends and axes. Figure 2's arrows were checked against Sections 4.1–4.2 and Appendix N; the inconsistent printed transform on pages 4 and 34 is explicitly disclosed. Real-robot count/color inconsistencies were checked on pages 8, 18 and 24; the low-dimensional Mobile Kitchen discrepancy was checked on pages 6 and 41. Every page supporting retained scientific details is included above. All 45 pages were read as text; pages outside this list were not part of the visual pass. No separate supplemental images, code or external release assets were inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 18, 19, 20, 22, 23, 24, 25, 28, 29, 31, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- 1 Introduction; 2 Related Work; 3 Problem Setup
- 4 Method, including 4.1–4.2; 5 Experiment Setup
- 6 Experiments, including 6.1–6.4; 7 Limitations; 8 Conclusion
- Acknowledgments and References
- Appendices A–E: overview, contributions, FAQ, limitations, related work
- Appendices F–J: robot/object transfer, real robots, demonstrators, alternative methods
- Appendices K–O: subtasks, tasks, transform derivation, generation and policy training
- Appendices P–U: generation rates, low-dimensional policies, bias, varied sources, generation seeds and pose error
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Identity and edition: the title and all eight authors match the catalog. The inspected title page identifies CoRL 2023, Atlanta, USA. No numbered revision or revision date is printed; no other edition was supplied for comparison. This report identifies the supplied proceedings PDF by its immutable hash.
- All 13 supplied text chunks, covering PDF pages 1–45, were read individually. Selected original pages and six final crops were visually inspected; the illustrated edition lists the exact page coverage.
- The supplied text extraction does not reconstruct figure images; this limitation was addressed by inspecting the retained PDF pages and crops. Not every illustration was separately cropped.
- Separate supplemental material availability has not been fully verified. No separate supplements were supplied.
- Code, released datasets, project website and external references were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
E01PDF p. 1, title/author block, abstract and venue footer
Exact catalog title and author order: Ajay Mandlekar; Soroush Nasiriany; Bowen Wen; Iretiayo Akinola; Yashraj Narang; Linxi Fan; Yuke Zhu; Dieter Fox. Affiliations are NVIDIA and The University of Texas at Austin; Nasiriany and Wen share equal contribution. CoRL 2023 is printed; the abstract reports over 50K demonstrations across 18 tasks.
Go to primary source ↓E02PDF pp. 1–2, Sections 1–3, contributions and BC objective
Motivates reuse of similar human manipulation trajectories and defines likelihood-based behavioral cloning. Generated data are used to train policies.
Go to primary source ↓E03PDF p. 3, Figure 2 and Section 3, Assumptions 1–3
Pipeline selects, transforms and executes object-centric segments. Requires delta end-effector actions, known ordered subtasks and relevant object poses during collection, not deployment.
Go to primary source ↓E04PDF p. 4, Section 4.2, segment selection, transformation and execution
Transforms controller target sequences, adds an interpolation bridge, executes delta actions with source gripper commands, and retains only successful complete trajectories. Defines data-generation rate.
Go to primary source ↓E05PDF p. 4, Section 4.2 transform; PDF p. 34, Appendix M, both displayed equations
The stated equality is (T_W^{O'_0})^{-1}T_W^{C'_t}=(T_W^{O_0})^{-1}T_W^{C_t}. The printed final equation uses T_W^{O_0}(T_W^{O'_0})^{-1}T_W^{C_t}; its order does not follow from the stated left multiplication. Both passages were visually checked.
Go to primary source ↓E06PDF pp. 28–29, Appendix K.1–K.3 and Figures K.1–K.2
Specifies object-centric examples and automatic subtask boundary metrics; discusses manual annotation as an alternative. Square uses grasp contact and final insertion success; these metrics are not required during policy execution.
Go to primary source ↓E07PDF p. 35, Appendix N.1–N.3; p. 36, Selection Strategy; pp. 37–38, Appendix N.5–N.6
Defines seven-dimensional delta-pose/gripper actions, positional/rotational interpolation and endpoint holding, per-episode/per-subtask source selection, nearest-neighbor scoring, mobile action copying, and task-specific generation settings. Square uses nearest three with per-subtask false; real interpolation is 25/25.
Go to primary source ↓E08PDF p. 36, Table N.1 and Appendix N.4; p. 37, replay-control continuation
Gaussian action noise excludes the gripper; default scale is 0.05. Threading D0 normal/no-noise/replay-with-noise generation rates are 51.0/84.5/53.8 and image-policy success rates are 98.0±1.6/59.3±6.8/74.0±3.3. Noisy replay resets to source initial states and collects 1,000 successes.
Go to primary source ↓E09PDF pp. 4–5, Section 5, task variants and Data Generation and Imitation Learning Methodology
D0 contains source demonstrations; D1/D2 broaden or alter resets. Most tasks use 10 sources, Mobile Kitchen 25; generation continues until 1,000 successes per variant, followed by BC-RNN training. Names both simulation frameworks.
Go to primary source ↓E10PDF p. 39, Appendix O, Observation Spaces, Training Hyperparameters and Policy Evaluation
Specifies front/wrist RGB plus proprioception, low-dimensional object poses, image sizes, pixel shifts, inherited robomimic hyperparameters and low-dimensional learning rate. Simulation selects maximum checkpoint success with 50 rollouts and three training seeds; real tasks evaluate the last checkpoint over 50 episodes. Does not name the ± statistic or enumerate all inherited settings.
Go to primary source ↓E11PDF p. 6, Figure 4 left table and caption
Image-policy success rates: Square source 11.3±0.9 and D0/D1/D2 90.7±1.9/73.3±3.4/49.3±2.5; Threading source 19.3±3.4 and D0 98.0±1.6; Three Piece Assembly source 1.3±0.9 and D0 82.0±1.6. Full table spans the 16 simulation tasks.
Go to primary source ↓E12PDF p. 18, Appendix C question 11; pp. 6–7, Figure 4 comparison plots and Sections 6.2–6.3
Scale breakdown: 175 simulation source demos, 36K reset-variant demos, 12K robot-transfer demos and 2K object-transfer demos. Image comparisons find 200 generated versus 200 human demonstrations often comparable; larger generated datasets show diminishing gains from 1,000 to 5,000.
Go to primary source ↓E13PDF p. 22, Appendix F, Figure F.1 and Tables F.1–F.2
Panda sources generate separate datasets for Panda, Sawyer, IIWA and UR5e. Square D0 image-policy results are 90.7±1.9, 86.0±1.6, 80.0±4.3, 84.7±0.9; generation rates are 73.7, 55.8, 37.7, 64.7. D1 table shows lower rates, especially IIWA 48.0±3.3.
Go to primary source ↓E14PDF p. 23, Appendix G, Table G.1 and Figures G.1–G.2
Object transfer covers an unseen mug and a 12-mug set, plus pan/carrot variants. The text limits transfer to geometrically similar, similarly scaled rigid objects with aligned canonical frames.
Go to primary source ↓E15PDF p. 8, Section 6.4; p. 24, Appendix H, Real Robot Data Collection Details
Main text gives 200 Stack successes/243 attempts and 100 Coffee successes/192 attempts, 82.3%/52.1% generation, and 36%/14% BC-RNN success over 50 evaluations. Source policies score zero on narrower resets. Appendix H describes calibrated RGBD, RANSAC plane removal, DBSCAN clustering, global pose initialization and ICP refinement.
Go to primary source ↓E16PDF p. 8, Figure 5 caption versus Section 6.4; p. 18, Appendix C question 11; p. 24, Appendix H
Section 6.4 gives 200 Stack and 100 Coffee generated demos. Figure 5 describes 100 successes per task, Appendix C says 200 real demos from 20 source demos, and Appendix H refers generically to 100 real demos. These counts cannot be silently harmonized. Figure 5 caption assigns blue to D0 and orange to D1; Section 6.4 later calls the source region orange.
Go to primary source ↓E17PDF p. 24, Figure H.1 and Gap in Policy Performance between Sim and Real
Compares simulated generation with 5 interpolation steps against 25 interpolation plus 25 hold steps. Text gives Stack D1 99.3% to 68.7% and Pick Place D0 50.7% to 11.3%. The plotted Coffee D1 comparison does not show a drop; no error bars are shown.
Go to primary source ↓E18PDF p. 25, Appendix H, Improved Performance with More Flexible Policy Models
Diffusion Policy trained on the real Stack dataset scores 76% over 50 evaluations compared with BC-RNN's 36%. Detailed Diffusion Policy configuration and across-seed uncertainty are not supplied.
Go to primary source ↓E19PDF pp. 19–20, Appendix D, limitations 1–9
Discusses fixed single-object subtask structure, pose requirements, success-only filtering, interpolation collision risk, rigid-object/quasi-static scope, untested dynamics changes, restricted mobile motions, and absence of multi-arm support.
Go to primary source ↓E20PDF p. 42, Appendix R, support coverage and artifacts
Measures occupancy of discretized initial-state bins; Three Piece Assembly D1 covers 43.5%. Coarse support does not capture nonuniform counts or repetitive behavior. Successful traces may contain unnatural interpolation or collisions.
Go to primary source ↓E21PDF p. 40, Table P.1, Gear Assembly row; p. 6, Figure 4; p. 41, Table Q.1
Gear Assembly D1 generation is 8.2%; its image-policy result is 74.0±2.8 and low-dimensional result 76.0±4.9. The metrics and observation modalities must be kept separate.
Go to primary source ↓E22PDF p. 45, Appendix U, Table U.1 and caption
Uniform pose noise is applied to source and new-scene object poses: 5 mm/5° or 10 mm/10°. Coffee D1 generation is 63.5/22.6/4.3%; image-policy success is 90.7±2.5/83.3±2.5/77.3±4.1. Threading D0 generation falls to 5.2% at level 2.
Go to primary source ↓E23PDF p. 44, Appendix T and Table T.1
Main experiments use one generation seed. Additional study uses three generated datasets for Stack Three and Square variants and three low-dimensional policy seeds per dataset. Results are close, but the generality of that check is task- and modality-limited.
Go to primary source ↓E24PDF p. 31, Appendix L, Square task; p. 8, Figure 5 reset illustration
Square grasps a nut and inserts it onto a peg. D0 fixes the peg, D1 moves peg and nut, and D2 expands regions and varies peg rotation. Figure 5 illustrates analogous reset distinctions for Threading.
Go to primary source ↓E25PDF p. 39, Appendix O, Hardware
Simulation generation/training machines use a Volta V100 GPU, 8 CPUs, 32 GB memory and 128 GB disk; sometimes 2–4 runs share a machine. Real experiments use RTX 3090, 36 CPUs, 32 GB memory and 1 TB storage. Total GPU-hours and software version pins are not stated.
Go to primary source ↓E26PDF p. 6, Section 6.1 Mobile Manipulation; p. 41, Table Q.1, Mobile Kitchen row
Main text gives the Mobile Kitchen low-dimensional source score as 2.7%, whereas the explicit table row gives 6.7±3.8%; both give generated 76.7%, with ±10.5 in the table.
Go to primary source ↓8.5 Primary sources
MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations ↗
PDF · 23,358 extracted words
Source fingerprint
5f65080fac4df7f9500ccd126dbca6071ca4289c566eb03704dad8b65426c509