EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data
1. Paper overview
In one sentence: Explicit human wrist and hand supervision improves dexterous robot learning at scale, while matched human–robot experience remains essential for efficient adaptation. e-actionse-datae-alignmente-architecturee-maine-one-shote-adapters
| At a glance | What to know |
|---|---|
| Research problem | Source description Human video scales more readily than dexterous robot teleoperation, but noisy motion estimates and different hand kinematics obstruct transfer. EgoScale asks whether explicit wrist and finger supervision at large scale provides a useful manipulation prior, and how much aligned robot experience is then needed. e-identitye-actionse-data |
| Core mechanism | Source description A 20,854-hour human pretraining mixture combines diverse egocentric recordings with higher-precision EgoDex motion supervision. e-data |
| A key reported result | Five-task R1Pro dexterous manipulation: Human pretrain + midtrain: 0.56 success, 0.83 completion. Average binary success rate; average task completion score. Shirt rolling, card sorting, tong fruit transfer, bottle unscrewing and syringe transfer; 100 robot demonstrations each except 20 for shirt. Two seeds; nominally 10 trials per checkpoint, with conflicting bottle counts. No pretrain: 0.02/0.24; midtrain only: 0.28/0.53; human pretrain: 0.38/0.71 (success/completion). The plotted success gain is 54 percentage points, clarifying the abstract’s “54%.” Card success decreases from 0.70 human-only to 0.65 combined. e-identitye-evaluatione-maine-rubric |
| Reading caution | Reader analysis Bottle evaluation conflicts: Section 3.1 calls it Task III with 16 trials; Appendix B calls it Task IV and says four configurations, four trials each, but 12 total. One-shot additive rubrics can exceed their stated unit maximum; milestone exclusivity/normalization is missing. G1 dish increments sum to 0.99 although the maximum is called 1.0. e-evaluatione-rubrice-one-shot-rubrice-g1-rubric |
Core contributions
- Source description
A 20,854-hour human pretraining mixture combines diverse egocentric recordings with higher-precision EgoDex motion supervision. e-data
- Source description
Human pretraining and aligned mid-training are complementary; task-specific post-training completes the deployment recipe. e-traininge-main
- Author claim
Validation loss tracks downstream completion across five data scales; one-shot and G1 experiments suggest transferable manipulation structure within the tested settings. e-scalinge-one-shote-g1
Figure 2. Matched sensing and explicit motion targets connect human demonstrations to a shared action policy. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start on the left: the head and wrist views belong to the aligned collection setup, rather than describing every in-the-wild pretraining video. On the right, follow image and instruction encoding into the VLM, then its conditioning arrows into the DiT action expert. The separate hand-pose extraction and retargeting branch constructs human action supervision. Noisy joint and end-effector inputs enter the action encoder; decoded outputs return through the dashed iteration loop. Section 2.3 additionally specifies proprioception and embodiment adapters, which this overview simplifies. N denotes the depicted iteration count, whose numerical value is not supplied. e-actionse-alignmente-architecturee-traininge-adapters
What it supports. The learned output is a chunk of executable wrist and joint targets. Human motion becomes an explicit supervision space, and shared VLM/DiT modules allow embodiment-specific interfaces to reuse it. This architecture supports the VLA and dual-system descriptions; the diagram does not show prediction of future world states.
Where the evidence stops. The hand-extraction branch supplies training labels, not a required human-hand observation at robot inference. The flame icons are not a complete freezing schedule: Section 2.4 changes trainable modules across stages.
2. Motivation
2.1 The problem and the proposed response
Human video scales more readily than dexterous robot teleoperation, but noisy motion estimates and different hand kinematics obstruct transfer. EgoScale asks whether explicit wrist and finger supervision at large scale provides a useful manipulation prior, and how much aligned robot experience is then needed. e-identitye-actionse-data
2.2 What this reading follows
A human video contains useful manipulation behavior, but a robot needs executable motion targets. EgoScale turns estimated wrist trajectories and hand articulation into supervision for a vision-language-action policy. It first learns from a large human-video mixture, then uses a much smaller collection with matched human and robot sensing before adapting to individual tasks. The figures below separate three questions: what the policy predicts, how human-data scale changes robot performance, and what alignment contributes to transfer. Read the gains alongside their supervision budgets: the one-shot experiments also use aligned human demonstrations, and the new robot embodiment receives its own adaptation data. e-actionse-datae-alignmente-architecturee-maine-one-shote-adapters
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | VLA |
| Architecture | Dual-system |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Conflicts with the recorded classification
VLA, dual-system VLM/DiT architecture and post-training/data augmentation are supported. However, “Latent action pretraining” misdescribes explicitly supervised wrist poses and joints: internal embeddings do not establish latent-action discovery. Prediction paradigm and quadrant being not applicable are supported because the policy predicts actions without a described future-state model. e-actionse-architecturee-adapters
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Turn observed motion into the robot’s training language
The transfer begins before the neural network sees a training target. Human camera motion is estimated so wrist poses can be expressed in a common world frame; the paper’s written transform then measures each wrist pose relative to the action chunk’s reference pose. Finger motion undergoes a different conversion: an optimizer fits Sharpa joint angles under URDF joint limits, then smooths the sequence. These labels teach the policy where to move and how to articulate the hand. Reader interpretation: the supervision is useful because it is organized around control, rather than because the model discovers an unspecified latent action vocabulary. This distinction also exposes reproduction dependencies. The source’s consecutive-frame wording differs from its chunk-reference equation, and retargeting weights and smoothing settings remain unspecified. e-actionse-datae-retargete-architecture
5.2 Use scale and alignment for different learning jobs
The large human mixture provides behavioral variety, while the aligned collection makes the sensing and action interfaces more comparable. The recipe therefore separates broad pretraining from matched human–robot mid-training and then task-specific post-training. Stage I updates all parameters. Stage II preserves the vision-language backbone while updating the separately described vision encoder, DiT and, for G1, state/action adapters. Stage III adapts to the task and changes whether the vision encoder is frozen according to prior mid-training. Reader interpretation: the Figure 4 comparisons are consistent with these stages doing complementary work, but they do not isolate camera matching from motion supervision or additional optimization. The three-view capture hardware and calibrated intrinsics are part of the alignment intervention, not incidental presentation details. e-datae-alignmente-traininge-adapterse-maine-hardware
5.3 Separate prediction quality, task progress and successful execution
The scaling result links two evaluations that should remain distinct. Human validation measures action MSE after averaging stochastic predictions; robot completion scores reward task-specific milestones after physical execution. Binary success is stricter still. A policy can grasp tongs or partially fold a shirt and receive credit while failing the full instruction. Reader interpretation: agreement across these measures is encouraging, but none substitutes for the others. The one-shot experiment additionally changes the available supervision: a single target-robot demonstration is accompanied by many aligned human demonstrations. G1 transfer adds embodiment-specific play and a separate lower-body controller. Together, the experiments support a reusable prior plus adaptation, while uncertainty, exact scoring conventions and claims beyond the tested data range remain open. e-scalinge-maine-rubrice-one-shote-one-shot-rubrice-g1e-adapters
5.4 Training and inference
During training
Stage I updates all VLA parameters for 100K steps on 256 GB200 GPUs, global batch 8,192, learning rate 5×10⁻⁵. e-training
Stage II runs 50K steps, batch 2,048, learning rate 3×10⁻⁵. The vision-language backbone is frozen while the separately named vision encoder and DiT update; Appendix D.1 also lists state/action adapters. e-traininge-adapters
Stage III uses robot task demonstrations for 10K steps, batch 512, learning rate 3×10⁻⁵. The vision encoder is frozen after mid-training and unfrozen otherwise. The introduction’s two-stage transfer description thus becomes three optimization stages including post-training. e-overviewe-training
During inference
The VLM conditions iterative action generation from noisy inputs; Figure 2 labels N iterations without specifying N. End-effector and joint targets control the R1Pro arms/hands with RGB feedback; its base and torso remain fixed. e-architecturee-control
On G1, EgoScale supplies upper-body targets; a separately trained Homie controller handles lower-body balance and locomotion. No rollout of predicted future images or world states is described. e-g1e-adapters
5.5 Implementation flow
- Build explicit action labels
Estimate camera motion with SLAM and hand pose from 30-FPS video. Transform the wrist into a world frame, then express motion relative to the chunk reference. Retarget hand keypoints to 22-DoF Sharpa joints. e-actionse-data
- Constrain hand retargeting
Appendix D uses URDF kinematics and joint limits in per-frame nonlinear optimization with CasADi/IPOPT. Warm-start from the previous solution and apply exponential smoothing. Objective terms, weights and filter coefficient are unspecified. e-retarget
- Condition the action expert
Image/language embeddings condition a DiT trained by flow matching. Embodiment-conditioned MLPs encode state/noisy actions and decode hand actions; the backbone, expert and wrist representation are shared. e-architecturee-adapters
- Align sensing before adaptation
Mid-training covers 344 tabletop tasks, approximately 30 human and 5 robot trajectories each: about 50 human hours and 4 robot hours. Matched cameras, calibrated intrinsics, Vive wrist tracking and Manus hand capture narrow the domain gap. e-alignment
6. Experiments & results
EgoScale learns dexterous robot policies from action-labeled egocentric human video, then aligns them with robot sensing and control using a small human–robot play dataset. A vision-language backbone conditions a flow-based action expert predicting wrist motion and hand joints. Real-robot success improves after both human pretraining and aligned mid-training; transfer still requires robot demonstrations and embodiment-specific adaptation.
The supplied paper presents quantitative results and ablations as Figures 4–8 and contains no original quantitative table. Six faithful figure crops therefore supply the method, numerical results and ablation coverage; no replacement table was fabricated. Appendix B’s evaluation rubrics are prose rather than a result table. e-visual-forme-rubric
6.1 Read the original evidence
Figure 4. Human pretraining and aligned mid-training deliver the strongest average results, with task-specific exceptions. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the legend to track the same four conditions across both panels: hatched no-pretraining, pale mid-training only, blue human pretraining, and red combined training. The left panel rewards partial progress according to Appendix B; the right requires end-to-end task completion. Compare the Average groups only within their own metric. The red averages are 0.83 completion and 0.56 success, whereas the hatched averages are 0.24 and 0.02. Every task receives downstream robot training: 100 demonstrations except 20 for shirt rolling. These conditions therefore test the value of earlier training stages within a task-adaptation protocol. e-identitye-evaluatione-maine-rubric
What it supports. The success-rate increase from 0.02 to 0.56 is 54 percentage points. Combining the stages improves the overall average over either alone, but not every task: card success is 0.70 for human pretraining and 0.65 after adding mid-training. Partial progress and reliable completion remain distinct outcomes.
Where the evidence stops. Section 3.1 and Appendix B conflict on bottle task numbering and trial counts. No error bars or per-seed values accompany the chart, so small differences should not be treated as established statistical effects.
Figure 5. More human data improves both offline action prediction and downstream completion over the tested range. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read left to right. The training curves show that the smallest human datasets begin to overfit, while larger mixtures continue reducing validation error. The middle panel summarizes optimal validation loss against a logarithmic data axis. The right panel is a separate real-robot measurement after direct post-training, with mid-training omitted from this scaling experiment. Its labels run from 0.30 at 1k hours through 0.45, 0.48 and 0.57 to 0.71 at 20k hours. The offline metric averages 16 sampled action chunks at each of 20 timesteps per episode, over 2,000 held-out human episodes. e-maine-scaling
What it supports. The measured points support a useful association between lower human action error and higher robot completion. They do not show that an offline action sample is itself a successful robot execution. The reported R² of 0.9983 describes the fitted loss-versus-data relationship, not an independently validated robot-performance forecasting model.
Where the evidence stops. Equation (1) defines D as hours, but its printed coefficients do not reproduce the plotted losses under literal-hour substitution. The normalization remains unresolved. Preserve the observed trend without silently repairing the equation or extrapolating beyond 20k hours.
Figure 6, quantitative panels. One robot demonstration becomes more useful after aligned mid-training, with substantial additional human supervision. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The crop isolates the two quantitative panels from Figure 6. Read folding and bottle bars separately, then distinguish completion on the left from binary success on the right. Red denotes the combined human-pretraining and mid-training condition, blue omits mid-training, and pale bars omit large-scale pretraining. For folding, post-training uses one robot demonstration plus 100 aligned human demonstrations. For bottles, that allowance applies to each of three geometries. The full figure’s nearby examples compare related mid-training motions with target-task behavior, so the claimed novel-task transfer still draws on shared motion structure. e-one-shote-one-shot-rubric
What it supports. Combined training reaches 0.88 folding success and a plotted 0.556 bottle success, versus 0.35 and 0.0 for human pretraining alone. The chart supports a strong benefit from alignment, but contradicts the prose’s blanket statement that both baselines fail: folding also reaches 0.15 with mid-training alone.
Where the evidence stops. The prose gives bottle success as 0.55 rather than the chart’s 0.556. Appendix B leaves additive one-shot milestone normalization unclear. The reported setting is one-shot in robot demonstrations, not in total demonstration count.
Figure 7. Shared human-derived representations transfer to G1 after embodiment-specific adaptation. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The photographs name the two G1 tasks; each lower plot then contains both tasks, rather than assigning one entire plot to one task. Read completion on the left and success on the right. Red combines human pretraining with mid-training that includes G1 play; pale and blue are the two incomplete training recipes. Appendix D.1 explains the interface change: embodiment-specific encoders and decoders surround a shared VLM and DiT, adapting a representation pretrained with 22-DoF Sharpa targets to a 7-DoF tri-finger hand. The upper-body policy is also paired with a separate Homie lower-body controller. e-g1e-adapterse-hardwaree-g1-rubric
What it supports. Combined success is 0.67 for Pen in Bin and 0.50 for Dish in Rack, compared with 0.30 and 0.20 for either displayed baseline. These plotted gains support adapted cross-embodiment reuse. They do not establish an embodiment-independent controller that can operate a new robot without its own data.
Where the evidence stops. The stated two seeds and ten trials each do not straightforwardly yield the plotted 0.67 binary success. G1 data quantities and detailed adapter configuration are unspecified, and no seed-level uncertainty accompanies the plot.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Five-task R1Pro dexterous manipulation Shirt rolling, card sorting, tong fruit transfer, bottle unscrewing and syringe transfer; 100 robot demonstrations each except 20 for shirt. Two seeds; nominally 10 trials per checkpoint, with conflicting bottle counts. | Human pretrain + midtrain: 0.56 success, 0.83 completion. Average binary success rate; average task completion score | No pretrain: 0.02/0.24; midtrain only: 0.28/0.53; human pretrain: 0.38/0.71 (success/completion). The plotted success gain is 54 percentage points, clarifying the abstract’s “54%.” Card success decreases from 0.70 human-only to 0.65 combined. e-identitye-evaluatione-maine-rubric |
| Human-data scaling without mid-training Pretrain on 1k, 2k, 4k, 10k or 20k hours, then directly post-train on robot tasks. | 0.30, 0.45, 0.48, 0.57 and 0.71, respectively. Average task completion score | Validation uses 2,000 held-out human episodes, 20 sampled timesteps each, averaging 16 action chunks per timestep before MSE. The within-range association supports scaling, but does not separate behavioral diversity from hours or establish performance beyond 20k hours. e-maine-scaling |
| One-shot shirt folding and water-bottle unscrewing One robot plus 100 aligned human demonstrations for folding; the same allowance per bottle for three geometries. | Figure 6: 0.88 folding and 0.556 bottle; prose reports bottle as 0.55. Binary task success rate | Midtrain only: 0.15/0.0; human pretrain only: 0.35/0.0 (folding/bottle). “One-shot” counts robot demonstrations. The text’s blanket baseline-failure claim conflicts with nonzero folding success in the chart. e-one-shote-one-shot-rubric |
| Cross-embodiment transfer to Unitree G1 7-DoF tri-finger hands; G1 play during mid-training and downstream adaptation. Two seeds, 10 trials per seed as stated. | Human pretrain + midtrain: 0.67 / 0.50. Binary success for Pen in Bin / Dish in Rack | Both midtrain-only and human-pretrain-only baselines: 0.30 / 0.20. Plotted gains are 37 and 30 percentage points. This is adapted transfer; 0.67 is not straightforwardly compatible with the stated two-by-ten binary-trial denominator. e-g1e-adapterse-g1-rubric |
| Human hand-action representation ablation Card, tong and bottle; wrist-only versus fingertip trajectories with an MLP joint mapping versus retargeted joints. | Retargeted joints: 0.74 / 0.79 / 0.61. Task completion score, ordered Card / Tong / Bottle | Wrist only: 0.56 / 0.24 / 0.26; fingertip: 0.17 / 0.76 / 0.55. Joint targets perform best here, but the fingertip condition changes the mapping mechanism as well as supervision. e-hand-ablation |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 8. Retargeted joint supervision is the most consistent action representation in the tested tasks. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each task group in purple, yellow and red order: wrist-only, fingertip-based, and full retargeted joints. The metric is completion score, so a higher bar can reflect better partial progress without full success. The clearest separation is card sorting: 0.56, 0.17 and 0.74. Tong scores are 0.24, 0.76 and 0.79; bottle scores are 0.26, 0.55 and 0.61. Section 3.6 explains that fingertip supervision predicts wrist and fingertip trajectories and then uses an MLP to obtain joint commands. The retargeted condition learns the joint targets produced by constrained preprocessing. e-hand-ablatione-maine-retarget
What it supports. Full joint targets lead in all three groups, while fingertip supervision is uneven: nearly matching joints on tongs but falling below wrist-only on cards. This pattern supports the practical importance of the hand representation and its decoding, rather than a simple rule that adding more geometric targets always improves manipulation.
Where the evidence stops. The comparison also changes the action-to-joint mapping, and reports no uncertainty. Figure 8’s 0.61 bottle score differs from the human-pretraining 0.63 in Figure 4; the paper does not explain that difference.
7. Analysis & limitations
7.1 What the evidence leaves open
Bottle evaluation conflicts: Section 3.1 calls it Task III with 16 trials; Appendix B calls it Task IV and says four configurations, four trials each, but 12 total. One-shot additive rubrics can exceed their stated unit maximum; milestone exclusivity/normalization is missing. G1 dish increments sum to 0.99 although the maximum is called 1.0. e-evaluatione-rubrice-one-shot-rubrice-g1-rubric
Figures 4–8 omit error bars and seed-level outcomes. Small evaluations with controlled initial scenes limit uncertainty and generalization conclusions. Figure 8’s bottle joint score is 0.61 versus human-only 0.63 in Figure 4, without explanation. e-maine-hand-ablatione-g1e-one-shote-evaluation
Joint model/data scaling, weaker unlabeled supervision and zero-shot novel-task execution remain future directions. Diverse human training scenes do not themselves establish broad robot deployment coverage. e-scalinge-conclusione-diversity
7.2 Questions for discussion
- Does matched sensing or shared motion structure account for more of the mid-training benefit?
- Would validation MSE retain its predictive ranking under new robot tasks and a different diversity mix?
- How much of the joint-target advantage comes from retargeting constraints rather than the prediction space?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the human mixture including 829 EgoDex hours, aligned collection, task demonstrations, calibration and hand URDFs. Appendix A specifies an OAK-D-Wide head camera and two OAK-1-Wide wrist cameras; training compute is given without a complete executable configuration. e-datae-alignmente-evaluatione-traininge-hardwaree-retarget
Missing details include exact checkpoint/model size, action horizon, inference iterations/control rate, mixture weights, filtering thresholds, split identities, optimizer schedule, retargeting weights and software versions. Clarify the 21-versus-25 keypoint wording, wrist reference convention and G1 adaptation data quantity. e-architecturee-traininge-datae-retargete-adapterse-scaling
Reader-proposed checks: isolate aligned mid-training at fixed data/update budgets, and compare action representations under controlled decoder and label-quality protocols. Predeclare binary success and report per-seed outcomes; neither check was run. e-alignmente-evaluatione-hand-ablation
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate alignment from additional training
Reader-proposed experiment, not performed: start from the same human-pretrained checkpoint and compare matched human–robot mid-training with a deliberately perturbed camera-calibration/viewpoint condition using the same trajectories, update count and sampling weights. Include a no-mid-training baseline with a matched total update budget. Keep the final one-robot-plus-100-human adaptation set fixed for folding. Predeclare end-to-end success, identical initial-scene sampling and per-seed reporting. If correct alignment reliably exceeds both controls, that supports a sensing-alignment contribution; if only extra training matters, the aligned condition should lose its advantage. e-alignmente-traininge-evaluatione-one-shot
Check 2: Test whether joint targets or the mapping produce the hand advantage
Reader-proposed experiment, not performed: use the same human clips and fixed wrist targets to train wrist-only, fingertip and retargeted-joint conditions at equal compute. Add a fingertip condition whose joint conversion uses a controlled constrained solver alongside the paper’s MLP mapping. Fix downstream demonstrations and evaluate card sorting and tong transfer with binary success plus separately reported grasp milestones. Publish retargeting weights, joint-limit violations and per-seed outcomes. If constrained fingertip conversion closes the joint-target gap, mapping quality explains part of the original ablation; a persistent gap would motivate testing the supervision space itself. e-actionse-retargete-hand-ablatione-evaluatione-rubric
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version page; all method, training, evaluation and conclusion pages; Figures 1–10; and Appendices A–D.1, including camera models, task rubrics, dataset distributions and retargeting/adaptation details. All six final original crops were individually viewed with axes, legends and labels retained. Figure 2 arrows and iteration loop were cross-checked with Sections 2.1–2.4 and Appendix D.1; stage-specific freezing follows the text rather than generic flame icons. Scaling-unit, one-shot prose/chart, trial-count and rubric discrepancies are retained. Reference pages 19–22 were completely read in the supplied text; external works, project pages and supplements were not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1. Introduction
- 2. Method
- 2.1. Human Action Representation
- 2.2. Human Data Sources and Processing
- 2.3. Model Architecture
- 2.4. Training Recipe
- 2.5. Robot Systems and Control
- 3. Experiment
- 3.1. Experiment Setup
- 3.2. Large-Scale Human Pretraining Is Key to Strong Dexterous Manipulation Policy
- 3.3. Policy Performance Scales with Pretraining Data Size
- 3.4. Aligned Human-Robot Mid-Training Enables One-Shot Transfer
- 3.5. Human Pretraining Enables Cross-embodiment Transfer
- 3.6. Hand Action Space Design for Human Pretraining
- 4. Related Work
- 5. Acknowledgement
- 6. Conclusion
- A. Details of Robot System
- B. Task Descriptions and Evaluation Rubric
- C. Details of In-the-wild Human Data Curation
- C.1. Dataset Statistics and Analysis
- C.2. Category Distribution
- C.3. Environment Diversity
- C.4. Task Coverage
- C.5. Object Vocabulary
- D. Hand Retargeting Algorithm
- D.1. Additional Details on Cross-Embodiment Transfer to G1
- References
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Identity/version: the title and all 15 authors match the catalog. The inspected artifact identifies itself as arXiv:2602.16710v1, 18 February 2026; its title-page header separately reads 2026-2-19. No later revision or separate edition was supplied or compared.
- All six supplied text chunks were read individually, including the complete body, Appendices A–D.1 and references on PDF pages 1–22. PDF pages 1–18 and Figures 1–10 were visually inspected; reference pages 19–22 were read as text.
- The acquisition warned that text extraction does not reconstruct figure images. This limitation was addressed by inspecting the retained PDF and all six final crops.
- Separate supplemental material availability has not been fully verified; no separate supplement was supplied.
- The linked project, code, datasets and external references were not opened. No paper scripts were executed and no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title block, arXiv margin, Abstract and Introduction
Exact EgoScale title and 15-author list; NVIDIA, University of California, Berkeley and University of Maryland. Margin states arXiv:2602.16710v1, 18 Feb 2026; header states 2026-2-19. Abstract says success improves by 54%.
Go to primary source ↓e-overviewPDF p. 2, Figure 1 and caption; p. 5, Section 2.4
Figure 1 depicts pretraining, mid-training and post-training while its caption calls the transfer framework two-stage; Section 2.4 explicitly specifies three optimization stages.
Go to primary source ↓e-actionsPDF p. 3, Section 2.1, Raw Sensor Streams, Wrist-level Arm Motion and Hand Articulation
Defines camera/world transforms, 21 keypoints, wrist index 1 and chunk-reference relative wrist transform; retargets to 22-DoF Sharpa joints. Prose calls wrist changes consecutive despite the time-0 equation.
Go to primary source ↓e-dataPDF pp. 3–4, Section 2.2, Stage I
20,854 total hours, diverse in-the-wild scenes, 30-FPS RGB, off-the-shelf SLAM/hand estimation, and 829 EgoDex hours covering 194 tasks. Precise processing configuration and mixture sampling weights are not supplied.
Go to primary source ↓e-alignmentPDF p. 4, Figure 2(a), caption and Section 2.2, Stage II
344 tasks, about 30 human and 5 robot trajectories each, 50 human and 4 robot hours. Matching cameras/intrinsics, Vive wrist poses and Manus 25-joint transforms synchronized with video.
Go to primary source ↓e-architecturePDF p. 4, Figure 2(b) and caption; p. 5, Section 2.3
VLM encodes image/language; DiT uses flow matching for action chunks. Figure shows noisy joint/EEF inputs and an N-iteration loop. Missing human proprioception uses a learnable placeholder; embodiment input/output MLPs surround shared modules.
Go to primary source ↓e-trainingPDF p. 5, Section 2.4
Stage I: 100K steps, 256 GB200 GPUs, batch 8,192, learning rate 5e-5, all parameters trainable. Stage II: 50K, batch 2,048, 3e-5; backbone frozen, vision encoder/DiT updated. Stage III: 10K, batch 512, 3e-5; vision freezing depends on prior mid-training.
Go to primary source ↓e-controlPDF p. 5, Section 2.5
R1Pro base/torso fixed; two 7-DoF arms receive relative end-effector commands; 22-DoF Sharpa hands receive joint targets. Head and two wrist RGB cameras supply observations.
Go to primary source ↓e-evaluationPDF pp. 6–7, Section 3.1, Tasks and Evaluation Metric
Five tasks use 100 demonstrations except shirt rolling with 20. Two training seeds, nominally 10 trials per checkpoint, image-overlay initialization. Bottle-specific four-by-four exception is labeled Task III.
Go to primary source ↓e-mainPDF p. 7, Figure 4(a–b), Average, Card and Bottle groups; Sections 3.2–3.3
Average completion/success: no pretrain 0.24/0.02, mid-only 0.53/0.28, human-only 0.71/0.38, combined 0.83/0.56. Card success 0.70 human-only versus 0.65 combined. Bottle human-only completion 0.63. Scaling checkpoints omit mid-training.
Go to primary source ↓e-scalingPDF p. 8, Figure 5, Section 3.3 and Eq. (1); p. 9, opening continuation
Completion at 1k/2k/4k/10k/20k hours is 0.30/0.45/0.48/0.57/0.71. Validation averages 16 samples at each of 20 timesteps from 2,000 episodes. Fit L=0.024−0.003 ln(D), R²=0.9983 defines D as hours, inconsistent with plotted losses under literal substitution. Continuation disclaims extrapolation.
Go to primary source ↓e-one-shotPDF p. 9, Figure 6 right-hand charts, caption and Section 3.4
One robot plus 100 aligned human demonstrations per object; three water bottles. Folding success mid-only/human-only/combined: 0.15/0.35/0.88; bottle: 0/0/0.556. Completion: 0.25/0.65/0.92 and 0.10/0.10/0.64. Prose gives bottle 0.55 and overstates baseline failure. Related motions exist in mid-training.
Go to primary source ↓e-g1PDF pp. 9–10, Section 3.5 and Figure 7
G1 has 7-DoF tri-finger hands. Combined completion 0.83/0.88 and success 0.67/0.50 for Pen in Bin/Dish in Rack; mid-only completion 0.45/0.54, human-only 0.68/0.58; both baselines success 0.30/0.20. Text states two seeds, 10 trials each and Homie lower-body control.
Go to primary source ↓e-hand-ablationPDF p. 10, Figure 8; p. 11, Section 3.6
Card/Tong/Bottle completion: wrist-only 0.56/0.24/0.26, fingertip 0.17/0.76/0.55, joints 0.74/0.79/0.61. Fingertip predicts wrist/fingertip SE(3) trajectories then maps to joint commands with an MLP. Authors describe unstable grasps and mapping errors.
Go to primary source ↓e-conclusionPDF p. 12, Section 6, final paragraph
Model/data scaling, weaker/unlabeled video objectives and possible zero-shot novel-task execution are future directions.
Go to primary source ↓e-hardwarePDF p. 13, Appendix A, Figure 9 and caption
R1Pro uses Sharpa hands; G1 uses tri-finger hands. Both have two OAK-1-Wide wrist cameras and an OAK-D-Wide head camera.
Go to primary source ↓e-rubricPDF pp. 13–14, Appendix B, R1 Pro Sharpa rubrics
Completion uses task-specific progress or additive milestones. Tong uses lemon/plum, five trials each. Bottle is Task IV: four configurations, four trials each, but 12 total stated. Syringe has seven additive substeps.
Go to primary source ↓e-one-shot-rubricPDF pp. 14–15, Appendix B, Tasks VIII–IX and binary-success definition
Folding adds 0.4, 0.4, 0.3 with messy-fold penalty. Bottle lists 0.1, 0.2, 0.5, 0.2, 0.2. Naive addition exceeds the stated [0,1] range; milestone exclusivity/normalization is unspecified. Binary success requires end-to-end completion.
Go to primary source ↓e-g1-rubricPDF p. 15, Appendix B, G1 Tasks
Pen in Bin is described as marker/canister manipulation. Dish assigns 0.11 per pickup, transfer and placement for three plates but calls the maximum 1.0; literal sum is 0.99.
Go to primary source ↓e-diversityPDF pp. 15–17, Appendix C.1–C.5 and Figure 10
Long-tailed category, environment, task and object distributions. Category/environment/task plots show groups above 50 hours. Broad human coverage is not a robot generalization evaluation.
Go to primary source ↓e-retargetPDF p. 17, Appendix D and D.1 Shared Wrist Action
Text says 21 human keypoints represented as 25, and 20 robot keypoint poses. Optimizes 22 joints with URDF limits, CasADi/IPOPT, previous-frame warm start and exponential smoothing; objectives/weights/filter settings absent. D.1 calls wrist changes frame-to-frame.
Go to primary source ↓e-adaptersPDF pp. 17–18, Appendix D.1
Embodiment-specific state/noisy-action encoders and joint decoders surround shared VLM/DiT. G1 play participates in mid-training; vision encoder, expert and state/action adapters update while vision-language backbone is frozen. G1 data quantity and detailed adapter configuration are absent.
Go to primary source ↓e-visual-formPDF pp. 7–10, Figures 4–8; pp. 13–18, Appendices A–D.1
Main results, one-shot, G1 transfer and action ablation appear as plots. The supplied paper contains no quantitative result table; appendices provide prose rubrics, system images and dataset plots.
Go to primary source ↓8.5 Primary sources
EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data ↗
PDF · 9,244 extracted words
Source fingerprint
39c691baf374a154e26ffc0098b97037875909495013d4017d97761716ff2735