EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Data
1. Paper overview
In one sentence: Action-guided optimal transport improves human-to-robot policy transfer by aligning observation features during training, while relying on tracked human actions and task-specific correspondence. problempairingobjectivereal-resultsuncertaintyablation
| At a glance | What to know |
|---|---|
| Research problem | Source description Human demonstrations can expand scene and behavior coverage, but appearance, kinematics and missing wrist-camera modalities create covariate shift. Sharing policy weights does not guarantee compatible latent features. EgoBridge aims to preserve action information while aligning domains. problemarchitecture |
| Core mechanism | |
| A key reported result | Drawer: overall and held-out quadrant: 47% overall, 77 points; 33% held-out-quadrant SR. Full-task success rate; total subtask points. 144 robot demonstrations in three quadrants; human coverage of all four. Evaluation: 48 rollouts, two per drawer. Co-train: 22% overall, 55 points, 0% held-out; ATM: 8% held-out. Physical transfer to unseen robot locations. Overall and held-out rates have different denominators; 47% is not a separately reported in-distribution rate. drawer-settingreal-results |
| Reading caution | Source description Demonstrated transfer is single-task. DTW can confuse stages or spatial targets, especially under extreme viewpoints; multitask training, multiple embodiments and unlabeled Internet video remain future extensions. limitationspair-diagnostics |
Core contributions
- Source description
Physical and simulated evaluations test settings shared by training domains and settings demonstrated only by the source embodiment. drawer-settingcoffee-settingsim-protocol
Figure 2. Alignment acts on encoded context; action supervision trains the complete policy. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read upward from the sensor stems. Purple marks components used by human and robot data; the blue wrist branch is available only to the robot. The shared trunk mixes modality tokens with learned context tokens. Its output context is both the input to the OT objective and the conditioning information supplied to the decoder. Learned action tokens enter the decoder separately. The loss labels describe training: the caption and Algorithm 1 specify that OT updates the encoder, whereas BC updates encoder and decoder. The wrist switch is consistent with a missing human modality, not evidence of a frozen branch. Appendix B further specifies embodiment-specific proprioceptive stems beneath this simplified diagram. architectureobjectivepairingreal-traininghuman-dimensions
What it supports. The method adds an alignment objective at the representation used for action prediction. It does not need to predict a future image or reconstruct a human trajectory at robot inference time. The robot still uses its wrist observations, and the shared decoder emits Cartesian pose-plus-gripper chunks conditioned on encoded observations.
Where the evidence stops. The graphic simplifies the input interfaces. Table 3 lists human pose orientation, but Appendix B says human supervision contains only xyz. Preserve that source inconsistency; the figure alone cannot resolve the exact human proprioceptive dimensionality.
2. Motivation
2.1 The problem and the proposed response
Human demonstrations can expand scene and behavior coverage, but appearance, kinematics and missing wrist-camera modalities create covariate shift. Sharing policy weights does not guarantee compatible latent features. EgoBridge aims to preserve action information while aligning domains. problemarchitecture
2.2 What this reading follows
A human and a robot can perform the same manipulation while producing very different camera views and motion timings. EgoBridge asks how their demonstrations can teach one policy without letting those differences separate its internal representations. The central move is to use action trajectories to decide which observations should be inexpensive to align. Dynamic Time Warping identifies related motions; optimal transport then shapes the observation encoder while behavior cloning preserves action prediction. The figures below connect that training mechanism to physical drawer, coffee and folding evaluations. Read the positive transfer results alongside the small evaluation budgets, missing uncertainty estimates and explicit disagreements between some tables and their surrounding prose. problempairingobjectivereal-resultsuncertaintyablation
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Foundational work |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
This table preserves the labels recorded at reading time. The current major category is Related resources. View the current classification.
3.1 Evidence-based assessment
Conflicts with the recorded classification
Foundational work and Action policy foundations fit; the Not applicable world-model quadrant is supported because there is no future-observation model, joint future/action predictor or inverse-dynamics action extraction. Latent action pretraining is unsupported: the latent encodes observations and measured actions supervise co-training. Shared encoder/decoder weights do not establish a One Model world-action architecture. problemarchitectureobjectivesim-training
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 First make human motion comparable to robot commands
The shared latent space depends on preprocessing before any optimal transport is computed. A robot command starts in joint space; forward kinematics and camera calibration express its label in the egocentric frame. Human tracking is harder because the camera moves with the head. EgoBridge projects future hand positions into the camera frame at the beginning of each chunk, giving that chunk a stable reference. It then normalizes proprioception and actions separately for each embodiment. The two chunks have the same number of samples but different physical durations: human trajectories span 0.9 seconds after interpolation, while robot trajectories span two seconds. This motivates tolerance to temporal shifts. The human supervision mask is also essential: the described loss supplies xyz targets rather than assuming the human recordings contain robot gripper and orientation labels. data-frameshuman-dimensionsreal-training
5.2 Use actions to shape a soft transport problem
During training, encode a human minibatch and a robot minibatch into observation features. Compare their demonstrated action sequences with DTW, then select the closest human trajectory for each robot trajectory. That selection does not replace the transport solver. Instead, multiply the selected pair’s squared feature distance by a small discount; all other pairs retain their feature distances. Sinkhorn must still distribute mass subject to both minibatch marginals, so a preferred pair is not a guaranteed exclusive match. The resulting cost encourages behavioral correspondence in the feature space. Behavior cloning remains active on both domains and trains the action decoder. At deployment, the robot supplies observations to the trained encoder and decoder; the action-based pairing calculation is no longer part of the policy’s observation-to-action path. otpairingobjectivearchitecture
5.3 Test transfer at the boundary of robot experience
The most informative comparison is not simply whether more demonstrations improve a training-domain task. Drawer withholds one spatial quadrant from robot demonstrations while retaining human examples there. Coffee withholds the grinder and harder scene from robot data. Reader interpretation: these protocols ask whether the policy can use human coverage to extend executed robot behavior, rather than merely fit the common training region better. Co-train is therefore a crucial control because it sees the same cross-domain information without the alignment objective. EgoBridge’s positive held-out rates and the Drawer ablation support the proposed mechanism. They do not isolate every causal factor or establish broad task generality. The supplemental Laundry tests add appearance shifts, while the authors explicitly leave multitask alignment and unlabeled human video to future work. drawer-settingcoffee-settingreal-resultsablationextra-laundrylimitationsbaselines
5.4 Training and inference
During training
Real training uses robot Smooth L1 losses on position/gripper and half-weighted orientation MSE; human loss masks everything except xyz. Alpha is 0.7. AdamW uses learning rate 5×10⁻⁵, batch size 32 and Sinkhorn blur 0.05. One L40s trains for 100k/110k/120k iterations on Drawer/Laundry/Coffee; about 24 hours is reported. real-training
PushT uses a ResNet-18/UNet diffusion policy, one observation and 16-step action chunks. Alpha is 0.2, blur 0.01; training takes 130k iterations on one A40, reported as roughly two hours. No separate frozen EgoBridge training stage is described. sim-trainingobjective
During inference
Real inference maps current robot observations through encoder and decoder; DTW/Sinkhorn operate during training. Simulation denoises Gaussian noise conditioned on observations. Real deployment replanning frequency and the Cartesian-to-joint execution interface are unspecified. architecturereal-trainingsim-trainingdata-frames
5.5 Implementation flow
- Put demonstrations into comparable frames
Robot command labels undergo forward kinematics and camera calibration. Future human hand positions are transformed into the current camera frame. Per-embodiment z-score normalization reduces scale differences. Equal 100-sample chunks still span different durations: 0.9 seconds for interpolated human motion and two seconds for robot motion. data-frames
- Encode available modalities
A shared image stem and separate proprioceptive stems feed a shared transformer trunk; wrist stems are robot-only. Eight output context tokens summarize observations. The real decoder attends to this context through alternating self- and cross-attention and predicts Cartesian action chunks. architecturereal-training
- Select behavioral correspondences
For every robot action chunk in the minibatch, select the human chunk with minimum DTW distance. Discount that pair’s squared latent distance, leaving other costs unchanged. This creates a preference within mass-constrained transport, rather than an exact one-to-one assignment. pairingot
- Optimize alignment and imitation
Sinkhorn computes a soft coupling from shaped costs. Its transport loss trains the encoder; behavior cloning trains encoder and decoder. Human labels guide training correspondences rather than supplying an inference-time trajectory database. objectivepairing
6. Experiments & results
EgoBridge co-trains robot policies with labeled egocentric human demonstrations, using motion similarity to guide optimal-transport alignment of observation features. Its contribution is training-time representation transfer into executed manipulation policies. Evidence includes transfer to robot-unseen drawer locations and visual settings, with unresolved implementation details and unreported uncertainty.
6.1 Read the original evidence
Figure 4. Human-only coverage defines the transfer test. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the legend, then compare the training row with the rollout row in each task column. Purple indicates settings represented in both human and robot demonstrations; red indicates human-only coverage. In Drawer, the red region is the top-right quadrant absent from robot demonstrations. Coffee changes the target object and, in the harder evaluation, the scene. Laundry in this main figure is an in-distribution bimanual task. The image therefore maps the meaning of the result columns rather than giving success measurements itself. The later white-table and shirt-color Laundry tests belong to Appendix J and should not be folded into this main figure’s protocol. drawer-settingcoffee-settinglaundry-settingreal-resultsextra-laundry
What it supports. Table 1’s 33% Drawer behavior-generalization rate concerns locations shown by humans but withheld from robot demonstrations. Coffee’s 27% scene-plus-object result similarly tests a setting absent from robot data. These are meaningful cross-domain transfers, although both evaluations remain within the paper’s specified manipulation tasks.
Where the evidence stops. Human-only does not mean unseen by the whole training process. Human demonstrations explicitly cover these settings. The layout cannot establish transfer to arbitrary new tasks, and the main Laundry results are not an out-of-distribution test.
Table 1. Physical success improves, but each column has its own task and denominator. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the column groups before comparing rows. Coffee has three success-rate columns: the original setting, a new object, and a new scene plus object. Drawer mixes total subtask points, total full-task success, toy-placement success and held-out-quadrant success. Laundry again pairs subtask points with full-task success. A value before the vertical bar in a points column is not a percentage. The bottom row is EgoBridge; compare within a column to Co-train to assess the benefit beyond pooling human and robot data. The evaluation budgets differ: Coffee uses 15 rollouts per setting, Drawer 48 overall, and main Laundry 18. real-resultsdrawer-settingcoffee-settinglaundry-settingbaselinesuncertainty
What it supports. EgoBridge reaches 72% Laundry success versus 33% for Co-train and EgoMimic, a 39-percentage-point gain. Its Drawer total success is 47% versus Co-train’s 22%. On human-only settings, Drawer reaches 33% and Coffee scene-plus-object reaches 27%. These results concern actual robot task completion rather than generated-video quality.
Where the evidence stops. Confidence intervals are absent. Baselines are the paper’s adapted implementations, including a modified ATM. The 44-point headline compares Laundry against 28% baselines; it is neither an average improvement nor the margin over the strongest Laundry baseline.
Table 7. The controlled pushing benchmark separates pooling data from choosing the alignment loss. Original paper, p. 21 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Every cell contains mean reward followed by success rate. Reward is the maximum overlap with the target shape during a rollout, averaged across evaluation seeds. Read left to right as the triangle target pusher encounters the base setting, a changed background, then both a changed background and mirrored T configuration. The circle source provides the broader training coverage. Compare EgoBridge with Co-train to isolate the reported gain over shared-data training, and with Standard OT or MMD to compare alignment objectives under the common ResNet/UNet policy family. Appendix H specifies 100 shared evaluation seeds, selected using a fixed random-sampling seed. sim-protocolsim-resultssim-trainingbaselinesuncertainty
What it supports. In the hardest setting, EgoBridge reaches 0.6520 mean reward and 39% success, versus Co-train’s 0.6214 and 31%. Standard OT gives 8% success and MMD 14%. Thus generic distribution alignment can be worse than simple co-training, while the action-guided version improves the reported transfer result.
Where the evidence stops. The circle is a simulated source embodiment, not recorded human behavior. Main text defines success with reward ≥0.9, but Appendix H says >0.9. These are evaluation seeds, not evidence of independent training repetitions or statistical significance.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Drawer: overall and held-out quadrant 144 robot demonstrations in three quadrants; human coverage of all four. Evaluation: 48 rollouts, two per drawer. | 47% overall, 77 points; 33% held-out-quadrant SR. Full-task success rate; total subtask points | Co-train: 22% overall, 55 points, 0% held-out; ATM: 8% held-out. Physical transfer to unseen robot locations. Overall and held-out rates have different denominators; 47% is not a separately reported in-distribution rate. drawer-settingreal-results |
| Scoop Coffee: in-distribution, object and scene transfer 50 robot demonstrations of a can in one scene; human data add a grinder and another scene. Each setting uses 15 rollouts across five target positions. | 67% / 60% / 27%, respectively. Success rate | EgoMimic: 60% / 53% / 0%; Co-train: 53% / 46% / 0%; robot-only BC: 33% / 40% / 7%. Joint object/scene transfer remains difficult. Robot-only BC’s 7% means saying all baselines fail is too broad. coffee-settingreal-results |
| Laundry: bimanual folding 300 robot and approximately 700 human demonstrations; 18 physical evaluations. | 72%; 48 points. Full-fold success rate; subtask points | Co-train and EgoMimic: 33% SR; robot-only BC, MimicPlay and ATM: 28%. The gain is 39 percentage points over the strongest listed human-augmented baselines and 44 over the 28% baselines; 44 is not an across-task average. laundry-settingreal-results |
| PushT: base, purple background and mirrored-T transfer Triangle target pusher; 100 common evaluation seeds per setting. Circle source data supply broader appearances and motions. | 0.7605 / 53%; 0.7206 / 48%; 0.6520 / 39%. Mean maximum goal IoU / success rate | Co-train: 0.7062 / 48%; 0.6899 / 42%; 0.6214 / 31%. Hardest-setting SR: Standard OT 8%, MMD 14%, target-only 0%. Hardest-setting improvement over co-training is eight percentage points. The 53-to-39 decline is 14 points. Main text uses ≥0.9 for success, whereas Appendix H says >0.9. sim-protocolsim-results |
| Drawer: action-aware alignment ablation Table 2 compares DTW pairing, MSE pairing, marginal OT and no alignment. | EgoBridge: 47% / 33%. Drawer SR / behavior-generalization SR | MSE: 14% / 17%; Standard-OT: 33% / 17%; Co-train: 22% / 0%. The table supports DTW-guided alignment. Adjacent prose gives MSE Drawer as 17% and calls the first column in-distribution; preserve the table’s 14% and the unresolved labeling conflict. ablationreal-results |
| Laundry: additional appearance generalization New white surface, then white surface plus human-only shirt color; 30 extra minutes of human data; 20 rollouts over five positions per setting. | 50% / 43; 30% / 35. Success rate / subtask points | Co-train: 25% / 25; 5% / 7. Both improve by 25 percentage points, under a supplemental protocol distinct from the 18-rollout main Laundry evaluation. extra-laundry |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 9. Inspect the motion correspondence that supplies alignment supervision. Original paper, p. 22 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare the DTW gallery on the left with MSE on the right, moving through Drawer, Coffee and Laundry from top to bottom. Each example places a robot view beside a human view with action trajectories overlaid. Examine whether the motions address similar targets and similar task phases; visual similarity of the room alone is not the pairing criterion. Section 4.2 computes DTW on action sequences and discounts the corresponding latent cost. The overlaid image traces help a reader interpret those numerical pairings, but are not themselves the matching algorithm. Appendix I describes these as randomly sampled minibatch pairs and explains several failure cases. pairingpair-diagnostics
What it supports. The examples make the intended advantage of temporal warping tangible: related actions can proceed at different rates, while pointwise MSE can prefer a different phase or target. The authors also show imperfect DTW matches and acknowledge trajectories spanning multiple stages and extreme viewpoints as sources of ambiguity.
Where the evidence stops. This is qualitative evidence, not a measured pairing-accuracy benchmark. The colored borders have no explicit legend or numerical scoring rule in the supplied caption. Do not turn their counts into success rates or infer population frequencies from this gallery.
Table 2. Both behavioral pairing and alignment affect Drawer transfer. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read downward through increasingly different training objectives. EgoBridge uses DTW to shape transport costs. MSE changes the behavioral pairing metric; Standard-OT aligns marginal features without DTW cost shaping; Co-train removes the auxiliary alignment objective. Compare the two columns separately: the first is labeled Drawer SR, and the second is behavior-generalization SR. The 47% and 22% first-column values match Table 1’s total Drawer results. The adjacent prose calls them in-distribution, so that stronger interpretation is not adopted here. It also states an MSE value of 17% for the first column, whereas the actual table clearly prints 14%. ablationreal-resultspairingbaselinesuncertainty
What it supports. Using the printed table, DTW-based EgoBridge exceeds MSE pairing by 33 points on Drawer SR and 16 points on behavior generalization. Standard OT retains some benefit over Co-train, but reaches only 17% behavior generalization versus EgoBridge’s 33%. Action-aware cost shaping is therefore supported by a mechanism-relevant comparison.
Where the evidence stops. The source’s prose/table disagreement remains unresolved. No repeated-training variance or statistical test is supplied, and these ablations do not measure how often pseudo-pairs are semantically correct. Treat them as reported performance evidence, not a proof of the proposed causal account.
7. Analysis & limitations
7.1 What the evidence leaves open
Demonstrated transfer is single-task. DTW can confuse stages or spatial targets, especially under extreme viewpoints; multitask training, multiple embodiments and unlabeled Internet video remain future extensions. limitationspair-diagnostics
No confidence intervals or significance estimates accompany results. T-SNE and nearest neighbors support the representation hypothesis but do not establish causality; the source differs on whether those displays use action tokens or encoder outputs. uncertaintylatent-diagnostics
Human dimensionality is inconsistent: Table 3 lists xyz plus Euler angles, while the decoder description says only 3D positions exist. Coffee human duration is two hours in Section 5.2 but 180 minutes in Appendix G/Table 6. These are unresolved discrepancies. human-dimensionscoffee-setting
7.2 Questions for discussion
- How much transfer survives controlled changes to human/robot temporal normalization?
- Does action-guided alignment remain useful when similar motions correspond to different tasks?
8. Reproducibility audit
8.1 Requirements and known gaps
Physical replication requires calibrated Aria frames, tracked hands, ViperX followers, WidowX leaders and wrist cameras. Exact splits, normalization statistics and controller implementation are prerequisites for comparing action-space transfer. hardwaredata-frames
Algorithm 1 names λ and ε, but configuration tables omit numerical λ and give GeomLoss blur without an explicit ε mapping; diffusion sampling steps and software versions are unspecified. PushT’s 350 listed demonstrations cover five configurations despite G.2 saying four. pairingreal-trainingsim-trainingsim-protocol
Respect baseline adaptations, including non-autoregressive ATM. The supplied edition promises later code/data release and mentions a supplementary proof not identified in the supplied material. Neither actual release nor a theoretical transfer guarantee is established here. baselinesreleaseproof-scope
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does behavioral cost shaping beat equally tuned marginal alignment?
Reader-proposed check, not an experiment performed here: rebuild the listed 350-demonstration PushT mixture and use the paper’s fixed evaluation seeds. Hold architecture, preprocessing, sampling, training budget and initialization seeds constant across Co-train, Standard OT, DTW-shaped OT and MSE-shaped OT. Repeat each configuration across several independent training seeds. Because numerical λ is unspecified, preregister the same discount grid for DTW and MSE and select configurations without using final transfer-test outcomes; include λ=1 as the unshaped control. Report each setting’s reward and SR with training-seed uncertainty, and test both documented threshold conventions. If DTW does not reliably exceed tuned marginal OT and Co-train on mirrored-purple evaluation, the reported action-aware advantage would be weakened. pairingsim-trainingsim-protocolsim-resultsablationuncertainty
Check 2: Does DTW tolerate timing differences while preserving task phase?
Reader-proposed check, not a completed reproduction: keep Drawer’s three-quadrant robot split and human coverage fixed, then compare DTW and MSE pairings under controlled temporal resampling of human trajectories. Include the published 0.9-second human/two-second robot construction as the control. On held-out demonstration pairs, independently annotate grasp, transport and closure phases and target drawer; measure phase and target agreement before retraining policies with otherwise identical settings. Evaluate full-task success separately for familiar drawers and the human-only quadrant, retaining matched rollout starts. DTW should retain pairing agreement and transfer success under modest timing changes. If it aligns phases no better than MSE, or if better pairing fails to improve robot success, the temporal-correspondence explanation needs qualification. data-framespairingpair-diagnosticsdrawer-settingablation
8.3 Reading coverage
Visual audit: All nine text chunks covering the complete 31-page PDF were read. Original page images were inspected for the title, all ten figures, all eight tables, method equations, Algorithm 1, architecture and training details, hardware and data processing, evaluation protocols, supplemental experiments, limitations, and relevant checklist disclosures. All six final original crops were inspected and retain their headers, legends or qualitative markings. Figure 2’s loss placement and wrist branch were cross-checked against its caption, Algorithm 1 and Appendix B. Table 2’s 14%/17% discrepancy and total/in-distribution labeling conflict are explicitly preserved. Reference pages and the remaining checklist pages were read as text; they supply no additional retained method or numerical claims. Separate supplements, project videos, code and data artifacts were outside this supplied reading; the checklist’s claimed supplementary proof was not identified in the supplied appendices.
PDF pages inspected for this edition: 1, 3, 4, 5, 6, 7, 8, 9, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 27. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; Sections 1–7, pp. 1–10
- References, pp. 10–12
- Appendix A: Algorithm Pseudocode, pp. 12–13
- Appendix B: Network Architecture, pp. 13–16
- Appendix C: Baselines, pp. 16–18
- Appendices D–F: robot/human data and alignment, pp. 18–19
- Appendix G: task details, pp. 19–21
- Appendices H–K: supplementary results and diagnostics, pp. 21–23
- Complete NeurIPS checklist, pp. 24–31
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Identity/version note: the inspected NeurIPS 2025 conference PDF matches the catalog title and author identities; it spells Lawrence Y. Zhu with a middle initial missing from the catalog. No separate revision number or revision history is supplied, so equivalence to another edition is not established.
- All nine supplied chunks were read individually, including references and checklist. Text extraction does not reconstruct figure images; all ten figures and eight tables were subsequently inspected in the retained PDF.
- Separate supplemental material availability has not been fully verified. The supplementary proof mentioned in checklist item 3 was not supplied as a separately identifiable proof.
- Code, external project pages, videos and data files were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title, author block and conference footer
The title matches the catalog. Authors are Ryan Punamiya, Dhruv Patel, Patcharapong Aphiwetsa, Pranav Kuppili, Lawrence Y. Zhu, Simar Kareer, Judy Hoffman and Danfei Xu, all at Georgia Institute of Technology. The footer identifies NeurIPS 2025. The catalog omits Zhu’s middle initial. No separate revision number appears.
Go to primary source ↓problemPDF p. 4, Section 3.3
A shared observation encoder and action policy are trained from labeled human and robot demonstrations despite latent covariate shift. Observation and behavior generalization are distinguished.
Go to primary source ↓otPDF p. 3, Section 3.1; pp. 4–5, Section 4.1
Entropy-regularized optimal transport supplies a differentiable coupling cost for latent/action joint alignment. This loss acts on the encoder.
Go to primary source ↓pairingPDF p. 5, Section 4.2, DTW and shaped-cost equations; p. 13, Algorithm 1, lines 7–20
For each robot j, i*(j) minimizes DTW over human trajectories. Squared latent Euclidean distance is multiplied by a small positive lambda only for selected pairs. Sinkhorn uses uniform minibatch masses. The phrase row-wise disagrees with the indexed argmin over i for fixed j; Algorithm 1 confirms the indexed rule.
Go to primary source ↓objectivePDF p. 5, Section 4.3; p. 13, Algorithm 1, lines 21–24
Training combines behavior cloning on both domains with alpha times joint OT. Both encoder and decoder receive BC gradients, while OT shapes the encoder.
Go to primary source ↓architecturePDF p. 6, Figure 2 and Shared Policy Architecture; pp. 14–15, Appendix B.1
A shared image stem, embodiment-specific proprioceptive stems and robot-only wrist stems feed the shared transformer trunk. OT acts on output context tokens, which condition a shared action decoder. The figure’s wrist switch corresponds to modality availability. Its caption assigns OT to the encoder and BC to the entire network.
Go to primary source ↓real-trainingPDF p. 14, Table 3; p. 15, Appendix B.1.2 and Table 4
Real settings include eight context tokens, 100 action tokens, a 16-block trunk and eight-block decoder. Robot loss is Smooth L1 on xyz and gripper plus 0.5 MSE on Euler angles; human loss is Smooth L1 on xyz. Alpha is 0.7. AdamW uses learning rate 5e-5, weight decay 0.0001, linear scheduling, batch size 32, ColorJitter and ImageNet normalization. GeomLoss Sinkhorn blur is 0.05. One L40s trains Drawer/Laundry/Coffee for 100000/110000/120000 iterations; about 24 hours is reported.
Go to primary source ↓human-dimensionsPDF p. 6, Egocentric Human Data; p. 14, Table 3, human proprioception row; p. 15, masked human loss; p. 18, Appendix E
Main text uses SE(3) notation and Table 3 lists human xyz plus Euler angles, but B.1.2 says human data contain only 3D positions and supervises only xyz. Appendix E describes positions while using SE(3) notation. This discrepancy is unresolved.
Go to primary source ↓data-framesPDF p. 18, Appendices D–E, data-processing and human-action equations; p. 19, Appendix F
Robot command labels undergo forward kinematics and camera calibration. Future human hand positions are projected into the current camera frame using device poses. Per-embodiment, per-dimension z-score normalization is used. Both real chunks contain 100 samples: human chunks interpolate ten samples spaced three frames apart at 30 Hz over 0.9 seconds; robot chunks use 100 successive samples at 50 Hz over two seconds.
Go to primary source ↓hardwarePDF p. 18, Appendices D–E; p. 19, Figure 6
The robot uses two ViperX 300s followers, two WidowX teleoperation leaders, Project Aria egocentric imaging and Intel RealSense D405 wrist cameras. Human recordings use Aria and its Machine Perception Services.
Go to primary source ↓sim-trainingPDF pp. 15–16, Appendix B.2 and Table 5
PushT uses 96×96 RGB, normalized 2D proprioception, one observation and 16-action chunks. ResNet-18 conditions a UNet diffusion policy. OT acts on concatenated image/proprioceptive features. Inference denoises Gaussian noise into trajectories. One A40 trains for 130000 iterations, approximately two hours; AdamW learning rate is 1e-4, batch size 32, alpha 0.2 and Sinkhorn blur 0.01.
Go to primary source ↓baselinesPDF pp. 16–17, Appendix C.1–C.2
Co-train removes alignment; Standard OT removes DTW cost shaping. Real baselines are adapted to the paper’s architecture. ATM uses stacked non-autoregressive tracks, then freezes the trunk and track head while training the Cartesian head, explicitly differing from original ATM. MimicPlay freezes a high-level policy to condition a robot-trained low-level policy.
Go to primary source ↓drawer-settingPDF p. 7, Section 5.2, Drawer; p. 8, Figure 4; p. 19, Appendix G.1; p. 21, Table 6, Drawer row
The task requires toy grasp, placement into an open drawer and drawer closure. Robot data comprise 144 demonstrations across three quadrants of a 6×4 array; 360 human demonstrations over 60 minutes cover all four. The top-right quadrant is withheld from robot data. Evaluation uses 48 rollouts, two per drawer; success requires all stages. Table 6 gives 29 robot minutes, while G.1 rounds to 30.
Go to primary source ↓coffee-settingPDF p. 7, Section 5.2, Scoop Coffee; p. 8, Figure 4; p. 19, Appendix G.1; p. 21, Table 6, Coffee row
Robot training uses 50 demonstrations of a can in one scene. Human data add a grinder and another scene. Each evaluation setting uses 15 rollouts across five target positions. Main text says two hours of human data; G.1 and Table 6 give 180 minutes, with Table 6 listing 720 demonstrations.
Go to primary source ↓laundry-settingPDF pp. 7–8, Section 5.2, Laundry; p. 21, Table 6, Laundry row
Bimanual folding includes both sleeves and folding the shirt in half. Robot data contain 300 demonstrations across three shirts; human data contain approximately 700 demonstrations, both collected over two hours. Main evaluation uses 18 rollouts; success requires all stages.
Go to primary source ↓real-resultsPDF p. 7, Table 1, all rows and columns
EgoBridge Coffee SR is 67/60/27% for in-distribution/object/scene-plus-object; Co-train is 53/46/0%, EgoMimic 60/53/0%, robot-only BC 33/40/7%. EgoBridge Drawer total is 77 points and 47% SR, Place Toy 72%, behavior generalization 33%; Co-train gives 55 points, 22% total SR and 0% behavior SR; ATM gives 8% behavior SR. Laundry gives EgoBridge 48 points and 72% SR; Co-train 41 and 33%, EgoMimic 38 and 33%, robot-only BC 38 and 28%. MimicPlay and ATM also have 28% Laundry SR.
Go to primary source ↓ablationPDF p. 9, Table 2 and adjacent Ablation paragraph
Table 2 gives Drawer SR / behavior-generalization SR: EgoBridge 47/33%, MSE 14/17%, Standard-OT 33/17%, Co-train 22/0%. Adjacent prose says 17% for the first MSE column and calls that column in-distribution; Table 1 labels the corresponding 47% as Drawer total.
Go to primary source ↓sim-protocolPDF pp. 6–7, Section 5.1 and Figure 3; p. 21, Appendices G.2–H
Circle source and triangle target differ in appearance and friction, with triangle friction multiplier 0.7. G.2 lists 100 original-white demonstrations per pusher plus 50 source examples for each of original-purple, mirrored-purple and mirrored-white: 350 total, despite saying four scenarios. Evaluation uses 100 seeds selected from 101–9999 with sampling seed 42. Mean reward averages maximum goal IoU; main text defines SR using ≥0.9, while H says over 0.9.
Go to primary source ↓sim-resultsPDF p. 21, Table 7, all rows and setting columns
For base/purple/purple-plus-mirrored-T, EgoBridge reward/SR is 0.7605/53%, 0.7206/48%, 0.6520/39%; Co-train 0.7062/48%, 0.6899/42%, 0.6214/31%; Standard OT 0.7009/38%, 0.5303/15%, 0.5109/8%; MMD 0.6439/45%, 0.4867/22%, 0.5876/14%; Target Only 0.5555/39%, 0.0904/0%, 0.0992/0%.
Go to primary source ↓pair-diagnosticsPDF p. 21, Appendix I; p. 22, Figure 9
Random minibatch pseudo-pairs compare DTW with MSE on Drawer, Coffee and Laundry. Authors describe improved temporal/spatial correspondence with DTW, but failures when trajectories span stages or viewpoints change extremely. Colored borders are qualitative annotations without a numerical pairing-accuracy metric or explicit color legend.
Go to primary source ↓latent-diagnosticsPDF p. 9, Figure 5 and H3 paragraph; p. 23, Appendix K and Figure 10
T-SNE, Wasserstein distances and nearest neighbors support the authors’ claim of improved domain overlap and behavioral correspondence. Main text describes action tokens, whereas K describes encoder outputs. The displays do not themselves establish causal sufficiency.
Go to primary source ↓extra-laundryPDF p. 22, Appendix J and Table 8
Additional Laundry testing uses a new white surface with 30 extra minutes of human data, then a harder human-only shirt-color combination. Each setting uses 20 rollouts over five positions. EgoBridge obtains 50% SR/43 points and 30%/35; Co-train obtains 25%/25 and 5%/7.
Go to primary source ↓limitationsPDF p. 9, Section 6; p. 19, Appendix G.1; p. 20, Figure 8
The demonstrated scope is single-task transfer; DTW may be less informative for multitask adaptation. Multiple embodiments and unlabeled Internet human data are future work. Failures include missed grasps, wrong targets and incomplete drawer closure or folds.
Go to primary source ↓releasePDF p. 26, NeurIPS checklist item 5
The supplied edition answers No to open data/code access and plans release after acceptance; this is not evidence of an actual release.
Go to primary source ↓uncertaintyPDF p. 27, NeurIPS checklist item 7
The authors answer No to statistical-significance reporting, citing cost and variability of repeated physical-robot trials. The retained tables supply no confidence intervals.
Go to primary source ↓proof-scopePDF p. 25, NeurIPS checklist item 3; supplied Appendices A–K, pp. 12–23
The checklist claims a detailed proof in supplementary material. The supplied appendices describe algorithms, architecture, data and experiments; no such separately identified proof was supplied.
Go to primary source ↓8.5 Primary sources
EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Data ↗
PDF · 15,765 extracted words
Source fingerprint
4ade87c62a205d66ba0b26deb0099c6880df707185d24b99e55da03c77e6861f