HumanX: Toward Agile and Generalizable Humanoid Interaction Skills from Human Videos
1. Paper overview
In one sentence: HumanX uses contact anchors and physics to expand human demonstrations into interaction references, then distills imitation policies for robot deployment, with generalization bounded by the synthesized data and available sensing. e02e04e07e10e12e15
| At a glance | What to know |
|---|---|
| Research problem | Source description Humanoid interaction learning needs both plausible robot–object contact data and feedback policies that tolerate changing object states. Independently reconstructing human and object trajectories can violate contact constraints, while task-specific RL rewards demand repeated engineering. HumanX replaces exact interaction reconstruction with physical synthesis and a shared interaction-imitation objective. e02e04e09 |
| Core mechanism | |
| A key reported result | Basketball Catch-Shot: Final +Tea-Stu: 64.7% GSR, 86.8% SR Generalization success rate (GSR); original-demonstration success rate (SR). Simulation; one video expanded to 50 clips for the final setting. GSR samples the augmented distribution with uniform ±0.3 m initial-ball perturbations and a new hoop target; success means landing within 20 cm of its center. HDMI: 2.4% GSR, 53.1% SR; XMimic Base: 4.9% GSR, 93.4% SR Generalization improves substantially, while original-trajectory success decreases relative to Base. This is a simulated distribution-specific comparison. e12e13 |
| Reading caution | Reader analysis GSR tests come from the augmented distribution; the source does not establish disjoint-distribution generalization or report GSR trial counts, seeds or confidence intervals. Table I's successive additions are not a factorial ablation: basketball GSR even falls from 13.5% at +DI to 10.9% at +IT. e12e13 |
Core contributions
- Author claim
XMimic combines interaction rewards, disturbed initialization, probabilistic interaction termination and teacher distillation. The authors attribute substantially better generalization to these coordinated choices; the experiments evaluate the combined system and staged variants. e07e09e10e13
Figure 2. XGen builds an interaction reference by assigning different physical constructions to contact and free motion. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow panels 1–3 down the left: the human video becomes estimated body motion and then retargeted humanoid motion. On the right, panels 4–6 define the object relative to an anchor, move it with that anchor and refine contact. The green contact interval in panel 1 corresponds to the period from contact start to contact end in Section III-B. The pink non-contact intervals use the simulator branch in panel 7; its arrows lead from initial object states through simulation to trajectories. Panel 8 assembles the phases. Yellow augmentation markers identify changes to object geometry, contact motion or free-motion initialization. e03e04e05
What it supports. The object trajectory need not be reconstructed independently in every video frame. The source instead imposes an anchor relationship during contact and a physics model outside contact. This creates controllable opportunities to vary object size, pose and trajectory while retaining a reference interaction for policy learning.
Where the evidence stops. The diagram includes estimated or manually defined object configuration and annotated phases. It does not demonstrate a fully automatic video-to-policy pipeline or prove that every synthesized trajectory will remain feasible under real robot dynamics.
2. Motivation
2.1 The problem and the proposed response
Humanoid interaction learning needs both plausible robot–object contact data and feedback policies that tolerate changing object states. Independently reconstructing human and object trajectories can violate contact constraints, while task-specific RL rewards demand repeated engineering. HumanX replaces exact interaction reconstruction with physical synthesis and a shared interaction-imitation objective. e02e04e09
2.2 What this reading follows
A human video gives a robot a motion example, but leaves contact geometry, object depth and feedback behavior unresolved. HumanX addresses these gaps in two stages. XGen constructs robot–object trajectories using retargeted body motion, contact anchors and simulated free flight. XMimic learns policies that imitate those trajectories while tolerating disturbed starts and imperfect sensing. Read the synthesis and training diagrams together, then compare the simulation tables with the hardware evidence. The key distinction is between matching an original demonstration, succeeding within an augmented distribution, and completing a physical interaction under a particular perception setup. e02e04e07e10e12e15
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Foundational work |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
This table preserves the labels recorded at reading time. The current major category is Datasets. View the current classification.
3.1 Evidence-based assessment
Supports the recorded classification
The recorded foundational-action-policy and data-generation relevance is supported. XGen uses an external physics simulator to synthesize references; XMimic learns control by imitation. Neither the teacher–student architecture nor a shared student constitutes joint learned future/action prediction. Keeping architecture, prediction paradigm and quadrant as Not applicable is appropriate for this world-action-model taxonomy. e04e06e07e10
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Build the object motion around contact
Start with the carrying example in XGen. Retargeting supplies the robot's body pose sequence, but it does not by itself place the box consistently between the hands. XGen defines an anchor, such as the palm midpoint, and associates an object pose with it at contact. Propagating that relationship gives a contact-phase trajectory; force-closure refinement adjusts the robot configuration. Free motion is generated separately from boundary states. For catching, the source simulates backward from the desired contact pose and reverses the resulting sequence, with damping inverted. These operations explain why one demonstration can support changed geometries and flight paths: the pipeline regenerates an interaction using specified physical relationships instead of requiring additional independently reconstructed object tracks. e03e04e05
Figure 5. The student retains a feedback control loop after privileged training information is removed. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the black action arrows left to right in each panel: policy output goes to the PD controller, then to the simulator or humanoid. Stage 1 inputs are privileged body information, proprioception and object state. Green dashed arrows connect reference interaction and simulator behavior to the imitation objective. In Stage 2, the teacher's lock indicates frozen parameters; orange dashed connections bring teacher and student actions into the BC loss described in Equation (4). The student still receives interaction-imitation training. On the right, reference and teacher branches are absent. Proprioception closes the robot loop, while the grey optional branch supplies external object state through MoCap. e06e07e08e11e14
What it supports. Privileged teachers make skill learning easier, while the student is trained under deployable observations. The final controller operates at 100 Hz and sends actions through a 1 kHz PD loop. These arrows describe executed feedback control; the architecture does not show a learned world model predicting future scenes for planning.
Where the evidence stops. Figure 5 summarizes the inputs. Table IV also lists history, optional target observations and a student skill label. The source does not fully explain the label's use during autonomous pattern selection, so the diagram should not be read as a complete observation specification.
5.2 Teach interaction, then restrict what the controller can observe
A teacher learns through PPO with privileged body and object information. Its reward combines body imitation with object tracking, relative motion and contact agreement, plus regularization. This matters because imitating the body alone can leave the object behind. Interaction termination adds a second signal: during expected contact, excessive object-to-body separation can probabilistically end the episode. The student then learns with PPO and behavior cloning from the pretrained teachers while losing privileged observations. Reference trajectories still supervise training, but are absent from student deployment observations. NEP removes external object observations entirely; MoCap retains object sensing and trains with simulated signal loss. Table IV gives the fuller observation list, including history and a student skill label, beyond the compact arrows of Figure 5. e06e07e08e09e10
5.3 Ask which generalization claim each experiment can support
Reader analysis: Table I's most revealing comparison is between success on the original reference and success over the augmented distribution. Basketball Base already succeeds at its original trajectory, yet its GSR is low; the final setting trades some original success for much greater variation tolerance. The testing distribution is still generated from the augmentation scheme, so this does not establish unrestricted transfer to new tasks or objects. Table II changes the question to learning three patterns within a skill and supports teacher distillation there. Table III changes both the environment and the scoring protocol: it reports physical trial counts, with external sensing for some tasks and sequence completion for NEP basketball. Those counts cannot be substituted for simulated target-accuracy scores or treated as evidence of an all-domain controller. e12e13e14e15
5.4 Training and inference
During training
Each skill-pattern teacher learns from its dedicated XGen dataset with PPO. The student trains on their union. The common reward tracks body motion, object state, relative body–object motion and contact graphs, with AMP and regularizers. Shared rewards still require contact labels and reference construction. e06e07e09
Disturbed initialization perturbs robot and object starting states. Interaction termination probabilistically ends an episode when object-to-key-body error exceeds a threshold during reference contact. Domain randomization varies physical properties and perception noise and adds sustained external forces; MoCap students also experience simulated tracking loss. e08e10
During inference
The student acts from current observations and history without phase or reference trajectories. Table IV additionally lists a student skill-label input and optional object/target fields. NEP removes object observations; MoCap supplies external poses. Policy and MoCap run at 100 Hz, while the PD loop runs at 1000 Hz. No learned future-state rollout is described in this deployment loop. e07e08e10e11
The authors motivate proprioceptive interaction by a torque-balance argument: measured positions/velocities, PD commands and velocity history could expose external forces implicitly. This is an observation-design argument, without direct validation of an internal force estimator. e17
5.5 Implementation flow
- Recover and retarget the body
GVHMR estimates SMPL human root and joint poses. GMR maps them to the humanoid through keypoint alignment, skeleton scaling and inverse-kinematics optimization. This establishes robot motion before synthesizing the object trajectory. e03
- Construct contact and free-motion phases
Annotate contact start/end. During contact, propagate an object pose relative to a palm-midpoint or body-part anchor and refine robot poses under force-closure constraints. SAM-3D or manual specification supplies object geometry/initial pose. After release, simulate forward from the release pose; before catching, simulate backward from contact and reverse the sequence, inverting damping. Interpolate body poses near transitions. e04
- Expand interaction coverage
Scale or substitute meshes, translate/scale contact trajectories and randomize free-motion initial velocities. These operations create physical reference variations from the same demonstration rather than recovering additional observed executions. e05
- Learn feedback control
An MLP parameterizes a Gaussian action policy with learnable covariance. Teachers receive proprioception, privileged body information and object states. The student removes privileged observations and is trained using PPO plus behavior cloning from pretrained teachers; Figure 5 marks the teacher as frozen during distillation. e06e07
6. Experiments & results
HumanX turns monocular demonstrations into humanoid control policies through XGen, which synthesizes interaction references using contact anchors and physics, and XMimic, which trains imitation policies with privileged teachers and deployable students. It supports proprioception-only basketball behaviors and externally tracked interactions. Its strongest evidence concerns simulated generalization within specified augmented distributions and physical Unitree G1 trials under distinct perception protocols (e02, e04, e07, e12, e13, e15, e16).
6.1 Read the original evidence
Table I. Original-trajectory imitation and generalization are different outcomes, and the staged additions affect them differently. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each task as a separate four-column block. SR measures success on the original demonstration; GSR measures success in the specified augmented distribution. Object error and key-body error are lower-is-better tracking metrics, with units that change across task blocks. The top rows compare prior methods. Below the divider, the source describes successive additions to XMimic Base: disturbed initialization, interaction termination, data augmentation and teacher–student training. Compare SR and GSR together rather than selecting only the best percentage. Section VI-B defines the basketball and cargo thresholds and the badminton hit criterion; page 7 defines the perturbation distributions behind GSR. e10e12e13
What it supports. The final row reaches 64.7%, 90.6% and 96.3% GSR for basketball, badminton and cargo, compared with HDMI's 2.4%, 25.3% and 1.8%. Basketball also exposes a tradeoff: Base has 93.4% SR, whereas the final setting has 86.8% despite its much stronger generalization score.
Where the evidence stops. The rows are a staged ablation, with nonmonotonic effects: basketball GSR drops from 13.5% at +DI to 10.9% at +IT. Cargo augmentation entries are unreported. GSR uncertainty, trial counts and the meaning of the error columns' ± notation are unspecified.
Table III. Hardware results must be read with both their perception mode and their trial denominator. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each entry is a count of successful trials, not a simulation GSR percentage. Read the mode in parentheses: the first four left-column tasks use MoCap, while Basketball Pickup at the bottom is NEP, like the right-column skills. MoCap provides object or human motion; NEP uses proprioception without external object sensing. The denominator is 50 for the MoCap tasks and 10 for the NEP tasks. Section VI-C says each skill uses one demonstration video. Its explicit NEP success definition is sequence completion without dropping the ball while maintaining balance; it does not define these entries as basket-scoring rates. e08e11e15e16
What it supports. The physical robot succeeds in 41 of 50 catch-pass trials and 43 of 50 cargo pickups under MoCap. NEP results include 8 of 10 jumpshots and 7 of 10 layups under the stated completion criterion. These counts demonstrate executed skills, while the two sensing regimes answer different capability questions.
Where the evidence stops. The prose lists five NEP skills, but the table adds Spin Move as a sixth. No badminton hardware row or physical baseline is provided. MoCap task-specific success thresholds and uncertainty intervals are not reported.
Figure 9. Proprioception-only control supports dynamic basketball sequences under the paper's initialization and success protocol. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow each labeled sequence separately rather than treating the montage as one continuous trial. The upper row shows jumpshot, dribble and pickup; the lower row shows the longer pump-fake turnaround fadeaway and layup. The source identifies these as NEP executions: the student receives no object observations, but still receives proprioceptive feedback and history. Section VI-C initializes the ball in hand, except for Pickup, where it begins on the floor. The appendix's torque-balance discussion motivates how commanded actions and measured joint motion could reveal interaction forces. The photos show the resulting behavior, while Table III supplies its trial counts. e08e15e16e17
What it supports. Removing explicit object perception does not remove feedback control. HumanX demonstrates a useful set of contact-rich behaviors under this restricted sensing mode, including a long turning maneuver. The figure makes the physical embodiment and motion variety visible; the quantitative reliability evidence remains the separate trial table.
Where the evidence stops. A successful montage is not a force-estimation measurement or an unselected trial distribution. The source explicitly says NEP cannot handle non-contact interactions such as catching a flying ball. No timestamps or timing benchmark can be inferred from these stills.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Basketball Catch-Shot Simulation; one video expanded to 50 clips for the final setting. GSR samples the augmented distribution with uniform ±0.3 m initial-ball perturbations and a new hoop target; success means landing within 20 cm of its center. | Final +Tea-Stu: 64.7% GSR, 86.8% SR Generalization success rate (GSR); original-demonstration success rate (SR) | HDMI: 2.4% GSR, 53.1% SR; XMimic Base: 4.9% GSR, 93.4% SR Generalization improves substantially, while original-trajectory success decreases relative to Base. This is a simulated distribution-specific comparison. e12e13 |
| Badminton Hitting Simulation; one demonstration, 50 augmented training clips in the final setting; GSR uses uniform ±0.3 m shuttlecock initial-position perturbations. Success is a hit. | Final: 90.6% GSR, 100% SR GSR; SR | HDMI: 25.3% GSR, 90.0% SR; Base: 41.6% GSR, 97.1% SR Measures interception, without a shuttlecock-placement accuracy requirement. e12e13 |
| Cargo Pickup in simulation Single-video task; GSR places cargo within a forward semicircle of radius 3 m. Success requires lifting within 10 cm of target height. | Final: 96.3% GSR, 99.3% SR GSR; SR | HDMI: 1.8% GSR, 95.8% SR; Base: 50.9% GSR, 100% SR The cargo +Data Aug row contains slashes, so its isolated augmentation contribution is unreported. e12e13 |
| Multi-pattern football kicking and badminton hitting Simulation; three demonstration videos/patterns per task, processed and augmented by XGen. | With teacher–student: football 93.1%; badminton 84.3% GSR | Without teacher–student: 74.2% and 52.4%, respectively Supports distillation for consolidating patterns within a skill; does not establish one policy spanning every reported domain. e14 |
| Physical MoCap interaction skills Unitree G1; one video per skill; externally tracked object/human motion; 50 trials per skill. | Catch-Pass 41/50; Cargo Pickup 43/50; Football Kicking 42/50; Reactive Fighting 37/50 Reported successful trials / total trials | No real-robot baseline row is reported. Real executions are demonstrated, but these counts use a different protocol from simulated GSR; detailed task-specific MoCap success thresholds are not stated. e11e15e16 |
| Physical NEP basketball skills Unitree G1 without external object perception; 10 trials per listed skill. Prose defines success as completing the sequence without dropping the ball while remaining balanced. | Pickup 10/10; Jumpshot 8/10; Dribble 8/10; Pump-fake 9/10; Layup 7/10; Spin Move 9/10 Reported successful trials / total trials | No real-robot baseline row is reported. These are sequence-completion counts, not an explicitly defined basket-scoring rate. The prose lists five skills; Table III adds Spin Move. e15e16 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table II. Teacher distillation improves the reported success of learning multiple patterns within a skill. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read vertically within each task column. Both tasks contain three distinct patterns derived from three human demonstration videos, unlike the single-video setting emphasized in the main evaluation. XGen processes and augments those demonstrations. The rows switch the teacher–student scheme off and on; each cell is a generalization success rate. Figure 7 on page 5 supplies qualitative context by showing distinct kicking and badminton postures. Read this table alongside Stage 2 of Figure 5: the experiment asks whether separate teachers help a student acquire the repertoire, rather than testing a single teacher on one reference trajectory. e07e14
What it supports. Football GSR rises from 74.2% to 93.1%, and badminton from 52.4% to 84.3%. These results support the teacher–student design for the reported multi-pattern setting. They extend the evidence beyond one demonstrated pattern, while remaining task-specific simulation comparisons with three patterns per task.
Where the evidence stops. The table supplies no seeds, confidence intervals or explicit compute-matching analysis. It does not establish one deployed policy covering every domain, nor a common three-video budget for the single-video results elsewhere in the paper.
7. Analysis & limitations
7.1 What the evidence leaves open
GSR tests come from the augmented distribution; the source does not establish disjoint-distribution generalization or report GSR trial counts, seeds or confidence intervals. Table I's successive additions are not a factorial ablation: basketball GSR even falls from 13.5% at +DI to 10.9% at +IT. e12e13
NEP cannot handle non-contact interception such as catching a flying ball. The force argument assumes remaining dynamics terms are approximately constant. Qualitative recovery and failure sequences support behavior examples, without isolating an internal reasoning or force-estimation mechanism. e08e16e17
The abstract describes ten skills across five domains transferring to hardware, but Table III contains four domains and no badminton rate. Its six NEP entries also differ from the five-skill prose list. Preserve these reporting inconsistencies rather than supplying a missing physical badminton experiment. e01e15
7.2 Questions for discussion
- How much generalization survives outside the XGen augmentation distribution?
- Does disturbed initialization interact with probabilistic termination when their effects are tested factorially?
- Which proprioceptive history variables causally support NEP robustness?
8. Reproducibility audit
8.1 Requirements and known gaps
The reported setup uses iPhone 16 clips, Isaac Gym, one NVIDIA RTX 4090, 16,384 parallel environments and normally 20,000 iterations per policy. Hardware deployment uses Unitree G1 and a 14-camera Noitom system over 5 × 5 × 2.6 m for MoCap experiments. e11
A reproduction needs the demonstrations, phase/anchor annotations, object assets and contact refinement. The PDF leaves the interpolation-window value, termination threshold/probability, reward coefficients, detailed PPO/network settings, history length and MoCap-loss distribution unspecified. Table IV lists observation/reward terms, not a complete executable configuration. e04e06e07e08e09e10
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate augmentation coverage from the DI–IT interaction
Reader-proposed simulation experiment: use Basketball Catch-Shot and one fixed set of 50 XGen clips. Train a 2 × 2 comparison with disturbed initialization and interaction termination independently enabled/disabled; hold rewards, architecture, training budget and teacher–student setting constant. Evaluate paired test cases both within the published ±0.3 m perturbation range and in a separately defined, wider range, using the same 20 cm target criterion. Report SR, GSR, contact failures and uncertainty across repeated seeds. A gain confined to the published range would support coverage-specific robustness; a negative DI–IT interaction would explain why the staged +IT row falls below +DI without implying that termination is universally beneficial. e05e10e12e13
Check 2: Test whether velocity history causes NEP force robustness
Reader-proposed check: train matched NEP basketball controllers with full proprioceptive history, velocity history removed, and velocity history temporally shuffled. Preserve current positions/velocities, action and PD-error observations and match model capacity and training conditions. In simulation, apply identical held-out sustained-force pulses and object-mass changes; measure sequence completion, balance loss and body–object separation with repeated seeds. This isolates temporal information from simply adding input dimensions. A full-history advantage that increases under disturbances would support the appendix's mechanism; no advantage would weaken that causal account even if baseline NEP success remains high. This proposed check does not assume the policy exposes a calibrated force estimate. e08e10e15e16e17
8.3 Reading coverage
Visual audit: Inspected the title/author/version page; all twelve figures and four tables; method and training equations on pages 3–5; simulation protocol and hardware configuration on pages 6–8; and appendix observation, reward and dynamics material on pages 12–13. All six final original crops were separately viewed at their delivered resolution. Figure 2 phase construction and arrows were checked against Section III-B; Figure 5's frozen-teacher marker, loss connections, optional perception and deployment feedback were checked against Section IV-A/B and Table IV. All retained numerical, method, configuration and proposed-check source facts are covered by these pages. Reference pages 9–11 were read as text, with no visual inspection claimed. Separate supplements and demonstration videos were not supplied for this pass. The source's NEP skill-count mismatch and missing physical badminton row are preserved in the explanations.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 12, 13. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, author block and abstract, PDF p. 1: arXiv:2602.02473v1 [cs.RO], 2 February 2026; title and all nine authors match the catalog
- I. Introduction and II. Related Work, PDF pp. 1–3
- III. XGen, including motion extraction, interaction synthesis and augmentation, PDF pp. 3–4
- IV. XMimic, including teacher/student training, perception, rewards and simulation settings, PDF pp. 4–6
- V. On Generalization of Interaction Skills, PDF p. 6
- VI. Experiments: settings, simulation evaluation/ablations and real robot experiments, PDF pp. 6–8
- VII. Conclusion, PDF p. 8; References, PDF pp. 9–11
- Appendix material: VIII. Unified Interaction Imitation Reward and IX. Perceiving External Force from Proprioception, PDF pp. 12–13
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Only the supplied v1 artifact was reviewed; no other edition was supplied or compared. No title or author discrepancy was observed in this version.
- Text extraction does not reconstruct figure images; the retained PDF was separately inspected on pages 1–8 and 12–13, including all figures and tables. Reference pages 9–11 were read as text only.
- Separate supplemental material availability has not been fully verified.
- The project page, code, training assets and demonstration videos were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/author/affiliation block, arXiv margin and abstract
Title and nine authors match the catalog; the margin identifies arXiv:2602.02473v1, 2 February 2026. Affiliations are The Hong Kong University of Science and Technology and Shanghai AI Laboratory. The abstract makes the broad ten-skill/five-domain transfer claim.
Go to primary source ↓e02PDF pp. 1–2, Abstract and Section I
Motivates replacing costly interaction data and task-specific reward engineering with XGen physical interaction synthesis and XMimic imitation learning.
Go to primary source ↓e03PDF p. 3, Section III-A, Eqs. (1)–(2)
GVHMR estimates SMPL human poses; GMR retargets using keypoint alignment, skeleton scaling and IK optimization.
Go to primary source ↓e04PDF p. 3, Figure 2 and Section III-B; p. 4, Section III-B.1–2
Contact segmentation, anchors, SAM-3D/manual object configuration, anchor propagation and force-closure refinement define contact references. Free motion is simulated forward after release and backward before contact with reversed damping; body transitions use k-frame interpolation without specifying k.
Go to primary source ↓e05PDF p. 3, Figures 3–4; p. 4, Section III-C.1–3
Augmentation changes mesh/size, contact trajectory geometry and non-contact initial velocity. The yellow augmentation markers in Figure 2 identify related synthesis operations.
Go to primary source ↓e06PDF p. 4, Section IV-A.1–2 and Eq. (3); p. 5, Figure 5 Stage 1
An MLP Gaussian policy with learnable covariance outputs actions mapped to torques by PD control. One privileged teacher is trained with PPO per skill-pattern dataset.
Go to primary source ↓e07PDF p. 5, Figure 5, Section IV-A.3 and Eq. (4)
Students train on combined datasets using PPO and BC from pretrained teachers, with privileged states excluded. Figure 5's lock marks the frozen teacher; deployment contains the student, PD controller and feedback, with optional MoCap.
Go to primary source ↓e08PDF p. 5, Section IV-B.1–2; p. 12, Table IV observation columns
NEP removes object observations and cannot catch unseen flying objects. MoCap training simulates tracking loss. Table IV lists current/history proprioceptive terms, teacher reference/delta body positions, a student skill label and optional object/target observations.
Go to primary source ↓e09PDF p. 5, Section IV-C; p. 12, Table IV and Section VIII, Eqs. (5)–(12)
Additive reward contains body, object, relative-motion, contact and regularization terms. Body terms include AMP; tracking uses exponential error rewards. Contact uses weighted binary graph mismatch. The supplied appendix does not assign numerical reward hyperparameters.
Go to primary source ↓e10PDF p. 5, Section IV-D.1; p. 6, Section IV-D.2–3 and Section V
Disturbed initialization changes robot/object start states. Interaction termination is probabilistic above an unspecified relative-error threshold during reference contact. Domain randomization includes physical properties, noise and sustained forces. Students exclude phase/reference observations.
Go to primary source ↓e11PDF p. 6, Section VI-A
Reports iPhone 16, Isaac Gym, one RTX 4090, 16,384 environments, normally 20,000 iterations, Unitree G1, 14 Noitom cameras over 5 × 5 × 2.6 m, and 100 Hz policy/MoCap with 1000 Hz PD.
Go to primary source ↓e12PDF p. 6, Section VI-B.1; p. 7, metric definitions and GSR sampling paragraph
Defines task successes and four metrics. One demonstration produces one clip, or 50 with augmentation. GSR samples the augmented distribution: ±0.3 m ball/shuttlecock perturbations and forward radius-3 m cargo placements. The ablation is described as incrementally adding components.
Go to primary source ↓e13PDF p. 6, Table I, all method rows and task groups
Final GSR is 64.7/90.6/96.3% versus HDMI 2.4/25.3/1.8% for basketball/badminton/cargo. Final SR is 86.8/100/99.3%. Base GSR is 4.9/41.6/50.9%. Basketball +DI and +IT GSR are 13.5 and 10.9%; +Data Aug is 60.6%. Cargo augmentation cells are slashes. Error-column units vary by task; GSR uncertainty is absent.
Go to primary source ↓e14PDF p. 5, Figure 7; p. 8, Table II and Section VI-B.2
Each multi-pattern task uses three demonstration videos/patterns. Teacher–student versus no teacher–student yields football GSR 93.1 versus 74.2%, and badminton 84.3 versus 52.4%.
Go to primary source ↓e15PDF p. 8, Table III and Section VI-C.1–2
Reports four MoCap counts (41/50, 43/50, 42/50, 37/50) and six NEP counts (Pickup 10/10, Jumpshot 8/10, Dribble 8/10, Pump-fake 9/10, Layup 7/10, Spin Move 9/10). Prose lists five NEP skills and defines sequence completion/balance without dropping. No badminton row, physical baseline or confidence intervals are supplied.
Go to primary source ↓e16PDF p. 7, Figures 9–10; p. 8, Figures 11–12 and Section VI-C
Shows NEP basketball and MoCap interaction sequences, cargo recovery and qualitative failure examples without sustained-force randomization or simulated MoCap signal loss. Text reports over ten consecutive passes and over fourteen return kicks; these are sequence examples rather than per-trial confidence estimates.
Go to primary source ↓e17PDF p. 12, Table IV and Section IX opening; p. 13, Section IX, Eqs. (13)–(14)
Rearranges commanded torque minus inertial, Coriolis, gravity and friction components into external joint-space force contribution. Positions/velocities, PD command approximation and velocity history motivate observations; other terms are assumed approximately constant. No direct force-estimation accuracy experiment is reported.
Go to primary source ↓8.5 Primary sources
HumanX: Toward Agile and Generalizable Humanoid Interaction Skills from Human Videos ↗
PDF · 8,618 extracted words
Source fingerprint
48dee856941dd7c078329e93b7b59c7a3d4674c2f169ecb3c479de1193f7c482