How to Learn from What a Human Would Avoid? Intervention-Aware World Models with Real-World RL for Dexterous Manipulation
1. Paper overview
In one sentence: WHIRL turns human takeovers into actor-side risk predictions, improving dexterous learning in the reported runs while inheriting the labeling operator's habits. e02e03e06e08e10e11e14
| At a glance | What to know |
|---|---|
| Research problem | |
| Core mechanism | |
| A key reported result | Autonomous success across five dexterous tasks: WHIRL: Cube 30/30 (100.0%); LEGO 29/30 (96.7%); Pick & Place 27/30 (90.0%); Pull Drawer 19/20 (95.0%); Long Horizon 17/20 (85.0%). Successful rollouts / total; success rate. Held-out in-region deployments; pedal disabled; one trained seed per method/task; budget-matched checkpoints; 30 trials per pick task and 20 per drawer task. Residual RL: 25/30, 23/30, 18/30, 15/20, 12/20. ResFiT: 25/30, 24/30, 19/30, 14/20, 13/20. BC: 20/30, 19/30, 13/30, 11/20, 7/20, respectively. WHIRL leads every row. Against Residual RL, differences are 16.7, 20, 30, 20 and 25 percentage points. Residual RL is not the best baseline in every row. Long Horizon requires all three stages without resets. e09e10e11e18 |
| Reading caution | Source description Generalization is within demonstrated workspace regions, not to new objects, layouts, hands or fine in-hand manipulation. One method-aware operator and one seed per cell limit causal confidence; smoothing cannot supply across-seed uncertainty. The predictor inherits that operator's takeover threshold. e14e18 |
Core contributions
Figure 2. Execution produces labeled experience; model predictions return through learning updates. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start in the upper panel with robot state and images. The snowflake marks the frozen behavior prior, while the residual policy provides the trainable correction; their arrows meet at the action composer. The human branch can override autonomous execution. Follow the dashed purple arrow into replay, then the blue arrow into the lower panel's four-head world model. Its two outgoing paths have different destinations: an imagined target reaches the base-aware critic, and a risk-shaping signal reaches the actor update. The dashed orange feedback arrow changes the residual policy. These are replay-driven learning routes; the diagram does not insert a world-model planner between the composer and robot. e03e04e05e06e07e08
What it supports. The takeover is useful twice: the executed correction becomes experience, and its label trains a predictor used in later actor updates. This separates actual human-controlled robot actions from learned risk prediction. The frozen prior continues supplying nominal behavior while the residual absorbs online improvement.
Where the evidence stops. Figure 2's replay shorthand uses current-index reward and termination, whereas Equation 3 supervises next-step quantities. Preserve the diagram's labels, but follow the equations when describing target timing. The diagram alone does not specify the learning schedule or exact gradient boundaries.
2. Motivation
2.1 The problem and the proposed response
Human takeovers prevent or recover costly manipulation failures but consume expert attention. The authors ask whether the binary decision to intervene can become reusable predictive supervision, helping a 16-DoF hand avoid states likely to require further assistance. e02e04
2.2 What this reading follows
A human can rescue a robot grasp before an episode is lost, but that rescue also reveals which situations the operator distrusts. WHIRL records the pedal signal and trains a small world model to predict takeover at the next step. A frozen imitation policy supplies the nominal motion, a residual actor learns corrections, and the model supplies both learning targets and risk-sensitive actor updates. This reading follows that separation from the execution diagram to autonomous deployment results. The central evidence is real-robot task completion and reduced intervention fractions during training, bounded by one seed, one operator and in-region testing. e02e03e06e08e10e11e14
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is WAMs. View the current classification.
3.1 Evidence-based assessment
Insufficient evidence to decide
The recorded taxonomy is entirely unassigned, so no existing quadrant is supported or contradicted. Architecture evidence shows separate prior, residual actor, critic and action-conditioned latent world model. This is modular model-assisted RL: the WM predicts consequences, while the actor generates commands. Joint head training does not establish a unified future-and-action generator; no inverse-dynamics action extraction is described. e03e05e06e07e17
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Turn a takeover into a correctly timed training example
The learning signal begins with reliable control transfer. WHIRL maps non-thumb glove joints affinely and solves thumb fingertip IK with opposition alignment, then rebases the command at takeover so the first commanded hand pose matches the robot. Replay records the action that actually executed, including the operator's command during intervention. The world model receives the encoded current state and action, but its takeover target is the next-step pedal state. This timing makes it a forward predictor rather than a classifier of whether control is already human. When the next pedal label is unavailable, the mask suppresses that loss. Demonstration segments without takeover logging therefore need not become false negative supervision. The same replay also trains dynamics, reward and termination predictions. e04e05e06e15
Figure 3. The intervention head enters actor utility and is excluded from imagined critic targets. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the numbered heads down the left, then compare the two brackets on the right. The critic path includes dynamics, reward and termination, with ensemble disagreement controlling its uncertainty gate. The actor path includes all four heads. Dynamics contribute disagreement; reward encourages task completion; termination and predicted intervention discourage undesirable outcomes. Equation 5 defines their weighted utility, and Equation 6 gates its contribution to the actor loss. Both right-hand paths also show the behavior-prior action because the critic and residual policy operate relative to nominal behavior. Setting the intervention coefficient to zero removes only the fourth head's actor contribution, not its training or the remaining model functions. e06e07e08e13e17
What it supports. This routing makes an informative ablation possible: retain the learned model and its critic support, but stop penalizing predicted takeovers in the actor. The design keeps operator behavior out of the explicit reward target while allowing it to bias which actions the policy learns to choose.
Where the evidence stops. The figure's reward label is r_t, but Equation 3 uses r_{t+1}. The prose says model gradients are detached without specifying the action-gradient boundary. These are implementation ambiguities; excluding a direct takeover reward does not establish the stronger optimality-invariance claim in Appendix C.
5.2 Separate what earns reward from what an operator would avoid
The critic learns task return using real transitions plus a one-step auxiliary target built from predicted dynamics, reward and termination. Its target excludes takeover probability. The actor, however, receives a utility that rewards predicted success and penalizes termination, intervention and disagreement, with uncertainty gating its weight. Appendix C calibrates reward probability onto success and step values before this combination. The separation motivates the α_i=0 ablation: the model can keep learning while its human-specific penalty stops influencing action updates. Reader analysis: keeping task reward unchanged is a useful architectural distinction, but it is not a proof that the learned policy remains optimal. Shaping changes actions and the replay distribution, so indirect effects on value learning remain possible. Exact gradient detachment must also be resolved before implementing the actor update. e07e08e13e16e17
5.3 Ask two different questions of success and intervention curves
Table 1 asks whether the trained policy completes a task without corrective takeover. Figures 5 and 6 ask how much of recent training interaction remains under human control. These outcomes should be read together, but their denominators and checkpoints differ. Success counts use held-out in-region deployments near a nominal interaction budget; traces aggregate completed episodes and may extend beyond it. The five-episode intervention fraction weights long recoveries more heavily than brief pedal taps, yet it is still a recent-window fraction rather than cumulative human labor. Reader analysis: higher autonomous success makes a simple giving-up explanation less plausible, but does not validate every claim about total operator savings. One method-aware operator and one training seed also leave uncertainty that trial-level statistics and within-run smoothing cannot remove. e10e11e12e14e18e19
5.4 Training and inference
During training
Each task supplies 15–30 demonstrations for both prior fitting and replay seeding, with no additional offline data during online learning. Nominal budgets are approximately 8,000 interaction steps for pick/drawer tasks and 50,000 for Long Horizon. Success evaluation uses the checkpoint closest to that budget; episode-end traces can extend beyond it. e10e18
Table 2 specifies 256-dimensional latents, two 512-unit hidden layers per head, Adam at 10^{-4}, batch size 128, discount 0.97, critic auxiliary weight 0.05 and actor auxiliary weight 0.40. The intervention coefficient is 0.4. Reward probability is mapped to success/step values 10.0 and −0.01 before actor utility use. e16e17
During inference
Online robot execution composes the prior and residual command unless the pedal selects human control. The model's depicted role is to update actor and critic from replay. Held-out success tests disable the pedal: an operator may emergency-stop, but cannot rescue trajectories; timeouts, drops and workspace violations fail. e03e05e10
5.5 Implementation flow
- Collect continuous corrections
A SpaceMouse controls the arm, a Manus Quantum glove controls the hand, and a pedal switches control. Non-thumb joints use calibrated affine maps; thumb IK includes opposition alignment. At takeover, rebasing anchors the first hand command to the current robot pose. Replay stores the human command during intervention, rather than the autonomous proposal. e04e15
- Constrain online exploration
The frozen action-chunk behavior prior provides nominal actions. Only the residual changes online; its hand component passes through a shared three-dimensional synergy basis into 16 joint coordinates. Separate arm and synergy-space L2 penalties keep corrections local. e05e16
- Predict consequences and takeover
The model takes the critic encoder's detached latent and action. Five residual-dynamics networks predict next latents; ensemble disagreement estimates uncertainty. Scalar heads predict reward, termination and next-step intervention. Missing pedal labels are masked, not treated as negative examples. e06e16
- Separate value targets from behavioral risk
A base-aware critic concatenates executed/proposed and prior actions. Real-transition TD learning receives an uncertainty-gated one-step model auxiliary target. Takeover probability enters only the actor utility alongside reward, termination and disagreement. No multi-step planner or inverse-dynamics action extraction is described. e07e08e17
6. Experiments & results
WHIRL learns when a human would take over a dexterous robot and uses that prediction to steer residual-policy training. A frozen imitation prior supplies nominal behavior; an action-conditioned, one-step latent world model supports critic learning and actor risk shaping. Five real-robot tasks show higher autonomous success, but the evidence is restricted to one training seed, one operator and familiar workspace regions.
6.1 Read the original evidence
Figure 4. Five physical tasks vary grasp geometry, placement demands and sequence length. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each labeled sequence from left to right. Cube and LEGO occupy the top row: one offers regular convex geometry and the other narrower, irregular grasp affordances. The middle row adds placement at a target and pulling a drawer through a handle grasp. The bottom sequence combines opening, placing inside and closing. Section 4.4 specifies that this Long Horizon task is evaluated as one rollout without stage resets. Appendix F defines the spatial randomization and workspace limits. These images identify the physical actions behind the success table; they are selected task illustrations rather than a complete record of every evaluation trial. e09e10e14e20
What it supports. The evaluation extends beyond a single pick motion to placement and prismatic contact, including a three-stage sequence. It remains a task-specific study on the same FR3 arm and LEAP Hand. The photographs help distinguish these executed manipulation tasks from purely visual world-model predictions.
Where the evidence stops. The source tests variation within the demonstration regions. These sequences do not establish transfer to unseen objects, layouts or hand morphologies, and a selected photograph sequence cannot establish autonomy or an aggregate success rate by itself.
Table 1. WHIRL leads all five autonomous deployment comparisons in the reported runs. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across a task row before comparing percentages between tasks. Each cell gives successes divided by trials, followed by its percentage; drawer tasks have fewer trials than pick tasks. BC is the frozen prior, ResFiT adds autonomous residual learning, Residual adds the authors' HIL mechanism, and WHIRL adds the model pathway. Appendix D clarifies that Residual is an in-house implementation, not an execution of the published HIL-SERL or SiLRI codebases. All success evaluations disable the pedal and select checkpoints near the nominal per-task budget. Thus the WHIRL counts represent deployment without corrective human takeovers, even though its policy was trained with them. e09e10e11e18e19
What it supports. For LEGO, WHIRL succeeds in 29/30 trials (96.7%), compared with Residual's 23/30 (76.7%) and ResFiT's 24/30 (80.0%). Pick & Place gives the largest gap over Residual: 90.0% versus 60.0%, or 30 percentage points. WHIRL also leads the end-to-end Long Horizon row at 17/20.
Where the evidence stops. The table evaluates one training seed per cell. Appendix D's one-sided Fisher tests are uncorrected across five tasks; Drawer and Long Horizon have p=0.091 and p=0.078. Deployment counts support task-level comparisons, but do not establish robustness across training seeds or operators.
Figure 5. Later training generally needs less human control, but the curves do cross. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the shared legend to compare blue WHIRL with red WHIRL without the world model, equivalent to Residual RL here. The vertical quantity is the sum of operator-controlled steps across five trailing episodes divided by their total lengths. It is not an unweighted average of episode fractions and not the number of pedal presses. Read each panel's axis independently: both the fraction scale and interaction horizon vary by task. Appendix E provides the noisier per-episode view, while Appendix D explains that logging occurs at episode termination. Consequently a final trace point may come after the nominal checkpoint used for Table 1. e10e12e14e18
What it supports. WHIRL generally reaches low intervention fractions sooner, including on the longer drawer sequence. Section 4.5 reports 0.018 versus 0.113 for Pull Drawer's final rolling fraction, approximately 84% lower. That numerical comparison comes from the prose; the plotted endpoint window is not sufficiently identified to verify those exact values visually.
Where the evidence stops. The caption's claim that WHIRL stays below the baseline is stronger than the plotted curves: early crossings are visible. The reported 84% is an endpoint-fraction reduction, not a measured reduction in cumulative human time. Neither smoothed curves nor Appendix E supplies across-seed uncertainty or numerical episode logs.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Autonomous success across five dexterous tasks Held-out in-region deployments; pedal disabled; one trained seed per method/task; budget-matched checkpoints; 30 trials per pick task and 20 per drawer task. | WHIRL: Cube 30/30 (100.0%); LEGO 29/30 (96.7%); Pick & Place 27/30 (90.0%); Pull Drawer 19/20 (95.0%); Long Horizon 17/20 (85.0%). Successful rollouts / total; success rate | Residual RL: 25/30, 23/30, 18/30, 15/20, 12/20. ResFiT: 25/30, 24/30, 19/30, 14/20, 13/20. BC: 20/30, 19/30, 13/30, 11/20, 7/20, respectively. WHIRL leads every row. Against Residual RL, differences are 16.7, 20, 30, 20 and 25 percentage points. Residual RL is not the best baseline in every row. Long Horizon requires all three stages without resets. e09e10e11e18 |
| Pull Drawer training intervention burden HIL training under a shared operator rule; trailing five episodes, weighted by their lengths. | Section 4.5 reports a final fraction of 0.018 for WHIRL. Rolling fraction of interaction steps under human control | The prose reports 0.113 for the no-WM baseline, an approximately 84% relative reduction. This is an author-reported endpoint comparison, not cumulative operator time saved. Figure 5 does not unambiguously reproduce these as its last plotted endpoints; the precise reporting window remains unresolved. Early curve crossings also preclude pointwise dominance. e12 |
| Actor-side intervention-shaping ablation Pick LEGO and Pull Drawer; full WHIRL versus α_i=0 versus removal of the WM pathway. | Full WHIRL reaches the near-zero regime earlier in both plotted tasks. Rolling-five step-level intervention fraction versus interaction steps | α_i=0 retains model training and critic support but converges later; no-WM is slower still. The curves support actor consumption of the intervention prediction beyond auxiliary head training. Drawer shows an early cost; no exact convergence threshold or across-seed uncertainty is supplied. e13e14 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 6. Turning off the actor's intervention penalty delays convergence despite retaining model learning. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First compare blue Full with orange α_i=0, holding the learned model pathway in place. The orange condition still trains the intervention head and retains dynamics, reward, termination and uncertainty-related functions; it removes actor consumption of predicted takeover probability. Then compare orange with red w/o WM, which removes the model pathway more broadly. Both panels show rolling-five step-level intervention fractions, with interaction steps on the horizontal axis. The cleanest visual distinction is when each curve approaches zero, rather than its ordering at every instant. In Pull Drawer, blue initially incurs more intervention than orange before crossing and declining faster. e08e13e14e16e17
What it supports. The ablation supports a contribution from predictive intervention shaping beyond simply training an extra head. Retaining the model but setting α_i=0 loses part of the convergence advantage in these runs. The slower no-WM curve also suggests that the rest of the model pathway contributes.
Where the evidence stops. Two tasks and single-seed traces cannot establish a universal gain. The early Drawer cost does not directly demonstrate predictor calibration, although the authors interpret it as a fitting delay. Removing the WM changes multiple functions; it cannot isolate the imagined critic target alone.
7. Analysis & limitations
7.1 What the evidence leaves open
Generalization is within demonstrated workspace regions, not to new objects, layouts, hands or fine in-hand manipulation. One method-aware operator and one seed per cell limit causal confidence; smoothing cannot supply across-seed uncertainty. The predictor inherits that operator's takeover threshold. e14e18
Appendix D reports one-sided, uncorrected Fisher p-values of 0.026, 0.026, 0.008, 0.091 and 0.078 in task order. These are per-task comparisons, with no correction for five tests. Binomial confidence intervals quantify deployment trials, not training-seed variability. e19
Keeping takeover probability outside reward backups removes its direct target contribution; it does not prove optimal-policy invariance or eliminate indirect effects through changed actions and replay. Appendix C's stronger invariance and no-leakage language exceeds that architectural guarantee. e07e08e17
Source notation needs care: Figure 3 labels reward r_t whereas Equation 3 trains on r_{t+1}; Equation 4 uses current proposed/prior actions at the imagined next latent, unlike Equation 12's next-state actions. The stated detached model gradients leave the precise actor-gradient boundary unspecified. e06e07e08
7.2 Questions for discussion
- Does intervention shaping retain its advantage across operators with different takeover thresholds?
- How much of the improvement survives when compared at identical cumulative human-controlled steps and autonomous interaction budgets?
8. Reproducibility audit
8.1 Requirements and known gaps
Recreate the FR3/LEAP rig with D405 wrist and D435 external RGB cameras at 30 Hz, 20-Hz arm impedance control, 120-Hz hand/glove streams and one RTX 4090. Table 3 gives workspace limits and task horizons; reproduce the operator calibration, thumb opposition solve and takeover rebasing. e15e20
The PDF defers exact residual-penalty schedules to supplementary configurations and does not fully specify synergy construction, prior chunk settings, software versions or actor/critic optimizer settings. Resolve gradient boundaries and action indexing before implementation. Also reconcile Section 4.4's pre-drop wording with Appendix D's dropped-object/lost-closure takeover trigger. e05e07e08e10e16e18
Proposed checks: repeat the α_i=0 comparison across seeds with matched demonstrations and operator rules; independently reconstruct rolling fractions from episode step counts and align endpoints to checkpoints. Neither check was run for this report. e12e13e18
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Repeat the intervention-consumption ablation with controlled exposure
Reader-proposed experiment, not performed: on Pick LEGO and Pull Drawer, compare full WHIRL with α_i=0 across multiple training seeds, holding demonstrations, frozen prior, synergy basis, model losses, replay capacities, other utility coefficients and nominal interaction budgets fixed. Retain intervention-head training in both arms. Use the same explicit takeover rule, counterbalance method order, and record total human-controlled steps plus pedal-disabled success at matched checkpoints. The mechanism predicts that enabling actor consumption improves the joint success/burden tradeoff beyond its seed variability. No consistent advantage, or lower intervention accompanied by worse autonomous success, would weaken that interpretation. The supplementary configuration and unresolved gradient boundaries must be obtained or specified before calling this a reproduction. e08e10e13e14e16e18e20
Check 2: Reconstruct the 84% claim from episode counts and checkpoint windows
Reader-proposed audit, not performed: obtain each episode's human-controlled step count u_i, total length T_i and ending interaction step for the two Pull Drawer runs. Recompute the trailing-five ratio as the sum of u_i divided by the sum of T_i; compare it with an unweighted episode mean and with cumulative human-controlled steps as explicit controls. Evaluate both the final recorded window and the window closest to the nominal 8,000-step checkpoint. Determine whether a documented common rule reproduces 0.113 versus 0.018 and aligns with Figure 5. Failure to locate such a window would leave the exact endpoint claim unresolved. Figure 7's plotted ratios alone cannot recover the necessary episode lengths. e10e12e18e20
8.3 Reading coverage
Visual audit: Actually inspected the original rendered title/author page, all main-text pages, and all appendix pages. This includes Figures 1–8, Tables 1–3, the unnumbered statistical table, equations, training settings, hardware and workspace details. All six final original crops were separately viewed at their rendered resolution. Checked routing arrows and head membership against Equations 3–6, reward calibration against Appendix C, and imagined versus real action indexing against Equations 4 and 12. The edition preserves reward-label and gradient ambiguities, early curve crossings, and the unresolved match between Q2's numerical endpoints and Figure 5. Pages 9–10 contain references read in the complete text pass; they were not visually inspected. No external supplement, project media, code execution or experiment reproduction was included.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 14, 15. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, author block and abstract (p. 1)
- 1 Introduction
- 2 Related Work
- 3 Method, including 3.1–3.3
- 4 Experiments, including 4.1–4.5
- 5 Limitations
- 6 Conclusion
- References
- A Hand Retargeter Implementation Details
- B World Model and RL Hyperparameters
- C Design Choices for the Intervention-Aware World Model
- D Detailed Training and Evaluation Protocol
- E Raw Per-Episode Intervention Traces
- F Hardware, Workspace, and Data Collection
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Acquisition omission: Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This gap was addressed by separately inspecting the original PDF pages and final crops.
- Separate supplemental material availability has not been fully verified.
- Only the supplied PDF was reviewed; linked project material and supplementary code were not inspected, and experiments were not reproduced.
- Identity: the inspected title and all eight authors match the catalog. The title page identifies arXiv:2609.06009v1 [cs.RO], 5 Sep 2026; no different revision or edition was supplied or substituted.
- All five supplied text chunks were read completely. Appendix F is present and reviewed although omitted from the acquisition section list. Reference pages 9–10 were read as text; visual inspection covers pages 1–8 and 11–15.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, author/affiliation block and arXiv margin stamp
The title matches the supplied identity; authors are Jiaju Yin, Zhenhui Zhang, Lixin Xu, Heng Zhang, Jun Shao, Yating Feng, Arash Ajoudani and Renjing Xu. Affiliations are HKUST (Guangzhou), Italian Institute of Technology and Zhejiang University. The stamp specifies 2609.06009v1, 5 Sep 2026.
Go to primary source ↓e02PDF p. 2, Section 1, motivation and contribution bullets
The paper proposes reusing binary human takeovers as prediction targets to reduce expert attention during dexterous real-robot learning.
Go to primary source ↓e03PDF p. 3, Figure 2 and caption
The execution diagram separates frozen prior, residual policy, action composer and human override from replay-driven world-model, critic and actor updates. Its replay label uses r_t and d_t, with next observation and i_{t+1}.
Go to primary source ↓e04PDF p. 4, Sections 3.1–3.2, Equation 1 and replay paragraph
SpaceMouse, Manus Quantum glove and pedal provide tri-channel control. Retargeting combines affine non-thumb maps, thumb IK and takeover rebasing; intervention replay retains operator actions.
Go to primary source ↓e05PDF p. 4, Section 3.2 and Equation 2
A frozen action-chunk behavior prior is corrected by an online residual. The hand residual uses W in R^{K×16}; arm and synergy coordinates receive separate squared-norm penalties.
Go to primary source ↓e06PDF pp. 4–5, Section 3.3, Equation 3, mask paragraph and Figure 3
The model consumes detached critic latents and actions. Dynamics predict residual next latents; reward, done and next-step takeover heads use binary supervision. m_t masks unavailable next-step labels. Figure 3 shows the reward as r_t, while Equation 3 indexes its target at t+1.
Go to primary source ↓e07PDF p. 5, critic path and Equation 4; p. 12, Appendix B, Equation 12
The base-aware critic conditions on action and prior action. Uncertainty gates the auxiliary model target; takeover probability is excluded. Equation 4 explicitly uses current proposed/prior actions at predicted next latent, while Equation 12 uses next-state actions and notes an omitted SAC entropy term.
Go to primary source ↓e08PDF p. 5, actor path, Equations 5–6 and risk-shaping discussion
Actor utility adds calibrated reward and subtracts weighted termination, takeover probability and disagreement. The loss is uncertainty-gated; α_i=0 removes intervention shaping. The prose says model gradients are detached without specifying the computational boundary.
Go to primary source ↓e09PDF p. 6, Sections 4.1–4.2 and Figure 4
Five FR3/LEAP tasks span cube/LEGO grasping, placement, drawer pulling and drawer-open/place/close. The baseline ladder adds autonomous residual RL, HIL residual RL and the world model.
Go to primary source ↓e10PDF p. 7, Sections 4.3–4.4
Success uses held-out in-region deployments with pedal disabled and nominal-budget checkpoints. Training uses 15–30 demonstrations, a frozen prior and one seed/operator per cell. Long Horizon is end-to-end without stage resets.
Go to primary source ↓e11PDF p. 7, Table 1, all task rows and method columns
WHIRL counts are 30/30, 29/30, 27/30, 19/20, 17/20; Residual counts are 25/30, 23/30, 18/30, 15/20, 12/20; ResFiT counts are 25/30, 24/30, 19/30, 14/20, 13/20; BC counts are 20/30, 19/30, 13/30, 11/20, 7/20 in Cube, LEGO, P&P, Drawer, Long Horizon order.
Go to primary source ↓e12PDF p. 7, Section 4.3 and Figure 5; p. 8, Section 4.5 Q2; p. 15, Appendix E and Figure 7
Rolling intervention fraction divides total human-controlled steps by total episode lengths across the trailing five episodes. Q2 reports Pull Drawer final values 0.113 versus 0.018 (84%). Figure 5 has early crossings and last plotted points not clearly matching those values. Figure 7 supplies raw episode fractions rather than numerical episode-count logs.
Go to primary source ↓e13PDF p. 8, Figure 6 and Section 4.5 Q3; p. 13, Appendix C, Clean ablation surface
The Pick LEGO/Pull Drawer comparison removes actor use of takeover probability by α_i=0 while preserving WM learning, or removes the WM pathway. Full WHIRL approaches zero earlier; Drawer has an initial cost before crossover.
Go to primary source ↓e14PDF p. 8, Section 5; p. 14, Appendix D, variance discussion
Limitations include in-region evaluation, one seed, one method-aware operator, weak statistical evidence on drawer tasks, and inheritance of operator takeover habits. Traces have no across-seed bands.
Go to primary source ↓e15PDF pp. 11–12, Appendix A, Equations 7–11 and solver schedule
Non-thumb calibration uses three reference postures. Thumb IK aligns fingertip position and opposition direction, with temporal smoothing, damped Gauss–Newton updates and joint limits. Rebase evaluation uses a memoryless glove mapping while continuous control retains a temporal warm start.
Go to primary source ↓e16PDF p. 12, Appendix B and Table 2
Hyperparameters include latent 256; heads 2×512 hidden; ensemble 5; Adam 10^{-4}; batch 128; reward values (10,−0.01); γ=0.97; λ_c=0.05; λ_a=0.40; gate (0.03,0.01); utility coefficients (1,0.4,1); synergy K=3. Exact residual-penalty schedules are deferred to supplementary configurations.
Go to primary source ↓e17PDF pp. 12–13, Appendix C, deterministic one-step design and actor-routing rationale
The authors use a one-step deterministic latent model instead of long imagined rollouts. Reward probability is calibrated as p_r R_succ+(1−p_r)R_step. Actor routing leaves the environment reward unchanged; claims of optimality invariance and no Q leakage appear as design arguments, not a formal proof.
Go to primary source ↓e18PDF pp. 13–15, Appendix D, demonstrations, checkpoint budget, operator procedure and baseline rationale
Demonstrations seed replay; no additional offline data are used. Nominal checkpoints are approximately 8k or 50k steps, while traces log at episode ends. One method-aware operator intervenes at imminent workspace violations or dropped objects/lost closure. Residual RL is an in-house mechanism implementation, not a run of published HIL-SERL or SiLRI code.
Go to primary source ↓e19PDF p. 14, Appendix D, statistical analysis and unnumbered table
WHIRL Wilson 95% intervals are [89,100]%, [83,99]%, [74,97]%, [76,99]%, [64,95]%. One-sided Fisher p-values versus Residual RL are 0.026, 0.026, 0.008, 0.091, 0.078. The five tests receive no multiple-comparison correction.
Go to primary source ↓e20PDF p. 15, Appendix F, Table 3 and Figure 8
FR3 with 16-DoF LEAP uses wrist D405/external D435 RGB at 30 Hz, arm impedance control at 20 Hz and glove/hand streams at 120 Hz; one RTX 4090 is reported. Table 3 specifies workspace/reset values, ±15-cm pick and ±5-cm drawer randomization, and task horizons of 300, 300, 600, 450 and 2100 steps.
Go to primary source ↓8.5 Primary sources
How to Learn from What a Human Would Avoid? Intervention-Aware World Models with Real-World RL for Dexterous Manipulation ↗
PDF · 7,048 extracted words
Source fingerprint
2a926aa63d8ecc5167034069586a7dd27f0a916b098835fab2dd1574da4e8a10