Motus2: A Self-Evolving General World Model for Dexterous Manipulation
1. Paper overview
In one sentence: Motus2 makes action generation, future simulation, and progress evaluation conditional interfaces of one model, enabling optional planning and policy improvement while memory and tactile feedback add distinct deployment costs. E02E03E04E08E09E13E15E16E17
| At a glance | What to know |
|---|---|
| Research problem | Source description Dexterous manipulation combines expensive embodiment-specific demonstrations, visual occlusion, and contact states that vision cannot reliably reveal. The paper seeks to exploit human interaction data and unsuccessful robot experience while preventing undesirable actions from becoming imitation targets and preventing future observations from leaking into deployed action prediction. E02E03E18 |
| Core mechanism | |
| A key reported result | Five-task main manipulation suite: Midtrain-SFT: Place Ball 100%, Multi-Finger 70%, Attach Eraser 100%, Screw Bulb 90%, Put Phone 60%; average 84%. Task success rate and task-macro-average. Physical robot; 20 rollouts per task; matched target-task SFT data and evaluation. WAN-SFT, Pretrain-SFT, and Midtrain-SFT differ in initialization. Pretrain-SFT averages 51%; WAN-SFT and π0.5 each average 0%. Mid-training adds 33 percentage points over Pretrain-SFT. Reported executed-robot results support the staged initialization recipe on this suite. The 84% result belongs to supervised Midtrain-SFT, not a combined demonstration of all proposed extensions. E12E13 |
| Reading caution | Author claim The authors identify wearable tactile noise and morphology mismatch as the primary limitation: glove deformation can produce signals without external contact, and human gloves do not transfer directly across differently shaped robot hands. E18 |
Core contributions
Figure 1. One shared core supports three queries, while tactile refinement forms an additional pathway. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the left half as the modeling interface. Language, action, stereo-video, value-query, and tactile tokens enter their indicated pathways; action and video generation share causal transformer blocks. The tactile expert is drawn separately and refines action sub-chunks. Read the right half as a sequence of conditional queries: the policy proposes action branches Aᵢ, the simulator produces corresponding future latents Zᵢ, and the evaluator assigns values Vᵢ. At the bottom, Maximum selects an action for test-time execution, whereas RL Loss returns a learning signal to the policy. These paths share the core model’s parameters but answer different conditional questions, so the drawing should not be interpreted as three independently trained networks. E02E04E06E09iv01
What it supports. The overview connects a world-action policy to an executable model-based loop: propose, predict, score, then select or improve. It also exposes the boundary of ordinary deployment. The action factor can run alone; imagined futures and evaluator calls are activated when planning or model-based policy optimization needs them.
Where the evidence stops. The diagram combines optional extensions in one overview. It does not mean every result uses planning, full history, and tactile feedback together, or that every control step generates future video.
2. Motivation
2.1 The problem and the proposed response
Dexterous manipulation combines expensive embodiment-specific demonstrations, visual occlusion, and contact states that vision cannot reliably reveal. The paper seeks to exploit human interaction data and unsuccessful robot experience while preventing undesirable actions from becoming imitation targets and preventing future observations from leaking into deployed action prediction. E02E03E18
2.2 What this reading follows
Dexterous manipulation demands more than a plausible action sequence: the hands can hide the object, contact can change rapidly, and a candidate movement can make the task worse. Motus2 builds a shared video–action model that can propose actions, simulate their consequences, and score progress. Its action-first dependency structure allows ordinary control to stop after generating actions; planning activates the additional interfaces. The illustrated evidence separates the large benefit of egocentric pretraining and robot grounding from the smaller measured gains of model-based policy improvement. Separate memory and tactile studies then examine what information the controller needs when recent images are insufficient. These are different controlled studies, with different tasks and operating settings. E02E03E04E08E09E13E15E16E17
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | One Model |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Insufficient evidence to decide
WAM and One Model are supported for the core: action, video, and read-only value tokens share a transformer with explicit causal masks. WM-RL, memory, and tactile subcategories are also supported; the tactile expert has separate parameters, so sharing should not be interpreted as covering every module. Other mechanisms/Outside quadrants remain uncertain without catalog quadrant definitions: the paper combines joint video–action pretraining with action-first conditional control, simulation, and evaluation, rather than inverse-dynamics action recovery. The supplied snapshot is preserved. E02E03E06E08E09
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Make failed experience useful without imitating it
Motus2 factorizes its shared model as p(A,Z,Y | c) = π(A | c) p(Z | c,A) p(Y | c,A,Z). The context c contains instruction, proprioception, and accessible visual history; A is an action chunk, Z its future observations, and Y a categorical progress target. The action-first mask enforces this order, but supervision gates determine which trajectories teach which factor. Successful curated demonstrations activate action and video learning. Simulation mode conditions on clean recorded actions while predicting future video; evaluation mode conditions on clean actions and video while learning progress. Failed or irrelevant behavior can therefore teach transitions and unfavorable outcomes without becoming an imitation target. My interpretation is that this separation is central to the “self-evolving” claim: information about what happened and preference for what should happen enter different supervised pathways. E02E03E04E05
Figure 2. The decisive mask change prevents action queries from reading their own predicted future. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Take a row as the querying group and a column as the group it can read. In the joint mask on the left, A₁ can read Z₁ and Z₁ can read A₁; their within-chunk modeling is bidirectional. In the action-first mask on the right, the A₁-to-Z₁ cell is white, but the Z₁-to-A₁ cell remains colored. U₁ can then read the action and its predicted consequence, while its column is hidden from other token groups. Repeat this inspection for later chunks. Context groups C provide clean observed history under the windowed causal structure. The muted U positions on the left indicate that value queries are inactive during joint pretraining. E03E04iv02
What it supports. The mask realizes the factorization policy → simulator → evaluator. Actions depend on available context, future predictions can depend on those actions, and values can inspect both. Removing the reverse future-to-action edge is what lets the adapted policy produce an action chunk without materializing that chunk’s future video during ordinary control.
Where the evidence stops. A mask specifies permitted access, not whether learned features exploit it well. Loss routing is a separate requirement: only successful trajectories supervise action imitation, while other trajectories can still supervise transitions or evaluation.
5.2 Turn imagined progress into selection or a policy update
For a candidate action Aᵢ, the simulator predicts Zᵢ and the evaluator estimates a progress value Vᵢ. Best-of-N planning executes the highest-valued branch and replans after a real observation. MBRL instead uses the same values to change action-related parameters through DiffusionNFT. Scores are centered and normalized within the generated candidate group, clipped, and mapped into weights between zero and one. Higher weights emphasize a positive velocity field around an EMA reference; lower weights emphasize its negative counterpart. A ground-truth action–future pair anchors each group positively and is excluded from score normalization. The predicted future affects the action update through its score, not as an input to the action predictor. This distinction explains Table 3: candidate selection and proposal learning can help separately, but both depend on the reliability of learned simulation and evaluation. E05E06E07E15E21
5.3 Separate missing history from missing contact information
A hand can obscure an object that was visible earlier, or conceal a contact force that was never observable in RGB. Motus2 addresses these as separate information problems. Global autoregression retains every past clean visual latent; hybrid memory trades some of that detail for persistent compressed tokens and a recent window. Their task-specific comparison favors global history, with cost growing over time. The tactile expert instead operates at a shorter temporal scale: the backbone denoises a 48-action chunk to an intermediate noise level once, and a small expert finishes successive six-action sub-chunks using newly acquired touch. Detached K/V features are reused across the eight refinements. Future force is predicted during training only. My reading is that these extensions support different causal explanations—remembering earlier evidence and measuring current contact—and their separate ablations should remain separate when interpreting the overall system. E08E09E16E17E22E23
5.4 Training and inference
During training
Initialization uses Wan 2.2-TI2V-5B. Video-only monocular training runs for 500K low-resolution and 340K high-resolution steps; joint stereo video–action training follows for 450K steps. The approximately 130K corpus hours describe raw recordings before filtering, not effective training hours. E10E20
Mid-training uses more than 100 hours of robot and supplementary alignment data. Simulation and evaluation receive 10% and 5% of samples; the remainder use policy mode. The evaluator uses 201 bins and value-loss weight 0.05. Target-robot supervised adaptation retains the action-first mask. E21
MBRL generates eight candidates per prefix plus a ground-truth anchor, with a one-chunk imagined horizon. DiffusionNFT updates only action-related parameters; the video backbone and evaluator stay frozen. A versioned EMA reference supports asynchronous rollout and optimization, with stale or excessively delayed samples discarded. E06E07
Long-history variants train on complete episodes packed with masks that prevent cross-episode attention. Tactile losses update the expert and its modality encoders/decoder using detached backbone states. Force prediction has weight 0.1 and is auxiliary to action refinement. E08E22E23
During inference
Ordinary control queries only the action factor; default deployment uses five action-denoising steps. Future video and values need not be generated. After execution, actual observations refresh the policy context. E04E08E21
Planning proposes N chunks, simulates and scores each, executes the highest-valued chunk, and replans after receiving the real observation. Tactile deployment outputs refined actions; it does not require future-force prediction. E06E09
5.5 Implementation flow
- Standardize human interaction data
Synchronize recordings, wrist poses, and language annotations. A 134-dimensional representation combines bilateral wrist poses with wrist-local finger keypoints. Human fingers are retargeted offline to first-generation 20-DoF Wuji joints using constrained geometric optimization; target-robot post-training instead uses recorded native joint angles. E10E11
- Construct the shared causal computation
Language enters through cross-attention; stereo video, proprioception, actions, and value queries participate in the shared transformer. Pretraining allows mutual video–action visibility within a chunk. Robot adaptation blocks action tokens from reading that chunk’s future video or value query. Video can read actions, and the read-only value query can read both. E02E03
- Route supervision by trajectory quality
Curated successful trajectories supply action targets. Simulation mode conditions on clean recorded actions and learns future video; evaluation mode conditions on clean actions and video and learns progress. Failed, suboptimal, and task-irrelevant interactions enter applicable simulation or evaluation modes without supervising imitation. E03E04
- Learn a branch evaluator
Successful segments receive positive relative-progress targets and negative trajectories receive their signed counterpart. A categorical readout predicts discretized targets, and its expectation ranks candidate action–future pairs. These values are explicitly not calibrated success probabilities. E02E05E24
- Retain observation history
The default sliding window stores actual observed latents, evicts old entries, and rebases temporal RoPE. Global autoregression retains every preceding observation at growing cache and attention cost. Hybrid memory retains initial anchors and recent frames at full resolution while compressing intermediate history into persistent tokens. E08E22
- Refine actions with tactile feedback
The backbone partially denoises a 48-action chunk to σc = 0.2 and supplies detached layer-wise caches. At 30 Hz action execution, the tactile expert refines eight successive six-action sub-chunks using the latest tactile window. Future-force tokens can read actions, but action refinement cannot read noised future forces. E09E23
6. Experiments & results
Motus2 adapts a jointly pretrained video–action transformer into an action-first controller: propose an action chunk, optionally simulate its visual consequences, and evaluate the branch using learned relative progress. Best-of-N planning selects branches, while DiffusionNFT updates action parameters using the same scores. Separate studies examine full-history context and tactile refinement. The main physical-task suite reaches 84% average success after robot-domain mid-training; policy improvement and the other extensions are evaluated separately. [E02, E06, E13, E15–E17]
6.1 Read the original evidence
Table 2. Egocentric pretraining and robot-domain grounding are separate steps in the main comparison. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The three rows ending in SFT use the same target-task supervised adaptation procedure but start from different checkpoints. WAN-SFT begins from the video foundation model, Pretrain-SFT adds the two-stage egocentric curriculum, and Motus2 begins after robot-domain mid-training. Each task entry is a success rate over twenty matched configurations; partial completion counts as failure. Read across the five tasks before comparing Avg. SR, which gives equal weight to each task. Place Ball and Put Phone test placement, Attach Eraser and Screw Bulb involve alignment and sustained contact, and Multi-Finger tests coordinated manipulation. The π0.5 row uses the matched target-task data and evaluation interface described in the setup. E12E13iv03
What it supports. The initializations produce 0%, 51%, and 84% mean success for WAN-SFT, Pretrain-SFT, and Motus2, respectively. Mid-training adds 33 percentage points over egocentric pretraining alone. Even the strongest row remains uneven: Place Ball and Attach Eraser reach 100%, while Put Phone reaches 60%.
Where the evidence stops. Zero success for π0.5 and WAN-SFT is specific to these tasks and adaptation conditions. This table is not a universal ranking, and it does not itself isolate planning, tactile feedback, or memory extensions.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Five-task main manipulation suite Physical robot; 20 rollouts per task; matched target-task SFT data and evaluation. WAN-SFT, Pretrain-SFT, and Midtrain-SFT differ in initialization. | Midtrain-SFT: Place Ball 100%, Multi-Finger 70%, Attach Eraser 100%, Screw Bulb 90%, Put Phone 60%; average 84%. Task success rate and task-macro-average | Pretrain-SFT averages 51%; WAN-SFT and π0.5 each average 0%. Mid-training adds 33 percentage points over Pretrain-SFT. Reported executed-robot results support the staged initialization recipe on this suite. The 84% result belongs to supervised Midtrain-SFT, not a combined demonstration of all proposed extensions. E12E13 |
| Policy optimization and test-time selection Put Phone and Multi-Finger; physical robot; 20 rollouts per task; factorial comparison from the same target-robot checkpoint. | Baseline: 60%/70%, average 65.0%; planning: 65%/70%, average 67.5%; MBRL: 65%/80%, average 72.5%; both: 70%/80%, average 75.0%. Success rate | Planning adds 2.5 percentage points with either policy; MBRL adds 7.5 points with either inference setting. The measured improvements distinguish changing candidate selection from changing policy weights. Their statistical reliability is unresolved because uncertainty estimates are not reported. E12E15 |
| Long-horizon context probes in simulation Find Square and Press Button; 25 rollouts per task; matched training and paired configurations. | Global autoregression: 84%/72%, average 78%. Success rate | Hybrid memory: 64%/40%, average 52%. Full observation retention outperforms the tested compressed representation in simulation. This table does not include the default sliding-window baseline. E16E22 |
| Long-horizon context probes on the physical robot Find Square and Press Button; 20 rollouts per task; matched training and paired configurations. | Global autoregression: 65%/50%, average 57.5%. Success rate | Hybrid memory: 30%/20%, average 25.0%. The ordering agrees with simulation, but these physical execution rates must remain separate from the simulation results. E12E16 |
| Contact-sensitive manipulation Pull Out Paper Cup with Sharpa Wave and Tear Paper with Wuji Hand 2; 20 physical rollouts per task; matched checkpoint and post-training data. | With tactile expert: 75%/70%, average 72.5%. Success rate | Without tactile expert: 65%/55%, average 60.0%; average gain 12.5 percentage points. This establishes a benefit for the complete tactile package. It does not isolate the contributions of refinement, auxiliary force prediction, or added model capacity. E17E23 |
| Stereo human-action prediction scaling Nested 2k, 4k, 10k, and 20k raw-hour subsets; common trajectory-disjoint held-out set; best validation epoch selected at each scale. | Reported fitted relation: L*val(D) ≈ 0.101 − 0.005 ln D. MSE between the mean sampled action prediction and normalized ground-truth wrist/hand actions | The text reports monotonically decreasing best validation error across the four scales. This is a fitted human-action prediction trend over the measured range, not an executed-robot success result. Individual curve values are omitted because the figure was not visually inspected. E14 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 3. Planning changes candidate selection; model-based learning changes the proposal policy. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The four rows form a two-by-two design from the same target-robot checkpoint. Compare the first and second rows to add planning without updating the policy; compare the first and third to add MBRL under direct inference. The last row activates both. Put Phone and Multi-Finger each have twenty evaluation rollouts, so one additional success changes a task’s entry by five percentage points and the two-task average by 2.5 points. Read the task columns as well as Avg. SR: planning raises Put Phone in this sample but leaves Multi-Finger unchanged. MBRL updates action-related parameters, whereas the simulator and evaluator remain fixed during that optimization. E06E07E15E21iv04
What it supports. Planning adds 2.5 points to the two-task mean both before and after MBRL. MBRL adds 7.5 points with or without planning, bringing the combined system to 75.0%. The table supports distinct observed contributions from selecting among candidates and improving the distribution that generates them.
Where the evidence stops. The planning increment corresponds to one additional successful rollout across forty trials. The table provides neither confidence intervals nor a matched wall-clock comparison, so broader reliability and compute-efficiency claims remain unresolved.
Table 4. Keeping all observations wins these memory probes, with a growing history cost. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Keep the Simulation and Real robot blocks separate: they use twenty-five and twenty rollouts per task, respectively. Within each block, compare hybrid memory against global autoregression under matched training and paired configurations. Both variants are trained on complete episodes, but their retained information differs. Global autoregression keeps all prior clean visual latents. Hybrid memory retains initial anchors and recent frames at full resolution while compressing older observations into memory tokens. Find Square and Press Button were selected because useful evidence can be delayed or occluded. Their averages diagnose these long-history mechanisms; the table does not include the bounded sliding-window default as another row. E08E16E21E22iv05
What it supports. Global autoregression reaches a 78% simulation mean and 57.5% real-robot mean, compared with 52% and 25% for hybrid memory. The direction agrees across both tasks and settings. The results favor retaining full visual history on these probes, while leaving the deployment cost of that additional context to a separate measurement.
Where the evidence stops. Global attention and KV storage grow with episode length. These variants also use fifty denoising steps in the reported comparison, unlike the default five-step controller; their success rates should not be paired with default-runtime assumptions.
Table 5. Fresh tactile refinement helps on two contact-sensitive tasks. Original paper, p. 15 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The rows begin from the same mid-trained checkpoint and use the same target-task post-training data. The upper row removes the tactile expert; the lower row includes both its action-refinement and future-force prediction objectives. Read the two task columns separately: Pull Out Paper Cup uses the Sharpa Wave hand, and Tear Paper uses Wuji Hand 2. Each rate comes from twenty rollouts, with complete task success as the criterion. The average weights both tasks equally. The mechanism behind the lower row uses recent tactile measurements to finish denoising short action sub-chunks while reusing detached backbone features; future-force prediction supplies a training target rather than a deployed output. E09E17E23iv06
What it supports. Tactile feedback raises paper-cup success by 10 percentage points and tear-paper success by 15 points, giving a 12.5-point mean gain. These outcomes support the complete tactile extension on the evaluated contact tasks. They do not yet identify how much of the gain comes from fresh sensing versus the auxiliary force-prediction objective.
Where the evidence stops. The comparison removes or adds the entire tactile expert. It does not separate sensor information, extra model capacity, refinement cadence, and force prediction, or establish transfer between human and robot tactile interfaces.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors identify wearable tactile noise and morphology mismatch as the primary limitation: glove deformation can produce signals without external contact, and human gloves do not transfer directly across differently shaped robot hands. E18
The physical studies use 20 rollouts per task without reported confidence intervals or repeated-training-seed estimates. Separate task suites test MBRL, memory, and tactile sensing, so the reported gains do not establish the performance of a single configuration combining every extension. E12E13E15E16E17
Self-evolution is demonstrated through policy updates with frozen dynamics and value components. The experiments do not quantify simulator error, imagined-branch ranking accuracy, or sustained improvement across repeated real-world data-collection cycles. Shared model biases could therefore influence both imagined outcomes and their scores. E07E15E24
Signing targets according to trajectory outcome can label useful prefixes of failed executions negatively. Whether this penalizes recoverable behavior is untested. The qualitative evaluator examples use executed actions and observed video, which does not directly establish reliability on generated futures. E05E24
The memory comparison lacks a sliding-window baseline and measured latency or memory costs. The tactile comparison changes both refinement and prediction machinery, leaving the auxiliary prediction objective's independent effect unresolved. E08E16E17E22
Qualitative evidence has internal documentation mismatches. Section 5.4 describes Screw Bulb examples that differ from Appendix F's four-task account. Appendix F also describes phone-box closure and separate-container sorting, whereas Appendix G specifies a phone stand and an orange basket. Those trajectories cannot be assumed to validate the main-suite task checkers. E25
7.2 Questions for discussion
- Would segment-level recovery labels improve over signing every segment by trajectory outcome? [E05, E24]
- How accurately do evaluator rankings on imagined branches predict real execution outcomes before and after NFT optimization? [E06, E15]
- Can hybrid memory approach full-history success under matched latency, and does force prediction help beyond tactile refinement alone? [E16, E17, E22]
8. Reproducibility audit
8.1 Requirements and known gaps
The supplied title, complete author list, and identifier match the catalog; the cover identifies arXiv:2608.30237v1, dated 31 August 2026. This review uses the supplied artifact provenance. No code was inspected and no experiments were reproduced. E01
Reproducing the full curriculum requires the Wan initialization, access to the mixed open and procured corpus, synchronized pose processing and retargeting, robot/alignment trajectories, and the relevant bimanual hardware. Source-specific filtering thresholds, exact target-task data quantities, and complete training compute are not specified sufficiently for an exact rerun. E10E11E19E20E21
Several settings are concrete: default action sampling uses five steps, while the memory comparison uses 50; MBRL rollout action/video sampling uses four/eight steps, β = 0.1, EMA decay 0.99, publication every ten updates, and a 600-second sample-age cutoff. Comparisons must preserve these distinct protocols rather than silently sharing one sampler configuration. E07E21E22
A focused reproduction would repeat the policy-optimization/planning factorial study from one checkpoint with paired physical configurations, then test whether branch rankings agree with actual outcomes. This directly probes the proposed feedback mechanism without requiring a new large-scale pretraining run. E06E15
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does predicted progress rank real consequences?
Proposed check: freeze the policy and generate the same candidate set for every matched task prefix. Compare learned-value ranking with random ranking, shuffled values, and an offline oracle based on realized outcomes in resettable evaluation states. Separately score real future observations and model-predicted futures to localize simulator versus evaluator error. Report pairwise ranking accuracy, calibration, success, and compute at a fixed candidate count. A learned ranking advantage that survives matched sampling would support the planning mechanism; systematic high scores for failed imagined branches would reveal exploitable model error. E05E06E15E24
Check 2: What history benefit survives a fixed inference budget?
Proposed check: compare the bounded sliding-window default, hybrid memory, and global autoregression using identical demonstrations, denoising steps, and paired Find Square/Press Button resets. Vary the delay between informative observation and required action, recording peak KV memory and control latency alongside success. Repeat with a strict per-decision compute budget. The falsifiable prediction is that full history helps increasingly after evidence leaves the recent window, but may lose its advantage when attention growth reduces feedback frequency. This would distinguish information retention from the operating budget used in the reported table. E08E16E21E22
8.3 Reading coverage
Visual audit: Original PDF pages 4, 6, 12, 14, and 15 and all six final crops were visually inspected. Figure 1 interfaces, Figure 2 attention cells, Table 2 main results, Table 3 factorial planning/MBRL comparison, Table 4 memory study, and Table 5 tactile study were read with their source protocols. Every table header and denominator context was checked; cropped graphics retain their original content. The PDF SHA-256 was recomputed and matches the base report. Other figures, external demonstrations, code execution, value calibration, and robot replication remain outside this illustrated pass.
PDF pages inspected for this edition: 4, 6, 12, 14, 15. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1. Introduction
- 2. Related Work
- 3. Method
- 3.1. Overview
- 3.2. General World Modeling
- 3.3. Value-Guided Closed-Loop Self-Evolution
- 3.3.1. Progress-Based Value Learning
- 3.3.2. Planning and Policy Optimization
- 3.4. Working Memory
- 3.5. Tactile Expert: Tactile Refinement and Prediction
- 4. Large-Scale Egocentric Dataset
- 5. Experiments
- 5.1. Experiment Setup
- 5.2. Main Results
- 5.3. Scaling Laws of Stereo Egocentric Human Data
- 5.4. Model-Based Policy Improvement and Test-Time Planning
- 5.5. Long-Horizon Context Mechanisms
- 5.6. Effect of Tactile Feedback
- 6. Conclusion
- 7. Limitations and Future Work
- Acknowledgments
- References
- A. Robot System and Teleoperation Stack
- A.1. Robot Platforms
- A.2. Teleoperation
- B. Human–Robot Alignment Data Collection
- C. Egocentric Data Curation and Offline Annotation
- C.1. Pose Standardization
- C.2. Quality-Aware Temporal Processing
- C.3. Language Annotation
- D. Hand Retargeting Algorithm
- E. Model and Training Details
- E.1. Pre-training
- E.2. Mid-training
- E.3. Post-training
- F. Qualitative Value-Model Trajectories
- G. Evaluation Tasks
Outside the original text pass
- Review covers the supplied text, including appendix prose and figure captions; no PDF figure images were visually inspected.
- Numerical curves were not reconstructed from extracted figure labels.
- Separate supplemental material availability and completeness remain unverified.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
E01PDF p. 1, title, author block, and arXiv version line
The title and ordered author list match the catalog. The cover identifies arXiv:2608.30237v1, 31 August 2026, and lists a project-page link.
Go to primary source ↓E02PDF pp. 4–5, Section 3.1, Eqs. (1)–(3)
The POMDP observation includes vision, proprioception, and optional tactile sensing. A shared density factors into action policy, action-conditioned future prediction, and categorical progress evaluation.
Go to primary source ↓E03PDF pp. 5–6, Section 3.2, Eqs. (4)–(7)
Token masks enforce action-first dependencies; all modalities use the shared transformer. Independent noise levels and loss gates select policy, simulation, and evaluation supervision.
Go to primary source ↓E04PDF p. 6, Section 3.2, supervision routing and Action-first inference
Only curated successes activate action supervision. Ordinary deployment reads the action factor, while planning or optimization can additionally generate future video and values.
Go to primary source ↓E05PDF p. 7, Section 3.3.1, Eqs. (8)–(9)
Successful segments receive Δt/(T−t); failed and task-irrelevant segments receive its negative. Targets are discretized for categorical value learning.
Go to primary source ↓E06PDF p. 8, Section 3.3.2, Model-Based Planning and Eqs. (10)–(13)
Planning executes the highest-valued candidate and replans with real observations. NFT normalizes generated scores, excludes the ground-truth anchor from statistics, and updates action velocities without future-video conditioning.
Go to primary source ↓E07PDF p. 9, Asynchronous MBRL Training; p. 25, Appendix E.3, MBRL and planning
The asynchronous pipeline freezes video and value components. It uses eight generated candidates plus an anchor, a one-chunk horizon, four/eight action/video denoising steps, β=0.1, EMA decay 0.99, ten-update reference publication, and a 600-second age cutoff.
Go to primary source ↓E08PDF pp. 9–10, Section 3.4
Sliding memory retains observed latents with eviction and RoPE rebasing; global autoregression retains complete history; hybrid memory compresses older observations. Episode packing prohibits cross-episode attention.
Go to primary source ↓E09PDF p. 10, Section 3.5, Eqs. (14)–(16)
A tactile expert completes partially denoised actions using detached backbone caches. Auxiliary future-force prediction cannot feed into action-refinement tokens and is used only during training.
Go to primary source ↓E10PDF pp. 10–12, Section 4 and Table 1
The corpus totals approximately 130,000 raw recording hours before filtering and annotation, combining open-source and procured monocular/stereo data. Alignment collection is separate.
Go to primary source ↓E11PDF pp. 22–24, Appendices C–D, Eqs. (20)–(25)
Data use a 134-dimensional bilateral pose representation, quality-aware segmentation, and language annotation. Human keypoints are retargeted to 20-DoF Wuji joints; target-robot data retain native recorded joints.
Go to primary source ↓E12PDF p. 12, Section 5.1
Physical evaluations use 20 configurations per task and the simulation memory study uses 25. Each configuration receives one rollout; all task predicates must be met, and averages weight tasks equally.
Go to primary source ↓E13PDF p. 12, Table 2, all method rows; p. 13, Section 5.2 Results
Midtrain-SFT reports 100/70/100/90/60% across the five tasks and 84% average. Pretrain-SFT averages 51%; WAN-SFT and π0.5 report zero throughout.
Go to primary source ↓E14PDF p. 13, Section 5.3, Eqs. (17)–(18)
Nested 2k/4k/10k/20k raw-hour subsets share a trajectory-disjoint validation set. Best-epoch normalized human-action MSE is fitted by 0.101−0.005 ln D.
Go to primary source ↓E15PDF p. 14, Section 5.4 and Table 3, all rows
Baseline, planning, MBRL, and combined variants report Put Phone/Multi-Finger success of 60/70, 65/70, 65/80, and 70/80%, respectively.
Go to primary source ↓E16PDF p. 14, Section 5.5 and Table 4, simulation and real-robot rows
Hybrid/global averages are 52/78% in simulation and 25/57.5% on the robot. The comparison includes Find Square and Press Button but no sliding-window row.
Go to primary source ↓E17PDF p. 15, Section 5.6 and Table 5
The complete tactile variant improves cup extraction from 65 to 75% and paper tearing from 55 to 70%, using matched starting checkpoint and post-training data.
Go to primary source ↓E18PDF pp. 15–16, Section 7
The stated limitations concern noisy glove signals from deformation and the inability to transfer one glove geometry directly between human and robot embodiments.
Go to primary source ↓E19PDF pp. 21–22, Appendices A–B
The paper describes Tianji dual-arm systems with Wuji, Wuji Hand 2, or Sharpa Wave hands and platform-specific teleoperation. Human alignment records stereo views and tracker-relative wrist poses without commanding a robot.
Go to primary source ↓E20PDF p. 24, Appendix E.1
Wan 2.2-TI2V-5B initializes video-only training for 500K and 340K steps, followed by 450K joint stereo video–action steps. Stage 2 uses 12 latent frames and two-frame chunks.
Go to primary source ↓E21PDF p. 25, Appendix E.2 and E.3, Sliding-window SFT
Mid-training uses over 100 hours, simulation/evaluation fractions 0.10/0.05, 201 value bins, and value weight 0.05. Default SFT uses an eight-frame window, two-frame chunks, a two-chunk cache, and five deployment denoising steps.
Go to primary source ↓E22PDF p. 25, Appendix E.3, Global autoregression and Hybrid working memory
Hybrid memory uses two anchor frames, four recent frames, and eight memory tokens per latent frame. Both long-context variants use 10K training steps and 50 evaluation denoising steps.
Go to primary source ↓E23PDF pp. 25–26, Appendix E.3, Tactile expert
The 30-layer expert uses width 128. A 48-action chunk at 30 Hz is divided into eight six-action sub-chunks after denoising to σc=0.2. Forces are sampled at 90 Hz; auxiliary prediction weight is 0.1.
Go to primary source ↓E24PDF pp. 26–27, Appendix F, Protocol and trajectory descriptions
Qualitative values condition on executed actions and observed video, without the eventual outcome as input. Values are relative-progress scores, explicitly not calibrated success probabilities.
Go to primary source ↓E25PDF p. 14, Section 5.4, Qualitative value trajectories; pp. 26–27, Appendix F; p. 30, Appendix G
Section 5.4 references Screw Bulb trajectories, while Appendix F discusses Cut Reeds, Make Tea, Multi-Finger, and Put Phone. Appendix F's box-closure and separate-container requirements differ from Appendix G's phone-stand and orange-basket descriptions.
Go to primary source ↓iv01PDF p. 4, Figure 1; original page and final crop visually inspected at 200 DPI
Figure 1 was visually inspected: shared Video-Action Causal DiT blocks sit beside a tactile expert; its right chain explicitly labels Policy (WAM), Simulator (AC-WM), Evaluator (VM), RL Loss, and Maximum. Candidate action, frame, and value branches are retained in the crop.
Go to primary source ↓iv02PDF p. 6, Figure 2; original page and final crop visually inspected at 200 DPI
Figure 2 was visually read row by row for the first action/video/value block and checked against later blocks. The right A1 row masks Z1 and U1; Z1 reads A1; U1 reads A1 and Z1; other queries cannot read U1. The left joint mask permits A1/Z1 mutual attention and leaves value groups inactive.
Go to primary source ↓iv03PDF p. 12, Table 2; original page and final crop visually inspected at 200 DPI
Table 2 was visually inspected, preserving all five tasks and the average column. π0.5 and WAN-SFT are zero throughout; Pretrain-SFT is 60/35/90/55/15, mean 51; Motus2 Midtrain-SFT is 100/70/100/90/60, mean 84. Source setup states twenty configurations per task and matched SFT.
Go to primary source ↓iv04PDF p. 14, Table 3; original page and final crop visually inspected at 200 DPI
Table 3 was visually inspected: baseline 60/70/65.0; planning 65/70/67.5; MBRL 65/80/72.5; both 70/80/75.0 for Put Phone/Multi-Finger/average. Caption and surrounding protocol specify the same checkpoint and twenty trials per task.
Go to primary source ↓iv05PDF p. 14, Table 4; original page and final crop visually inspected at 200 DPI
Table 4 was visually inspected: Simulation hybrid/global results are 64/40/52 and 84/72/78; Real robot results are 30/20/25.0 and 65/50/57.5. The original table contains no sliding-window row. Appendix E.3 specifies matched long-history training and fifty denoising steps.
Go to primary source ↓iv06PDF p. 15, Table 5; original page and final crop visually inspected at 200 DPI
Table 5 was visually inspected with all task and average headers: without tactile, 65/55/60.0; with tactile, 75/70/72.5. The source protocol identifies matched checkpoint/data, twenty trials per task, and the complete refinement-plus-prediction extension.
Go to primary source ↓8.5 Primary sources
Motus2: A Self-Evolving General World Model for Dexterous Manipulation ↗
PDF · 13,291 extracted words
Source fingerprint
11e5a000cd5064307707e6e5ee842bb3a4f887c5f7833b4fbb22d1df772def97