Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics
1. Paper overview
In one sentence: RWM exposes a recurrent simulator to its own prediction errors during training, enabling long imagined PPO rollouts at the cost of sequential training and continued reliance on simulation. e-are-architecturee-policye-ablatione-hardwaree-online
| At a glance | What to know |
|---|---|
| Research problem | Source description Low-level robotic control combines partial observability, stochasticity and abrupt contact changes. Small prediction errors can compound during imagined policy training, allowing a controller to exploit an inaccurate simulator. RWM targets useful long rollouts with limited environment interaction. e-probleme-policy |
| Core mechanism | Source description A dual-autoregressive GRU combines recurrent processing of observation-action history with feedback of predicted observations during training. e-are-architecture |
| A key reported result | Real tracking reward and training cost: MBPO-PPO: 0.90 ± 0.04; policy training 5 min; inference 1 ms. RWM pretraining: 6M transitions and 50 min. Real tracking reward, training time, state transitions and step inference time. Table 1 compares RWM pretraining plus MBPO-PPO with simulator-trained PPO; no robot-specific split or hardware trial protocol is identified. PPO: 0.90 ± 0.03; 250M transitions; 10 min training; 1 ms inference. Displayed mean tracking rewards are equal. Pretraining adds cost; the dash for MBPO-PPO transitions is not zero. Uncertainty type, trial count and reward normalization are unspecified, preventing equivalence or end-to-end sample-efficiency claims. e-table |
| Reading caution | Source description The authors acknowledge advantages of well-tuned model-free methods with high-fidelity simulation. Hardware online learning remains deferred: model exploitation causes collisions, recovery is difficult, and privileged signals may require unavailable sensors. Appendix A.4.4 reports more than 20 failures on average during online learning in simulation. e-tablee-online |
Core contributions
- Source description
A dual-autoregressive GRU combines recurrent processing of observation-action history with feedback of predicted observations during training. e-are-architecture
- Source description
MBPO-PPO trains a separate policy on long imagined trajectories; evaluations span manipulation and locomotion prediction, with physical policy deployment on ANYmal D and Unitree G1. e-policye-generalitye-hardware
- Reader analysis
The authors emphasize avoiding domain-specific architectural biases. This concerns the predictor: the control system still uses selected proprioceptive variables, privileged contact information and task-designed rewards. e-architecturee-spacese-rewards
Figure S6. Two recurrence loops expose the model to its own forecast history. Original paper, p. 16 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the upper window labeled 'from step t.' The yellow observation circles and action labels feed the gray hidden-state squares, whose horizontal arrows carry recurrent history. The first predicted observation branches from the current hidden state; the dashed continuation feeds that prediction, together with the next action, into the next hidden state and forecast. Vertical L_o links compare predictions with ground-truth targets. The lower window shifts the training start by one step. Read these as two sliding training examples, not two separate networks. Section 3.2 and Equation (2) explain the additional privileged-information loss that this simplified diagram omits. e-are-architecture
What it supports. The model learns both to summarize history and to continue from imperfect predicted observations. This makes the forecast distribution part of training, providing the mechanism behind the paper's long-rollout objective. Teacher forcing retains the history encoder but stops after a single predicted step; it does not supply the same multi-step feedback.
Where the evidence stops. The drawing illustrates the information flow, not a guarantee of stability. It omits the Gaussian output heads and privileged-information branch. No stop-gradient marker is shown; the text explicitly describes reparameterized end-to-end training.
2. Motivation
2.1 The problem and the proposed response
Low-level robotic control combines partial observability, stochasticity and abrupt contact changes. Small prediction errors can compound during imagined policy training, allowing a controller to exploit an inaccurate simulator. RWM targets useful long rollouts with limited environment interaction. e-probleme-policy
2.2 What this reading follows
A robot controller can learn to exploit a simulator's mistakes as readily as it can learn useful motion. RWM addresses this problem by training a recurrent world model on the observations it predicts itself, so errors encountered in imagination also appear during learning. A separate PPO policy then learns inside that model. The important reading path runs from the two recurrence loops, through the horizon ablation, to policy reward and physical deployment. The experiments support useful long-rollout prediction and hardware transfer, while the appendix makes clear that safe online policy learning on hardware remains an unfinished objective. e-are-architecturee-policye-ablatione-hardwaree-online
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Supports the recorded classification
The recurrent predictor and separate MLP actor support Dual-system. Action-conditioned forward dynamics supplies PPO training experience; neither joint future/action generation nor inverse dynamics produces controls. Other mechanisms and Outside quadrants are appropriate. Memory and WM-RL fit, although the actor is learned from scratch rather than post-trained from an existing policy. e-architecturee-policye-traininge-pretraining
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Separate history memory from forecast feedback
Imagine predicting the next joint and base measurements after a commanded joint-position target. A single observation may leave relevant dynamics hidden, so RWM first processes an observation-action history through a GRU. That is the inner recurrence. It then samples its next observation and uses that prediction as input for another forecast: the outer recurrence. Training compares the resulting sequence against recorded future observations and privileged signals. This distinction matters because a recurrent network can still be trained with teacher forcing and encounter unfamiliar prediction errors when rolled out. Figure 2 makes that comparison with a short example; its caption calls the history horizon H, while the method equations and hyperparameter tables call it M. Figure S6's arrows agree with the text's two-loop description. e-are-architecture
5.2 Turn a predictor into a simulator for PPO
During world-model training, recorded actions condition the prediction targets. During policy optimization, the action source changes: a separate actor reads an imagined observation and supplies the next joint target. RWM predicts the resulting observation and privileged signals, from which the specified reward terms are computed. Predicted failures also end imagined episodes. PPO updates the actor from these trajectories, while new environment data correct the model in the alternating training loop. The reported locomotion setup first pretrains RWM on simulation data and then fine-tunes it with a single simulation environment. The actor itself starts from scratch in imagination. Thus the method's online-data loop is part of training; the physical demonstration is deployment of the learned controller. e-policye-spacese-rewardse-pretraininge-onlinee-hardware
5.3 Ask which experiment isolates the central mechanism
The broad task chart is persuasive about the combined recipe, but it mixes architecture and training regime. The paper itself says an autoregressively trained RSSM becomes comparable to RWM. Figure S8 offers a more direct test of forecast exposure: hold history fixed and vary how many predicted steps enter the training loss. Its error-and-time pair reveals both the benefit and its computational price. Figure 5 then asks a different question: whether the resulting model supports a useful policy when optimization can exploit its mistakes. Reader interpretation: these experiments form a chain of evidence, not interchangeable success metrics. Low prediction error supports the simulator, simulator reward supports policy learning, and physical deployment supports transfer. None by itself establishes safe autonomous learning on hardware. e-generalitye-ablatione-policy-resultse-hardwaree-online
5.4 Training and inference
During training
Locomotion models are pretrained on simulation trajectories from policies for similar tasks under varied dynamics, then fine-tuned with a single simulation environment. The policy starts from scratch in imagination. Manipulation experiments do not need this pretraining. e-pretraininge-online
World-model training uses M=32, N=8, forecast decay 1.0, batch size 1024, learning rate 10⁻⁴ and weight decay 10⁻⁵. Reparameterization supports gradients through sampled predictions. Table S10 reports 2500 iterations and five seeds. e-are-training
Policy and value networks are three-layer 128-unit ELU MLPs. Table S11 gives 4096 imagination environments, 100 steps per iteration at 0.02 seconds, PPO learning rate 0.001, discount 0.99 and clip range 0.2. Both predictor and policy are updated across the training loop. e-policye-training
During inference
In imagination, the policy reads predicted observations and supplies actions; the model supplies resulting observations. On hardware, the trained policy receives robot observations and outputs joint targets. No online world-model planning is described for deployment. e-policye-spacese-hardware
Contact prediction supplies auxiliary supervision and imagined rewards/terminations. It is not an inverse-dynamics action decoder. e-are-rewardse-pretraining
5.5 Implementation flow
- Represent the robot
World-model observations have 45 dimensions for ANYmal D and 96 for G1; actions are 12 and 29 joint-position targets. Policy inputs differ, including velocity commands and last actions. Privileged targets cover contacts and, for G1, foot height and velocity. e-spaces
- Encode history and forecast recursively
A two-layer 256-unit GRU feeds 128-unit ReLU MLP heads for distribution means and standard deviations. Inner recurrence processes history; outer recurrence feeds sampled predictions back with subsequent actions. Sliding windows contain M history steps and N forecast targets. e-are-architecture
- Train a controller in imagination
Algorithm 1 alternates data collection, world-model updates, buffer-initialized imagination and PPO updates. Rewards are computed from predictions using tracking and regularization terms. Predicted failures end imagined episodes and affect returns. e-policye-rewardse-pretraining
6. Experiments & results
RWM trains a recurrent neural simulator on its own multi-step predictions, then uses it to train a separate PPO controller. The paper demonstrates trajectory prediction and zero-shot locomotion-policy transfer, while retaining simulation pretraining and simulation-based online data collection. Its central tradeoff is better long-rollout fidelity at greater world-model training cost.
6.1 Read the original evidence
Figure 4. Broad task coverage supports autoregressive training, with an architectural comparison caveat. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each group corresponds to one named task, beginning with arm reaching and object manipulation and continuing through quadruped and humanoid velocity tracking. The vertical axis is relative autoregressive error e, so shorter bars are better. Use the legend to separate the two RWM variants: the yellow RWM-TF bars use teacher forcing, while the orange RWM-AR bars use multi-step autoregressive training. The gray bars are other architectures. The most direct comparison is between the two RWM variants within a group. Section 4.3 states that models receive equal context, but the gray baselines use their customary teacher-forced training. e-generality
What it supports. Orange RWM-AR bars are the lowest throughout the displayed task configurations. The result shows that the proposed training recipe works across several robot/task settings. The accompanying text also says RSSM becomes comparable when trained autoregressively, so the visual should not be read as proving that the GRU is uniquely capable.
Where the evidence stops. The comparison changes training regime as well as architecture. Exact bar values and error-bar semantics are not specified. These are separate task evaluations, not evidence of a single trained model transferring unchanged among robots.
Figure 5. An improving imagined reward matters only when simulator reward improves with it. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the first two panels as an ANYmal D pair and the last two as a G1 pair. Within each pair, the first panel tracks model error and the second tracks mean reward across learning iterations. Color identifies the algorithm. In the reward panels, line style carries a different meaning: solid curves are simulator ground truth, and dashed curves are model predictions. Those simulator rewards are for evaluation only; MBPO-PPO trains with rewards computed from model outputs. Compare the orange solid and dashed curves to see whether the policy's apparent progress survives evaluation outside its learned simulator. e-policy-resultse-training
What it supports. MBPO-PPO reduces model error and learns higher-reward behavior than the reported SHAC and Dreamer baselines. Its initially optimistic predicted reward becomes closer to simulator reward, especially on ANYmal D. The remaining G1 gap makes the distinction between imagined success and evaluated control performance particularly useful.
Where the evidence stops. The curves report simulation training, not hardware reward histories. Shading semantics are unspecified. G1 runs to 10000 plotted iterations, whereas Table S11 lists a 2500-iteration maximum without an explicit G1-specific schedule.
Table 1. Equal displayed tracking rewards accompany very different training accounts. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the first two method columns together: RWM pretraining creates the learned simulator, and MBPO-PPO subsequently learns the policy. The PPO column is the model-free comparator trained with a high-fidelity simulator. The pretraining column lists 6M transitions and 50 minutes, while the policy stage lists five minutes. The transition entry for MBPO-PPO is a dash, so it cannot be treated as zero online data. At the bottom, both policy columns report a mean real tracking reward of 0.90 with different plus-minus terms. The one-millisecond entries refer to step inference, not the whole training process. e-tablee-policy-results
What it supports. The reported tracking reward is 0.90 ± 0.04 for MBPO-PPO and 0.90 ± 0.03 for PPO. The five-minute policy stage is shorter than PPO's ten minutes, but requires the separately listed 50-minute pretraining stage. This table motivates careful amortization analysis rather than an unconditional claim of faster end-to-end training.
Where the evidence stops. Robot-specific evaluation conditions, trial counts, uncertainty type and tracking-reward normalization are missing. Do not equate this scalar with Figure 5's mean reward, treat the dash as zero, or infer statistical equivalence.
Figure 1. The final evidence step is a deployed controller, with online adaptation still unresolved. Original paper, p. 2 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow each labeled row horizontally through time, then compare vertically within its robot group. 'Imagination' visualizes states forecast by RWM, while 'Ground Truth' shows simulator evolution. The lower groups add 'Deployment' rows containing actual hardware images for ANYmal D and Unitree G1. These row labels are essential: visual similarity between two rendered trajectories is a different kind of evidence from a physical robot executing the learned controller. Read the montage alongside Section 4.4 for the zero-shot transfer claim and Appendix A.4.4 for the explanation that online policy learning was performed in a single simulation environment. e-hardwaree-pretraininge-online
What it supports. The paper provides physical locomotion demonstrations in addition to predictive-state visualizations. That supports the practical utility of a separately trained policy using RWM imagination. The images illustrate transfer and trajectory behavior; they do not independently establish the frequency of successful deployments or the severity of disturbances tolerated.
Where the evidence stops. These are visualizations of low-dimensional predictions, not generated-video outputs. Static examples provide no trial denominator. The appendix explicitly defers online learning on hardware because collisions, recovery and privileged sensing remain unresolved.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Real tracking reward and training cost Table 1 compares RWM pretraining plus MBPO-PPO with simulator-trained PPO; no robot-specific split or hardware trial protocol is identified. | MBPO-PPO: 0.90 ± 0.04; policy training 5 min; inference 1 ms. RWM pretraining: 6M transitions and 50 min. Real tracking reward, training time, state transitions and step inference time | PPO: 0.90 ± 0.03; 250M transitions; 10 min training; 1 ms inference. Displayed mean tracking rewards are equal. Pretraining adds cost; the dash for MBPO-PPO transitions is not zero. Uncertainty type, trial count and reward normalization are unspecified, preventing equivalence or end-to-end sample-efficiency claims. e-table |
| History and forecast horizon ablation Figure S8, fixed M=32; evaluation split and exact rollout-error aggregation are unspecified. | N=8: e=0.47, 1.07 h. Relative autoregressive prediction error e; training hours | N=1: e=3.99, 0.62 h; N=32: e=0.47, 2.27 h. Multi-step training improves displayed error over teacher forcing, with diminishing gains and greater cost. No per-cell uncertainty is provided. e-ablation |
| Prediction across robotic environments Figure 4 covers 15 Isaac Lab task configurations with matched context; baseline architectures ordinarily use teacher forcing. | RWM-AR has the lowest displayed error across the plotted tasks; exact bar values are not tabulated. Relative autoregressive prediction error e, lower is better | MLP, RSSM, Transformer and RWM-TF. The text reports comparable RSSM performance when RSSM also receives autoregressive training. This supports the training regime more cleanly than unique GRU superiority; it does not establish one model transferring unchanged between robots. e-generality |
| Noisy autoregressive trajectory prediction ANYmal D study at 50 Hz; Gaussian perturbations affect observations and actions. Section 4.1 identifies hardware trajectories. | Figure 3 shows sustained alignment and lower RWM errors under plotted noise conditions; no exact endpoint values are claimed. Trajectory alignment and relative error e | The noise-study MLP also uses autoregressive training with matched history and forecast horizons. A better-controlled architectural diagnostic than Figure 4, but error normalization and noise scaling remain unspecified. e-prediction |
| Imagined policy learning and hardware transfer ANYmal D and G1 velocity tracking; simulator rewards evaluate training, followed by reported zero-shot physical deployment. | Figure 5 shows improving MBPO-PPO reward and declining model error; Figure 1 depicts deployment on both robots. Model error e, mean reward r and qualitative deployment | SHAC is unstable and Dreamer learns less effectively; authors report neither baseline yielded a deployable policy. Predicted and simulator rewards are separate curves. Executed locomotion is demonstrated, but safe learning directly on hardware is not. e-policy-resultse-hardwaree-online |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure S8. Longer training forecasts help until added computation stops buying lower displayed error. Original paper, p. 18 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read M along the horizontal axis and N up the vertical axis: the teacher-forcing row N=1 is at the bottom. Each coordinate identifies the same model configuration in both panels. The left panel reports relative autoregressive error e; the right gives training time in hours. Hold M=32 fixed and move upward to isolate the forecast-horizon comparison. Then move horizontally within a row to inspect the value of more history. Use the printed cell values for comparisons because the two colorbars encode different quantities and scales. The authors select M=32 and N=8 as their practical compromise. e-ablation
What it supports. At M=32, increasing N from 1 to 8 changes error from 3.99 to 0.47 and time from 0.62 to 1.07 hours. Increasing N further to 32 retains the displayed error of 0.47 while raising time to 2.27 hours. This is a concrete reason to prefer a moderate training forecast.
Where the evidence stops. Cells have no uncertainty estimates, and the PDF does not fully specify error normalization or evaluation sampling. The values are not strictly monotonic: at M=32,N=16, error is 0.53. Avoid claiming every horizon increase helps.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors acknowledge advantages of well-tuned model-free methods with high-fidelity simulation. Hardware online learning remains deferred: model exploitation causes collisions, recovery is difficult, and privileged signals may require unavailable sensors. Appendix A.4.4 reports more than 20 failures on average during online learning in simulation. e-tablee-online
Evaluation gaps include exact error normalization, explicit train/test partitions, Figure 4 error-bar definitions and Table 1 uncertainty semantics. Figure 5 plots G1 training through 10000 iterations, whereas Table S11 lists 2500 maximum iterations without a robot-specific override. e-generalitye-policy-resultse-tablee-training
7.2 Questions for discussion
- Does the advantage persist when RSSM and GRU use the same autoregressive loss and compute budget? [e-generality]
- How closely does open-loop prediction error predict simulator reward and collision frequency under policy-induced distribution shift? [e-policy-results, e-online]
8. Reproducibility audit
8.1 Requirements and known gaps
The stated stack is PyTorch 2.4.0, CUDA 12.6 and an NVIDIA RTX 4090. Reproduction needs observation/action layouts, reward weights, termination logic, pretraining trajectories and single-environment fine-tuning conditions. e-softwaree-spacese-rewardse-pretraining
Resolve loss forms, normalization and splits, noise scaling, dynamics-shift ranges, buffer-size units and robot-specific schedules before exact replication. Preserve distinct time reports: Table 1 gives 50-minute pretraining, Table S10 approximately one hour, and Figure S8 configuration-specific hours. e-are-predictione-traininge-pretraininge-tablee-ablation
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Match the training regime before attributing the gain to GRU
Reader-proposed check, not performed: use identical ANYmal D trajectories, trajectory-level held-out splits, preprocessing and history M=32 to compare GRU and RSSM under N=1 and N=8 training. Keep data and parameter budgets as close as possible, document residual capacity differences, and report both matched-update and matched-wall-clock comparisons across five seeds. Measure a prespecified observation-error metric over 100-step held-out rollouts, plus GPU memory and training time. Resolve the source's undefined e normalization before claiming exact replication. If autoregression improves both models and closes the architecture gap, that supports the training-regime explanation. A persistent GRU advantage under both budgets would provide stronger architectural evidence. e-generalitye-architecturee-traininge-ablation
Check 2: Test whether lower forecast error predicts better control
Reader-proposed check, not performed: pretrain otherwise identical M=32 models with N=1,8,32 on the same locomotion data, then run the same MBPO-PPO schedule with matched single-environment interaction budgets and 100-step imagination. Hold rewards, reset rules and domain shifts fixed; use five seeds. Record held-out forecast error, predicted versus simulator reward, base-contact failure counts, and pretraining plus fine-tuning time. Evaluate actor checkpoints in simulation only. If N=8 lowers open-loop error but does not improve actual simulator reward or reduce failures, prediction fidelity alone is insufficient for the control claim. Compare N=32 against N=8 to test whether its extra training cost buys downstream robustness despite equal displayed ablation error. e-ablatione-policy-resultse-traininge-rewardse-pretraininge-online
8.3 Reading coverage
Visual audit: The title/author/version page, all original figures (1–5 and S6–S9), Table 1, Tables S2–S11, method equations, Algorithm 1, software/hardware statement, reward definitions and safety discussion were visually inspected. Every retained claim's supporting method, numerical and configuration pages are included above. All six final crops were individually viewed at their original helper-produced resolution; their labels, arrows, axes, legends and table entries are retained. Figure S6 was cross-checked against Section 3.2 and Equations (1)–(2); Figure 2's H/M history-notation difference is disclosed in the walkthrough. Figure 5's G1 iteration range differs from Table S11, as noted in its caution. Text-only pages 3 and 10–13 were read in the complete chunk pass but not separately rendered. External videos, code and separate supplements were outside the supplied material.
PDF pages inspected for this edition: 1, 2, 4, 5, 6, 7, 8, 9, 14, 15, 16, 17, 18, 19, 20, 21. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1 Introduction
- 2 Related work
- 2.1 World Models for Robotics
- 2.2 Model-Based Reinforcement Learning
- 3 Approach
- 3.1 Reinforcement Learning and World Models
- 3.2 Self-supervised Autoregressive Training
- 3.3 Policy Optimization on Learned World Models
- 4 Experiments
- 4.1 Autoregressive Trajectory Prediction
- 4.2 Robustness under Noise
- 4.3 Generality across Robotic Environments
- 4.4 Policy Learning and Hardware Transfer
- 5 Limitations
- 6 Conclusion
- References
- A Technical Appendices and Supplementary Material
- A.1 Task Representation
- A.1.1 Observation and action spaces
- A.1.2 Reward functions
- A.2 Network Architecture
- A.2.1 RWM
- A.2.2 Baselines
- A.2.3 MBPO-PPO
- A.3 Training Parameters
- A.3.1 RWM
- A.3.2 MBPO-PPO
- A.4 Additional Experiments and Discussions
- A.4.1 Dual-autoregressive Mechanism
- A.4.2 Visualization of Imagination Rollouts
- A.4.3 Collision Handling and Model Pretraining
- A.4.4 Challenges in Real-World Online Learning
- A.5 Ethics and Societal Impacts
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Identity/version: the inspected title page identifies arXiv:2501.10100v5 [cs.RO], 14 December 2025, and 'Preprint. Under review.' The exact title and Chenhao Li, Andreas Krause, Marco Hutter match the catalog. Its CoRL workshop venue and alternative workshop BibTeX labels are not established by this PDF. No venue edition or earlier revision was supplied for content comparison.
- All six supplied chunks, including references and Appendices A.1–A.5, were read. Text extraction did not reconstruct figure images; the retained PDF resolved this limitation through inspection of every figure and table. Text-only pages 3 and 10–13 were not separately visually inspected.
- Separate supplemental material availability has not been fully verified. No separate supplement was supplied; linked project videos and code were not inspected.
- No paper scripts were executed and no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title/author block and arXiv margin
The exact observed title is Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics. Authors are Chenhao Li, Andreas Krause and Marco Hutter, all ETH Zurich, Switzerland. The artifact is arXiv:2501.10100v5, 14 December 2025, labeled Preprint. Under review.
Go to primary source ↓e-problemPDF pp. 1 and 4, Introduction and Section 3.1
The motivation is sample-efficient control with a learned world model under partial observability, stochasticity and accumulating prediction errors.
Go to primary source ↓e-arPDF pp. 4–5, Section 3.2, Equations (1)–(2) and Figure 2
Historical observation-action pairs and prior predictions condition multi-step forecasts. The loss combines observation and privileged-information discrepancies weighted by forecast decay. Training uses sliding M+N windows and reparameterization. Figure 2 contrasts autoregression with N=1 teacher forcing; its caption uses H=3 for the history that the equations call M.
Go to primary source ↓e-architecturePDF p. 5, Section 3.2; p. 16, Figure S6 and Section A.2.1; p. 17, Table S7
Inner autoregression updates GRU history states; outer autoregression feeds back predictions. The GRU base has hidden shape 256,256 and the ReLU MLP heads 128 units, predicting means and standard deviations. Figure S6 shows observation losses, not the additional privileged-information head.
Go to primary source ↓e-policyPDF p. 5, Section 3.3 and Equation (3); p. 6, Algorithm 1; p. 17, Figure S7
The policy samples actions from imagined observations. The algorithm collects data, trains the model, seeds imagination from the buffer, rolls out T steps and updates the separate policy using PPO. Predicted observations and privileged information supply rewards.
Go to primary source ↓e-spacesPDF p. 14, Tables S2–S4; p. 15, Table S5
World-model observations contain base velocities, gravity, joint positions, velocities and torques (45/96 dimensions). Actions are joint-position targets (12/29). Privileged targets include contacts and G1 foot quantities. Policy observations include velocity command and previous actions (48/99 dimensions).
Go to primary source ↓e-rewardsPDF pp. 14–16, Section A.1.2 and Table S6
Rewards sum velocity tracking and robot-specific weighted terms for vertical/rotational motion, torque, acceleration, action changes, feet air time, contacts, orientation, clearance and joint deviation.
Go to primary source ↓e-trainingPDF pp. 18–19, Tables S9–S11
Policy/value MLPs have hidden shape 128,128,128 with ELU. RWM uses 0.02-second steps, 2500 iterations, learning rate 1e-4, weight decay 1e-5, batch 1024, M=32,N=8,alpha=1, approximately one hour and five seeds. MBPO-PPO lists 4096 imagined environments, 100 steps, buffer size 1000, 2500 iterations, learning rate 0.001, five epochs, four mini-batches, KL target 0.01, gamma=0.99, clip 0.2, entropy 0.005 and five seeds.
Go to primary source ↓e-softwarePDF p. 17, Section A.3
The implementation uses PyTorch 2.4.0 with CUDA 12.6 and is trained on an NVIDIA RTX 4090 GPU.
Go to primary source ↓e-predictionPDF p. 6, Section 4.1; p. 7, Figure 3 and Section 4.2
Section 4.1 identifies ANYmal D hardware trajectories at 50 Hz with M=32,N=8. Figure 3 begins autoregression at t=32. Its noise panel compares RWM against an autoregressively trained MLP using identical history and forecast horizons; perturbations affect observations and actions.
Go to primary source ↓e-generalityPDF pp. 7–8, Section 4.3 and Figure 4
The 15 plotted tasks span manipulation and legged velocity tracking. RWM-AR has the lowest bars. The text states equal context, teacher-forced baselines, comparable performance from autoregressively trained RSSM, and memory constraints for autoregressive transformer training.
Go to primary source ↓e-policy-resultsPDF pp. 8–9, Figure 5 and Section 4.4
Figure 5 plots model errors and rewards for ANYmal D and G1. Solid curves are simulator ground truth used only for evaluation; dashed curves are predictions. MBPO-PPO improves more than SHAC and Dreamer. The G1 axis extends to 10000 iterations.
Go to primary source ↓e-tablePDF p. 9, Table 1 and Section 5
Table 1 gives 6M pretraining transitions and 50 minutes; MBPO-PPO takes 5 minutes, with 1 ms step inference and real tracking reward 0.90±0.04. PPO reports 250M transitions, 10 minutes, 1 ms and 0.90±0.03. MBPO-PPO transitions are a dash. The text acknowledges broader model-free performance advantages.
Go to primary source ↓e-ablationPDF p. 18, Figure S8 and Section A.4.1; p. 19, continuation of A.4.1
At M=32,N=1/8/32, error is 3.99/0.47/0.47 and training time 0.62/1.07/2.27 hours. M increases left-to-right and N bottom-to-top. The discussion selects M=32,N=8 as a cost/accuracy compromise.
Go to primary source ↓e-pretrainingPDF p. 19, Section A.4.3
Base-ground contact causes termination/reset, and predicted failures terminate imagined episodes. Locomotion RWM is pretrained with simulation data from related policies under varied dynamics; the actor starts from scratch in imagination while RWM is fine-tuned on a single-environment dataset. Suboptimal-policy pretraining can stabilize learning. Manipulation does not require this pretraining.
Go to primary source ↓e-onlinePDF p. 20, Section A.4.4; p. 21, Section A.5
Online model exploitation produces more than 20 failures on average. Recovery and measurement/estimation of privileged information remain challenges. A single simulation environment with domain shifts substitutes for hardware online learning. Policy training occurs in simulation and hardware online adaptation is deferred.
Go to primary source ↓e-hardwarePDF p. 2, Figure 1; p. 9, Section 4.4; p. 20, Figure S9 caption
Figure 1 distinguishes imagined states, simulator ground truth and physical ANYmal D/G1 deployment. Section 4.4 reports zero-shot transfer and disturbance robustness, with neither SHAC nor Dreamer producing deployable policies. Figure S9 explains pose/velocity markers in state visualizations; these visualize trajectories rather than pixel-prediction outputs.
Go to primary source ↓8.5 Primary sources
Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics ↗
PDF · 8,753 extracted words
Source fingerprint
613d81eb472e437a76447f408f46ddeb0d786da29fae81e66089db1bfb1b7a14