SimWAM: A Simple World Action Model for End-to-End Autonomous Driving
1. Paper overview
In one sentence: SimWAM uses future-video prediction to train a direct trajectory policy, trading explicit test-time imagination for dependence on the quality of learned current-observation features. e04e07e09e10e11e13e16e19
| At a glance | What to know |
|---|---|
| Research problem | Source description Imagine-then-act driving planners place future-video synthesis inside the planning loop. SimWAM asks whether video prediction can instead teach a useful observation representation during training, preserving dynamics-informed trajectory prediction while removing future-frame generation at deployment. e02e04 |
| Core mechanism | Source description An isolated attention mask makes action tokens independent of future-video tokens while both use current-observation features. e07 |
| A key reported result | NAVSIM navtest trajectory planning: 91.5 PDMS (higher is better). 103,288 navtrain scenes; 12,146 navtest scenes; single front camera; imitation plus RL; non-reactive benchmark. SGDrive 91.1; DriveWAM 90.1; DriveLaW 89.1; Human Agent 94.8. Highest learned score listed in Table 1, with a 1.4-point advantage over DriveWAM. Different sensors and training recipes prevent a controlled architectural comparison. e09e10e13 |
| Reading caution |
Core contributions
- Source description
An isolated attention mask makes action tokens independent of future-video tokens while both use current-observation features. e07
- Source description
Flow-GRPO refines continuous trajectory generation using a compositional driving reward and action-only LoRA updates. e08
Figure 2. Separate video and action experts share attention to the present, while the mask blocks future-to-action dependence. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the left-hand inputs: the current frame and training-time future frames enter the video VAE, noisy actions enter the action encoder, and text and ego state provide conditioning. In the center, f0 marks current-frame tokens, f1 through fn mark future frames, and a1 through an mark actions. Follow the colored attention cells: future tokens and action tokens can use f0, but their cross-block cells are empty. The smaller inference matrix keeps the current frame and actions only. The right-hand dashed video outputs are removed during inference and RL; the trajectory output and reward branch remain relevant to action generation and policy training. e04e05e07e08
What it supports. The diagram makes the deployment simplification concrete. Future-video prediction is a supervised training task whose influence passes through learned current-observation features. It is not a required intermediate prediction before each trajectory. The two DiTs retain separate parameters despite their common attention interface, supporting the paper's modular architecture claim.
Where the evidence stops. Removing future-frame generation does not mean removing every computation inherited from the video expert. Current-image representation is still required. The reward symbols depict policy training, not a claim that the learned video generator supplies the NAVSIM reward.
2. Motivation
2.1 The problem and the proposed response
Imagine-then-act driving planners place future-video synthesis inside the planning loop. SimWAM asks whether video prediction can instead teach a useful observation representation during training, preserving dynamics-informed trajectory prediction while removing future-frame generation at deployment. e02e04
2.2 What this reading follows
A driving planner can benefit from learning how scenes evolve without rendering those futures every time it acts. SimWAM makes that separation explicit with two diffusion transformers and an attention mask: future video and trajectories are supervised together, but action tokens cannot read future-frame tokens. This edition follows the information flow before examining what the experiments establish. The strongest NAVSIM result includes reinforcement learning, the harder NAVSIM-v2 evaluation uses imitation alone, and nuScenes transfer measures open-loop predictions. The central lesson is a useful training/inference separation, supported by component and timing studies whose small differences and deployment implications need careful interpretation. e04e07e09e10e11e13e16e19
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Supports the recorded classification
Dual-system is supported by two parameter-independent experts linked through attention. Other mechanisms / Outside quadrants is reasonable when classifying deployment: future modeling is training supervision, with neither joint future/action generation nor inverse dynamics required at inference. Joint flow training alone does not imply a One Model architecture. The driving, efficient-inference and policy-post-training labels are supported; RL reward evaluation uses NAVSIM rather than generated video. e04e07e08
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 How future supervision can help an action branch that cannot see futures
During co-training, the current frame is the common conditioning input for two supervised problems: predicting future visual latents and predicting the expert trajectory. Both use rectified flow, which asks a network to recover a velocity field from a noisy target. The losses are added with equal weight in the NAVSIM setup. The attention mask prevents action tokens from reading future-frame tokens, and the current-frame row in Figure 2 does not read those future tokens either. My interpretation is that this separates two routes of influence: future-video errors can shape observation-processing parameters through training gradients, while actual future representations cannot enter the action computation during a forward pass. This explains why removing future tokens at inference need not create a missing conditioning input. It also motivates checking both the mask and the claimed training benefit separately. e03e05e06e07e13
5.2 What the two experts retain when future generation is removed
The video expert begins with Wan2.2-5B, its VAE and T5 encoder, while the action expert is a distinct diffusion transformer. Navigation and ego state join the current visual representation as action conditions. At inference, integrating the action flow produces position-and-heading waypoints directly; there is no need to extract actions from a generated movie. This modularity is narrower than discarding the entire visual backbone: the planner still needs the representation of the current observation. The source tests the interface with several video backbones and action-model sizes, but those comparisons also change capacity or pretraining. My reading is that they establish practical replaceability within the tested family, while leaving open how much benefit comes specifically from learned temporal dynamics. Joint optimization should therefore not be mistaken for a single shared-parameter architecture. e04e05e07e15
5.3 Why reinforcement changes the training sampler and the driving tradeoff
Imitation matching rewards agreement with logged trajectories. The RL stage instead evaluates groups of candidate trajectories with NAVSIM's compositional PDM reward. To obtain per-step stochastic transitions with tractable likelihoods, the paper replaces the flow ODE with a marginal-preserving SDE and uses group-relative clipped policy updates. Determinism here should be understood conditional on an ODE's starting noise; the SDE adds transition noise useful for this policy-gradient construction. Only the action expert's LoRA adapters are updated, and future-frame generation remains absent. The authors concentrate training on scenes whose imitation PDMS is below 90, with the strongest reported point at 15k updates. The component table then shows the tradeoff: aggregate score and progress improve, while no-collision and time-to-collision metrics fall slightly. A reward gain is therefore not evidence that every safety dimension improved. e03e08e13e18
5.4 Training and inference
During training
NAVSIM uses 103,288 training scenes, 100 imitation epochs and equal action/video loss weights. Defaults are 384×672 images, AdamW with initial learning rate 1e-4 and cosine decay. The selected action expert has 1.02B parameters. A separate PhysicalAI run uses 65K samples for 15 imitation-only epochs. e06e09e12e15
RL converts the flow ODE to a marginal-preserving SDE with Gaussian transition likelihoods. For each scenario, eight sampled trajectories receive NAVSIM PDM rewards and group-relative clipped updates. Only rank-32 action-attention LoRA adapters are updated, with alpha=16 and learning rate 5e-5. RL selects navtrain scenes with imitation PDMS below 90; evaluation still covers full navtest. e08
The hard-subset RL curve peaks at 91.5 PDMS after 15k updates, then declines slightly. This is a reported training trajectory, without repeated-run uncertainty. e18
During inference
Keep current-observation processing and sample the action flow directly. Future-frame tokens and future decoding are unnecessary, including during RL. This does not establish removal of all video-expert computation. The paper describes planned trajectories, leaving low-level actuation and deployed feedback unspecified. e04e07e21
Ten sampling steps produce 90.3 PDMS in the imitation ablation; five steps already reach 90.1. The latency comparison below belongs to this checkpoint, not an independently timed 91.5-PDMS RL model. e19
5.5 Implementation flow
- Encode the present
Wan2.2-5B supplies the video DiT, VAE and T5 encoder. The VAE produces image tokens; navigation enters through text cross-attention, and an MLP embeds ego state. The current image stays clean while future latents are noised. e05e06
- Couple supervision, isolate information
The 1024-wide action DiT and video DiT use distinct weights. Both future and action tokens attend to current-observation tokens; neither attends to the other. Joint losses therefore train shared observation features without giving the action branch future targets as inputs. e05e06e07
- Generate trajectories directly
The action expert predicts a flow velocity field conditioned on current features, ego state and navigation. Integrating from Gaussian noise produces position-and-heading waypoints; there is no inverse-dynamics action extraction or future-video rollout in this policy interface. e03e04e05e07
6. Experiments & results
SimWAM trains separate video and trajectory diffusion transformers together, then plans from current observations without synthesizing future frames. Its strongest NAVSIM result includes action-expert reinforcement learning; additional experiments test imitation-only planning and open-loop transfer. The central evidence concerns efficient trajectory prediction, with limited support for broader deployment claims.
6.1 Read the original evidence
Table 1. The reinforced model leads the listed learned planners in aggregate NAVSIM score. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the sensor column before ranking methods: C denotes camera and L denotes LiDAR, so some rows use more views or another sensor modality. SimWAM uses one front camera. Move across its shaded row from NC, no collision, through DAC, drivable-area compliance, EP, ego progress, TTC, time-to-collision, and C, comfort, to the aggregate PDMS. These metrics are higher-is-better. The reported setup trains on 103,288 navtrain scenes and evaluates all 12,146 navtest scenes. This table uses SimWAM after reinforcement learning; its score should not be exchanged with the imitation-checkpoint values in the attention, timing or NAVSIM-v2 experiments. e09e10e13
What it supports. SimWAM reports 91.5 PDMS versus 91.1 for SGDrive, 90.1 for DriveWAM and 89.1 for DriveLaW. The gain over DriveWAM is 1.4 score points. Human Agent remains higher at 94.8. SimWAM's strongest aggregate result does not mean it leads every individual submetric.
Where the evidence stops. NAVSIM is a non-reactive planning benchmark. The rows differ in sensors, pretraining and optimization, so this ranking does not isolate the attention mask's causal effect or establish physical driving safety. The table supplies no repeated-run uncertainty.
Table 3. The two-stage benchmark reveals both a comparative gain and substantial remaining difficulty. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each method occupies two rows. S1 evaluates real-world scenarios; S2 re-evaluates corresponding synthesized scenarios with reactive traffic agents, as described in the benchmark setup. Read each submetric vertically to see the change between stages, then use the rightmost EPDMS entry as the overall two-stage score. It is not a separate S2 score or an arithmetic average of the visible columns. Focus on NC for no collision, DAC for drivable-area compliance, DDC for driving-direction compliance, TTC for time-to-collision, and LK for lane keeping. SimWAM uses its imitation checkpoint here, before the RL stage that supplies the headline NAVSIM result. e09e11
What it supports. SimWAM's overall 37.6 EPDMS exceeds DriveLaW's 30.6 by 7.0 points. Its S2 DAC of 78.6 and LK of 49.5 exceed DriveLaW's 67.6 and 45.8. Yet SimWAM's own NC falls from 98.0 to 81.8 and LK from 97.3 to 49.5 between stages.
Where the evidence stops. The benchmark has a specified two-stage structure, rather than unrestricted physical deployment. The large changes across stages are not an isolated test of reactivity because the scenarios also change. EPDMS values here should not be compared numerically with NAVSIM PDMS.
Table 9. Transfer improves the reported collision metric without leading trajectory-imitation accuracy. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First separate rows by the Finetune column. The bottom three methods are evaluated without nuScenes fine-tuning; upper rows include target-dataset adaptation and often auxiliary supervision. The retained asterisk note marks front-camera-only input. Read the two metric groups independently: L2 is displacement from logged expert trajectories in meters, while collision rate is a percentage evaluated against logged surrounding-agent futures. Each group reports 1-, 2- and 3-second horizons plus the paper's average. SimWAM's model is transferred from NAVSIM. Compare it primarily with the other zero-shot rows, while keeping their unstandardized pretraining histories in mind. e16
What it supports. SimWAM reports 0.96 m average L2 and 0.05% average collision. DriveWAM has the same average L2 and 0.06% collision; DriveVA has better L2 at 0.84 m and 0.06% collision. Thus the table supports a favorable collision metric under domain shift, rather than uniform superiority on both measures.
Where the evidence stops. These are open-loop predictions against logged futures, not collisions observed during executed autonomous driving. The 0.01-percentage-point average difference has no accompanying event counts or uncertainty, and the rounded zero values do not establish zero underlying risk.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| NAVSIM navtest trajectory planning 103,288 navtrain scenes; 12,146 navtest scenes; single front camera; imitation plus RL; non-reactive benchmark. | 91.5 PDMS (higher is better) | SGDrive 91.1; DriveWAM 90.1; DriveLaW 89.1; Human Agent 94.8. Highest learned score listed in Table 1, with a 1.4-point advantage over DriveWAM. Different sensors and training recipes prevent a controlled architectural comparison. e09e10e13 |
| NAVSIM-v2 planning Imitation checkpoint before RL; navtest and the official two-stage safety-critical navhard protocol. | navtest 90.2; navhard overall 37.6 EPDMS (higher is better) | DriveLaW: 88.6 and 30.6 respectively; DriveFine navtest: 89.7. The navhard gain over DriveLaW is 7.0 points. Overall EPDMS is distinct from per-stage submetrics; SimWAM's NC drops from 98.0 in S1 to 81.8 in reactive S2. e09e11 |
| PhysicalAI open-loop trajectory prediction Separate imitation-only training on 65K samples; 15 epochs; 1,000-clip test subset shared with the cited DriveWAM evaluation. | 3 s: 0.40 / 1.08; 4 s: 0.69 / 1.96 ADE / FDE, lower is better; units not labeled in Table 4 | DriveWAM: 0.47 / 1.35 at 3 s and 0.83 / 2.47 at 4 s. Better logged-trajectory agreement at both horizons, with differing model sizes and pretraining; this does not measure executed driving. e12 |
| Zero-shot nuScenes planning NAVSIM-trained model; no nuScenes fine-tuning; front camera; open-loop metrics against logged futures. | 0.96 m / 0.05% Average L2 error (m) / collision rate (%) | DriveVA: 0.84 m / 0.06%; DriveWAM: 0.96 m / 0.06%. Lowest reported average collision rate among listed methods, but no L2 advantage over DriveVA. The collision improvement is 0.01 percentage points; event counts and uncertainty are absent. e16 |
| Video supervision and attention ablation NAVSIM component sequence and imitation-checkpoint mask comparison. | Action-only 86.6; +Video 90.3; +RL 91.5. Isolated mask 90.3. PDMS | Bidirectional mask 90.2; Action→video mask 90.1. Video adds 3.7 points and RL adds 1.2. Small mask differences lack uncertainty; the clear structural distinction is avoiding future tokens. RL raises progress while reducing NC and TTC slightly. e13e14 |
| Action sampling efficiency NAVSIM imitation ablation; single NVIDIA A100; default 384×672 input. | 1 step: 68.9 / 115; 5: 90.1 / 297; 10: 90.3 / 518; 20: 90.2 / 968. PDMS / latency (ms) | Five steps save 221 ms versus ten for 0.2 fewer PDMS points. More steps cease improving reported accuracy beyond ten. Timing scope, batch size and repeated-run variation are not specified, limiting deployment-rate conclusions. e19e21 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Tables 5–6. Training-stage gains are much larger than the reported score differences between attention masks. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Treat the left table as a cumulative sequence: Action-only is the starting configuration, +Video adds joint video supervision, and +RL refines the co-trained policy. Compare PDMS first, then inspect NC, DAC, EP and TTC to see which driving attributes move together. The right table answers a different question using attention configurations: bidirectional, the paper's Action-to-video alternative, and isolated attention. The surrounding method discussion states that the first two require future-frame representations at inference, while the isolated configuration does not. These mask scores belong to the imitation configuration; the shaded 91.5 on the left includes an additional RL training stage. e07e13e14
What it supports. Video co-training adds 3.7 PDMS points and RL adds 1.2. RL increases EP from 83.9 to 86.4 and DAC from 98.0 to 98.7, while NC and TTC decline slightly. Isolated attention reaches 90.3, only 0.1–0.2 points above the alternatives, while removing the future-token dependency.
Where the evidence stops. The component comparison does not establish a capacity- or compute-matched control for the video expert. The small mask gaps lack uncertainty estimates. The strongest supported mask conclusion is compatibility with good planning scores and a simpler inference dependency.
Table 13. Five steps recover most reported accuracy; extra integration beyond ten adds cost without a gain. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read downward to increase the action sampler's integration budget. The middle columns report navtest driving submetrics and PDMS, while the right column reports latency in milliseconds. Section 5.4 specifies an imitation-trained model for configuration ablations and one NVIDIA A100 for timing; the default input is 384×672. The one-step row shows that cheap inference can still be inaccurate when the flow is insufficiently integrated. Compare five and ten steps for a practical tradeoff, then compare ten and twenty to look for saturation. The shaded ten-step row identifies the adopted balance rather than a universal optimum for all hardware and checkpoints. e06e19e21
What it supports. Five steps yield 90.1 PDMS at 297 ms versus 90.3 at 518 ms for ten steps, saving 221 ms for a 0.2-point reduction. Twenty steps cost 968 ms and score 90.2. One step is faster at 115 ms but falls sharply to 68.9 PDMS.
Where the evidence stops. These measured pairs concern the imitation checkpoint; they do not directly pair 518 ms with the reinforced 91.5 score. Batch size, timing boundaries and variance are unspecified, so a deployed control frequency cannot be inferred confidently from this table alone.
7. Analysis & limitations
7.1 What the evidence leaves open
NAVSIM is non-reactive, navhard uses a bounded two-stage protocol, and PhysicalAI/nuScenes are open-loop. Together these support benchmark planning gains, not physical autonomous-driving reliability. e09e11e12e16
Co-training gains are consistent with useful video supervision, but the component table does not isolate dynamics knowledge from added capacity or pretraining. Backbone comparisons also change model and data priors. Small mask and collision differences lack statistical uncertainty. e13e14e15e16
No dedicated limitations section is supplied. The two qualitative examples highlight farther RL progress but do not characterize a failure distribution or prove safety across scenarios. e20
7.2 Questions for discussion
- Would video co-training retain its gain against a capacity- and compute-matched observation-only control?
- Does hard-scene RL improve held-out safety submetrics when training budgets and scenario selection are controlled?
8. Reproducibility audit
8.1 Requirements and known gaps
Required resources include Wan2.2-5B with VAE/T5, the action DiT, the NAVSIM splits and PDM evaluator, and the specified PhysicalAI samples/test subset. Training hardware/time, batch size, exact subset identities, VAE/T5 freezing and SDE noise/endpoints need clarification. e05e08e09e12e21
Proposed checks should first test future-token independence with fixed noise, then compare hard-subset and full-data RL under matched reward-evaluation budgets. Track submetrics and repeated seeds rather than relying only on aggregate PDMS. e07e13e18
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test the promised independence from future tokens
Reader-proposed check: hold the checkpoint, current image, ego state, navigation, action-noise seed and integration schedule fixed in evaluation mode. Compare trajectory outputs with future-token slots populated by normal noised targets, replaced by unrelated future tokens, and removed using the inference mask. The isolated model should agree within a predefined numerical tolerance. Run the same interventions on a deliberately unmasked or bidirectional control to verify that the perturbation reaches an available attention path. Any systematic action change in the isolated model would falsify the claimed forward-pass independence or expose an implementation leak. This checks the inference mechanism; it does not by itself establish the value of video supervision. e07e14
Check 2: Separate hard-scene selection from additional RL effort
Reader-proposed check: start several seeds from the same imitation checkpoint and freeze the hard subset using only its navtrain scores below 90. Compare hard-subset SDE RL with full-navtrain SDE RL at equal numbers of candidate-trajectory reward evaluations, keeping eight candidates, LoRA rank, learning rate and update schedule fixed. Use a prespecified validation rule for checkpoint selection and then evaluate the complete held-out navtest split. Report PDMS plus NC, DAC, EP and TTC with uncertainty at the planned 5k, 10k, 15k and 20k checkpoints. If the subset advantage disappears under matched budgets, the claimed benefit of more informative hard-scene rewards weakens; if progress gains repeatedly coincide with safety losses, report that tradeoff explicitly. e08e09e13e18
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version page, all method and experiment pages, Figures 1–4 and Tables 1–13. Six original crops were individually inspected, including a revised Table 9 crop retaining the front-camera asterisk note. Pages 3, 5 and 7 supply equations and training settings used in the explanations; page 9 supplies the imitation/A100 timing protocol. All five text chunks, including reference-only pages 12–15, were read. No separate supplement, code or experiments were inspected or executed.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title and Abstract (p. 1)
- 1 Introduction (pp. 1–2)
- 2 Related Work, 2.1–2.3 (pp. 2–3)
- 3 Preliminary (pp. 3–4)
- 4 Method, 4.1–4.3 (pp. 4–6)
- 5 Experiments, 5.1–5.5 (pp. 6–10)
- 6 Conclusion (p. 11)
- References (pp. 11–15)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction limitation was addressed by inspecting original PDF pages 1–11, all four figures and all thirteen tables; reference-only pages 12–15 were read as text.
- Reviewed artifact: arXiv:2608.07468v4, dated 27 August 2026. The catalog submittedDate is 26 August 2026; earlier versions and a revision history were not supplied, so changes between editions cannot be established. Title and all nine authors match.
- No appendix occurs in the supplied 15-page PDF. No separate supplement was supplied.
- Code, model weights and external links were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, author block, affiliation block and arXiv margin
Title matches the catalog. The title page lists Zongchuang Zhao, Xin Zhou, Tianyang Xu, Zhengyang Sun, Kaixuan Zhou, Yu Wu, Honglin Li, Dingkang Liang and Xiang Bai, with Huazhong University of Science & Technology and Dongfeng Research & Development Institute. The margin identifies arXiv:2608.07468v4 [cs.CV], 27 Aug 2026.
Go to primary source ↓e02PDF pp. 1–2, Abstract and Section 1; p. 2, Figure 1
The authors motivate replacing costly test-time future imagination with training-time video supervision and present SimWAM as a simple baseline. Figure 1 compares reported PDMS and latency across world-model-based planners; exact baseline timing values are not tabulated there.
Go to primary source ↓e03PDF p. 3, Section 3, Eq. (1) and sampling paragraph
Rectified flow interpolates a clean target with Gaussian noise, predicts the velocity epsilon minus the target with squared error, and integrates from noise at tau=1 to data at tau=0.
Go to primary source ↓e04PDF p. 4, Section 4.1, Problem formulation, Eqs. (3)–(4); p. 5, opening paragraph
Inputs are a front image, ego velocity/acceleration/yaw rate, and navigation command. Outputs are ego-frame position-and-heading waypoints. Eq. (4) conditions the policy on the current observation representation instead of generated future latents.
Go to primary source ↓e05PDF p. 4, Figure 2; p. 5, Section 4.1, Video expert and Action expert
Wan2.2-5B initializes the video DiT, with video VAE and T5 text encoder. A 1024-hidden-size action DiT predicts a trajectory velocity field. The current image is clean conditioning; future latents are noised targets. Ego state is embedded with an MLP and navigation uses text cross-attention.
Go to primary source ↓e06PDF p. 5, Section 4.1, Co-training, Eq. (5); p. 7, Section 5.1, Implementation details
The loss adds action and lambda-weighted video flow matching. Defaults use 384×672 front images, eight waypoints and eight future frames across four seconds at 2 Hz. NAVSIM imitation training lasts 100 epochs with lambda=1. AdamW and cosine decay start at 1e-4.
Go to primary source ↓e07PDF p. 4, Figure 2 attention matrices; p. 5, Sections 4.2–4.3; p. 6, flexibility paragraphs
Future-frame and action tokens both attend to current-observation tokens but are mutually invisible. Experts share no weights and interact through attention. Future-frame generation and its VAE decoder can be omitted at inference and RL; current-observation processing remains.
Go to primary source ↓e08PDF p. 4, Eq. (2) and From ODE to SDE; p. 5, Reinforcement; p. 7, Implementation details
Flow-GRPO uses a marginal-preserving SDE with Gaussian transitions and tractable likelihoods. Eight trajectories per scenario receive NAVSIM PDM rewards for group-relative clipped updates. Only rank-32 action-attention LoRA adapters, alpha=16, are updated at 5e-5, on navtrain scenes whose imitation PDMS is below 90.
Go to primary source ↓e09PDF pp. 6–7, Section 5.1, Datasets and benchmarks and Implementation details
NAVSIM uses 103,288 navtrain and 12,146 navtest scenes and is non-reactive. PDMS combines NC, DAC, EP, TTC and Comfort. NAVSIM-v2 adds compliance and comfort terms; navhard has real-scenario S1 followed by synthesized reactive S2. NAVSIM-v2 uses the imitation checkpoint before RL.
Go to primary source ↓e10PDF p. 6, Table 1, SimWAM, SGDrive, DriveLaW, DriveWAM and Human Agent rows
NAVSIM navtest PDMS is 91.5 for SimWAM, 91.1 for SGDrive, 89.1 for DriveLaW, 90.1 for DriveWAM and 94.8 for Human Agent. SimWAM's NC/DAC/EP/TTC/C are 98.4/98.7/86.4/95.5/100.0. The table includes heterogeneous sensor configurations.
Go to primary source ↓e11PDF p. 7, Tables 2–3, SimWAM, DriveFine and DriveLaW rows; Section 5.1 implementation paragraph
Imitation-trained SimWAM obtains 90.2 EPDMS on NAVSIM-v2 navtest versus DriveFine 89.7 and DriveLaW 88.6. On navhard its overall EPDMS is 37.6 versus DriveLaW 30.6. SimWAM S1/S2 NC is 98.0/81.8 and LK 97.3/49.5; S2 DAC/DDC/TTC are 78.6/87.3/78.6.
Go to primary source ↓e12PDF pp. 6–7, PhysicalAI dataset and implementation paragraphs; p. 8, Table 4
PhysicalAI contains 306,152 clips of 20 seconds, split 153,625/90,928/61,599. SimWAM trains by imitation on 65K training samples for 15 epochs and evaluates on the 1,000-clip subset used by DriveWAM. SimWAM ADE/FDE is 0.40/1.08 at 3 s and 0.69/1.96 at 4 s; DriveWAM is 0.47/1.35 and 0.83/2.47. Table 4 reports 6B parameters for SimWAM but does not label displacement units.
Go to primary source ↓e13PDF p. 8, Table 5, all rows; Section 5.3, Component analysis
Action-only, +Video and +RL obtain 86.6, 90.3 and 91.5 PDMS. From +Video to +RL, EP rises 83.9 to 86.4 and DAC 98.0 to 98.7, while NC falls 98.7 to 98.4 and TTC 95.9 to 95.5.
Go to primary source ↓e14PDF p. 8, Table 6; pp. 8–9, Section 5.3, Attention mask
Bidirectional, Action→video and Isolated masks obtain 90.2, 90.1 and 90.3 PDMS. The paper says the first two require future-frame representations at inference. No repeated-run uncertainty is supplied in the table.
Go to primary source ↓e15PDF p. 8, Tables 7–8; p. 9, Video backbone flexibility and Action expert scalability
Video backbones LTX-Video, Wan2.1-1.3B, Cosmos2.5 and Wan2.2-5B obtain 88.7/90.2/90.4/90.3 PDMS. Action experts of 0.21B/0.45B/1.02B obtain 89.9/90.1/90.3; 1.02B is adopted for remaining experiments. The authors attribute Cosmos's result to driving-relevant pretraining.
Go to primary source ↓e16PDF p. 9, Table 9, zero-shot rows and caption; Cross-dataset generalization paragraph
NAVSIM-trained SimWAM transfers without nuScenes fine-tuning or auxiliary supervision. Its L2 at 1/2/3 s is 0.29/0.82/1.77 m, average 0.96 m; collision is 0.00/0.03/0.11%, reported average 0.05%. DriveVA averages 0.84 m/0.06%; DriveWAM averages 0.96 m/0.06%. Collision is measured against logged surrounding-agent futures.
Go to primary source ↓e17PDF p. 9, Tables 10–11; p. 10, Exploration sampler and Prediction horizon and frame density
Random-noise exploration and SDE achieve 91.3 and 91.5 PDMS. Future targets of 4 frames/2 s/2 Hz, 4 frames/4 s/1 Hz and 8 frames/4 s/2 Hz yield 89.9, 90.2 and 90.3 PDMS.
Go to primary source ↓e18PDF p. 10, Figure 3 and RL training dynamics
The imitation checkpoint is 90.3 PDMS. At 5k/10k/15k/20k updates the hard subset scores 91.1/91.2/91.5/91.4, while all training data scores 90.9/90.7/91.4/91.2. Both peak at 15k and decline thereafter.
Go to primary source ↓e19PDF p. 9, Section 5.4 opening; p. 10, Tables 12–13 and related paragraphs
Configuration ablations default to the imitation model and latency uses one NVIDIA A100. Table 13 reports steps/PDMS/ms: 1/68.9/115, 5/90.1/297, 10/90.3/518, 20/90.2/968. Table 12 reports 192×352:88.9/509 ms, 384×672:90.3/518 ms, 768×1344:90.6/573 ms.
Go to primary source ↓e20PDF pp. 10–11, Section 5.5 and Figure 4; p. 11, Section 6
Two qualitative navtest scenes contrast imitation and RL trajectories, with red ellipses highlighting farther progress after RL. These are illustrative comparisons; the paper ends with a conclusion and references on pp. 11–15, with no appendix or dedicated limitations section.
Go to primary source ↓e21PDF pp. 4–7, Sections 3–5.1; p. 10, Figure 3 and sampling discussion
The paper specifies losses, data counts, epochs, learning rates and LoRA settings, but does not specify training GPU count/time, batch size, SDE noise-scale value or endpoint handling, or detailed VAE/T5 freezing policy. No low-level vehicle controller or deployed feedback implementation is described.
Go to primary source ↓8.5 Primary sources
SimWAM: A Simple World Action Model for End-to-End Autonomous Driving ↗
PDF · 8,646 extracted words
Source fingerprint
6f55da6df120d809adf3d739a3b03b6d385d49a126445836a06ccd189303abcf