SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
1. Paper overview
In one sentence: SV-WAM uses future-video supervision to train a surround-view planner whose causal mask removes future-video computation at deployment, while a differentiable footprint penalty improves benchmark road compliance. e02e04e06e09e10e12e13e14e17e19
| At a glance | What to know |
|---|---|
| Research problem | Source description Six-camera context can expose lateral and rear hazards, but denoising six-view futures is expensive. SV-WAM asks whether predictive visual supervision can improve planning without requiring future-video generation during deployment. NAVSIMv2 evaluates this through pseudo-simulation with pre-generated shifted-state observations; nuScenes measures open-loop trajectory predictions. e02e09 |
| Core mechanism | |
| A key reported result | NAVSIMv2 navtest planning: 91.0 EPDMS (%) ↑. Human penalty enabled; 12,146-scenario navtest; two-step C×6 planner; pseudo-simulation. DriveLaW and EponaV2: 88.9; AutoDrive-P3: 89.9. Controlled SV-WAM C×1/C×3 variants: 89.3/90.5. Best aggregate in Table 1, not best on every submetric. Authors rerun DriveLaW/DriveVLA-W0; other baseline scores come from papers or the leaderboard. e09e10 |
| Reading caution |
Core contributions
Figure 4. Action queries cannot read future-video keys, while future-video queries can read actions. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read Q down the rows and K across the columns. The colored headers identify ego-state tokens in yellow, reference-video tokens in green, future-action tokens in blue, and future-video tokens in purple. Plain gray cells permit attention; crossed cells forbid it. The first two rows can read only the two clean conditioning groups. The blue action row can read those conditions and actions, but its purple future-video cell is crossed. The purple video row can read all four groups, including actions. This asymmetry agrees with the caption and Equation (8). Because the conditioning groups also cannot read noisy actions or futures, their representations can be cached across action denoising steps. e04e05e06e07e08e13e14
What it supports. The matrix makes the deployment argument explicit: removing future-video tokens removes no permitted input to the action predictor. Video prediction can still train the shared transformer through its own loss. The resulting planner retains historical surround-view conditioning while omitting future-video generation; the appendix separately reports matching trajectories with that branch present or absent.
Where the evidence stops. This diagram specifies attention permissions, not measured attention weights or planning quality. The performance and latency consequences require the separate ablations and timing protocol. It also does not make the training-time footprint regularizer an online safety filter.
2. Motivation
2.1 The problem and the proposed response
Six-camera context can expose lateral and rear hazards, but denoising six-view futures is expensive. SV-WAM asks whether predictive visual supervision can improve planning without requiring future-video generation during deployment. NAVSIMv2 evaluates this through pseudo-simulation with pre-generated shifted-state observations; nuScenes measures open-loop trajectory predictions. e02e09
2.2 What this reading follows
SV-WAM separates two uses of a driving world model: learning from what happens next and generating that future online. It trains a shared transformer on action and six-view video targets, but makes action tokens depend only on observed history and other actions. This enables action-only deployment with all six cameras retained. The paper's strongest result is NAVSIMv2 navtest planning, supported by component and attention-mask ablations. Reading the evidence carefully also reveals limits: pseudo-simulation differs from physical deployment, latency starts after input preprocessing, and the attention visualization is less supportive of its caption's ranking than the caption suggests. e02e04e06e09e10e12e13e14e17e19
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is WAMs. View the current classification.
3.1 Evidence-based assessment
Insufficient evidence to decide
The recorded catalog is unassigned, so there is no existing quadrant judgment to endorse. Architecture evidence supports a One Model assessment: shared DiT blocks jointly predict action and future-video velocities. The mechanism is joint prediction with an action-centered dependency, not inverse dynamics. Future prediction supervises training; deployed control uses the learned action marginal without future rollout. This is an editorial assessment, not a catalog change. e04e05e07e08
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Separate the forward dependency from the training signal
Imagine freezing the observed scene and sampling a noisy action sequence. The action predictor needs to read the clean history and coordinate the action tokens with one another. Under SV-WAM's mask, it cannot read the future-video tokens. Those video tokens nevertheless pass through the same trainable transformer and incur their own prediction loss. Consequently, learning to forecast future visual structure can alter parameters later used by actions, without supplying future-video activations to actions in that forward pass. The condition prefix is also isolated from noisy actions and futures, allowing its keys and values to remain reusable during denoising. At deployment, only the permitted action dependency is evaluated. The appendix's branch-on versus branch-off trajectory check is the corresponding empirical test of this architecture. e04e05e07e08
5.2 Teach the planner about the vehicle it must keep on the road
A predicted path is a geometric reference; the footprint reached by a tracking vehicle can differ from that reference. SV-WAM therefore reconstructs a clean action estimate from the flow velocity, denormalizes it, and simulates tracking with an LQR controller and bicycle dynamics. The implementation interpolates the reference onto a finer rollout grid and evaluates all four corners at each pose. A positive signed distance means a corner lies inside the drivable region. The softplus penalty grows as clearance falls below the margin, and log-mean-exp concentrates the objective on larger violations. The authors reimplement the tracker and map queries with PyTorch tensors so the penalty can update the action generator. This is training supervision, while NAVSIM's DAC metric remains an evaluation measure. e06e09
5.3 Match every conclusion to its evaluation endpoint
The strongest evidence combines three distinct comparisons. Table 1 evaluates the final planner and camera coverage; Table 3 measures sequential training additions; Table 4 tests the attention mask across denoising budgets. Together they support the proposed design, although they do not isolate every interaction or quantify training-seed variability. The deployment argument then requires a separate timing protocol: H20 results end at trajectory output and exclude future-video decoding. Transfer to nuScenes uses open-loop validation with no target adaptation, which the authors explicitly frame as a sanity check. Finally, navhard and the failure examples constrain the headline: the model is not the highest-scoring method on every split, and conservative behavior can still cause a collision. My reading is that the paper establishes a promising benchmark operating point, with physical deployment still untested. e09e10e11e12e13e14e15e18e19
5.4 Training and inference
During training
The full approximately 5B-parameter, 30-block Wan2.2 DiT is trainable; the VAE stays frozen. Train on NAVSIM trainval for 10k AdamW iterations, batch 80, learning rate 10^-4 and weight decay 10^-2. Fine-tune for 1k iterations with effective batch 640 and learning rate decaying from 10^-5 to 10^-6. Each stage takes roughly one day on 16 H800 GPUs. e08
Both action and video targets share a shifted noise time with coefficient 5.0. Their flow losses have equal weight; compliance weight is 0.01. Training uses twelve future actions and twelve-frame six-view video supervision. e05e08
During inference
Encode the observation/state prefix once, cache its keys and values, initialize Gaussian action noise, and perform two Euler flow steps. Denormalize and accumulate actions in SE(2). The authors report identical trajectories up to numerical precision when optional future-video tokens are included at the same step count. This is trajectory generation; the LQR rollout is a tracking model, not evidence of physical action execution. e03e07
5.5 Implementation flow
- Encode the observed surround
Resize each view to 455 × 256 and center-crop to 448 × 224. Concatenate front-left, front, front-right, rear-right, rear and rear-left along width. A frozen Wan2.2-TI2V-5B causal VAE and shared patch tokenizer encode history and training futures; MLPs embed ego states and actions. e03
- Restrict information flow
The clean state/history prefix attends only within itself. Action queries attend to that prefix and actions. Future-video queries can attend to all groups. Thus video losses update shared parameters without introducing a forward dependency from actions to future video. e04e05
- Penalize the tracked footprint
Recover clean actions, denormalize them and roll out an LQR tracker with a kinematic bicycle model. The differentiable PyTorch implementation samples a signed-distance field at all four vehicle corners. Distances are positive inside the drivable area; a soft margin penalty and log-mean-exp aggregation emphasize dangerous corners. e06
6. Experiments & results
SV-WAM trains one shared transformer to denoise driving actions and future surround-view video, but blocks action attention to future-video tokens. Deployment retains six-camera history while generating only trajectories. A differentiable vehicle-footprint loss improves road compliance. The strongest reported result is 91.0 EPDMS on NAVSIMv2 navtest; the evidence concerns benchmark planning, with weaker hard-split performance and no demonstrated real-vehicle deployment.
6.1 Read the original evidence
Table 1. Six views improve the controlled SV-WAM variants' aggregate score, with mixed changes in individual metrics. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the Input column before comparing scores: C×1 means one front camera, C×3 three front-facing cameras, C×6 six surround cameras, and L LiDAR. The dagger marks world-model methods; bold and underlining denote the best and second-best values. All scores are percentages on navtest with human penalty enabled. The shaded rightmost column is EPDMS. First compare the three Ours rows, which the authors describe as sharing architecture and training schedule. Then inspect individual columns: NC, DAC, DDC and TLC concern safety/compliance; EP, TTC, LK, HC and EC concern progress, collision margin, lane keeping and comfort. e09e10
What it supports. The controlled Ours rows rise from 89.3 to 90.5 to 91.0 EPDMS as coverage expands. Six-view SV-WAM exceeds the 88.9 reported for DriveLaW and EponaV2, but its 98.8 DAC is below the three-view variant's 99.1. The table supports an aggregate advantage, not uniform superiority.
Where the evidence stops. Other methods use different sensing and training pipelines. Only some baseline planning results were rerun by the authors. NAVSIMv2 uses pseudo-simulation, so these scores do not establish performance on a real vehicle or under fully reactive traffic.
Table 5. Action-only inference reduces the reported H20 trajectory-output latency to 341.6 ms. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each row from visual encoding through core generation and Other to Total. Components are means in milliseconds; Total reports mean plus or minus standard deviation. Timing starts at preprocessed inputs and ends at a trajectory, excluding optional video rendering and future-video VAE decoding. Measurements use a single H20, batch one, bf16 where supported, and thirty samples after five warm-ups. The most informative comparison is the last two rows: both are SV-WAM variants, but removing the mask requires future-video computation. The authors use official code, checkpoints and default inference settings for the external baselines. e14e19
What it supports. Masked SV-WAM reports 341.6 ± 2.1 ms versus 848.0 ± 1.8 ms without the mask. Its remaining cost includes 174.7 ms visual encoding and 151.4 ms core generation. The mask removes a dependency, but does not make six-camera perception free.
Where the evidence stops. The table is not raw-sensor-to-actuator timing. The separate H800 Table 11 compares action-only output with joint action and decoded-video output, so its speedup is not directly interchangeable with this trajectory-only H20 comparison. Neither timing validates onboard real-vehicle operation.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| NAVSIMv2 navtest planning Human penalty enabled; 12,146-scenario navtest; two-step C×6 planner; pseudo-simulation. | 91.0 EPDMS (%) ↑ | DriveLaW and EponaV2: 88.9; AutoDrive-P3: 89.9. Controlled SV-WAM C×1/C×3 variants: 89.3/90.5. Best aggregate in Table 1, not best on every submetric. Authors rerun DriveLaW/DriveVLA-W0; other baseline scores come from papers or the leaderboard. e09e10 |
| Training-component ablation NAVSIMv2 navtest; shared C×6 architecture and evaluation. | 83.1/93.4 → 87.7/95.6 → 90.1/98.6 → 91.0/98.8 EPDMS / DAC (%) ↑ | Successively enable future-video supervision, DAC regularization and final fine-tuning. Video supervision adds 4.6 EPDMS points in this sequence. The ordered ablation does not estimate every interaction between components. e12 |
| Causal mask and denoising steps NAVSIMv2 navtest; C×6 variants trained with matched architecture and training settings except mask. | Masked 2/5/10 steps: 91.0/90.9/90.8 EPDMS (%) ↑ | Unmasked 2/5/10 steps: 87.3/87.9/88.2. Extra refinement helps the unmasked variant but does not improve the masked one; no seed-level uncertainty is supplied. e13 |
| Trajectory-output latency Single H20; batch 1; bf16 where supported; 30 samples after 5 warm-ups; starts from preprocessed inputs. | 341.6 ± 2.1 Latency mean ± standard deviation (ms) ↓ | Unmasked SV-WAM: 848.0 ± 1.8; DriveVLA-W0: 514.9 ± 3.8. Excludes optional video rendering and future-video VAE decoding. This is not raw-sensor-to-actuator latency; H800 video-output timings use a different endpoint. e14 |
| Zero-shot nuScenes planning Validation split, no target-domain adaptation; average over 1/2/3 s open-loop metrics. | 0.89 / 0.16 Average L2 (m) ↓ / collision rate (%) ↓ | Zero-shot DriveVLA-W0: 1.43/0.77; PWM: 3.99/0.36. Fine-tuned rows are a separate protocol. The authors treat this as a transfer sanity check because ego-state shortcuts affect these metrics, not as proof of interactive driving safety. e11e19 |
| Additional NAVSIM settings NAVSIMv1 navtest and NAVSIMv2 two-stage navhard with human penalty; distinct metrics/protocols. | v1: 90.2; v2 navhard: 36.1 PDMS / EPDMS (%) ↑ | v1 SafeDrive: 91.6; navhard EponaV2: 36.1 and RAP: 39.6. The headline navtest advantage does not establish dominance across benchmark settings. e15 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 3. The gains accumulate across video supervision, footprint regularization and final fine-tuning. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Move down the rows, reading each checkmark as an enabled training component. Video Sup. is future-video supervision, DAC-reg the drivable-area penalty, and FT the final fine-tuning stage. All rows use six-camera input and the same underlying architecture and evaluation setting. The first transition asks whether future-video co-training helps the action predictor. The second adds direct road-compliance supervision. The third adds the final training stage. Compare both right-hand columns: DAC isolates drivable-area compliance, while EPDMS aggregates safety and driving-quality terms. This paired reading avoids attributing every aggregate gain to richer visual prediction alone. e09e12
What it supports. Video supervision raises EPDMS from 83.1 to 87.7. Adding the footprint loss raises it to 90.1 while DAC rises from 95.6 to 98.6. Fine-tuning produces 91.0 EPDMS and 98.8 DAC. These are sequential absolute score improvements, not relative percentage gains.
Where the evidence stops. The rows form an ordered sequence, not a complete factorial experiment. They do not show DAC regularization without video supervision or separate all interaction effects. No repeated-training variance is given, and the predictive mechanism still needs more discriminating controls.
Table 4. More denoising steps benefit the unmasked model but do not improve the masked action path. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Split the table at the horizontal line. The upper block permits action attention to noisy future-video tokens; the lower block uses the action-centered causal mask. Within each block, Steps changes from two to five to ten, while Input remains C×6. The authors state that architecture, training configuration and hyperparameters are otherwise matched. Start with the rightmost EPDMS values to see the interaction between mask and step count, then check individual columns for tradeoffs. For example, the unmasked two-step row has higher ego progress than the masked two-step row despite a lower aggregate score. e04e07e13
What it supports. The unmasked model improves from 87.3 to 88.2 EPDMS between two and ten steps. The masked model starts at 91.0 and reaches 90.8 at ten steps. This supports the reported two-step operating point and is consistent with avoiding the noisy future-video dependency.
Where the evidence stops. The authors' noise-propagation explanation is an interpretation of this trend, not a direct measurement of corrupted action features. The small masked decline lacks uncertainty estimates. Branch-removal invariance is separately reported in the appendix, not established by these six rows alone.
Figure 7. The heatmap shows attention across all cameras, but its printed ranking differs from the caption's emphasis. Original paper, p. 15 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the camera labels on the heatmap itself: its rear columns are ordered Rear Right, Rear, Rear Left, unlike the image montage above. Rows a1 through a8 are the displayed future-action tokens; columns are observed camera views. Values are averaged over thirty transformer blocks. Warmer colors denote larger attention weights, but the printed numbers permit a more precise comparison. The caption and accompanying prose say front-left and rear receive the strongest attention. Inspect the Front and Rear columns directly: Front ranges from 0.20 to 0.23, whereas Rear ranges from 0.17 to 0.18. e03e17
What it supports. All displayed rows assign nonzero attention to side and rear cameras. However, Front exceeds Rear in every row, and the two largest values occur in the Front Left and Front columns. This supports use of surround-view tokens while requiring a narrower interpretation than the caption's ranking.
Where the evidence stops. The faithful crop is retained despite the caption/prose discrepancy. An averaged attention map cannot establish which view causally changes a trajectory, and the figure displays eight rows although the main planner predicts twelve tokens. The source does not explain that visualization selection.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors identify model size as an onboard deployment constraint. Failure examples include ambiguous turns, rain-obscured signals, and overly conservative acceleration followed by a rear-end collision. Real-vehicle closed-loop evaluation is future work. e18e19
Figure 7's caption and surrounding prose emphasize front-left and rear as strongest, but its displayed values give front greater weight than rear in every row. Attention is evidence of access, not a causal attribution of driving success. e17
Planning tables lack repeated-training uncertainty, and sequential component ablations leave interactions unresolved. Cross-method comparisons also differ in camera coverage and baseline provenance. e10e12e13
Figure 3 sketches history as −H:0, whereas Equation (2) indexes H frames as −H+1:0. The implementation explicitly uses four history frames; the schematic indexing should not be used to infer a different input count. e03e04e08
7.2 Questions for discussion
- Does correct future-video supervision outperform temporally shuffled supervision under matched training compute?
- Would removing rear or lateral views causally worsen lane-change performance despite their nonzero attention?
- How much of the DAC gain survives imperfect maps and tracker mismatch?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs NAVSIM/OpenScene data and drivable maps, the specified Wan backbone/VAE, source-only normalization statistics, and differentiable tracker parity. The appendix specifies a 0.1 s rollout grid, 60 poses, LQR horizon 10, and a 0.5 m signed-distance grid over x=[−20,80] m, y=[−40,40] m. e03e06e08
Preserve unclipped fixed-range action normalization versus clipped ego-state normalization. Verify controller equivalence and boundary gradients before large-scale training. The implementation section does not pin PyTorch/CUDA versions or provide every tracker coefficient, so compatibility and exact parity remain reproduction questions. e03e06e08
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test whether the action path is truly independent of future tokens
Reader-proposed check: fix a checkpoint, observation prefix, action-noise seed, precision and two-step schedule. Compare action velocities at every step and final trajectories with future-video tokens absent, present, and replaced by different video noise. Repeat with prefix caching enabled and disabled. Require agreement within a declared numerical tolerance for the masked model; use the unmasked variant as a dependency control. Record trajectory-only latency under the Table 5 timing boundary. Systematic masked differences would falsify implementation-level independence even if average EPDMS remained similar. e04e07e13e14
Check 2: Distinguish predictive supervision from a generic auxiliary training effect
Reader-proposed check: train matched six-view variants using correct future-video targets, temporally mismatched targets, or zero-weight video loss. Retain video-branch computations in the controls and match backbone initialization, data order, optimizer updates, DAC regularization and fine-tuning schedule. Report training compute as well as repeated-seed navtest EPDMS, DAC and collision components. A consistent advantage for correct futures over both controls would support the dynamics-supervision interpretation. Similar gains from mismatched futures would weaken that interpretation despite reproducing the original component-ablation trend. e05e08e09e12
8.3 Reading coverage
Visual audit: Read all ten supplied text chunks, including the complete 23-page PDF extraction, references and marked derived label transcription. Personally inspected the title/author page, every method and experiment page supporting retained claims, all original figures (1–16), all tables (1–11), and all six final crops. Pages 8–9 contain references and were read in text. The selected architecture visual is the complete Figure 4 attention matrix and legend, cross-checked against Equation (8) and its caption. Figure 3 was also inspected for the loss labels and history-index discrepancy. Figure 7’s numerical ranking differs from its caption/prose; the discrepancy is discussed explicitly. The appendix training, preprocessing, tracker, timing, benchmark and failure pages are included even when uncropped. No separate supplement, external video, code or physical experiment was inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and Introduction (pp. 1–2)
- Related Work (p. 2)
- Methodology: tokenization, causal mask, flow matching, compliance regularization, inference (pp. 3–5)
- Experiments, ablations, efficiency, qualitative analysis and Conclusion (pp. 5–8)
- References (pp. 8–9)
- Appendix: Implementation Details (pp. 10–11)
- Appendix: Dataset and Metric Details (pp. 11–12)
- Appendix: Additional Quantitative Results, Ablations and Inference-Efficiency Details (pp. 12–14)
- Appendix: qualitative analyses, failure cases, Limitations and Future Work (pp. 12–23)
- Separately marked derived transcription of three Figure 3 loss labels
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Poppler layout text is faithfully retained without manual repairs. Mathematical symbols, table ordering, figure labels and ligatures require inspection against the original PDF; successful extraction is not scientific reading.
- The prior full-text HTML source at https://arxiv.org/html/2609.03602 and its extracted text are preserved unchanged in the private recovery history. The complete original PDF is also preserved and remains available for visual reading; no replacement characters have been edited or guessed.
- The original PDF is complete and readable. Its unchanged raw extraction retains 3 replacement characters in diagram labels. A separately marked, independently image-reviewed TeX transcription covers every affected label in the derived reading text. These annotations are reading support, not author prose or restoration of the original font encoding. Original PDF pages 4 remain required visual evidence and must be inspected and included in the report visual audit.
- Identity: the inspected title, all 15 authors, and printed arXiv:2609.03602v1 stamp dated 3 Sep 2026 match the supplied identity. This report concerns v1 only; no alternate revision was compared.
- The supplied text extraction does not reconstruct figure images. Original PDF figures and tables were therefore inspected separately; reference-only pages 8–9 were read as text.
- The original layout extraction remains unrepaired; mathematical symbols, table ordering, labels and ligatures were checked against original page images where used.
- Separate supplemental material availability has not been fully verified.
- A prior full-text HTML source and extraction are retained in the supplied provenance history; this reading uses the complete original v1 PDF and does not infer equivalence to another edition.
- The unchanged extraction contains three replacement characters in Figure 3 labels. The separately marked TeX transcription is derived reading support, not author prose or repaired original encoding. PDF page 4 was personally inspected to verify the three loss labels.
- Code, checkpoints, external links and separate media were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title block and left-margin arXiv stamp
Exact title and 15-author byline match the supplied catalog; the artifact prints arXiv:2609.03602v1 [cs.CV], 3 Sep 2026, and five affiliations.
Go to primary source ↓e02PDF pp. 1–2, Abstract and Introduction
The paper motivates retaining six-camera observations while shifting future-video generation to training supervision.
Go to primary source ↓e03PDF p. 3, Input Tokenization, Eqs. (1)–(5); p. 10, Inputs and Outputs, Eqs. (20)–(22); pp. 11–12, Data Processing and Zero-Shot Protocol
Camera ordering, causal VAE, state/action representations, resizing/cropping and normalization are specified; twelve action increments are accumulated in SE(2), with eight used for 4 s evaluation.
Go to primary source ↓e04PDF p. 3, Action-Centered Causal Mask, Eqs. (7)–(10); p. 4, Figures 3–4 and captions
The prefix cannot read action/future-video tokens; action queries cannot read future-video keys/values; future-video queries can read actions. Equation (8) gives the action-centered factorization.
Go to primary source ↓e05PDF p. 3, Eqs. (6), (9)–(10); p. 4, Eq. (11); p. 5, Eq. (17)
Flow training matches action and future-video velocities, combining their squared errors with the compliance regularizer.
Go to primary source ↓e06PDF pp. 4–5, Drivable Area Compliance Regularization, Eqs. (12)–(17); p. 10, Drivable Area Regularization Details
Positive-inside signed distances at four footprint corners feed softplus and log-mean-exp penalties. The PyTorch LQR/bicycle rollout preserves gradients; configuration includes 60 poses at 0.1 s, horizon 10, margin 0.2, beta 0.2, rho 0.1 and the stated grid.
Go to primary source ↓e07PDF p. 5, Efficient Action-Only Inference, Eqs. (18)–(19); pp. 10–11, Inference
Deployment caches prefix keys/values and runs two action denoising steps without future-video tokens. The authors report trajectory identity up to numerical precision with/without the optional branch.
Go to primary source ↓e08PDF p. 5, Implementation Details; p. 10, Architecture and Training, Eq. (23)
Frozen causal VAE, trainable approximately 5B/30-block DiT, full training/fine-tuning schedule, 16 H800 hardware, equal flow-loss weights and flow-shift coefficient 5.0 are specified.
Go to primary source ↓e09PDF p. 5, Benchmarks and Metrics; p. 11, NAVSIM Overview and Eq. (25); p. 12, nuScenes Metrics
NAVSIMv2 uses pseudo-simulation and EPDMS safety gates; navtest has 12,146 scenarios and navhard 450. nuScenes uses open-loop L2/collision metrics.
Go to primary source ↓e10PDF p. 6, Table 1, EPDMS and component columns; p. 5, NAVSIMv2 Main Results; p. 6, Surround-View Coverage
The six-view score is 91.0, versus 89.3/90.5 for controlled one/three-view variants. Table caption specifies human penalty, camera/LiDAR notation and percentages; prose distinguishes rerun baselines from borrowed results.
Go to primary source ↓e11PDF p. 6, Table 2, zero-shot rows; pp. 11–12, nuScenes Zero-Shot Protocol and Metrics
SV-WAM validation averages are 0.89 m L2 and 0.16% collision, with no target normalization or adaptation; DriveVLA-W0/PWM zero-shot rows differ from the fine-tuned block.
Go to primary source ↓e12PDF p. 6, Table 3 and Training Components discussion; p. 5, Ablation Studies
Sequential video/DAC/fine-tuning additions yield EPDMS 83.1, 87.7, 90.1, 91.0 and DAC 93.4, 95.6, 98.6, 98.8 under common C×6 architecture.
Go to primary source ↓e13PDF p. 7, Table 4; p. 6, Action-Centered Causal Mask and Inference Steps
Masked scores at 2/5/10 steps are 91.0/90.9/90.8; unmasked scores are 87.3/87.9/88.2. Authors describe matched training except mask; no repeated-training uncertainty is tabulated.
Go to primary source ↓e14PDF p. 7, Table 5, caption and Efficiency Analysis; p. 12, Inference-Efficiency Details; p. 14, Table 11 and caption
H20 timing uses batch 1, 30 measured samples after five warm-ups and preprocessed-input-to-trajectory scope: 341.6±2.1 ms masked, 848.0±1.8 ms unmasked. Optional future-video decoding is excluded; H800 Table 11 instead includes decoded video in its joint path.
Go to primary source ↓e15PDF p. 13, Tables 6–7; pp. 11–12, NAVSIMv1 and navhard protocols/results
SV-WAM scores 90.2 PDMS on v1 and 36.1 EPDMS on navhard. SafeDrive reaches 91.6 on v1; RAP reaches 39.6 on navhard, where EponaV2 ties SV-WAM.
Go to primary source ↓e16PDF p. 14, Tables 9–10; p. 12, Additional Ablations
Action-token counts 8/12/16 give 90.4/91.0/90.5 EPDMS. DAC weights 0.01/0.05/0.1 give 91.0/91.0/90.5; fine-tuning batches 320/640/1280 give 90.4/91.0/90.9.
Go to primary source ↓e17PDF p. 15, Figure 7 heatmap and caption; pp. 12–13, Surround-View Attention Analysis
The displayed eight action rows average attention over 30 blocks. Front values are 0.20–0.23, rear 0.17–0.18, and front-left 0.21–0.26. Caption/prose emphasize front-left and rear; front exceeds rear in every displayed row.
Go to primary source ↓e18PDF p. 14, Failure Cases; p. 23, Figure 16 and caption
Three illustrated failures concern wrong-branch left turns, rain-obscured traffic signals, and SV-WAM's conservative acceleration/rear-end collision while both displayed baselines succeed.
Go to primary source ↓e19PDF p. 14, Limitations and Future Work; p. 12, nuScenes (full)
Authors identify the approximately 5B backbone as large for onboard platforms, propose real-vehicle testing, and explicitly caution that open-loop nuScenes results are sensitive to ego-status shortcuts.
Go to primary source ↓8.5 Primary sources
SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving ↗
PDF · 11,679 extracted words
Source fingerprint
dbc51e64e73b220d005016441af70dd00760fccc823f671598a494cb800be0eb