Awesome-WAMs
GitHub
PAPER REPORTENAll readings ↗

Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation

English reading report: Method, equations, original figures, experiments and reproducibility.

Authors: Xingtai Gui; Meijie Zhang; Tianyi Yan; Wencheng Han; Jiahao Gong; Feiyang Tan; Cheng-zhong Xu; Jianbing Shen

Affiliations: SKL-IOTSC, CIS, University of Macau; Afari Intelligent Drive

Source: 2603.14948 ↗ · Catalog record

Reading: 246 / 558 · 6 original figures & tables · ~20 min ·

1. Paper overview

In one sentence: WorldDrive transfers trajectory-conditioned video representations into a separate planner and distills future latents into a fast candidate scorer, trading online video simulation for learned future estimation. e-motivatione-architecturee-fare-representatione-reward-ablatione-navsim

At a glanceWhat to know
Research problem
Source description

Video-generation features need not encode the motion distinctions a driving planner requires. The authors seek reusable visual and trajectory representations while avoiding the latency of generating a future video for every candidate. The resulting system predicts and selects trajectories; the paper evaluates benchmark planning, not physical vehicle deployment. e-identitye-motivatione-protocol

Core mechanism
Source description

TA-DWM conditions diffusion on trajectory anchors plus residuals, making the motion encoder reusable by a downstream planner alongside the adapted visual encoder. e-architecturee-motion

A key reported resultNAVSIM navtest trajectory planning: 88.1 ordinary; 89.0 full navtrain

PDMS, higher is better. Single-view camera; ordinary and full-navtrain settings kept separate

ImagiDrive: 87.4 ordinary; full-navtrain PWM: 88.1 and DriveVLA-W0: 88.4 Strong listed vision-only results. The separate oracle best-of-six score is 93.6 versus 93.0 for DriveVLA-W0; it is an upper-bound diagnostic, not deployed selection. e-navsim

Reading caution
Reader analysis

Benchmark and generative evidence support planning utility and motion sensitivity, not verified physical dynamics. Figure 4 measures latent cosine similarity versus trajectory distance; Figures 6 and 9 show selected synthetic futures without observed counterfactual ground truth. The result tables give no run-to-run uncertainty. e-motion-diagnostice-qualitativee-navsime-generation

Core contributions

  • Source description

    TA-DWM conditions diffusion on trajectory anchors plus residuals, making the motion encoder reusable by a downstream planner alongside the adapted visual encoder. e-architecturee-motion

  • Source description

    FAR transfers future-latent supervision into lightweight attention decoders and a preference-trained scorer. The world model supplies training targets; candidate-conditioned latent estimation remains active during inference. e-fare-loss

Figure 2. Scene-generation training supplies reusable encoders; planning uses a separate candidate generator and rewarder. Original paper, p. 3 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read each band from left to right. In the upper band, historical images pass through the frozen VAE and trainable adapter, while the trajectory vocabulary and expert path enter the motion encoder. Their representations condition TA-DiT, whose future latents can be decoded into video. In the lower band, snowflakes mark inherited frozen encoders. The planner additionally receives ego status and produces candidate trajectories. Follow these candidates into FAR, but notice that the enclosed TA-DWM is labeled train-only. The flame icons summarize trainable components across phases; the implementation text further separates planner training from FAR training, freezing the planner before optimizing FAR. e-architecturee-motione-plannere-fare-implementatione-traininge-protocol

What it supports. The bridge is representation reuse plus distillation. WorldDrive does not run the upper video-generation pipeline for every planning decision. Its lower pipeline preserves candidate-conditioned future reasoning through a learned approximation, which is why world-model training can affect planning without appearing as online diffusion sampling.

Where the evidence stops. Shared encoders do not make this a single jointly generating action/video model. The figure compresses sequential training stages, and its final car icon denotes a selected trajectory rather than evidence of physical vehicle execution.

2. Motivation

2.1 The problem and the proposed response

Source description

Video-generation features need not encode the motion distinctions a driving planner requires. The authors seek reusable visual and trajectory representations while avoiding the latency of generating a future video for every candidate. The resulting system predicts and selects trajectories; the paper evaluates benchmark planning, not physical vehicle deployment. e-identitye-motivatione-protocol

2.2 What this reading follows

A driving video model can imagine a changing street without learning the motion distinctions needed to choose a trajectory. WorldDrive gives its video model a structured trajectory vocabulary, then reuses the resulting vision and motion encoders for planning. A second transfer happens inside the Future-aware Rewarder: candidate-conditioned future latents become training targets for a lightweight scorer. The important reading question is which part supplies the planning gain—generic video pretraining, driving-specific representation transfer, candidate quality, or future-aware selection. The architecture and two ablation tables help separate these contributions, while the benchmark footnotes keep deployable selection distinct from an oracle upper bound. e-motivatione-architecturee-fare-representatione-reward-ablatione-navsim

3. Research context

We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.

Catalog dimensionRecorded classification
Major categoryWAMs
ArchitectureDual-system
Prediction paradigmOther mechanisms
QuadrantOutside quadrants

3.1 Evidence-based assessment

Supports the recorded classification

Reader analysis

Dual-system is supported by a generative TA-DWM and separate planner/rewarder trained in stages with frozen representation transfer. Actions are proposed before candidate-conditioned future estimation; this is neither joint future/action generation nor inverse dynamics. Other mechanisms and Outside quadrants fit this arrangement. The latent-prediction tag applies to FAR, but JEPA is not established: its teacher is a diffusion world model with pixel-decodable latents. e-architecturee-plannere-fare-loss

This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.

4. Problem formulation

4.1 Inputs and outputs

InputsOutputs
  • Historical single-view camera frames
  • Ego velocity, acceleration and driving command
  • A clustered trajectory vocabulary; expert trajectories and future frames during training
  • Multiple refined trajectory candidates and a selected ego trajectory
  • Trajectory-conditioned future video when TA-DWM is separately invoked

4.2 Equations and their role

c=Ea(VK)+Eo(Y−VK)c=\mathcal{E}_a(\mathcal{V}_K)+\mathcal{E}_o(Y-\mathcal{V}_K)
Equation (1): Y is the expert trajectory, V_K denotes its nearest vocabulary anchors, and the anchor and offset encoders produce motion conditioning c. K here counts conditioning anchors, which should not be conflated with FAR's candidate count. e-motione-candidates
z^k=Dscene(Qs,[f,ck],[f,ck]),Lalign=Ek[∥z^k−SG⁡(zk)∥22]\hat z^{k}=\mathcal{D}_{\mathrm{scene}}(Q_s,[f,c^{k}],[f,c^{k}]),\qquad \mathcal{L}_{\mathrm{align}}=\mathbb{E}_{k}\left[\lVert\hat z^{k}-\operatorname{SG}(z^{k})\rVert_2^2\right]
Equations (3) and (6): scene queries Q_s attend to historical visual features f and candidate embedding c^k. Their output approximates teacher latent z^k. SG stops gradients into the frozen world model; Figure 3's double-headed MSE connector depicts matching, not two-way teacher training. e-fare-loss
Lreward=−E(vpos,vneg)∼V^K[log⁡σ(rpos−rneg)]\mathcal{L}_{\mathrm{reward}}=-\mathbb{E}_{(v_{\mathrm{pos}},v_{\mathrm{neg}})\sim\hat{\mathcal{V}}_K}\left[\log\sigma(r_{\mathrm{pos}}-r_{\mathrm{neg}})\right]
Equation (7) trains preferred candidate v_pos to receive a higher scalar reward than v_neg; sigma is the sigmoid. This learns relative preference, not a calibrated collision probability. e-loss

5. Method in detail

5.1 Make motion reusable before asking it to plan

Reader analysis

The motion encoder is the important connection between the two tasks. During scene-generation training, the expert trajectory is represented relative to nearby vocabulary anchors: the anchor supplies a coarse motion template and the residual supplies its adjustment. Their embeddings are added before conditioning the diffusion transformer. During planning, the complete vocabulary's anchor embeddings become queries that search visual history and ego state for relevant context. Learned heads then score and refine those anchors. Reader interpretation: this transfers a motion coordinate system learned while explaining visual change, rather than merely attaching a generic image encoder to a trajectory regressor. Table 4 supplies the corresponding incremental evidence, but it also shows why generic VAE pretraining must remain a separate part of the explanation. e-motione-plannere-representation

Figure 3. Two attention steps turn a candidate trajectory into an estimated future and then a reward. Original paper, p. 4 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Start at the bottom right: green future-scene queries are Q, while the combined pink motion and yellow visual features provide K and V. The Future-scene Decoder produces the green estimated future features. Move upward: the pink candidate-motion embedding now supplies Q, and those estimated futures become K and V for the Future-aware Decoder. A reward MLP finishes the candidate score. On the left, TA-DiT generates the teacher target only during training. The dashed MSE connector has arrowheads at both ends; Equation (6), however, explicitly stops gradients into the teacher. Read it as a feature-matching link, not reciprocal model updating. e-fare-losse-pairs

What it supports. FAR remains sensitive to each candidate because motion enters both future estimation and future-feature querying. Removing diffusion from inference therefore does not remove every future-oriented computation: the student directly estimates the representations on which its ranking depends. This is a learned approximation to the teacher's foresight.

Where the evidence stops. Latent alignment and preference ranking do not guarantee calibrated safety probabilities. The diagram omits the Bradley–Terry supervision path, and the text leaves preference-pair orientation ambiguous when planner rankings and oracle scores disagree.

5.2 Distill candidate consequences, then learn preferences

Reader analysis

FAR performs two different learning jobs. First, its scene decoder predicts a future representation from historical visual features and a candidate trajectory. Squared-error alignment compares that prediction with a stopped target from the frozen world model. Second, the future-aware decoder and reward MLP must turn those features into a useful ordering of candidates, trained with a Bradley–Terry preference objective. Accurate imitation of a teacher latent alone does not mathematically guarantee the right planning order. The implementation freezes the planner before training FAR, making selection a distinct stage. Reader interpretation: Table 5's future-feature gain is consistent with helpful foresight, but could also include benefits from the added decoder path. The unresolved pair-label details matter because a mistaken preference order would train a competent feature predictor to choose the wrong candidate. e-fare-losse-implementatione-pairse-reward-ablation

5.3 Separate candidate quality, selected behavior and video quality

Reader analysis

Three evaluation questions should remain separate. The ordinary NAVSIM result evaluates the selected trajectory. Oracle best-of-six evaluates whether a stronger candidate exists in the set under privileged scoring. The generation metrics evaluate decoded videos and do not measure the selected trajectory's driving success. A fourth check, NAVSIM-v2, adds reactive pseudo-simulation: its aggregate EPDMS improves, yet progress and some comfort scores expose remaining tradeoffs. The supplement's candidate sweep is especially informative because five candidates outperform ten under the learned rewarder. Reader interpretation: the next useful improvement may concern ranking robustness rather than simply producing more alternatives. Finally, the reported 53 ms belongs to the encoder–planner–FAR forward pass on one A800; it does not include video generation, training, or a complete vehicle control stack. e-navsime-protocole-navharde-generatione-candidatese-latency

5.4 Training and inference

During training

Source description

The supplement expands the main text's combined-data description into 200k nuPlan iterations followed by 100k nuScenes iterations: 256×512 video, eight historical and 17 future frames at 10 Hz, learning rate 10⁻⁴. The main text reports 16 A100 GPUs and batch size 32 for world-model training. e-implementatione-training

Source description

Before planner training, TA-DWM representations undergo 100k NAVSIM adaptation iterations at 512×1024 and learning rate 5×10⁻⁵: four historical frames over 2 seconds predict nine frames over 4.5 seconds, with padding. Frozen encoders then support 50 planner epochs, cosine scheduling with peak 6×10⁻⁴, eight NVIDIA 3090 GPUs and batch size 256. FAR follows for 10 epochs at 3×10⁻⁴ with the planner frozen. e-implementatione-training

Source description

Planner supervision combines cross-entropy against distance-derived imitation targets, binary cross-entropy for five simulator scores, and L1 offsets for the nearest expert anchor. FAR combines stopped-teacher latent alignment with Bradley–Terry preference loss. PDMS supplies oracle supervision, but the supplement's planner-top-1 sampling rule leaves pair orientation incompletely specified. e-losse-supp-plannere-pairs

During inference

Source description

The default FAR candidate count is five. Encoders, planner and rewarder run without TA-DiT sampling or video decoding. The selected output is a planned trajectory; a downstream physical controller is not specified. TA-DWM is separately invoked for qualitative future-scene analysis. e-fare-candidatese-qualitative

5.5 Implementation flow

  1. Encode scene and motion

    A frozen CogVideoX 3D causal VAE and trainable adapter encode visual history. K-means supplies 256 trajectory anchors. For an expert trajectory, the motion encoder adds embeddings of nearby anchors and their residual offsets, preserving both coarse intention and finer geometry. e-motione-implementatione-supp-world

  2. Learn conditional dynamics

    TA-DiT predicts diffusion noise from noised latents, visual context, timestep and motion embedding. Section 3.1 describes a future-frame latent target; Section 6.1 specifies concatenated historical and future frozen-VAE latents, with the adapter applied to context. This distinction matters when implementing the target pipeline. e-motione-losse-supp-world

  3. Propose trajectories

    Frozen anchor embeddings query visual features concatenated with an MLP ego-state embedding. Transformer features feed imitation-score, simulation-score and offset heads. High-scoring anchors receive predicted offsets to form candidates; Appendix Equation (9)'s score sign needs clarification. e-plannere-supp-planner

  4. Estimate futures and select

    Learnable scene queries attend to historical visual features and each candidate's motion embedding. A second decoder uses that motion embedding to query the estimated future features; an MLP gives a scalar reward. FAR selects the highest-reward candidate. e-fare-loss

6. Experiments & results

WorldDrive first learns trajectory-conditioned video dynamics, then transfers frozen vision and motion encoders into a trajectory planner. A separately trained Future-aware Rewarder approximates the world model's future latents to rank candidates without diffusion sampling at planning time. Its strongest evidence combines representation and rewarder ablations with single-camera NAVSIM results; generated counterfactual videos and oracle candidate scores have narrower evidential roles.

6.1 Read the original evidence

Table 1. The three WorldDrive scores answer three different evaluation questions. Original paper, p. 6 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read the sensor and footnote columns before comparing the rightmost PDMS column. SV means one camera view, MV multiple views, and L LiDAR. The unmarked WorldDrive row is the ordinary reported setting; its 88.1 can be compared with the unmarked single-view ImagiDrive row at 87.4. Dagger rows use the full navtrain split: WorldDrive reaches 89.0, against 88.1 for PWM and 88.4 for DriveVLA-W0. Double-dagger rows instead use an oracle to choose the best of six candidates. The retained footnotes are essential because the 93.6 row measures candidate-set potential under privileged selection, not FAR's ordinary decision quality. e-navsime-protocol

What it supports. WorldDrive reports strong single-view planning in both ordinary and full-navtrain settings. Its oracle result also shows that candidate generation contains better trajectories than the learned selection result alone reveals. These are complementary findings, but combining their conditions would exaggerate practical planning performance.

Where the evidence stops. The table mixes sensor suites and training scopes and supplies no uncertainty estimates. PDMS is a benchmark aggregate, not a physical-road safety certificate; the paper explicitly excludes oracle selection from inference.

Figure 6, first scenario. Changing the supplied trajectory changes the generated view of the same road bend. Original paper, p. 9 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Use the small trajectory plot at left as the key to the three video rows. The green expert path bends toward the positive horizontal side, the blue alternative remains comparatively straight in the plot, and the orange alternative bends toward the opposite side. Match those colors to the labels at the right edge, then compare the road boundary, chevron signs and building position as each row advances from left to right. This crop preserves one complete scenario from Figure 6, including all three conditions and their legend; the other scenarios remain on the inspected page. The differing views illustrate the model's response to motion conditioning. e-qualitativee-motione-far

What it supports. The video generator can express different visual consequences for alternative trajectory inputs, complementing the planner's use of the same motion representation. This is the interpretable scene-generation capability retained by the framework even though ordinary planning relies on distilled latent features rather than these decoded videos.

Where the evidence stops. These are selected synthetic futures, not recorded outcomes of executing the alternatives. Apparent geometric responsiveness cannot establish counterfactual dynamics accuracy or collision prediction. No numeric physical-error metric can be read from this visual.

6.2 Results and evaluation conditions

Task & protocolReported resultComparison & interpretation
NAVSIM navtest trajectory planning

Single-view camera; ordinary and full-navtrain settings kept separate

88.1 ordinary; 89.0 full navtrain

PDMS, higher is better

ImagiDrive: 87.4 ordinary; full-navtrain PWM: 88.1 and DriveVLA-W0: 88.4

Strong listed vision-only results. The separate oracle best-of-six score is 93.6 versus 93.0 for DriveVLA-W0; it is an upper-bound diagnostic, not deployed selection. e-navsim

NAVSIM-v2 navhard trajectory planning

Reactive traffic, pseudo closed-loop simulation; WorldDrive single-view, baselines multi-view

34.9

EPDMS, higher is better

DiffusionDrive: 27.5; LTF: 23.1

Aggregate gains coexist with lower progress: S1/S2 EP is 60.5/53.1 versus DiffusionDrive's 84.0/85.9. Comfort also remains difficult; the aggregate does not imply dominance on every sub-metric. e-protocole-navhard

nuScenes validation open-loop planning

SparseDrive evaluation convention; reported Avg. columns

0.42 m; 0.16%

L2 error in metres and collision rate in percent, lower is better

World4Drive: 0.50 m/0.16%; SparseDrive: 0.61 m/0.08%

Lower displacement error does not establish lower collision rate or closed-loop driving safety. e-openloop

Representation and rewarder ablations

Section 4.4 planning ablations; Tables 4–5

Representation sequence: 31.4 → 84.9 → 85.8 → 86.9; rewarder sequence: 86.9 → 87.0 → 88.1

PDMS

No pretraining → CogVideoX VAE → TA-DWM vision → TA-DWM motion; then no rewarder features → trajectory features → trajectory plus distilled future features

Most improvement comes from VAE pretraining. Motion transfer and the future-feature rewarder each add 1.1 points in their adjacent comparisons; the latter bundles queries and distillation. e-representatione-reward-ablation

FAR candidate-count sensitivity

Supplementary Table 8

K=1/3/5/10: 86.9/87.9/88.1/87.6

PDMS

Five candidates outperform ten in this experiment

Additional candidates can degrade learned selection; more diversity is not automatically better. e-candidates

Planning inference latency

One NVIDIA A800; WorldDrive batch size one and FAR K=5

53 ms: encoders 17.9, planner 18.9, FAR 16.2 ms

Forward latency

PWM: 850 ms with forecasting; 570 ms without, as reported in Table 6

Measured inference excludes the substantial training cost and does not establish end-to-end vehicle response time. e-latency

nuScenes validation video generation

nuScenes-adapted TA-DWM; 256×512 output

12.8/131.7

FID/FVD, lower is better

Drive-WM: 15.2/122.7 at 192×384

Best listed FID, but not best FVD; different resolutions and training regimes preclude a controlled architecture comparison. Qualitative trajectory control does not validate physical counterfactual accuracy. e-generatione-traininge-qualitative

6.3 Ablations and diagnostic examples

Read component removals and qualitative examples within their stated evaluation conditions.

Table 4. Generic video pretraining supplies the largest gain; driving-specific vision and motion transfer add smaller increments. Original paper, p. 7 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Follow the checkmarks down the first three columns. VAE Pretrain means CogVideoX 3D-VAE initialization. TA-DWM Vision and TA-DWM Motion mean inheriting the corresponding encoders learned through scene generation. The all-cross row scores 31.4 PDMS, and VAE initialization alone raises it to 84.9. Adding the driving-adapted visual representation gives 85.8; adding motion transfer gives 86.9. Read the rightmost aggregate alongside NC, DAC, TTC and EP, which describe collision avoidance, drivable-area compliance, time-to-collision and progress. The last step improves the aggregate while EP changes from 80.7 to 80.4, so the gain is not uniform across every component. e-representatione-protocole-training

What it supports. This table supports a specific incremental claim: after a strong generic visual foundation is present, transferring the motion representation contributes another 1.1 PDMS points. Attributing the entire rise from 31.4 to 86.9 to trajectory-aware world modeling would erase the much larger contribution of existing video pretraining.

Where the evidence stops. The sequential ablation does not isolate every training-budget or representation-capacity effect, and no run-to-run variability is given. The motion-transfer increment supports usefulness in this setup, not a unique causal explanation for all planning improvements.

Table 5. Trajectory features alone barely change the aggregate; the future-feature configuration supplies the larger gain. Original paper, p. 7 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read the first two columns as switches for the scoring inputs. Traj feat. means candidate trajectory features from the planner. Future feat. is a bundled setting that introduces the future-scene queries and future-latent distillation. The first row is the 86.9-PDMS baseline. Trajectory features alone yield 87.0, with a small decline in NC and TTC despite improved progress. The final row adds the future-feature machinery and reaches 88.1 while recovering or improving the listed safety-related scores. Keep this experiment separate from Table 4: it studies selection after the inherited representations are in place, rather than the value of representation pretraining itself. e-reward-ablatione-representatione-fare-loss

What it supports. The rewarder's useful change is associated with its future-feature configuration, not merely another trajectory-feature scoring head. The adjacent 87.0-to-88.1 comparison contributes 1.1 PDMS points. This supports the motivation for future-aware selection while leaving the individual roles of decoder capacity and teacher supervision unresolved.

Where the evidence stops. Future feat. changes both the query-based feature path and its distillation training. A matched-capacity scorer with no teacher or shuffled teacher targets would be needed to isolate whether dynamics-specific information causes the gain.

7. Analysis & limitations

7.1 What the evidence leaves open

Reader analysis

Benchmark and generative evidence support planning utility and motion sensitivity, not verified physical dynamics. Figure 4 measures latent cosine similarity versus trajectory distance; Figures 6 and 9 show selected synthetic futures without observed counterfactual ground truth. The result tables give no run-to-run uncertainty. e-motion-diagnostice-qualitativee-navsime-generation

Reader analysis

Two implementation ambiguities remain: Equation (9) prints a negative weighted log-sum despite Section 3.2 selecting the highest combined scores; Section 6.3 selects planner-top-1 plus hard/random alternatives, while the main text describes oracle-defined preferences. Neither score ordering nor complete pair-label construction should be silently repaired. e-plannere-supp-plannere-losse-pairs

7.2 Questions for discussion

  1. How much of FAR's gain survives a scorer with matched capacity but shuffled future-latent targets?
  2. Does motion-sensitive latent geometry predict counterfactual dynamics accuracy beyond the shown qualitative cases?
  3. Can reward calibration improve navhard progress and comfort without sacrificing its safety gains?

8. Reproducibility audit

8.1 Requirements and known gaps

Reader analysis

A faithful attempt needs CogVideoX weights, nuPlan/nuScenes/NAVSIM training data, the 256-anchor vocabulary and simulator-derived targets, plus separate video-generation and planning-adapted checkpoints. The supplied text does not specify the exact CogVideoX variant, optimizer details, FAR query count, teacher sampling configuration, or software versions. e-implementatione-supp-worlde-supp-plannere-traininge-far

Reader analysis

Proposed checks: hold the candidate set and scorer capacity fixed while removing or shuffling latent supervision; separately hold video training and planner settings fixed while ablating anchor-residual motion transfer. Measure learned selection against oracle candidate quality rather than conflating candidate coverage with ranking accuracy. e-representatione-reward-ablatione-candidatese-motion

8.2 Proposed reproduction checks

The following checks are proposals motivated by the paper. They have not been run as part of this reading.

Check 1: Does the teacher contribute useful dynamics beyond scorer capacity?

Reader-proposed experiment, not performed: freeze one planner checkpoint and its same five candidates per scene. Train equal-capacity FAR variants with correct teacher latents, shuffled candidate-to-latent targets, and no alignment loss while keeping the future-decoder path, preference labels, data and optimization budget fixed. Resolve the paper's preference-pair orientation first. Compare navtest PDMS, NC/TTC/EP and the difference between learned selection and oracle selection over those identical candidates across repeated seeds. If correct alignment consistently improves selection while shuffled and absent alignment do not, that supports dynamics-specific transfer. Similar performance among variants would weaken the claim that the Table 5 gain comes from teacher foresight itself. e-fare-losse-pairse-reward-ablatione-candidatese-navsim

Check 2: Does anchor-residual conditioning create a transferable motion representation?

Reader-proposed experiment, not performed: hold the VAE initialization, video datasets, training iterations, anchor vocabulary and planner capacity fixed. Compare TA-DWM trained with anchor-plus-residual conditioning against an anchor-only variant, then evaluate each planner both with its inherited motion encoder and with a newly initialized encoder under the same planner-training protocol. Report selected PDMS and oracle quality over an identical candidate-count budget. Also measure latent sensitivity to small and large trajectory perturbations using the paper's Figure 4 diagnostic. A selective advantage from inheriting the residual-trained encoder, together with improved selected planning, would support transferable motion detail. Greater latent sensitivity without planning benefit would show that controllability and decision utility can diverge. e-motione-traininge-representatione-motion-diagnostice-navsim

8.3 Reading coverage

Visual audit: Inspected the title/version/author page, main method and loss pages, every quantitative table (Tables 1–9), the motion-sensitivity plot, and qualitative Figures 5–6 and 8–9. Supplementary pages 13–14 supply the retained training, hardware, sampling and inference details. Figure 7(b) is visible; the upper Figure 7(a) panel is blank/obscured at both 200 and 300 DPI, so its extracted labels were not treated as a visually verified diagram. Its readable Section 6.1 text and the legible Figures 2–3 support the method account. Figure 3's MSE arrow was checked against the explicit stop-gradient in Equation (6). All six final original crops were individually viewed, including legends and the main-results footnotes. Pages 10–12 were read in the text chunks (conclusion continuation and references) but were not part of the visual pass; no retained method or numerical claim depends on their images. No external videos, code or separate supplemental artifact was inspected.

PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 13, 14, 15, 16. Appendix coverage: reviewed.

Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.

Text reading scope & known omissions
  • Abstract and title block, PDF p. 1
  • 1. Introduction, PDF pp. 1–2
  • 2. Related Works, including Sections 2.1–2.3, PDF pp. 2–3
  • 3. Method, including Sections 3.1–3.4 and Equations (1)–(7), PDF pp. 3–5
  • 4. Experiments, including Sections 4.1–4.6, PDF pp. 5–9
  • 5. Conclusion, PDF pp. 9–10
  • References, PDF pp. 10–12
  • Embedded Supplementary Material: 6. Further Implementation Details, Sections 6.1–6.4 and Equations (8)–(9), PDF pp. 13–14
  • 7. Further Ablation Study, PDF p. 14
  • 8. Further Qualitative Comparison, PDF pp. 14–16

Outside the original text pass

  • Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
  • Separate supplemental material availability has not been fully verified.
  • Identity/version note: the inspected title page identifies arXiv:2603.14948v1 [cs.CV], 16 March 2026. Title and all eight authors match the supplied catalog. No other revision or edition was supplied or compared.
  • Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This acquisition limitation was addressed through direct page and crop inspection, subject to the Figure 7(a) visibility limitation below.
  • Separate supplemental material availability has not been fully verified. The supplementary material embedded on PDF pages 13–16 was read in full; no separate supplement was supplied.
  • Figure 7(a)'s upper diagram is blank/obscured in both inspected page renders; its extracted labels do not establish a visible layout. Section 6.1 and Figures 2–3 remain readable. No claim is based on unseen details of Figure 7(a).
  • All six supplied text chunks were read individually without truncation. Code and external project material were not inspected, and no experiments were reproduced. References were read as part of this paper, not independently verified.

The visual audit above records the subsequent illustrated pass.

8.4 Traceable evidence

e-identityPDF p. 1, title/authors/affiliations, arXiv margin and AbstractInspect

Exact catalog title and eight authors; University of Macau and Afari Intelligent Drive affiliations; arXiv:2603.14948v1 [cs.CV], 16 Mar 2026. Abstract describes representation transfer and FAR.

Go to primary source ↓
e-motivationPDF pp. 1–2, Section 1 and Figure 1Inspect

The stated problem is scene-generation/planning representation misalignment and the cost of online future-video generation.

Go to primary source ↓
e-architecturePDF p. 3, Figure 2 and Section 3Inspect

Separate scene-generation and planning phases; frozen VAE, inherited encoders and training-only TA-DWM inside the rewarder diagram.

Go to primary source ↓
e-motionPDF p. 4, Section 3.1, Equation (1)Inspect

Motion conditioning adds anchor and residual embeddings; adapted visual history and motion condition diffusion. Main-text target is written as encoded future frames.

Go to primary source ↓
e-plannerPDF p. 4, Section 3.2, Equation (2)Inspect

Anchor queries attend to frozen visual features and ego state; imitation/simulation heads score anchors and regression predicts offsets. Text selects highest combined scores.

Go to primary source ↓
e-farPDF pp. 4–5, Figure 3, Section 3.3, Equations (3)–(4)Inspect

Scene queries attend to history and candidate motion; motion then queries distilled future features for reward prediction. TA-DiT is training-only. Figure's MSE connector is double-headed; Equation (6) specifies stopped teacher gradients.

Go to primary source ↓
e-lossPDF p. 5, Section 3.4, Equations (5)–(7)Inspect

Noise-prediction world loss, planner imitation/simulation/regression losses, stopped-teacher squared-error alignment and Bradley–Terry oracle-preference ranking.

Go to primary source ↓
e-protocolPDF p. 5, Sections 4.1–4.2Inspect

NAVSIM PDMS, NAVSIM-v2 reactive pseudo closed-loop EPDMS, nuScenes open-loop L2/collision evaluation, and nuScenes validation FID/FVD.

Go to primary source ↓
e-implementationPDF p. 5, Section 4.3Inspect

CogVideoX initialization, 256 K-means anchors, 16 A100 GPUs/batch 32 for TA-DWM, eight 3090 GPUs/batch 256 for planner, frozen planner for FAR and PDMS oracle supervision.

Go to primary source ↓
e-navsimPDF p. 6, Table 1, WorldDrive/ImagiDrive/PWM/DriveVLA-W0 rows and dagger footnotes; Section 4.4Inspect

Ordinary WorldDrive 88.1, ImagiDrive 87.4; full-navtrain WorldDrive 89.0, PWM 88.1, DriveVLA-W0 88.4; oracle best-of-six WorldDrive 93.6 versus 93.0. Oracle is explicitly excluded from inference.

Go to primary source ↓
e-navhardPDF p. 6, Table 2, EPDMS and S1/S2 EP columns; Section 4.4Inspect

WorldDrive EPDMS 34.9 versus DiffusionDrive 27.5 and LTF 23.1. WorldDrive EP 60.5/53.1 versus DiffusionDrive 84.0/85.9; authors acknowledge comfort difficulties.

Go to primary source ↓
e-openloopPDF p. 7, Table 3, Avg. L2/CR columns and captionInspect

WorldDrive 0.42 m/0.16%; World4Drive 0.50 m/0.16%; SparseDrive 0.61 m/0.08%; nuScenes validation, SparseDrive metric convention.

Go to primary source ↓
e-representationPDF p. 7, Table 4 and Impact of Pre-training StrategyInspect

No pretraining, VAE, TA-DWM vision, and TA-DWM motion yield 31.4, 84.9, 85.8 and 86.9 PDMS. Last transition changes EP from 80.7 to 80.4 while improving aggregate score.

Go to primary source ↓
e-reward-ablationPDF p. 7, Table 5 and Trajectory Rewarder Strategy AnalysisInspect

Neither feature: 86.9 PDMS; trajectory only: 87.0; trajectory and future: 88.1. Future feature means scene queries plus latent distillation. Trajectory-only NC/TTC slightly decrease.

Go to primary source ↓
e-latencyPDF p. 7, Table 6; PDF p. 14, Table 9 and Section 7 Feasibility AnalysisInspect

Single A800: PWM 850 ms, PWM without forecasting 570 ms, WorldDrive 53 ms. WorldDrive uses batch one/K=5; component times 17.9/18.9/16.2 ms.

Go to primary source ↓
e-generationPDF p. 8, Table 7, resolution/FID/FVD rowsInspect

WorldDrive 256×512, FID 12.8, FVD 131.7; Drive-WM 192×384, 15.2, 122.7. Resolution varies across methods.

Go to primary source ↓
e-motion-diagnosticPDF p. 7, Figure 4 and Section 4.5; PDF pp. 8–9, Motion sensitivity continuationInspect

Cosine similarity of generated latents falls as trajectory distance increases; plotted comparisons concern Top-1/Top-5 conditioning and 10k/100k training, with stronger discrimination after longer training.

Go to primary source ↓
e-qualitativePDF pp. 8–9, Figures 5–6 and Section 4.6; PDF pp. 14–16, Section 8 and Figures 8–9Inspect

Planning examples compare expert, no-FAR and FAR trajectories with optionally generated scenes. Scene-generation examples condition on expert and two non-expert paths; Figure 6's first group depicts a bend and divergent motion conditions.

Go to primary source ↓
e-supp-worldPDF p. 13, Section 6.1 and Figure 7Inspect

Text specifies frozen VAE encoding of historical/future frames, context adaptation, temporal latent concatenation and trajectory conditioning of TA-DiT. Figure 7(b) is visible; its upper panel is visually obscured.

Go to primary source ↓
e-supp-plannerPDF p. 13, Section 6.2, Equations (8)–(9)Inspect

Imitation soft targets use softmax of negative anchor-expert distance; five simulator rewards use BCE; positive-anchor offsets use L1. Equation (9) prints a negative weighted log-sum with weights 0.1, 0.5, 0.5, 1.

Go to primary source ↓
e-pairsPDF pp. 13–14, Section 6.3Inspect

From planner top-16, select planner-top-1, three lowest-simulation-score hard negatives, and three random remaining candidates; selected trajectories form BT preference pairs without a complete orientation rule.

Go to primary source ↓
e-trainingPDF p. 14, Section 6.4Inspect

Sequential nuPlan/nuScenes world-model training; generation checkpoint is nuScenes-adapted. NAVSIM representation adaptation precedes frozen-encoder planner and FAR stages. Iterations, resolutions, frame counts, horizons and learning rates are specified.

Go to primary source ↓
e-candidatesPDF p. 14, Table 8 and Section 7 Top-K Candidates in FARInspect

K=1,3,5,10 yields PDMS 86.9,87.9,88.1,87.6. Default K=5; larger candidate sets can hurt selection.

Go to primary source ↓

8.5 Primary sources

Scroll across the image to inspect details. Press Esc to close.