Epona: Autoregressive Diffusion World Model for Autonomous Driving
1. Paper overview
In one sentence: Epona shares a causal history representation across separate trajectory and image diffusion heads, gaining flexible video rollouts and cheaper planner-only inference while still accumulating prediction error. e03e04e05e10e13e14e15e20
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors seek flexible-duration driving simulation together with direct motion planning. They argue that fixed-length video diffusion complicates temporal extension, while discrete token autoregression sacrifices visual detail and action precision. Their proposed solution separates causal historical reasoning from continuous future generation; the broad critiques of competing model families remain author arguments, not isolated experimental conclusions. e02 |
| Core mechanism | |
| A key reported result | NAVSIM trajectory planning: 86.2 PDMS, higher is better. Test set; two-second observation history, four-second future; camera input. DRAMA 85.5 with camera and lidar; LAW/PARA-Drive 84.6 with camera; human 94.8. Best overall score among Table 4's learned planners, not best in every component or a physical deployment result. e13 |
| Reading caution | Reader analysis Table 1's 120s/600-frame maximum denotes plausible generation, not a common-horizon quantitative test; Figure 10 additionally shows selected frames through 140s. Neither validates traffic-rule understanding or sustained safe vehicle execution. No uncertainty intervals accompany the highlighted comparisons. e10e12e13e14e15e20 |
Core contributions
Figure 2. A shared memory of the past supports two separately sampled predictions. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the blue camera branch and green motion branch from left to right. DCAE converts historical observations into continuous latents; MST combines them with historical relative motions. Its compact representation F feeds both purple diffusion heads. The upper head predicts the next image latent, while the lower head predicts a sequence of future poses. The dashed connection from the trajectory output into image prediction agrees with Section 3.3: VisDiT can condition on the next predicted action. An external action can replace it. The top feedback path denotes training on predicted history; the circular arrow indicates repeated autoregressive prediction. Neither arrow represents a vehicle physically executing the generated path. e03e04e05e18
What it supports. The architecture separates the duration of a planned trajectory from the one-frame visual prediction step. Sharing F lets visual and trajectory objectives train the same historical representation, while the distinct heads let planner-only inference omit image generation. This flexibility follows from the specified module connections, not merely from the phrase joint training.
Where the evidence stops. The overview does not specify the attention mask inside MST or every sampling dependency. Section 3.3 supplies causal temporal attention; Appendix A and Figure 9 clarify that action modulation belongs only to VisDiT. Separate diffusion heads should not be mistaken for a single joint denoiser.
2. Motivation
2.1 The problem and the proposed response
The authors seek flexible-duration driving simulation together with direct motion planning. They argue that fixed-length video diffusion complicates temporal extension, while discrete token autoregression sacrifices visual detail and action precision. Their proposed solution separates causal historical reasoning from continuous future generation; the broad critiques of competing model families remain author arguments, not isolated experimental conclusions. e02
2.2 What this reading follows
A driving world model must represent how a scene evolves and how the vehicle might move through it. Epona assigns these jobs to two continuous diffusion predictors fed by the same history encoder. The trajectory head proposes a future path; the visual head predicts the next camera image under a chosen action. The distinction matters: video prediction can improve the representation during training without being executed when planning. Read the architecture first, then separate three kinds of evidence: visual-generation scores, benchmark trajectory quality and robustness to repeated self-prediction. The long-video examples illustrate plausibility, while the ablations offer narrower tests of the proposed mechanisms. e03e04e05e10e13e14e15e20
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Insufficient evidence to decide
The source supports WAM, autonomous driving, joint video-action modeling and long-horizon modeling. Distinct TrajDiT/VisDiT predictors make a modular Dual-system reading defensible despite shared MST. The mechanism is jointly trained, asynchronously coupled future/action prediction, not inverse dynamics; joint training alone does not establish One Model. Other mechanisms and Outside quadrants cannot be confirmed without the catalog's quadrant criteria. Preserve the snapshot and flag those labels for review. e03e04e05e18
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Why one historical representation can support two prediction horizons
Epona defines each historical action through relative orientation and ego-frame displacement, rather than a discrete driving token. Camera images and those motions first become a common sequence for MST. Causal temporal attention restricts historical information flow; spatial attention combines modalities within the representation. The final-frame latent F then conditions two different generation problems. TrajDiT samples a whole future trajectory, whereas VisDiT samples only the next camera frame and additionally receives a next-step action. This lets an externally chosen action steer the visual branch without asking the trajectory head to invent that command. It also explains why the presence of video prediction does not force the planner to render every candidate future: the two heads are separately usable, although trained together. e03e04e05e18
Figure 4. Chain-of-Forward trains on some of the model's own imperfect context. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the solid portion as next-step supervision: a green input sample leads to a yellow prediction, paired with the corresponding green target below. The source uses x for either image latents or trajectories. At the right, dashed return paths carry predicted samples into later input positions, exposing subsequent predictions to imperfect history. Section 3.4 explains how those samples are obtained: Equation (7) estimates clean data using one velocity prediction from a noisy target. This is cheaper than performing a complete diffusion sampling loop for each added training pass. The training configuration applies this procedure every ten steps with three forward passes. e06e08e15e21
What it supports. The intended intervention addresses the gap between ground-truth history during ordinary training and generated history during video rollout. It changes what later predictions must tolerate. Figure 4 explains the intervention, while Figure 8 supplies the quantitative evidence that the resulting model degrades more slowly over the tested rollout.
Where the evidence stops. The dashed paths do not specify gradient detachment or a full training algorithm. Equation (7) agrees with the data-directed regression target, but Equation (1)'s time-direction notation remains ambiguous when compared with Equation (2); the report preserves that implementation question.
5.2 How predicted training context changes the rollout problem
Ordinary next-frame training gives the model ground-truth history, while a long generated video repeatedly supplies its own outputs. Chain-of-Forward deliberately narrows that mismatch. Instead of fully sampling a fresh image for every extra training pass, it estimates clean data from a noisy target and the predicted velocity, then reuses the estimate as later context. Reader interpretation: this teaches recovery from the model's approximate predictions, but it does not reproduce every error distribution encountered when sampling from pure noise. The paper runs three passes every ten training steps. Figure 8 shows a widening FID advantage over the untreated model, while Table 6 separately shows the benefit of temporal decoding. These interventions address related degradation symptoms through different parts of the pipeline. e06e07e08e15e16
5.3 What remains when the video generator is switched off
At planning time, the shared encoder and trajectory head remain active while VisDiT can be omitted. Table 5 makes this distinction scientifically useful: removing video prediction from training reduces NAVSIM performance, even though image generation is unnecessary for planner-only inference. Reader interpretation: the demonstrated benefit can reside in a representation shaped by visual prediction; it need not come from evaluating imagined futures online. Runtime requires a second distinction. Ten diffusion steps yield roughly 0.02 seconds for MST and 0.03 seconds for TrajDiT, whereas the main experiments specify 100 steps. NAVSIM also changes the stated planning horizon to four seconds. A reproduction should therefore verify both the horizon adaptation and the quality-versus-sampling-step curve before associating the headline planning score with 20 Hz. e05e08e11e13e14
5.4 Training and inference
During training
The approximately 2.5B model uses 12-layer MST and VisDiT modules plus a 2-layer, 50M-parameter TrajDiT. World-model training uses NuPlan and 700 NuScenes scenes at 512×1024 resolution: 600k iterations, batch 96, AdamW, learning rate 10^-4 and weight decay 0.05, on 48 A100 GPUs for nearly two weeks. e08
Trajectory and image rectified-flow losses jointly supervise the world model. Every ten training steps, Chain-of-Forward performs three passes using single-step denoised estimates as subsequent context. DCAE is pretrained; its encoder stays fixed when adding temporal decoder attention. Thus training the world model from scratch does not mean that every component is randomly initialized. e05e06e07e08
During inference
The video evaluation uses ten past frames, repeatedly generates the next frame, and sets diffusion sampling to 100 steps. The generic planner and timing test use three-second trajectories, whereas NAVSIM explicitly uses two seconds of observation and a four-second future horizon; the source does not fully describe this adaptation. e08e09e11e13
On a single NVIDIA 4090, ten-step MST plus TrajDiT timings sum to about 0.05s, consistent with the claimed 20 Hz. At the 100-step experimental setting they sum to about 0.32s. These are module timings, and matching ten-step planning accuracy is not reported. e11
5.5 Implementation flow
- Compress and fuse history
DCAE encodes images with 32-fold spatial downsampling. MST embeds visual patches and three motion components, interleaving triangular causal temporal attention with multimodal spatial attention. Its last-frame representation F summarizes the supplied history; it is a learned latent, not an explicit map or physical state. e04e07
- Predict two different futures
TrajDiT denoises an entire continuous trajectory conditioned on F. VisDiT denoises one next-frame latent using F and the predicted or supplied next action. Both use dual-stream then single-stream transformer blocks; only VisDiT receives action modulation. The image head therefore is not an inverse-dynamics action extractor. e03e05e18
- Decode and continue
The temporal-aware DCAE decoder converts predicted latents to imagery. Autoregressive simulation conditions subsequent predictions on generated history. Planner-only inference uses MST and TrajDiT without rendering future images; predicting a trajectory does not by itself demonstrate execution by a physical vehicle. e03e05e06e07
6. Experiments & results
Epona compresses camera and ego-motion history into a shared temporal representation, then uses separate diffusion transformers to predict a continuous future trajectory and one action-conditioned image. Repeating image prediction produces long videos; disabling it enables cheaper trajectory inference. Its strongest mechanism evidence is improved planning after joint visual training and reduced rollout degradation after Chain-of-Forward training. Reported video quality, qualitative duration and benchmark planning scores establish different capabilities, with important protocol and implementation gaps.
6.1 Read the original evidence
Table 1. Lower FVD accompanies longer reported plausible rollouts, with an FID tradeoff. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each metric row separately rather than treating the rightmost bold values as a complete ranking. Lower is better for both FID and FVD. Epona's FVD is 82.8 compared with Vista's 89.4, but its FID is 7.5 compared with Vista's 6.9. The final row measures a different property: the maximum duration judged to produce plausible results, as the retained asterisk note explains. It lists 120 seconds and 600 frames for Epona. Section 4.1 describes ten-frame conditioning and 1,646 NuScenes validation clips; Section 4.2 says the comparison mostly uses scores reported in earlier papers rather than a uniform rerun. e08e09e10e19e20
What it supports. The table supports an approximately 7.4% relative FVD reduction against Vista, calculated from its displayed scores, together with a longer reported plausible horizon. It also shows a less favorable image-level FID. The three rows therefore support a tradeoff, not an improvement in every aspect of visual quality.
Where the evidence stops. Maximum duration is not a shared-horizon FVD test. Appendix B additionally lists InfinityDrive at FVD 70.06, below Epona's 82.83, qualifying the caption's broad superiority claim. Separate selected supplementary examples extend to 140 seconds without supplying a matching quantitative evaluation.
Table 4. Epona leads the listed learned planners in aggregate NAVSIM score. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the Input column to distinguish camera-only methods from camera-and-lidar methods, then move to the rightmost PDMS column. The preceding columns explain why the aggregate should not be read as dominance on every criterion: NC denotes no at-fault collision, DAC drivable-area compliance, TTC time-to-collision, Comf. comfort and EP ego progress. Epona scores 86.2 PDMS, with component values 97.9, 95.1, 93.8, 99.9 and 80.4. Section 4.3 specifies two seconds of observed history and four-second predicted trajectories for this evaluation. Keep that horizon distinct from the three-second trajectory used in the runtime table. e11e13
What it supports. Epona's 86.2 is higher than DRAMA's 85.5 and the 84.6 reported for camera-only LAW and PARA-Drive. The human row remains higher at 94.8. The result supports competitive aggregate planning on this benchmark, while the component columns show that other methods retain individual advantages.
Where the evidence stops. Different sensor inputs prevent a clean architectural comparison. These are benchmark planning scores rather than physical vehicle trials. Table 2's fast ten-step timings do not establish that this same NAVSIM accuracy is retained at the claimed 20 Hz.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| NuScenes video generation 1,646 validation clips; ten-frame conditioning; most baseline scores quoted from prior papers. | 82.8 / 7.5 FVD / FID, lower is better | Vista: 89.4 / 6.9; DrivingWorld: 90.9 / 7.4. The FVD reduction versus Vista is about 7.4% (reader calculation); FID is worse. This does not establish universal video superiority. e09e10e19 |
| NAVSIM trajectory planning Test set; two-second observation history, four-second future; camera input. | 86.2 PDMS, higher is better | DRAMA 85.5 with camera and lidar; LAW/PARA-Drive 84.6 with camera; human 94.8. Best overall score among Table 4's learned planners, not best in every component or a physical deployment result. e13 |
| NuScenes trajectory planning Front-camera input; no listed auxiliary perception supervision; 1/2/3-second horizons. | 1.25 m / 0.36%; one-second collision 0.01% Mean L2 error / mean collision rate | Doe-1: 1.26 m / 0.53%; UniAD: 1.03 m / 0.31%. Competitive under differing inputs and supervision; the strongest one-second collision result is not the strongest average result. e12 |
| Joint visual-training ablation NAVSIM test; remove video prediction during training. | 86.2 with joint training; 78.1 without PDMS | 8.1 score-point gain (reader subtraction); comfort stays 99.9. Supports the benefit of visual supervision, but does not isolate shared-latent architecture from additional training signal. e14 |
| Chain-of-Forward rollout ablation NuPlan test; Figure 8's final plotted point between 35 and 40 autoregressive frames. | 18.54 with Chain-of-Forward; 25.16 without FID | Both curves worsen as rollout length grows. Shows reduced degradation rather than its elimination; the source does not clarify the off-tick horizon coordinates. e15 |
| Temporal decoder ablation NuPlan test; generated-video evaluation at 40 frames. | 74.88 with temporal module; 100.11 without FVD40 | At 10 frames: 50.77 versus 52.95. The larger long-horizon gap supports temporal decoding, without independently measuring physical consistency. e16 |
| Conditioning-history ablation NuPlan test; 2, 5 or 10 conditioning frames. | 103.70 / 86.76 / 74.88 FVD40 | Ten frames perform best among tested lengths. Historical context helps within this sweep; ten is the reported computational limit, not a demonstrated universal optimum. e17 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 5. Visual supervision improves planning even when image generation can be disabled at inference. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare the two rows column by column. Section 4.4 defines the upper variant as training solely for trajectory prediction, with video prediction disabled during training. The lower row retains the joint visual and trajectory objective over shared latent F. Its improvements span no at-fault collision, drivable-area compliance, time-to-collision and ego progress; comfort remains unchanged at 99.9. The aggregate PDMS increases from 78.1 to 86.2, an 8.1-point difference calculated from the table. This ablation concerns how the model is learned, so it should be read separately from the option to disable VisDiT when running the already trained planner. e04e05e14
What it supports. The result supports visual prediction as useful supervision for the planning representation. It is especially relevant because Epona's planner need not synthesize images at inference: knowledge shaped by the visual objective can be used through F and TrajDiT. The table measures this training benefit rather than a benefit from online imagined rollouts.
Where the evidence stops. Removing video prediction also removes an objective and its training signal. The comparison therefore does not isolate shared-latent coupling from auxiliary supervision or establish a compute-matched advantage. Repeated-seed uncertainty and an independent visual-backbone control are not supplied.
Figure 8. Chain-of-Forward reduces the growth of rollout error, but both curves still deteriorate. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the legend before tracing either curve: solid blue circles use Chain-of-Forward, while dashed red squares omit it. The horizontal axis is autoregressive frames and the vertical axis is FID, where lower values indicate better image-distribution agreement. The curves begin close together and their separation grows across the displayed rollout. At the final plotted pair, the readable annotations give 18.54 with Chain-of-Forward and 25.16 without it. Both curves rise, which is central to the interpretation. The accompanying text identifies the NuPlan test set; Figure 7 provides selected visual comparisons, while this graph supplies the measured trend over the plotted range. e06e15
What it supports. The widening gap supports the claim that exposure to self-predicted history improves robustness to repeated generation. It does not show error-free dynamics: the treated model's FID still increases substantially. This is evidence for reduced distributional degradation over the evaluated rollout, rather than a guarantee of indefinite simulation stability.
Where the evidence stops. The first blue annotation overlaps a red marker in the source, so its exact value is not used. Final markers fall between 35 and 40 ticks; the caption does not explain their horizon convention. These FID scores measure image-distribution agreement, not traffic safety.
7. Analysis & limitations
7.1 What the evidence leaves open
Table 1's 120s/600-frame maximum denotes plausible generation, not a common-horizon quantitative test; Figure 10 additionally shows selected frames through 140s. Neither validates traffic-rule understanding or sustained safe vehicle execution. No uncertainty intervals accompany the highlighted comparisons. e10e12e13e14e15e20
Appendix B reports InfinityDrive FVD 70.06, Epona 82.83 and MagicDriveDiT 94.84. Its statement that MagicDriveDiT has better visual quality is not explained by those FVD values. The authors acknowledge compression artifacts. Preserve this tension rather than repeating an unrestricted state-of-the-art claim. e19
Equation (2)'s increasing-t derivative points toward noise, but the velocity target points toward data. Equation (7) is consistent with that target; Equation (1)'s integration convention remains ambiguous. Decoder temporal window/causality, training gradient detachment and detailed evaluation sampling procedures are insufficiently specified for an exact reproduction. e21e06e07e09e18
7.2 Questions for discussion
- Would planning still improve with an independent visual-training backbone, given the same compute and observations?
- How much of the reported planning quality survives the ten-step configuration associated with 20 Hz?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the stated datasets, trajectory coordinate conversion, pretrained DCAE and substantial training compute. Recover frame sampling, diffusion solver/time convention, trajectory normalization, NAVSIM horizon adaptation and decoder fine-tuning settings before interpreting mismatches. The source promises future code availability, which establishes no runnable release. e03e07e08e09e13e18e21e22
Proposed checks should separate extra visual supervision from shared-latent coupling and compare Chain-of-Forward against a compute-matched baseline. Keep data, horizon, decoder and evaluation noise fixed; measure planning and rollout quality separately. e14e15e16
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate the value of visual supervision from sharing F
Reader-proposed experiment: use the same training clips, planner capacity and NAVSIM split to compare trajectory-only training, the paper's shared-MST joint training, and a visual predictor trained with its own history backbone whose gradients do not enter the planner. Report both matched-iteration and matched-compute comparisons, with repeated seeds. Keep the two-second observation/four-second prediction protocol fixed and disable image generation for all planner evaluations. The discriminating observation is whether shared joint training improves PDMS and its component scores beyond both controls. If gains disappear under compute matching, or independent visual training produces the same planner improvement, attributing the Table 5 gap specifically to shared-latent coupling would be weakened. e04e05e08e13e14
Check 2: Test whether Chain-of-Forward changes degradation rather than only training cost
Reader-proposed experiment: train paired models with and without the stated three-pass, every-ten-step procedure, and add an untreated baseline with matched total forward-pass compute. Hold DCAE, decoder, ten-frame context, dataset split and diffusion sampling steps fixed. Resolve the paper's noise-time convention before implementation; document whether predicted context is detached. Evaluate identical NuPlan sequences and random seeds at explicitly declared integer horizons, measuring FID and FVD over rollout length, with confidence intervals. A persistently smaller degradation slope would support the intended robustness mechanism. A gap confined to unmatched compute, or only to near-ground-truth training contexts and absent during pure-noise rollout sampling, would challenge that interpretation. e06e07e08e09e15e16e17e21
8.3 Reading coverage
Visual audit: Rendered and visually inspected the title/author page, all method and experiment pages, and all three embedded supplementary pages. This covers Figures 1–10, Tables 1–8, equations, implementation settings, runtime, evaluation horizons and supplementary comparison caveats. All six final original crops were separately opened and inspected at their helper-returned dimensions; narrow crops were rendered directly from the PDF at higher DPI. Figure 8 was re-cropped at 600 DPI and inspected at 1734×1115 pixels. Its first blue annotation overlaps a red marker in the original graphic; the exact starting value is therefore not used. The readable endpoints and increasing separation remain the basis of the retained rollout result. Table 1 retains its caption because the necessary maximum-duration asterisk note is embedded there. Figure 2's action arrow was cross-checked with Sections 3.2–3.3 and Figure 9; Figure 4's feedback paths were checked against Equation (7), with the time-direction ambiguity in Equations (1)–(3) disclosed. The complete text, including references on PDF pages 10–12, was read in all five supplied chunks; those reference-only pages were not visually inspected. No separate video supplement, code or physical experiment was inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 13, 14, 15. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1. Introduction
- 2. Related Work (2.1–2.2)
- 3. Method (3.1–3.5)
- 4. Experiment (4.1–4.4)
- 5. Conclusion
- References
- Supplement A. Detailed Architecture of Dual-Single-Stream DiT
- Supplement B. More Discussions with Related Works
- Supplement C. More Long-term Video Generation Results
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction-only visual gap was addressed by inspecting original PDF pages and all final crops. The supplement embedded in PDF pages 13–15 was read; no separate videos or other supplements were supplied.
- Identity/version: the observed title and twelve authors match the catalog. This report reviews arXiv:2506.24113v1, 30 June 2025 (e01). The catalog identifies ICCV 2025; that venue edition was not supplied, so differences from the venue publication are unverified.
- All five supplied text chunks were read individually without truncation. Code was not inspected and experiments were not reproduced.
- Figure 8's first blue numerical annotation overlaps a red marker in the original PDF, including the higher-resolution crop. Its exact starting value is not used; the readable final values and rollout trend are retained (e15).
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, author/affiliation block and arXiv margin
The title and all twelve authors match the catalog. The artifact identifies arXiv:2506.24113v1, dated 30 June 2025; seven affiliations are printed.
Go to primary source ↓e02PDF pp. 2–4, Introduction, Figure 3 and Section 3.2
The authors motivate continuous frame-wise autoregression as an alternative to fixed-length video diffusion and discrete token prediction; these are their design arguments.
Go to primary source ↓e03PDF p. 4, Section 3.2, action definition and the two distributions under Our approach; p. 3, Figure 2
Historical front-camera observations and relative ego motions condition a future-trajectory policy; the next observation additionally depends on the next action, predicted or externally supplied.
Go to primary source ↓e04PDF p. 5, Section 3.3, Multimodal Spatiotemporal Transformer
Visual latents and three action components are embedded together; interleaved causal temporal and multimodal spatial attention produce the final-frame historical representation F.
Go to primary source ↓e05PDF p. 5, Section 3.3, TrajDiT/VisDiT, Equations (4)–(6)
Distinct diffusion heads generate continuous trajectories and next-frame visual latents, sharing F. VisDiT adds action modulation. Joint training sums trajectory and visual rectified-flow losses; inference may disable either prediction task.
Go to primary source ↓e06PDF p. 5, Figure 4 and Section 3.4; p. 6, Section 3.4, Equation (7)
Chain-of-Forward feeds model estimates into subsequent training passes. A single velocity-based estimate x-hat(0)=x(t)+t v is used instead of a full noise-to-data rollout.
Go to primary source ↓e07PDF p. 6, Section 3.5
DCAE uses 32-fold spatial downsampling, reducing token count sixteenfold relative to an 8-fold encoder. Temporal self-attention is inserted before the decoder; the encoder stays fixed during this fine-tuning.
Go to primary source ↓e08PDF p. 6, Section 4.1, World Model
The approximately 2.5B model comprises 12-layer/1.3B MST, 12-layer/1.2B VisDiT and 2-layer/50M TrajDiT. Training uses NuPlan and 700 NuScenes scenes, 512×1024 images, 48 A100 GPUs, nearly two weeks, 600k iterations, batch 96 and AdamW. Chain-of-Forward runs every ten steps with three passes; experiments use 100 diffusion steps.
Go to primary source ↓e09PDF pp. 6–7, Section 4.1 evaluation paragraphs; p. 7, Section 4.2 quantitative comparison
Video evaluation uses 1,628 NuPlan test clips and 1,646 NuScenes validation clips with ten conditioning frames. FID/FVD measure generated imagery; trajectory metrics are L2 error, collision rate and NAVSIM PDMS. Video baseline values mostly come from their original papers.
Go to primary source ↓e10PDF p. 6, Table 1, FID/FVD and maximum-duration rows, including asterisk note
NuScenes: Epona FID 7.5/FVD 82.8; Vista 6.9/89.4; DrivingWorld 7.4/90.9. Maximum durations are 120s/600 frames, 15s/150 frames and 40s/400 frames respectively. The note defines duration by plausible results.
Go to primary source ↓e11PDF p. 7, Table 2 and caption; p. 6, Section 4.1 sampling-step statement; p. 2, real-time planning claim
On one NVIDIA 4090, a three-second trajectory uses about 0.02s MST plus 0.03s TrajDiT at ten sampling steps; VisDiT takes about 0.3s. At 100 steps TrajDiT takes about 0.3s and VisDiT about 2s.
Go to primary source ↓e12PDF p. 8, Table 3, Ours, Doe-1 and UniAD rows; pp. 7–8, Section 4.3
Epona uses the front camera without listed auxiliary perception supervision: mean L2 1.25m, mean collision 0.36%, one-second collision 0.01%. Doe-1 reports 1.26m/0.53%; UniAD reports 1.03m/0.31% with additional supervision.
Go to primary source ↓e13PDF p. 8, Section 4.3, NAVSIM horizon statement; p. 9, Table 4 and caption
NAVSIM uses two seconds of past observations to predict four-second trajectories. Epona PDMS is 86.2 versus DRAMA 85.5, camera-only LAW/PARA-Drive 84.6 and human 94.8. NC, DAC, TTC, comfort and ego progress are constituent factors.
Go to primary source ↓e14PDF p. 8, Section 4.4, Effect of Shared Latent; p. 9, Table 5
Removing video prediction during training reduces NAVSIM PDMS from 86.2 to 78.1. The full model improves NC 94.5→97.9, DAC 89.7→95.1, TTC 88.1→93.8 and EP 74.7→80.4; comfort remains 99.9.
Go to primary source ↓e15PDF p. 8, Figures 7–8 and Section 4.4, Effect of Chain-of-Forward
NuPlan FID rises with rollout length in both variants. Figure 8 labels the final plotted values 18.54 with Chain-of-Forward and 25.16 without; the final markers lie between the 35 and 40 frame ticks. The first blue annotation overlaps a red marker, so its exact value is not used. Figure 7 gives selected minute-long qualitative comparisons.
Go to primary source ↓e16PDF p. 9, Table 6 and surrounding decoder ablation paragraph
NuPlan temporal-decoder ablation reports FVD at 10/25/40 frames: 50.77/61.46/74.88 with the module versus 52.95/76.46/100.11 without.
Go to primary source ↓e17PDF p. 9, Table 7 and Effect of Different Context Length paragraph
With 2/5/10 conditioning frames, NuPlan FVD40 is 103.70/86.76/74.88. The authors select ten frames as the computational upper limit in their setting.
Go to primary source ↓e18PDF p. 13 (supplement p. 1), Appendix A; p. 14 (supplement p. 2), Figure 9 and caption
Dual-stream blocks initially process noisy data and historical conditions separately, interacting through attention; single-stream blocks concatenate them. Action-control modulation applies only to VisDiT, as the caption clarifies.
Go to primary source ↓e19PDF p. 13 (supplement p. 1), Appendix B and Table 8
Table 8 gives DrivingGPT/Epona PDMS 82.4/86.2. Appendix B lists NuScenes FVD MagicDriveDiT 94.84, InfinityDrive 70.06 and Epona 82.83, yet describes MagicDriveDiT as visually better. It acknowledges possible artifacts from the compressed autoencoder.
Go to primary source ↓e20PDF p. 15 (supplement p. 3), Figure 10; p. 13, Appendix C
Three illustrated rollouts show selected frames through 140 seconds. These are qualitative examples; the text asserts long-term consistency without supplying a corresponding 140-second quantitative benchmark.
Go to primary source ↓e21PDF p. 4, Section 3.1, Equations (1)–(3); p. 6, Equation (7)
The noisy path is (1−t)x(0)+t epsilon, while the regression target is x(0)−epsilon. Equation (7) uses a plus-sign denoising estimate. Equation (1)'s differential notation leaves its time-direction convention insufficiently reconciled with the path derivative.
Go to primary source ↓e22PDF p. 2, abstract continuation, code-availability sentence
The source says code will be publicly available; it does not establish a released or runnable implementation.
Go to primary source ↓8.5 Primary sources
Epona: Autoregressive Diffusion World Model for Autonomous Driving ↗
PDF · 8,526 extracted words
Source fingerprint
2f4348183ec8e8c5c445cd61587ac217589b0bbdbcf9a930b6e088797e0804bb