Awesome-WAMs
GitHub
PAPER REPORTENAll readings ↗

RISE: Adaptive Imagination for World Action Models

English reading report: Method, equations, original figures, experiments and reproducibility.

Authors: Hongbo Lu; Liang Yao; Chenghao He; Hao Han; Fan Liu; Wenlong Liao; Tao He; Pai Peng

Affiliations: COWARobot Co. Ltd; Shanghai Jiao Tong University; Hohai University

Source: 2608.20430 ↗ · Project page ↗ · Catalog record

Reading: 61 / 558 · 6 original figures & tables · ~23 min ·

1. Paper overview

In one sentence: RISE learns when another imagined latent could improve a driving plan enough to justify its cost, trading exhaustive offline supervision for selective inference. e02e03e04e05e08e14e15e22

At a glanceWhat to know
Research problem
Author claim

The authors argue that globally fixed imagination schedules overlook differences in the planning value of additional prediction. Some prefixes already support an adequate trajectory, while others leave interactions unresolved. Factual driving logs also provide only realized outcomes, limiting supervision for plausible alternative hazards. e02

Core mechanism
Source description

A sequential stopping framework distills planning outcomes at every valid rollout depth into prefix-conditioned continuation decisions, with an explicit computation preference. e05e08

A key reported resultNAVSIM v2 planning: 90.8

EPDMS. Reported perception-free benchmark comparison; exact split identifier unspecified.

DriveFuture: 89.9. A 0.9-point improvement under the v2 metric, assessed separately from v1 PDMS. e12

Reading caution
Author claim

The authors limit demonstrated applicability to autonomous driving and identify generation/filtering costs as preventing complete factual–counterfactual coverage. e19

Core contributions

  • Source description

    A sequential stopping framework distills planning outcomes at every valid rollout depth into prefix-conditioned continuation decisions, with an explicit computation preference. e05e08

  • Source description

    CounterDrive combines generated futures, recovered ego motion and human incident annotations. Accepted training/test clips number 2,432/511 for nuScenes and 5,013/1,000 for NAVSIM; pairing covers selected source contexts. e03

  • Reader analysis

    The authors present scheduler transfer to DAWN as evidence of architectural generality. This demonstrates one additional driving architecture, leaving broader applicability open. e16e19

Figure 2. Offline supervision teaches a scheduler that chooses one more prediction or a single final planning call. Original paper, p. 7 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read the right-hand inference loop first: the Scheduler routes Roll to the Predictor and Stop to the Planner. In the center, the Latent Evaluator produces Risk Profile R_h for available prefixes and Future Planning Gain Profile B_h for possible continuations; the Gate emits x_h. Follow the caption, Eq. (9) and Algorithm 1: a positive x_h requests one more latent while depth remains available; x_h ≤ 0 stops. The center’s Planner-side label instead prints “0 ≤” to the left of x_h, conflicting with that rule. The crop faithfully retains this defect. In the training panel, GT and CounterDrive compress different supervision paths: factual planning outcomes train gains and the gate, while verified counterfactual pairs contribute localized risk supervision. e04e05e07e08e09e27

What it supports. The learned decision controls how much latent prediction precedes planning. It is neither a direct prediction of the final horizon nor a repeated online search through all candidate trajectories. Algorithm 1 invokes the final planner after stopping and returns its highest-confidence trajectory; exhaustive depth evaluation belongs to training.

Where the evidence stops. The central Planner branch visually reads “0 ≤ x_h”; the caption (p. 7), Eq. (9) (p. 8) and Algorithm 1 (p. 32) instead specify Stop for x_h ≤ 0. Figure 2 also omits post-stop refinement and conditioning inputs; it alone establishes neither a shared prediction/action backbone nor real vehicle execution.

2. Motivation

2.1 The problem and the proposed response

Author claim

The authors argue that globally fixed imagination schedules overlook differences in the planning value of additional prediction. Some prefixes already support an adequate trajectory, while others leave interactions unresolved. Factual driving logs also provide only realized outcomes, limiting supervision for plausible alternative hazards. e02

2.2 What this reading follows

A driving model can spend more time imagining without producing a better trajectory. RISE turns that observation into a decision made repeatedly: stop with the current prefix, or predict one more latent and reassess. The distinction matters because the useful horizon can change as an interaction becomes visible. Its evaluator learns both risk already exposed and the gains available from deeper prediction, while CounterDrive adds generated alternatives to otherwise factual driving logs. Read the architecture alongside the stopping comparison: the evidence concerns planning quality and computation allocation. Counterfactual hazard recognition and selected images provide additional evidence, with narrower implications for actual driving safety. e02e03e04e05e08e14e15e22

3. Research context

We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.

Catalog dimensionRecorded classification
Major categoryWAMs
ArchitectureNot applicable
Prediction paradigmOther mechanisms
QuadrantOutside quadrants

3.1 Evidence-based assessment

Insufficient evidence to decide

Reader analysis

The scheduler contribution supports 'Other mechanisms' and 'Outside quadrants': it learns stopping over future prediction, rather than jointly emitting futures/actions or explicitly solving inverse dynamics. However, 'Architecture: Not applicable' fits the scheduler only; the instantiated WAM has distinct predictor and diffusion-planner modules, providing no One Model evidence. Driving and inference-efficiency tags are supported. Staged planner post-training is present, but reinforcement learning and demonstrated real-time control are not established. The snapshot's overall interpretation therefore depends on whether it classifies the scheduler or the complete WAM. e04e05e08e09e14e16

This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.

4. Problem formulation

4.1 Inputs and outputs

InputsOutputs
  • Four front-camera frames, sampled at 2 Hz and resized to 256 × 512
  • Observed ego-pose history and current ego kinematics
  • Dataset-specific maximum latent horizon and computation preference λ
  • Training only: factual trajectories, future video, geometry-based risk targets and verified CounterDrive incident pairs
  • One confidence-selected ego trajectory containing eight poses at 0.5-second intervals
  • An adaptively selected latent rollout depth
  • Internal prefix-risk and continuation-gain estimates

4.2 Equations and their role

K(c;λ)=min⁡ ⁣({h∣dh=Stop}∪{H})K(c;\lambda)=\min\!\left(\{h\mid d_h=\mathrm{Stop}\}\cup\{H\}\right)
Equation (4): context c and cost preference λ determine the first stopping depth K through decisions d_h; H caps rollout. K = 0 permits observation-only planning. This imagination horizon differs from the number of planned poses. e05
q=1−Erisk(τ,τ∗),Erisk=4Rcol+Rnear+Rtraj+0.2Rcomf6.2q=1-\mathcal{E}_{\mathrm{risk}}(\tau,\tau^*),\qquad \mathcal{E}_{\mathrm{risk}}=\frac{4R_{\mathrm{col}}+R_{\mathrm{near}}+R_{\mathrm{traj}}+0.2R_{\mathrm{comf}}}{6.2}
Equations (35)–(36): τ is the selected trajectory and τ* its factual target. Collision, near-miss, normalized trajectory-error and comfort risks form a higher-is-worse composite; q reverses it for training. Exact component thresholds are not supplied. e07
gh∗=max⁡j∈{h+1,…,HD}[(qj−qh)−λ(cj−ch)],yh∗=I[gh∗>0]g_h^*=\max_{j\in\{h+1,\ldots,H_D\}}\big[(q_j-q_h)-\lambda(c_j-c_h)\big],\qquad y_h^*=\mathbb{I}[g_h^*>0]
Equations (19)–(20): q_h is the stop-at-h training score, j a continuation depth, H_D the dataset maximum, and c_h cumulative cost. The gate label y_h* requests continuation only for positive best remaining utility; ties stop. The experiments use c_h = h and default λ = 0.005. e08e21

5. Method in detail

5.1 Learn the value of continuing, then decide one step at a time

Reader analysis

During training, the final planner is evaluated at every valid prefix, giving a score curve for each real sample. The gain profile compares every deeper score with the score available now. The gate target then asks whether any continuation has positive gain after subtracting its extra cost. This implies that a poor immediate next step need not force stopping if a deeper continuation is worthwhile. At inference, however, the gate has no access to that complete curve or to ground-truth-dependent scores: it receives only the current prefix and learned predictions. It appends one latent after Roll and asks again. The computation preference therefore changes a sequence of local decisions, while the resulting horizon emerges from them. Figure 3 motivates the problem; Table 5 tests practical stopping strategies. e05e08e09e14e18

Figure 8. A generated alternative future illustrates the visual material used for CounterDrive supervision. Original paper, p. 29 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Follow the sequence from left to right across successive rows. Early frames establish the intersection and road markings; later frames bring a crossing silver car close to the viewpoint. The panel presents a generated future, not a RISE trajectory rollout. Section 3 explains its role: a source key frame and scene/incident prompt condition video generation, recovered ego poses provide motion conditions, and human checks filter distorted or inconsistent samples. Incident-onset annotations then support temporal risk supervision. None of those annotations is overlaid here, so use the images to understand the data modality and evolving interaction, rather than to recover a numerical risk target. e03e07e15e22

What it supports. The sample makes the counterfactual-data idea concrete: training can expose alternatives beyond the single future in a factual log. Appendix B uses verified incident pairs for risk ranking because generated clips lack reliable future-agent geometry. The panel illustrates a possible visual sequence; it does not itself validate the annotations or learned evaluator.

Where the evidence stops. The figure provides no timestamps, onset labels, factual counterpart or calibrated geometry. It cannot establish a collision time, legal responsibility or a successful avoidance action. The held-out discrimination result in Table 6 is separate evidence, not something measurable from this montage.

5.2 Separate hazard supervision from planning-gain supervision

Source description

The evaluator’s two outputs answer different questions. Factual trajectory outcomes supply scalar geometric-risk targets for the initial planner. CounterDrive cannot supply equally reliable future-agent geometry, so verified factual–counterfactual pairs contribute an ordering instead: after the annotated incident appears, the hazardous alternative should rank as riskier; before onset, paired predictions should agree. The learned risk branch then supplies gradients for small, bounded changes to nonempty latent prefixes. A final planner trains on these detached, refined prefixes. Only after that planner is trained are all-depth factual outcomes used to supervise continuation gains. Appendix B defines those scores as one minus a weighted risk composite. They are not the EPDMS column of Table 3, and CounterDrive avoidance recommendations remain metadata rather than direct planner action labels. e03e07e12

5.3 Read ablations at their own configuration and evidence scale

Reader analysis

The main benchmark lead belongs to the complete system, so mechanism assessment requires several narrower comparisons. Table 4 separates CounterDrive from the Scheduler, and Table 8 separates evaluator and gate contributions. Table 10 tests the number of refinement updates: in its reported configuration, two steps reach 90.9 PDMS and 90.2 EPDMS, with no further gain at four or eight. These values should not replace the full system’s 91.5/90.8 headline scores. Table 9 favors coverage of all prefix depths over training only at the extremes, while Table 11 shows that an excessive cost penalty can reduce quality. My reading is that adaptive computation depends on both a planner that tolerates variable context and a useful stopping signal; none of these individual tables demonstrates a deployment deadline or a safety guarantee. e11e12e13e17e23e24e25

5.4 Training and inference

During training

Source description

Stage I trains the predictor on real and accepted CounterDrive futures using LayerNorm-normalized latent L1 targets from the frozen encoder. OpenVO-recovered poses supply generated-video motion conditions. The initial planner uses only factual trajectories with diffusion corruption, position/yaw regression and mode-confidence supervision across prefix lengths. Appendix B specifies uniform sampling; Appendix A differs. e03e06e20

Source description

Stage II fits factual risk targets from the initial planner’s highest-confidence candidate. CounterDrive lacks reliable future-agent geometry, so verified pairs instead impose higher counterfactual risk at/after incident onset and aligned risks before onset. The final planner learns on detached, risk-refined factual prefixes. Its all-depth outcomes supervise gains; Appendix B defines the training score as one minus geometric risk, not NAVSIM EPDMS. e07

Reader analysis

Stage III freezes the other modules and distills whether any deeper prefix offers positive cost-adjusted improvement. Full-horizon enumeration constructs offline labels only. The detailed algorithm is supervised gate training, with no policy-gradient update or reinforcement-learning interaction loop. e08

During inference

Source description

Start with an empty prefix. Below the maximum depth, Roll requires x_h > 0; x_h ≤ 0 stops, as specified in Eq. (9) and Algorithm 1 despite Figure 2’s reversed Planner-side comparator. At each Roll, append one predicted latent using only observed ego-pose history, then reevaluate. Maximum depth is four on NAVSIM and three on nuScenes. No future ground-truth pose enters inference. Stop at zero skips refinement; otherwise use two refinement steps with step size 0.05 and residual norm cap 0.25. The planner runs once afterward, using 20 DPM-Solver++ sampling steps. e04e08e09e21e27

5.5 Implementation flow

  1. Encode and predict

    Four front-camera frames become two observed latent steps. Section 4.2 specifies frozen V-JEPA 2 ViT-L features, with 512 tokens of dimension 1024 per step. A separate 12-layer, width-384, 12-head causal Transformer extends the prefix using ego-motion conditions. Appendix A calls the encoder Drive-JEPA and adds a frozen 128-token Token-AE; the checkpoint/compression interface is not reconciled. e04

  2. Estimate risk and continuation gain

    Spatial pooling, LayerNorm and a causal GRU produce risks for available prefixes and score-gain predictions for valid deeper prefixes. The gate receives pooled latents, padded profiles, normalized depth, cumulative/incremental costs and the computation preference. Its positive output requests one further latent; it does not directly choose a final horizon. e05e08

  3. Refine the stopped prefix

    For a nonempty prefix, gradients through the frozen learned Risk Profile branch reduce accumulated predicted risk, with token-wise bounded residuals. This branch is distinct from the fixed geometry-based evaluator used to construct training labels. The refined prefix is detached before planner training. e07e09

  4. Generate and select a trajectory

    A separate diffusion Transformer conditions on observed/refined future tokens, observed motion history and current ego kinematics. It generates six candidates, each containing eight poses at 0.5-second intervals; highest predicted confidence selects the output. The source specifies a planned trajectory, not a downstream vehicle controller. e04e09e21

6. Experiments & results

RISE decides how long an autonomous-driving world-action model should imagine before producing a trajectory. A learned evaluator predicts prefix risk and continuation gains; a cost-conditioned gate repeatedly chooses Roll or Stop. CounterDrive supplies generated incident alternatives for prediction and localized risk supervision. RISE reports 90.8 EPDMS on NAVSIM v2 and a better score/latency combination than latent-convergence stopping. The evidence supports adaptive computation in the tested driving systems, while implementation conflicts and unverified deployment performance limit reproduction claims.

6.1 Read the original evidence

Table 3. RISE leads the listed NAVSIM v2 aggregate scores, with different strengths across components. Original paper, p. 11 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Start with the final EPDMS column and compare RISE’s bottom row with DriveFuture immediately above it. Those aggregate values support the paper’s principal v2 comparison. Then scan horizontally to see that an aggregate lead does not mean every component is best. For example, RISE’s DAC is 97.7 while DriveFuture’s is 99.1; the numerical cells deserve priority over inconsistent boldface. Section 5.2 calls this a perception-free comparison. Keep this v2 table separate from Table 2’s v1 PDMS and Table 1’s nuScenes displacement/collision metrics: the aggregate names, settings and scales are not interchangeable. e12e13e14

What it supports. RISE reports 90.8 EPDMS against DriveFuture’s 89.9, a 0.9-point lead over the strongest aggregate baseline listed here. This establishes a reported benchmark advantage for the complete system. The component and stopping-policy ablations are needed to investigate which parts contribute to it.

Where the evidence stops. The paper does not provide an exact split identifier or repeated-run uncertainty for this comparison. A benchmark aggregate does not establish an on-road safety guarantee, and this table alone cannot attribute the improvement specifically to stopping decisions.

6.2 Results and evaluation conditions

Task & protocolReported resultComparison & interpretation
nuScenes trajectory planning

Reported perception-free comparison; exact split identifier is not stated in the supplied experimental description.

0.31 m; 0.10

Average L2 error; average collision rate

DAWN: 0.33 m; 0.11.

Reported aggregate improvements. The collision-rate unit is unspecified in Table 1 and is not converted to a percentage here; these are benchmark planning results. e10

NAVSIM v1 planning

Reported perception-free benchmark comparison; exact split identifier unspecified.

91.5

PDMS

DriveFuture: 90.7.

A 0.8-point improvement over the strongest listed aggregate baseline; uncertainty is not reported. e11

NAVSIM v2 planning

Reported perception-free benchmark comparison; exact split identifier unspecified.

90.8

EPDMS

DriveFuture: 89.9.

A 0.9-point improvement under the v2 metric, assessed separately from v1 PDMS. e12

CounterDrive and scheduler ablation

NAVSIM; authors state the same backbone and training settings. Exact split identifier is not supplied.

Both components: 90.8 / 91.5

EPDMS / PDMS

Neither: 88.9 / 89.7; CounterDrive alone: 89.8 / 90.5; scheduler alone: 90.4 / 91.2.

The original table checkmarks and accompanying prose agree on component identities. Results support complementary contributions, but do not isolate every operation within the scheduler. e13

Adaptive stopping strategy

NAVSIM, maximum depth four; timing hardware and measurement protocol unspecified. Exact split identifier is not supplied.

90.8; 2.40; 287.429 ms

EPDMS; mean rollout depth; latency

Latent Margin: 89.7, 2.98 rollouts, 308.532 ms. Random Stop: 89.5, 2.03 rollouts, 264.075 ms.

The learned scheduler improves both score and latency over latent convergence. Random stopping remains faster, so RISE is not the lowest-latency option. e14

Counterfactual hazard discrimination

Held-out CounterDrive test set; identical models with and without CounterDrive training. The dataset-specific test subset is not identified in Table 6.

0.96 / 0.95 / 0.96 / 0.93; 0.96

AUC at 1/2/3/4 seconds; accident-recognition accuracy

Without CounterDrive: 0.49 / 0.52 / 0.51 / 0.50; 0.51.

Large discrimination gains on generated incident examples. These values do not measure accident avoidance by an executed driving policy. e15

Scheduler transfer

DAWN on NAVSIM; authors report leaving its Predictor and Planner unchanged. Exact split identifier is not supplied.

90.3

PDMS

DAWN baseline: 89.1.

A 1.2-point gain supports transfer to this additional architecture, without establishing cross-domain generality. e16

Risk-refinement ablation

NAVSIM configuration reported in Appendix C.4. Exact split identifier is not supplied.

Two refinement steps: 90.9 / 90.2

PDMS / EPDMS

Zero steps: 90.5 / 89.8; four and eight steps retain 90.9 / 90.2.

This configuration supports a small benefit that saturates after two steps. Its scores should not be substituted for the complete system's main results. e17

Evaluator and gate ablation

NAVSIM, Appendix C.2; other components held fixed. Exact split identifier is not supplied.

Both: 90.8 / 91.5

EPDMS / PDMS

Neither: 89.8 / 90.5; evaluator only: 90.3 / 90.9; gate only: 90.6 / 91.2.

Original checkmarks agree with the prose. The gate-only variant’s substitute inputs are not fully specified, limiting mechanistic interpretation. e23

6.3 Ablations and diagnostic examples

Read component removals and qualitative examples within their stated evaluation conditions.

Table 4. Counterfactual supervision and the scheduler each improve the shared baseline. Original paper, p. 11 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Use the checkmarks to identify the experimental change before reading the scores. The first row has neither component. The second enables CounterDrive alone, and the third enables the Scheduler alone. The final row enables both. Read EPDMS and PDMS as separate outcome columns rather than averaging them. The accompanying paragraph states that backbone and training settings are shared. CounterDrive affects future-prediction learning and verified-pair risk supervision; the Scheduler includes an evaluator and a gate. Consequently, this comparison separates two broad system contributions, while Table 8 is the more specific attempt to separate the two scheduler components. e03e07e13e23

What it supports. The baseline scores 88.9 EPDMS and 89.7 PDMS. CounterDrive alone reaches 89.8/90.5, the Scheduler alone 90.4/91.2, and both 90.8/91.5. The combined configuration is strongest in this table, supporting complementary contributions from additional supervision and adaptive imagination within the reported setup.

Where the evidence stops. These broad toggles do not isolate stopping from every evaluator or refinement effect. Table 8 supplies a finer ablation, but the gate-only variant’s replacement inputs are not fully described. No confidence intervals accompany the component differences.

Table 5. Planning-oriented stopping improves on latent convergence, while random stopping remains faster. Original paper, p. 11 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read each row as a joint quality-and-cost outcome. Random Stop assigns a reproducible horizon from zero through four. Latent Margin stops when consecutive latent predictions converge. The learned Scheduler instead conditions stopping on prefix information and predicted planning gains. The middle columns show average rollout count and reported latency; the rightmost column shows planning quality. Comparing only EPDMS hides computation, while comparing only milliseconds favors a weaker planner outcome. The Scheduler lies between the other methods in average rollout count and latency, yet has the highest EPDMS. This is the table’s clearest evidence that the stopping criterion matters beyond simply performing more prediction. e14e08

What it supports. The Scheduler reports 90.8 EPDMS with 2.40 rollouts and 287.429 ms, versus 89.7, 2.98 and 308.532 ms for Latent Margin. Random Stop reports 89.5 with 2.03 rollouts and 264.075 ms. RISE therefore improves both reported quality and latency over latent convergence, but does not minimize latency across these rows.

Where the evidence stops. Timing hardware, batching, warm-up, synchronization and tail latency are not specified. The policies also have different average budgets, so this is not a matched-compute causal test. It does not supply a measured speedup over full-depth rollout.

Figure 3. The best fixed rollout depth varies across retrospectively grouped scenes. Original paper, p. 12 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Rows group samples by the fixed depth that gives their highest planning score; columns apply each depth from zero through four to that group. Read across a row to see how its plans respond to additional imagination. The outlined diagonal cells correspond to each group’s defining best depth. The bars on the right give group sizes, not scheduler prediction counts, and the bottom color scale encodes the score values printed in the cells. In particular, the zero-depth row contains 1,248 scenes and the depth-four row 2,180. This is an outcome-based diagnostic of horizon sensitivity, not a confusion matrix for the learned gate. e18

What it supports. For the group favoring zero rollout, EPDMS changes from 89.9 at depth zero to 88.4 at depth four. For the group favoring four steps, the corresponding values are 88.5 and 91.1. These opposing trends motivate reassessing the value of prediction rather than assuming deeper rollout always helps.

Where the evidence stops. Groups are defined using their best observed outcome, so diagonal superiority is partly built into the grouping. This figure does not independently measure gate accuracy, prove causal effects of traffic complexity, or show that the scheduler identifies each best depth online.

7. Analysis & limitations

7.1 What the evidence leaves open

Author claim

The authors limit demonstrated applicability to autonomous driving and identify generation/filtering costs as preventing complete factual–counterfactual coverage. e19

Reader analysis

Linear rollout cost is a proxy for computation, and the latency comparison lacks timing hardware, batching and tail-latency details. The supplied results therefore do not establish a deployment deadline or real-time vehicle-control capability. e08e14e09e21

Reader analysis

Risk refinement optimizes a learned proxy, while gate targets optimize a hand-weighted risk score rather than the reported benchmark aggregate. Neither construction guarantees safety or optimal EPDMS. Generated-test discrimination also leaves sensitivity to real incidents and generation artifacts unresolved. e03e07e15

Reader analysis

The supplied text reports train/test clip counts but does not document source-scene separation sufficiently to audit counterfactual leakage. Baseline tables provide no repeated-run uncertainty, and the implementation specifies one default seed. e03e10e11e12e21

Source description

Figure 2’s central Planner-side comparator visually reads 0 ≤ x_h, reversing the stopping inequality in its own caption. Equation (9) and Algorithm 1 consistently stop when x_h ≤ 0 and continue only for a positive output below the maximum horizon. The illustration preserves this source defect and explicitly identifies the rule followed in this report. e27

7.2 Questions for discussion

  1. Would gate decisions change materially if targets used native benchmark scores and measured latency instead of the geometric-risk proxy and c_h = h? [e07, e08]
  2. Does CounterDrive improve recognition of independently collected real incidents, or partly recognition of generator-specific artifacts? [e03, e15]
  3. How much of the complete scheduler's gain comes from stopping decisions versus risk-guided latent changes when both receive matched training and inference budgets? [e13, e14, e17]

8. Reproducibility audit

8.1 Requirements and known gaps

Source description

Reproduction needs visual/Token-AE checkpoints, camera and ego-motion data, factual trajectories and geometric-risk inputs, plus accepted CounterDrive clips, source pairing and onset labels. Appendix B reports predictor/initial-planner/final-planner training for 80/20/50 epochs. Appendix A specifies eight A100s for the WAM/planners, effective batch 32, and one A100 for offline evaluator/gate training; seed 239 and fixed validation diffusion noise are documented. e03e04e07e21

Reader analysis

Resolve Appendix A/B conflicts before choosing a configuration: velocity loss versus none; full-prefix probability 0.25 versus uniform sampling; short FP32 evaluator phases versus 50-epoch bfloat16 training; local perturbation ordering versus onset-based pair calibration; and an extra Smooth-L1 gate utility loss versus BCE alone. The exact risk thresholds/normalization and encoder interface remain insufficiently specified. e04e07e20

Reader analysis

Proposed check: freeze predictor, planner, data and refinement, then compare every fixed depth, learned stopping and a matched-budget random policy on identical held-out scenes with fixed diffusion noise. Report native planning metrics, depth distributions and measured mean/tail latency. Separately test onset supervision with source-disjoint factual–counterfactual pairs and an independent real-incident evaluation. e03e13e14e18e21

8.2 Proposed reproduction checks

The following checks are proposals motivated by the paper. They have not been run as part of this reading.

Check 1: Does the gate beat random allocation at the same measured budget?

Reader-proposed check, not an experiment performed here: freeze the predictor, final planner, evaluator, CounterDrive training and refinement rule. Evaluate all fixed depths, the learned gate, and random stopping calibrated to match the gate’s average rollout count on a separate validation set. Use identical held-out scenes and diffusion noise. Measure native EPDMS/PDMS, collision metrics, mean and tail latency, and selected-depth distributions; document hardware and timing boundaries. Repeat with measured latency replacing the proxy cost c_h = h when constructing labels. The planning-gain explanation would weaken if matched-budget random stopping achieved the same quality, or if the gain disappeared under actual timing costs. e08e13e14e18e21

Check 2: Does incident-onset supervision generalize beyond generated appearance?

Reader-proposed check: after resolving the conflicting local-calibration descriptions, hold accepted clips, source-scene splits, predictor and planner fixed. Train evaluator variants with factual regression only, verified-pair ranking alone, and ranking plus pre-onset alignment. Keep every factual source and all its generated variants within one split. Measure horizon-specific AUC, accident-recognition accuracy, and factual–counterfactual risk differences before and after annotated onset. Add independently labeled real incidents as a separate transfer test. A benefit confined to generated test clips, or persistent excess risk before onset, would weaken the claim that localized supervision teaches hazard timing rather than generator-specific cues. Report these as proposed controls, not reproduced results. e03e07e15e20

8.3 Reading coverage

Visual audit: The title/author page, every method and experiment page, implementation and training appendices, all original figures and tables, both CounterDrive sample pages, prompt tables and inference/training algorithms were rendered and visually inspected. All six final crops were separately inspected for legibility, boundaries and retained labels. Evidence pages include uncropped supporting pages for metrics, hardware, hyperparameters, inconsistencies and proposed checks. Contents and reference pages were read in the complete supplied text; external code, datasets, videos and supplements were not inspected. No experiments were reproduced. Figure 2’s original Planner-side comparator was checked against its caption on page 7, Eq. (9) on page 8 and Algorithm 1 on page 32; the discrepancy is disclosed in the architecture reading guide and caution.

PDF pages inspected for this edition: 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33. Appendix coverage: reviewed.

Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.

Text reading scope & known omissions
  • Title page and Abstract (p. 1); Contents (pp. 2–3)
  • 1 Introduction (pp. 4–5); 2 Related Works, including 2.1–2.2 (pp. 5–6)
  • 3 CounterDrive, including 3.1–3.3 (p. 6)
  • 4 RISE, including 4.1–4.4.3 (pp. 6–10)
  • 5 Experiments, including 5.1–5.4.4 (pp. 10–13)
  • 6 Conclusion; 7 Limitations; 8 Acknowledgments (pp. 13–14)
  • References (pp. 15–17)
  • Appendix A Implementation Details (pp. 18–19)
  • Appendix B More Training Details, including B.1–B.3 and all equations (pp. 19–24)
  • Appendix C More Experimental Results, including C.1–C.5 (pp. 25–27)
  • Appendix D Samples of CounterDrive (pp. 28–29), including original images
  • Appendix E Prompts for CounterDrive Generation, including E.1–E.2 (pp. 30–31)
  • Appendix F Algorithms of RISE, including F.1–F.2 (pp. 32–33)

Outside the original text pass

  • Text extraction does not reconstruct figure pixels or preserve equation/table layout; downstream reading must inspect the retained original PDF.
  • The CounterDrive example images on PDF pages 28–29 contain spatial and trajectory information not reconstructed by plain text.
  • Availability of external supplements, code, datasets and videos beyond the retained PDF has not been fully verified.
  • The complete supplied text was read. Original PDF pages 1, 4–14 and 18–33 were visually inspected, resolving the extraction-only figure and layout gaps, including both CounterDrive sample pages. Contents and reference pages were read as text only.
  • Identity/version note: the observed title and all eight author identities match the catalog. The PDF bears arXiv:2608.20430v1, stamped 20 August 2026, while its title-page Date is August 24, 2026. The catalog submittedDate is August 19, 2026. These labels are not reconciled by the supplied material; this report concerns the supplied v1 artifact only. No different revision was examined. [e01]
  • No separate supplement was supplied. External code, dataset, project pages and videos were not inspected; availability and installation remain unverified. Referenced works were not independently read. No experiments were reproduced.

The visual audit above records the subsequent illustrated pass.

8.4 Traceable evidence

e01PDF p. 1, title page, author list and arXiv stampInspect

The observed title is RISE: Adaptive Imagination for World Action Models. Credits are Hongbo Lu, Liang Yao, Chenghao He, Hao Han, Fan Liu, Wenlong Liao, Tao He and Pai Peng, matching the catalog identities. Affiliations are COWARobot Co. Ltd; Shanghai Jiao Tong University; Hohai University. The arXiv stamp is 2608.20430v1, 20 August 2026; the separate title-page Date is August 24, 2026.

Go to primary source ↓
e02PDF pp. 4–5, Section 1 IntroductionInspect

The authors motivate repeated prefix-local stopping decisions through scene-dependent planning gains and distinguish missing counterfactual risk supervision from all-horizon planning-gain supervision.

Go to primary source ↓
e03PDF p. 6, Sections 3.1–3.3 and Eqs. (1)–(2); pp. 30–31, Appendix E, Tables 12–13Inspect

Wan 2.7 produces key-frame-conditioned 10-second, 1080p counterfactual videos sampled into 20 frames at 2 Hz. OpenVO supplies ego poses. Human filtering checks motion and distortion and labels incident onset and cause. Accepted nuScenes train/test counts are 2,432/511 and NAVSIM counts are 5,013/1,000; avoidance recommendations remain clip metadata.

Go to primary source ↓
e04PDF p. 7, Figure 2 and Sections 4.2.1–4.2.3; p. 18, Appendix A, World Action Model and Dynamic planners; pp. 19–20, Appendix B.1Inspect

Figure 2 distinguishes prediction/planning supervision and scheduler training from the inference routing loop. Section 4.2 specifies frozen V-JEPA 2 ViT-L features, two observed steps with 512 tokens of dimension 1024, and separate 12-layer, width-384, 12-head predictor and diffusion planner. Appendix A names Drive-JEPA and a frozen Token-AE with 128 tokens; B.1 limits inference motion conditioning to observed pose history.

Go to primary source ↓
e05PDF pp. 7–8, Sections 4.1 and 4.3, Eqs. (4)–(9); p. 21, Appendix B.2, Latent EvaluatorInspect

The effective horizon is the first stop depth, capped by the maximum. Risk profiles cover observed prefix depths; gain profiles cover valid continuations. The gate conditions on pooled latents, profiles, depth and costs, requesting one additional latent when its output is positive.

Go to primary source ↓
e06PDF pp. 19–21, Appendix B.1, Eqs. (22)–(34)Inspect

Predictor training uses normalized latent L1 loss on real and accepted generated sequences. Initial planner training uses factual trajectories, diffusion corruption, position/yaw losses and mode-confidence supervision across sampled prefix depths. The detailed objective explicitly excludes velocity loss.

Go to primary source ↓
e07PDF pp. 21–23, Appendix B.2, Eqs. (35)–(50)Inspect

Highest-confidence candidates receive normalized geometric risk with weights 4, 1, 1 and 0.2, divided by 6.2; planning score is its complement. Generated clips lack reliable future-agent geometry and instead supervise onset-masked ranking and pre-onset alignment. Frozen-risk gradients refine bounded latent residuals; detached refined prefixes train the final planner, whose all-depth outcomes supervise gains.

Go to primary source ↓
e08PDF p. 10, Eqs. (19)–(20); pp. 23–24, Appendix B.3, Eqs. (51)–(57); p. 33, Algorithm 2, Stage IIIInspect

Gate labels use the maximum future score improvement minus incremental cost; ties stop. The detailed objective is BCE with other modules frozen. Costs are c_h = h and λ is chosen from 0, 0.001, 0.005, 0.01 and 0.05, defaulting to 0.005. Exhaustive evaluation constructs training targets only.

Go to primary source ↓
e09PDF p. 32, Appendix F.1, Algorithm 1, lines 1–23; p. 24, Appendix B.3, HyperparametersInspect

Inference repeatedly evaluates the current prefix and appends one latent using observed motion history. After stopping, nonempty prefixes receive risk refinement and the planner is invoked once. Highest predicted confidence selects the returned trajectory. Default refinement uses two iterations, step size 0.05 and radius 0.25.

Go to primary source ↓
e10PDF p. 10, Table 1, DAWN and RISE rows, average L2 and collision columns; p. 11, Section 5.2Inspect

RISE reports average L2 0.31 m and collision rate 0.10; DAWN reports 0.33 m and 0.11. The table does not label collision rate as a percentage. Section 5.2 describes the comparison as perception-free. The cited table cells were also checked in the original PDF layout.

Go to primary source ↓
e11PDF p. 10, Table 2, DriveFuture and RISE rows, PDMS column; p. 11, Section 5.2Inspect

NAVSIM v1 aggregate PDMS is 91.5 for RISE and 90.7 for DriveFuture; the text states the resulting 0.8-point lead. The cited table cells were also checked in the original PDF layout.

Go to primary source ↓
e12PDF p. 11, Table 3, DriveFuture and RISE rows, EPDMS column; Section 5.2Inspect

Table 3 reports NAVSIM v2 EPDMS 90.8 for RISE and 89.9 for DriveFuture, a 0.9-point lead. The DAC cells are 97.7 and 99.1 respectively, despite boldface on RISE’s lower value. Section 5.2 specifies the perception-free comparison; no exact split identifier or repeated-run uncertainty accompanies the table.

Go to primary source ↓
e13PDF p. 11, Section 5.3.1 and Table 4, visually verified component checkmarks and score columnsInspect

The prose explicitly assigns EPDMS/PDMS 88.9/89.7 to the baseline, 89.8/90.5 to CounterDrive alone, 90.4/91.2 to scheduler alone and 90.8/91.5 to both, and states that backbone and training settings are held constant. The cited table cells were also checked in the original PDF layout.

Go to primary source ↓
e14PDF p. 11, Section 5.3.2 and Table 5Inspect

Table 5 gives Random Stop 2.03 average rollouts, 264.075 ms and 89.5 EPDMS; Latent Margin 2.98, 308.532 ms and 89.7; Scheduler 2.40, 287.429 ms and 90.8. Random Stop selects a reproducible depth in 0–4; Latent Margin tests consecutive-latent convergence. Timing hardware and measurement protocol are not specified.

Go to primary source ↓
e15PDF p. 12, Section 5.4.2 and Table 6Inspect

On the held-out counterfactual test set, CounterDrive training changes 1/2/3/4-second AUC from 0.49/0.52/0.51/0.50 to 0.96/0.95/0.96/0.93, and accident-recognition accuracy from 0.51 to 0.96. The cited table cells were also checked in the original PDF layout.

Go to primary source ↓
e16PDF p. 13, Section 5.4.3 and Table 7, PDMS columnInspect

The authors report integrating the scheduler into DAWN without modifying its Predictor or Planner; PDMS increases from 89.1 to 90.3.

Go to primary source ↓
e17PDF p. 26, Appendix C.4, Table 10 and accompanying paragraphInspect

The reported refinement configuration scores PDMS/EPDMS 90.5/89.8 with zero steps and 90.9/90.2 with two steps; four and eight steps provide no further improvement.

Go to primary source ↓
e18PDF p. 12, Section 5.4.1 and Figure 3, heatmap rows 0–4, columns 0–4 and scene-count bars; Figure 4Inspect

Samples are grouped retrospectively by their best fixed depth. Scene counts for groups 0–4 are 1,248, 2,775, 1,907, 4,036 and 2,180. Group 0 scores 89.9 EPDMS at depth 0 and 88.4 at depth 4; group 4 scores 88.5 and 91.1 respectively. The heatmap is not a gate-prediction confusion matrix. Figure 4 supplies representative scene images.

Go to primary source ↓
e19PDF pp. 13–14, Section 7 LimitationsInspect

The authors identify driving-only experiments and incomplete CounterDrive coverage due to generation and filtering cost as limitations.

Go to primary source ↓
e20PDF pp. 18–19, Appendix A, Dynamic planners, Evaluator and Rollout Gate; pp. 20–24, Appendix B, Eqs. (33)–(34), (47), (56) and Hyperparameters; p. 33, Algorithm 2Inspect

Appendix A lists a velocity-loss weight, full-prefix probability 0.25, short FP32 evaluator phases, Gaussian-perturbation local ordering, and a Smooth-L1 gate utility loss weighted 0.5. Appendix B excludes velocity loss, gives uniform prefix sampling, defines local calibration as pre-onset pair alignment, writes a BCE-only gate objective, and specifies 50-epoch bfloat16 evaluator training. These are unreconciled source descriptions.

Go to primary source ↓
e21PDF p. 10, Section 5.1; pp. 18–19, Appendix A, Compute and data, Dynamic planners and optimization; p. 24, Appendix B.3, HyperparametersInspect

Main models use eight A100 GPUs, bfloat16 and per-GPU batch size four; Appendix A separately places offline evaluator/gate training on one A100 in FP32. Predictor and planner epoch counts are 80, 20 and 50. The source specifies fixed validation diffusion noise, default seed 239, six candidate modes, 20 sampling steps and dataset horizons four and three.

Go to primary source ↓
e22PDF pp. 28-29, Appendix D, Figures 7-8 (visually inspected)Inspect

Figures 7 and 8 each show twelve CounterDrive frames. Sample 1 progresses toward roadside vegetation. Sample 2 shows an intersection and a silver car crossing close to the camera. Neither panel supplies timestamps, incident-onset labels, recovered poses or factual-pair images; visual plausibility alone does not establish annotation correctness.

Go to primary source ↓
e23PDF p. 26, Appendix C.2, Table 8 and accompanying paragraphInspect

The original Gate-column-only checkmark corresponds to 90.6 EPDMS / 91.2 PDMS; Evaluator-only corresponds to 90.3 / 90.9. Neither gives 89.8 / 90.5 and both 90.8 / 91.5. The text states other components are fixed but does not fully specify replacement gate inputs when the evaluator is absent.

Go to primary source ↓
e24PDF p. 26, Appendix C.3, Table 9, all rowsInspect

Uniform depth sampling [0.20,0.20,0.20,0.20,0.20] gives 89.8 EPDMS; extremes-only gives 87.4, short-heavy 89.6 and no-full 89.2. These are a separate training-distribution ablation, not the full model’s headline scores.

Go to primary source ↓
e25PDF p. 27, Appendix C.5, Table 11, all rowsInspect

EPDMS is 90.0, 90.7, 90.8, 88.6 and 88.4 for λ = 0, 0.001, 0.005, 0.01 and 0.05 respectively. The table does not give corresponding rollout counts or latency.

Go to primary source ↓
e26PDF pp. 13 and 25, Figures 5–6 and Sections 5.4.4 / C.1Inspect

Figure 5 overlays human, Drive-JEPA and RISE trajectories on camera/BEV examples. Figure 6 shows near-overlapping fixed-depth predictions on simple lane-following examples and different trajectories on turning/intersection examples. These are selected qualitative examples, not executed-vehicle success measurements.

Go to primary source ↓
e27PDF p. 7, Figure 2, central Planner-side comparator and caption; p. 8, Eq. (9); p. 32, Algorithm 1, lines 4–14Inspect

The central Planner branch in Figure 2 prints “0 ≤” immediately to the left of x_h, visually reading 0 ≤ x_h. This conflicts with the same figure’s caption, which sends x_h ≤ 0 to planning, and Eq. (9), which chooses Roll only for x_h > 0 below the maximum horizon. Algorithm 1 explicitly breaks at x_h ≤ 0 before appending a latent. The original crop preserves the comparator inconsistency; the reported stopping rule follows the caption, equation and algorithm.

Go to primary source ↓

8.5 Primary sources

Scroll across the image to inspect details. Press Esc to close.