Percept-WAM: Perception-Enhanced World-Awareness-Action Model for Robust End-to-End Autonomous Driving
1. Paper overview
In one sentence: A shared VLM learns spatial perception tokens and trajectory queries, improving reported driving benchmarks while leaving important decoder, cache and training details in unsupplied appendices. e-identitye-architecturee-confidencee-actione-plan-resultse-conf-ablatione-speede-missing
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors argue that language-based spatial question answering gives insufficient geometric supervision: localization errors and overconfident detections can compromise driving decisions. They propose reusable spatial features inside a VLM so that trajectory prediction can benefit from explicit perception learning. e-probleme-architecture |
| Core mechanism | Source description World-PV and World-BEV place 2D/3D detection and segmentation within a shared VLM, with optional LiDAR initialization for BEV features. e-architecturee-pve-bev |
| A key reported result | NAVSIM v1 trajectory planning: 90.2, reported PDMS (higher is better). Table 3 benchmark comparison; starred model uses two-stage training and NAVSIM-specific trajectory scoring. Exact evaluation partition is not named in the supplied body. 88.6 unstarred; 88.1 DiffusionDrive; differences of 1.6 and 2.1 score points. The paper describes NAVSIM as data-driven pseudo-simulation. The result supports benchmark planning, not demonstrated on-road execution. The abstract's PMDS spelling differs from the table's PDMS. e-plan-resultse-probleme-architecture |
| Reading caution | Reader analysis Qualitative examples support crowded-scene, bird and shopping-cart detection, but provide no quantified long-tail guarantee. Reasoning preservation is asserted without a dedicated reasoning evaluation in the supplied results. e-architecturee-qualitativee-pv-resultse-plan-results |
Core contributions
- Source description
World-PV and World-BEV place 2D/3D detection and segmentation within a shared VLM, with optional LiDAR initialization for BEV features. e-architecturee-pve-bev
- Source description
Grid-conditioned parallel object decoding, localization-sensitive confidence training, and modality-specific trajectory queries address different perception and planning bottlenecks. e-confidencee-beve-action
Figure 2. Spatial perception and action decoding share the VLM's features. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read upward from the inputs. Camera frames pass through the image encoder into World-PV features; optional LiDAR provides a separate route into World-BEV features. The caption and Section 3.2 clarify that BEV queries can also learn from PV features without LiDAR. The center separates prefill from autoregressive language decoding and the parallel action head. At the top, perception tasks and waypoint prediction use different output interfaces. The two arrows beside the KV cache indicate streaming reuse around prefill, but do not specify an eviction or recomputation schedule. Treat the language examples as illustrations of an interface, rather than measured reasoning performance. e-architecturee-beve-actione-streame-missing
What it supports. The figure supports the shared-backbone interpretation of One Model: perception features and action queries are handled within the VLM architecture, even though modality encoders and an action head remain distinct modules. The central spatial representation is learned for scene understanding; the illustrated branches do not implement an action-conditioned future-world rollout.
Where the evidence stops. The diagram does not give a complete attention mask or prove that language reasoning improves planning. Sections 3.2–3.3 supply the verifiable feature interactions; detailed masks and streaming implementation are deferred to appendices absent from this artifact.
2. Motivation
2.1 The problem and the proposed response
The authors argue that language-based spatial question answering gives insufficient geometric supervision: localization errors and overconfident detections can compromise driving decisions. They propose reusable spatial features inside a VLM so that trajectory prediction can benefit from explicit perception learning. e-probleme-architecture
2.2 What this reading follows
A driving model needs geometry it can use: where objects are, which regions are traversable, and how much confidence to place in a localization. Percept-WAM brings those targets into an InternVL2-8B backbone through World-PV and World-BEV tokens, then learns action queries for trajectory prediction. This reading follows three connections: localized features to structured perception, predicted box quality to confidence, and modality-specific features to an all-feature planner. The original tables show gains from realistic confidence examples and two-stage training, alongside metric-specific weaknesses and a latency–accuracy tradeoff. The supplied CVPR Open Access paper supports benchmark conclusions, with cited implementation appendices absent. e-identitye-architecturee-confidencee-actione-plan-resultse-conf-ablatione-speede-missing
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | VLA |
| Architecture | One Model |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The VLA and spatial-perception categories fit. Figure 2 and Sections 3.2–3.3 support One Model through a shared VLM carrying perception and action tokens, alongside modality encoders and MLP heads. The method predicts current scene structure and future ego trajectories; it does not describe action-conditioned future-world generation or inverse-dynamics action extraction, supporting the recorded not-applicable prediction quadrant. e-architecturee-beve-action
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 1. Turn image features into reusable spatial evidence
The method begins by making perception an explicit learning target inside the VLM. World-PV features support localized grid queries, which generate object attributes instead of answering only descriptive questions about a scene. A separate ego-centered World-BEV grid queries those image features through cross-attention; optional LiDAR features provide a metric initialization. Detection and segmentation then supervise complementary information: boxes locate objects, while masks describe occupied semantic regions. The segmentation interface retrieves masks from feature similarities using 16 mask tokens, with independent binary classes for overlapping BEV map regions. The source places these representations upstream of trajectory queries. This information flow explains how spatial supervision can affect the planner without requiring a generated future video or an explicit rollout of world dynamics. e-pve-beve-action
Figure 3. Confidence learning separates accurate box supervision from localization-quality supervision. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow panel (a) from the ground-truth dataset toward the two example-generation branches. Random perturbations yield the schematic uniform distribution; actual model predictions provide the proposed training distribution. Section 3.1.2 says these predictions come from an intermediate checkpoint on training images. In panel (b), white cells marked Attended identify supervised tokens, while gray Masked cells are ignored by the loss. Ground-truth examples supervise category and box fields, and confidence examples supervise the confidence field. The caption makes clear that this is loss masking: it should not be confused with the modality-attention restrictions used for action queries in Figure 5. e-confidencee-conf-ablation
What it supports. The scheme avoids teaching confidence exclusively from perfect ground-truth boxes. It instead conditions confidence prediction on the category and geometry of imperfect model outputs. During inference, the class score is multiplied by predicted IoU, giving the detector a localization-sensitive ranking signal whose empirical effect is tested in Table 5.
Where the evidence stops. The graphic labels bins 0, 1, 2, …, 20, whereas Section 3.1.2 specifies 20 bins. The endpoint/index convention is unresolved. The illustrated histograms and example detections are explanatory, not a numerical calibration evaluation.
5.2 2. Separate recognizing a class from trusting its box
A confident class token need not imply accurate geometry. Percept-WAM therefore trains an additional IoU prediction conditioned on a box's class and coordinates. Ground-truth examples teach the correct category and geometry, but their confidence fields are ignored by the loss; otherwise the training signal would repeatedly favor perfect scores. Prediction-derived examples teach only confidence, exposing that objective to the model's own imperfect boxes. At inference the two scores are multiplied. Table 5 makes the data choice consequential: random perturbations and uniform sampling of model predictions both reduce AP, whereas keeping the realistic prediction distribution improves AP and AP75. My interpretation is that matching the errors encountered at inference is the main lesson, but the supplied table does not isolate all sampling and optimization details. e-confidencee-conf-ablation
Figure 5. Four supervised trajectory branches train one all-feature inference output. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the square feature groups along the bottom: ego state, World-PV and World-BEV. The circles are the four point-query sets, labeled Q_ego, Q_pv, Q_bev and Q_full. Arrows pass upward through Percept-WAM to MLP trajectory decoders. The diagram does not draw every allowed cross-token edge, so use its caption and Section 3.3: the first three query sets access only their corresponding modalities, while Q_full accesses all features. Each set contains N point queries of feature dimension C. All four trajectories receive Smooth-L1 supervision during training; the continuation on page 6 selects Q_full at inference. e-actione-plan-resultse-bev-ablatione-speed
What it supports. The auxiliary branches make each modality responsible for predicting trajectory structure during training, while the deployed query head combines the modalities. That is the authors' alignment mechanism. My interpretation is that the branches could discourage an easy single-modality shortcut, but the figure alone does not establish that causal effect.
Where the evidence stops. The supplied ablations do not isolate removal of these three auxiliary trajectory losses. Also, NAVSIM uses static-vocabulary trajectory scoring with distinct decoder settings; this diagram alone cannot specify the complete system behind the 90.2 PDMS result.
5.3 3. Distinguish training branches, inference heads and benchmark protocols
During training, four query groups turn ego state, PV features, BEV features and their combination into separate trajectory predictions. Their Smooth-L1 objectives ask each restricted branch to carry useful trajectory information; only the all-feature branch supplies the final query-based output at inference. That training design is separate from the choice to reuse a streaming KV cache. It is also separate from NAVSIM's static-vocabulary scoring and selection, which the paper describes alongside distinct benchmark decoder settings. Consequently, read the evidence in three layers: Table 3 compares planning performance and the two-stage curriculum, Table 7 compares trajectory decoders and latency, and Figure 5 explains the query-head mechanism. The supplied paper does not establish that every best number comes from one identical inference configuration. e-actione-streame-plan-resultse-speed
5.4 Training and inference
During training
The curriculum first consolidates PV/BEV perception, then performs end-to-end VLA fine-tuning. Detection uses token cross-entropy; segmentation combines cross-entropy, sigmoid focal and Dice losses. AdamW uses learning rate 0.0002, weight decay 0.01, cosine decay and 1000 warmup steps, with mixed precision and gradient checkpointing. e-pve-beve-training
Table 2 combines driving datasets, general detection/segmentation and grounding datasets, plus driving QA. PV detection/segmentation uses a 10×10 grid; BEV detection uses 40×40 and segmentation 10×10. Further data composition and hyperparameters are deferred to missing Appendix B. e-training
During inference
World features can be computed during prefill; spatial proposals decode in parallel across grids while retaining autoregression within each proposal. Streaming uses a KV cache, longer-clip training and a named dual-recomputation mechanism, whose implementation is deferred to absent Appendix A.2. e-beve-stream
Keep benchmark decoders separate: the query-based trajectory head is described in Section 3.3, whereas NAVSIM additionally scores and selects trajectories from a static vocabulary. Table 3 explicitly refers to distinct decoder settings in missing Appendix A.3. e-actione-plan-results
5.5 Implementation flow
- Encode and localize image evidence
InternVL2-8B provides the backbone. Dynamic tiling encodes high-resolution image tiles with shared ViT weights and aligns their features globally. Interpolated grid queries from World-PV features predict local objects. Boxes are serialized as category, geometry and confidence tokens, with continuous labels discretized for token prediction. e-architecturee-pv
- Learn localization-sensitive scores
An intermediate model predicts boxes on training images; matched boxes receive IoU targets. Mix these with ground-truth examples: ground-truth samples train category/box tokens while masking confidence loss; prediction-derived samples train confidence while masking category/box loss. Inference multiplies class-token softmax confidence by predicted IoU. Figure 3's loss mask agrees with this text. e-confidence
- Lift into BEV and retrieve masks
Ego-centered learnable BEV grid tokens query World-PV features by cross-attention. Camera-only embeddings are learned from random initialization; optional PointPillars features pass through PixelUnshuffle and an MLP to initialize BEV tokens. Bilinearly sampled grid queries decode objects independently. PV/BEV segmentation retrieves masks through dot products with 16 mask tokens; BEV map classes use independent binary masks because regions can overlap. e-pve-bev
- Align action queries with modalities
Four point-query sets attend to ego state, PV, BEV, or all features, respectively. An MLP decodes each set into a trajectory. Training supervises all four with Smooth-L1; inference selects the all-feature query output. The output is a predicted waypoint trajectory; the supplied method does not specify a vehicle actuator controller. e-action
6. Experiments & results
Percept-WAM strengthens an InternVL2-8B driving model with supervised image-plane and bird's-eye-view perception tokens, then predicts trajectories through action queries. Its strongest planning row reaches 90.2 NAVSIM PDMS; its central evidence concerns perception and trajectory benchmarks, rather than learned future-world rollouts or physical driving execution.
6.1 Read the original evidence
Table 3. Two-stage training improves planning averages without leading every metric. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Keep the two benchmark blocks separate. The left block reports nuScenes trajectory L2 error at increasing horizons and its average, with downward arrows. The right block reports NAVSIM v1 scores, with upward arrows; use PDMS in the last column for the aggregate comparison. The star on the final row denotes two-stage training, as defined in the original caption. Compare the last two rows to read the curriculum effect, then compare with named baselines. Missing entries are dashes, not zeros. The source uses different decoder settings across benchmarks and points to Appendix A.3, which is not supplied. e-plan-resultse-architecturee-missing
What it supports. Percept-WAM* reaches 90.2 PDMS versus 88.6 without the star and 88.1 for DiffusionDrive: gains of 1.6 and 2.1 score points. Its nuScenes average improves from 0.38 to 0.36 m, while BEV-Planner remains lower at 0.35 m. Improvement in the overall score should not be read as leadership in every component.
Where the evidence stops. The paper calls nuScenes open-loop evaluation and describes NAVSIM as data-driven pseudo-simulation, although its results prose also says closed-loop metrics. Neither column demonstrates physical execution. Exact evaluation partitions, uncertainty and full decoder configurations are not recoverable from this table and supplied body.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| NAVSIM v1 trajectory planning Table 3 benchmark comparison; starred model uses two-stage training and NAVSIM-specific trajectory scoring. Exact evaluation partition is not named in the supplied body. | 90.2, reported PDMS (higher is better) | 88.6 unstarred; 88.1 DiffusionDrive; differences of 1.6 and 2.1 score points. The paper describes NAVSIM as data-driven pseudo-simulation. The result supports benchmark planning, not demonstrated on-road execution. The abstract's PMDS spelling differs from the table's PDMS. e-plan-resultse-probleme-architecture |
| nuScenes open-loop trajectory planning Table 3; L2 at 1, 2 and 3 seconds; exact evaluation partition not named there. | 0.16 / 0.33 / 0.60; reported average 0.36 L2 distance in meters (lower is better) | Unstarred average 0.38; UniAD 0.46; BEV-Planner 0.35. Two-stage training improves the reported average but does not beat every baseline. Open-loop trajectory agreement does not measure interactive driving success. e-plan-results |
| COCO 2D object detection Table 1 final AD+Gen model; official split policy in Table 2, but COCO evaluation partition not identified explicitly. | 51.7, reported mAP (higher is better) | LMM-Det 47.5; UFO-InternVL2-8B 48.9. A broad generalist result with different backbones and training mixtures across baselines, not a controlled architecture-only comparison. e-pv-resultse-training |
| nuScenes BEV perception Table 4, nuScenes val; no sequential information and 448×796 image input. Table does not annotate each row's sensor configuration. | 0.589 / 0.645; drivable area 87.0, pedestrian crossing 70.9, lane divider 62.7, vehicle 60.2 Detection mAP/NDS; map IoU | PointPillars detection 0.523 / 0.613; BEVFusion 0.685 / 0.714 and map IoU 85.5 / 60.5 / 67.7 for the first three classes. Percept-WAM leads selected map categories, while BEVFusion remains stronger in detection and lane-divider IoU. Avoid labeling this row camera-only. e-bev-results |
| IoU-confidence data ablation Table 5, nuImages val 2D detection. | Real model-prediction distribution: 49.6 / 70.4 / 53.7 AP / AP50 / AP75 | Class-only baseline 48.1 / 70.9 / 51.4; random perturbations AP 46.9; uniform model-prediction sampling AP 46.2. Realistic sampling improves AP by 1.5 points and AP75 by 2.3, while AP50 decreases by 0.5. Adding an IoU head alone is insufficient. e-conf-ablation |
| Streaming trajectory decoding Table 7, nuScenes val; timing hardware and detailed cache settings absent. | Query decoder: 0.3822 / 1174; with streaming: 0.3839 / 707 Average L2 / latency in milliseconds (both lower is better) | Direct AR: 0.3970 / 2700; clustered AR: 0.3919 / 1470. Streaming saves 467 ms, approximately 39.8% calculated from the table, with a small L2 increase. This timing result is separate from Table 3's best planning row. e-speede-stream |
| BEV component ablation Table 6, nuScenes val, cumulative modifications. | Camera baseline 25.0 / 25.7; LiDAR 33.2 / 32.2; augmentation 41.3 / 39.2; denser grids 50.4 / 46.6; MLP decoding 50.4 / 43.7. mAP / NDS | The final change reports 16× speedup while maintaining mAP. NDS decreases with MLP decoding. The cumulative setup does not isolate interactions, and its 50.4 mAP is distinct from Table 4's 58.9-equivalent result. e-bev-ablation |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 5. The confidence-example distribution determines whether IoU scoring helps. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the first row as the class-confidence baseline. Each following row adds IoU-based confidence but changes how confidence-training examples are distributed: randomly perturbed ground-truth boxes, uniformly sampled model predictions, or model predictions retaining their realistic distribution. Read across AP, AP50 and AP75 rather than treating the bold AP entry as the whole result. The caption fixes the task to 2D detection on nuImages val. Section 4.3 explains that uniform model-prediction sampling balances samples across IoU levels; this differs from simply taking model predictions. Figure 3 provides the shared loss-mask mechanism behind these comparisons. e-conf-ablatione-confidencee-training
What it supports. The realistic model-prediction row raises AP from 48.1 to 49.6 and AP75 from 51.4 to 53.7, but lowers AP50 from 70.9 to 70.4. Both alternative confidence datasets reduce AP below baseline. The evidence therefore favors the specific data construction, rather than an unconditional benefit from adding a confidence token.
Where the evidence stops. The table reports detection metrics, not a planner ablation or a safety guarantee. Sample counts, repeated-run variability and detailed confidence-data mixing are not reported here, limiting how precisely the source isolates distribution matching from other training differences.
Table 7. Parallel queries and streaming reduce latency with a measurable accuracy tradeoff. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each indented Streaming Infer. row as a modification of the decoder immediately above it. Direct AR generates trajectory text autoregressively. AR(cluster) instead uses clustered trajectory segments as action tokens, according to Section 4.3. Query-base refers to the point-query approach described in Section 3.3. Both columns have downward arrows: lower trajectory error and lower latency are preferable. To isolate the reported cache effect for the query head, compare only the last two rows. The 16× speedup in the separate BEV detection ablation belongs to Table 6 and must not be attached to this trajectory table. e-speede-streame-actione-bev-ablatione-plan-resultse-missing
What it supports. Query-based streaming reduces latency from 1174 to 707 ms, a calculated saving of 467 ms or about 39.8%, while average L2 rises from 0.3822 to 0.3839. Direct AR remains slower at 2700 ms without streaming. These results support a useful benchmark tradeoff, rather than an accuracy-neutral or hardware-independent speed claim.
Where the evidence stops. Timing hardware, batch conditions and the detailed dual-recomputation policy are absent from the supplied body. Table 7 also does not establish that the 707 ms configuration achieves Table 3's 90.2 NAVSIM PDMS.
7. Analysis & limitations
7.1 What the evidence leaves open
Qualitative examples support crowded-scene, bird and shopping-cart detection, but provide no quantified long-tail guarantee. Reasoning preservation is asserted without a dedicated reasoning evaluation in the supplied results. e-architecturee-qualitativee-pv-resultse-plan-results
The main tables give point estimates without seed variability or confidence intervals. Architecture, sensor and training differences limit causal interpretations of cross-method rankings; prediction quality does not establish physical deployment safety. e-pv-resultse-plan-resultse-bev-resultse-speed
Figure 3 labels confidence bins 0 through 20, whereas Section 3.1.2 says 20 bins. Endpoint/index conventions remain unresolved. Missing appendices also prevent verification of detailed attention masks, cache recomputation and geography-separated evaluation. e-confidencee-missing
7.2 Questions for discussion
- Does modality-specific trajectory supervision reduce reliance on ego state when evaluated under controlled sensor degradation?
- Do better localization-sensitive confidence scores improve planning once perception quality and training budget are controlled?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs the InternVL2-8B initialization, optional PointPillars encoder, Table 2 datasets and an intermediate detector checkpoint for confidence examples. Missing Appendix B prevents recovery of exact mixtures, batch size, training duration, freezing policy, hardware and software configuration from this artifact. e-architecturee-confidencee-beve-traininge-missing
Proposed checks: hold detection data and optimization fixed while varying only the confidence-example distribution; separately ablate modality-specific action-query supervision while keeping the all-feature inference head and training budget fixed. Measure calibration, AP and trajectory metrics rather than assuming perception improvements transfer to planning. e-confidencee-conf-ablatione-actione-plan-results
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Hold confidence examples fixed before changing their distribution
Reader-proposed check, not performed: generate one nuImages training prediction pool from a fixed intermediate checkpoint. From that pool, compare realistic sampling with uniform IoU sampling at equal sample counts, optimization steps and ground-truth mixing ratio; retain the same confidence head and complementary loss masks. Include the class-only baseline and a matched-size random-perturbation arm. Evaluate AP/AP50/AP75 on nuImages val and plot predicted scores against realized IoU, following Figure 6. If the real-distribution advantage disappears under these controls, the claim that distribution matching drives the gain is weakened. Specify the unresolved confidence-bin indexing before running the comparison. e-confidencee-conf-ablatione-training
Check 2: Test whether auxiliary modality queries reduce planner shortcuts
Reader-proposed check, not performed: start both arms from the same perception checkpoint, use the same nuScenes data and compute budget, and retain the Q_full inference head. Compare the four supervised query sets against a control that keeps the query tokens but removes the three modality-specific trajectory losses. Measure open-loop L2 on identical validation scenes, then apply matched camera degradation and ego-state perturbations to both arms. Stronger robustness in the fully supervised arm would support the proposed alignment mechanism; no improvement would weaken the claim that auxiliary branches prevent over-reliance. Keep streaming disabled and avoid changing the NAVSIM decoder in this isolation test. e-actione-traininge-plan-resultse-speed
8.3 Reading coverage
Visual audit: Visually inspected the title/author/affiliation block and CVF version notice on page 1; introduction on page 2; architecture and confidence training on page 3; detailed PV losses and confidence scoring on page 4; grid/BEV and action-query mechanisms on page 5; training datasets, settings and PV results on page 6; planning, BEV and confidence tables on page 7; confidence diagnostics, BEV/latency ablations and qualitative results on page 8; and the final reference page 14. All six final original crops were viewed. Figure 3's loss-mask colors were checked against its caption and Section 3.1.2; its bin-index discrepancy is disclosed. Figure 5's query restrictions and inference selection were checked against the caption and pages 5–6. These pages cover every scientific claim and proposed-check premise retained here. References on pages 9–13 were read as text only. Cited appendices are absent from the supplied artifact; no appendix, code or external edition was inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 14. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract (PDF p. 1)
- 1. Introduction (PDF pp. 1–2)
- 2. Related Work (PDF pp. 2–3)
- 3. Method, including 3.1.1–3.1.2, 3.2 and 3.3 (PDF pp. 3–6)
- 4. Experiments, including 4.1–4.3 (PDF pp. 6–8)
- 5. Conclusion (PDF p. 8)
- References [1]–[118] (PDF pp. 9–14)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All seven supplied text chunks were read individually, covering the complete 14-page artifact. The extraction's image limitation was addressed by inspecting PDF pages 1–8 and 14 and all six final crops.
- Appendices A.1, A.2, A.3, B and C.1 are cited in the body but absent from the supplied PDF, which ends with references on printed p. 10655. No separate supplement was supplied; its availability remains unverified.
- Identity: the observed title and all 19 authors match the catalog. The title-page notice identifies this as the CVPR Open Access version, identical to the accepted version except for its watermark; it distinguishes the final IEEE proceedings version. That other version was not inspected, and no numbered revision or revision date is established.
- Code, checkpoints and external resources were not inspected; experiments were not reproduced. Reference pages 9–13 were read as text, not visually inspected.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1 (printed p. 10642), title, author block and CVF notice
Exact title and all 19 catalog authors are present. Affiliations are Yinwang Intelligent Technology Co. Ltd. and Fudan University. The notice distinguishes the Open Access accepted-version copy from the final IEEE proceedings.
Go to primary source ↓e-problemPDF pp. 1–2, Abstract and Section 1
Spatial localization and confidence weaknesses motivate explicit perception supervision. The abstract spells the NAVSIM metric PMDS.
Go to primary source ↓e-architecturePDF p. 3, Figure 2, Section 3 overall architecture and Section 2 continuation
Shared InternVL2-8B backbone, PV/BEV features, optional LiDAR, action head, language branch and KV cache; NAVSIM is described as data-driven pseudo-simulation.
Go to primary source ↓e-pvPDF p. 4, Section 3.1 and 3.1.1
Grid queries, dynamic tiling, box serialization and discretization; 16 mask tokens retrieve segmentation masks. Detection uses cross-entropy and segmentation adds focal and Dice losses.
Go to primary source ↓e-confidencePDF p. 3, Figure 3(a–b) and caption; PDF p. 4, Section 3.1.2
Prediction-derived IoU examples, complementary loss masks and class-confidence times predicted-IoU inference scoring. Text specifies 20 bins while the figure depicts labels 0, 1, 2, …, 20.
Go to primary source ↓e-bevPDF pp. 4–5, Section 3.2 and Figure 4
BEV queries cross-attend to PV features; optional PointPillars initialization uses PixelUnshuffle and MLP. Grid proposals decode independently; attributes are discretized into [0,1024). BEV masks use independent binary class segmentation.
Go to primary source ↓e-actionPDF p. 5, Figure 5 and Section 3.3; PDF p. 6, Section 3.3 continuation
Four query sets have modality-restricted or full access; each is N by C and decoded by MLP. Smooth-L1 supervises all trajectories; Q_full provides the inference output.
Go to primary source ↓e-streamPDF p. 6, Section 3.3, Streaming Inference
Longer-clip training and dual-recomputation KV caching are named, with details deferred to Appendix A.2.
Go to primary source ↓e-trainingPDF p. 6, Table 2 and Section 4.1
Task datasets, official split default, optimizer settings, grid sizes and two-stage perception-to-VLA curriculum; Appendix B is referenced for further configuration.
Go to primary source ↓e-pv-resultsPDF p. 6, Table 1, COCO mAP column and AD/AD+Gen rows; Section 4.2
AD+Gen is identified as the final model. COCO detection mAP is 51.7 versus LMM-Det 47.5 and UFO 48.9; the table lists different visual and language backbones.
Go to primary source ↓e-plan-resultsPDF p. 7, Table 3 and Section 4.2 E2E Trajectory Planning Results
Starred two-stage row: nuScenes L2 0.16/0.33/0.60, average 0.36; NAVSIM v1 PDMS 90.2 versus 88.6 unstarred and 88.1 DiffusionDrive. Distinct decoder settings are deferred to A.3; NAVSIM uses static-vocabulary scoring and selection.
Go to primary source ↓e-bev-resultsPDF p. 7, Table 4 and Section 4.2 BEV Perception Results
nuScenes val detection and four map IoUs; no sequence input and image resolution 448×796. Percept-WAM detection is below BEVFusion, while selected map IoUs are higher.
Go to primary source ↓e-conf-ablationPDF p. 7, Table 5 and Section 4.3; PDF p. 8, Figure 6 and continuation
nuImages val comparison of class-only, random perturbation, uniform model-prediction and real model-prediction confidence training. AP/AP50/AP75 are 49.6/70.4/53.7 for real model predictions versus 48.1/70.9/51.4 baseline.
Go to primary source ↓e-bev-ablationPDF p. 8, Table 6 and Section 4.3 BEV 3D Detection
Cumulative camera, LiDAR, augmentation, 20×20-to-40×40 grid and MLP-decoding settings. Final mAP remains 50.4 but NDS falls from 46.6 to 43.7; 16× speedup is reported.
Go to primary source ↓e-speedPDF p. 8, Table 7 and Section 4.3 E2E training
Direct AR, clustered AR and query-based decoding trade accuracy against latency. Query-based streaming changes 0.3822/1174 ms to 0.3839/707 ms on nuScenes val.
Go to primary source ↓e-qualitativePDF p. 8, Figures 7–9 and Section 5
Selected PV, BEV and trajectory examples include bird and shopping-cart detection and a construction-zone scenario. Conclusion proposes future offline/online RL.
Go to primary source ↓e-missingPDF p. 5, Section 3.2 reference to A.1; p. 6, references to A.2 and B; p. 7, references to A.3 and C.1; p. 14 (printed p. 10655), final reference [118]
The body delegates attention masks, streaming details, decoder settings, further training configuration and geographical split results to appendices that do not occur in the supplied 14-page artifact.
Go to primary source ↓8.5 Primary sources
Percept-WAM: Perception-Enhanced World-Awareness-Action Model for Robust End-to-End Autonomous Driving ↗
PDF · 9,967 extracted words
Source fingerprint
5b1d5889b4e6e5ed91d99ea09f5ad0478a2fda23804af1c64231a8f6130686df