UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving
1. Paper overview
In one sentence: UniDriveVLA lets three specialized experts exchange information inside one model, improving selected driving and understanding metrics while leaving comfort, motion forecasting and full semantic retention unresolved. e-architecturee-attentione-benche-planninge-decouplinge-generale-diagnostic
| At a glance | What to know |
|---|---|
| Research problem | Author claim Driving needs geometric precision and semantic interpretation. The authors argue that adding spatial tokens to a shared language decoder makes heterogeneous representations interfere, weakening reasoning. Their proposal separates expert parameters while allowing planning to consume both semantic and spatial information. e-probleme-diagnostic |
| Core mechanism | Source description Three specialized experts share masked attention but retain separate projections and feed-forward pathways, forming one integrated driving model. e-architecturee-attention |
| A key reported result | Bench2Drive closed-loop driving: 78.37; 51.82%; 198.86; 11.78 Driving Score ↑; Success Rate ↑; Efficiency ↑; Comfortness ↑. CARLA benchmark; comparison group trained without PDM-Lite demonstrations. Orion: 77.74; 54.62%; 151.48; 17.38. Best listed score and efficiency within this training-data group, but lower success and comfort than Orion. No uncertainty is reported. e-benche-evaluation |
| Reading caution | Reader analysis Preservation is relative: Table 8's 8B model trails Qwen3-VL on every listed general benchmark, including MMStar 43.3 versus 63.0. Table 7 supports better retention than a shared decoder, not unchanged foundation-model ability. e-generale-decoupling |
Core contributions
- Source description
Three specialized experts share masked attention but retain separate projections and feed-forward pathways, forming one integrated driving model. e-architecturee-attention
- Source description
Sparse perception is refined before and after semantic interaction; progressive training adds spatial and action supervision, then freezes the language pathway. e-sparsee-training
Figure 3. Separate expert parameters exchange information through one controlled attention operation. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read from the bottom upward. Camera features enter the vision encoder, whose rightward arrow feeds the sparse encoder; detection and map anchors and occupancy queries identify the spatial branch. The language prompt enters a tokenizer, while a noised trajectory enters the action encoder. Purple, green and orange blocks retain separate QKV projections and feed-forward networks. The horizontal attention band joins their information, but Figure 4 specifies which directions are allowed. At the top, the LM head produces a driving-language output, the sparse decoder produces scene estimates, and the action decoder produces a trajectory. Section 3.3 further explains that spatial features are refined after semantic interaction. e-architecturee-attentione-sparsee-inputs
What it supports. The figure supports an integrated model with specialized pathways. Perception is available as an intermediate, interpretable representation, and the action expert can use both semantic and spatial context. This architecture underlies the report's disagreement with the catalog's Dual-system label; the evidence is the shared layer computation, not merely a common training objective.
Where the evidence stops. The attention band does not imply unrestricted communication. The displayed planning-language example does not establish that a complete verbal chain must be generated before each action. Sampling order and the downstream actuator controller are unspecified.
2. Motivation
2.1 The problem and the proposed response
Driving needs geometric precision and semantic interpretation. The authors argue that adding spatial tokens to a shared language decoder makes heterogeneous representations interfere, weakening reasoning. Their proposal separates expert parameters while allowing planning to consume both semantic and spatial information. e-probleme-diagnostic
2.2 What this reading follows
A driving model must interpret a scene, locate its geometry and generate a trajectory. UniDriveVLA assigns those jobs to separate transformer experts, then controls what each expert can read. The central idea is to protect the language pathway while making its semantic context useful to spatial perception and action generation. Read the architecture together with its attention mask: the shared attention bar alone hides this directionality. Then separate three experimental questions: whether the model drives well in simulation, whether it predicts logged trajectories accurately, and whether expert decoupling preserves understanding. The tables support qualified gains, with different winners across metrics and several unresolved reporting discrepancies. e-architecturee-attentione-benche-planninge-decouplinge-generale-diagnostic
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | VLA |
| Architecture | Dual-system |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Conflicts with the recorded classification
The VLA and spatially aware driving labels are supported. The recorded Dual-system architecture conflicts with the inspected design: three expert pathways participate in the same masked joint-attention layers and directly generate trajectories. This is architectural evidence for one integrated model, beyond joint training alone. No fast–slow planner handoff, future-world rollout planner or inverse-dynamics action extraction is specified; the world-action prediction quadrant remains not applicable. e-architecturee-attentione-inputse-sparse
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Separate parameter pathways while sharing relevant evidence
Begin with the distinction between representation and communication. UniDriveVLA constructs understanding tokens from images and instructions, spatial tokens from sparse perception, and action tokens from a noised velocity sequence. Each group receives its own projections and feed-forward transformations, but the attention operation uses concatenated queries, keys and values. The mask then determines what can be read: semantics can inform spatial estimates, and both can inform actions, while the understanding prefix cannot read later spatial/action tokens. The source describes this as decoupling heterogeneous objectives within one framework. Reader interpretation: the useful design principle is selective sharing, because fully isolated experts would lose cross-task context. The experiments must still establish whether this structural choice improves learning; the presence of separate expert names is insufficient evidence by itself. e-inputse-architecturee-attentione-decoupling
Figure 4. The mask sends semantic context toward perception and action while restricting the understanding prefix. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the column groups as the tokens available to each row, consistent with the query–key attention formulation in Section 3.2. The upper-left prefix is triangular: later tokens can use earlier context, while empty regions prevent the prefix from reading perception or suffix tokens. Green perception rows cover the prefix and the full perception block. Orange suffix rows cover prefix, perception and suffix columns, including status and action tokens. The legend separates token types from their visibility. This schematic agrees with the accompanying prose's main direction: understanding supplies semantic context to perception, and actions aggregate semantic and spatial information. The figure does not provide token counts for an actual implementation. e-attentione-training
What it supports. The architecture combines causal language processing with denser interactions inside spatial and continuous-action groups. It explains how a shared attention operation can remain directional: integrating information for planning does not require feeding every spatial or action token back into the language prefix. The intended protection concerns the structure of forward information flow.
Where the evidence stops. A forward attention mask is not a stop-gradient operation or a proof that optimization interference disappears. The paper specifies VLM freezing only in the final training stage; earlier joint optimization still requires careful empirical assessment.
5.2 Follow spatial information through both refinement passes
The spatial branch begins with multi-scale image features and sparse queries initialized from dataset-level K-Means instance banks. Temporal interaction carries scene context across observations, while intra-task reasoning and inter-task communication let the query groups use task structure and each other's evidence. Deformable aggregation retrieves visual information, followed by task-wise refinement. Occupancy is an auxiliary latent branch alongside detection, mapping, ego-status and motion representations. The important next step is that first-pass outputs are projected into the expert hidden space. There they receive semantic context through the masked attention mechanism and provide spatial context for the action expert. The resulting perception features are projected back into the sparse space and refined by another decoder. Thus the source describes a semantically enriched spatial pathway, rather than a single geometric feature extraction step. e-sparsee-attention
5.3 Distinguish staged learning from the incomplete execution recipe
The training schedule gradually changes which pathways can adapt. Stage one fully fine-tunes the VLM on a mixture dominated by general multimodal data. Stage two adds structured perception and flow-matching action objectives, uses language-model LoRA, and applies a lower learning-rate multiplier to the VLM than to the rest of the model. Stage three freezes the VLM and specializes the perception/action experts while adding motion forecasting. This is a training prescription, not a complete inference algorithm: the PDF does not specify the flow solver, sampling steps, language-output schedule or actuator controller. Reader interpretation: Table 5 makes the final motion addition especially worth investigating, because its intended dynamic prior does not improve the displayed open-loop planning metrics. Closed-loop effects would require a controlled comparison rather than extrapolation from that table. e-traininge-inputse-attentione-componentse-evaluation
5.4 Training and inference
During training
Stage 1 fully fine-tunes the VLM for 3 epochs at 4×10⁻⁵ using driving/general data in a 3:7 ratio, with general data mainly from FineVision. Driving frames are resized to 960×544. e-training
Stage 2 jointly optimizes language, perception and flow-matching action objectives for 30 epochs with AdamW: base learning rate 2×10⁻⁴, VLM rate 1×10⁻⁴, language-model LoRA and EMA. Stage 3 freezes the VLM and trains perception/action experts for 15 epochs at 1×10⁻⁴, retaining EMA and adding motion supervision. e-traininge-attention
During inference
The source establishes observation/instruction encoding, sparse perception, expert interaction and continuous trajectory output. It does not specify a complete sampling algorithm, flow solver or step count, velocity-to-position integration, language-generation schedule, replanning frequency or trajectory-to-actuator controller. Closed-loop results concern CARLA simulation; physical vehicle execution is not demonstrated. e-inputse-attentione-evaluation
5.5 Implementation flow
- Construct complementary tokens
The implementation describes Qwen3-VL with a SigLIP-2 vision encoder, MLP merger and Qwen3 language model. Images and instructions produce understanding tokens; sparse visual queries produce perception tokens. The action encoder embeds interpolations between Gaussian noise and target velocity sequences during training. e-inputse-training
- Extract and refine spatial priors
K-Means instance banks initialize queries. Temporal interaction, task-specific reasoning, cross-task communication and deformable feature aggregation update detection, map, ego and motion representations; occupancy uses an auxiliary latent branch. First-pass outputs enter the expert hidden space, receive semantic information, then return to a subsequent sparse decoder. e-sparse
- Control cross-expert visibility
Each expert computes its own queries, keys and values. Concatenated representations undergo masked joint attention, then separate output projections, normalization and feed-forward updates. Understanding cannot read later perception/action tokens; perception reads semantic context; actions aggregate both. Separate parameters therefore coexist with shared information flow. e-attention
6. Experiments & results
UniDriveVLA combines semantic understanding, sparse spatial perception and continuous trajectory generation in one Mixture-of-Transformers model. Separate expert parameters and directed attention aim to limit interference. Reported gains are strongest for selected planning and understanding comparisons; comfort, motion forecasting and preservation of foundation-model capability remain weaker.
6.1 Read the original evidence
Table 1. Driving Score and efficiency lead the non-PDM-Lite group, with weaker success and comfort. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the notes above the columns: dagger-marked models use PDM-Lite demonstrations, and the paper limits its bold best-result claims to models without that source. The left metric evaluates logged trajectory error over a two-second horizon at two hertz; the four columns to its right evaluate closed-loop behavior. Follow UniDriveVLA's bottom row, then compare it with Orion immediately above. Higher is marked as better for Driving Score, success, efficiency and comfort. A model can lead one column and trail another. In particular, the smallest open-loop L2 belongs to DriveMOE, while the best listed comfort in the comparison group belongs to MomAD. e-benche-evaluation
What it supports. UniDriveVLA's Driving Score is 78.37 versus Orion's 77.74, and efficiency is 198.86 versus 151.48. However, its success is 51.82% versus 54.62%, and comfort is 11.78 versus 17.38. These results support a specific driving-quality/efficiency gain within the stated demonstration group, not dominance across all behavior measures.
Where the evidence stops. Bench2Drive is CARLA simulation. The table reports neither repeated-run uncertainty nor the full controller and route configuration. Its efficiency metric is a driving metric, not a measurement of neural-network inference speed.
Table 3. Trajectory accuracy and collision rate tell different stories across the two protocols. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Choose an ego-status group before comparing methods, then stay inside either the ST-P3 or UniAD metric block. Each block separately lists displacement error in meters and collisions in percent at three horizons and as an average. The bottom group evaluates models without ego status. Within it, UniDriveVLA-Large uses Qwen3-VL-8B; Base uses the 2B backbone. Read the Large and SparseDrive rows across both average columns. The retained notes matter: SparseDrive's double-dagger indicates re-evaluation under the GPT-Driver protocol. Asterisks mark ego-state access, so the upper and lower halves are different input settings and cannot be merged into one ranking. e-planninge-evaluation
What it supports. Without ego status, Large has the lowest listed average L2: 0.51 m under ST-P3 and 0.90 m under UniAD. Its collision averages, 0.11% and 0.27%, exceed SparseDrive's 0.08% and 0.21%. With ego status, Large's average L2 of 0.42/0.74 m trails FSDrive's 0.28/0.45 m.
Where the evidence stops. These are open-loop comparisons against logged trajectories, not executed-drive safety rates. The two metric protocols should remain separate. The source explains the asterisk and SparseDrive marker, but does not explain the additional dagger on FSDrive's ego-status row.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Bench2Drive closed-loop driving CARLA benchmark; comparison group trained without PDM-Lite demonstrations. | 78.37; 51.82%; 198.86; 11.78 Driving Score ↑; Success Rate ↑; Efficiency ↑; Comfortness ↑ | Orion: 77.74; 54.62%; 151.48; 17.38. Best listed score and efficiency within this training-data group, but lower success and comfort than Orion. No uncertainty is reported. e-benche-evaluation |
| nuScenes planning without ego status Large/Qwen3-VL-8B; Table 3 reports ST-P3 and UniAD protocols separately, at 1/2/3 seconds; exact planning split size is unstated. | ST-P3: 0.51 m / 0.11%; UniAD: 0.90 m / 0.27%. Average L2 (m) ↓; average collision (%) ↓ | SparseDrive‡: 0.55 m / 0.08%; 0.99 m / 0.21%, with its re-evaluation qualification. Lowest listed average L2 in this input group, but higher collisions than SparseDrive. The with-ego group does not show the same L2 leadership. e-planninge-evaluation |
| nuScenes perception and motion forecasting Validation set; Large model. | 0.407/0.460; 0.535; 1.264/2.121 m. Detection mAP/NDS ↑; map mAP ↑; motion minADE/minFDE (m) ↓ | SparseDrive: 0.418/0.525; 0.551; 0.600/0.960 m. The unified model supplies multiple useful outputs but trails this baseline throughout these metrics, especially motion forecasting. e-perception |
| Expert decoupling ablation Shared decoder versus MoT in Table 7; general VQA, DriveBench and nuScenes. Parameter/compute matching and exact planning protocol are unstated. | MoT: 45.5/54.9; 0.533/0.140. General VQA/DriveBench (%) ↑; L2 (m)/collision (%) ↓ | Shared decoder: 31.1/50.8; 0.641/0.175. Map mAP is tied at 0.516. Supports a benefit from the tested architecture change, especially understanding, without establishing universal improvement or isolating parameter count. e-decoupling |
| Cumulative planning-component ablation nuScenes Table 5; protocol and L2 unit are not explicitly identified. | Baseline 0.75/0.27; +ego 0.61/0.21; +detection 0.58/0.10; +map 0.58/0.14; +occupancy 0.53/0.14; +motion 0.54/0.17. Average L2 ↓; collision rate (%) ↓ | Best collision occurs before mapping; best L2 occurs before motion. Additional tasks do not monotonically improve planning. These cumulative rows do not isolate every task's independent effect. e-components |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 5. More supervised components do not produce monotonic planning gains. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the checkmarks before the numbers. Each row retains the preceding components and adds the next one: ego status, detection, mapping, occupancy, then motion. Compare adjacent rows to understand the reported incremental changes. Detection takes collision rate from 0.21% to 0.10%; mapping leaves L2 at 0.58 and raises collision to 0.14%. Occupancy improves L2 to 0.53 at the same collision rate. Finally, the motion row moves to 0.54 and 0.17%. The bold entries therefore occur in different rows. This is a cumulative sequence, so it does not measure every component independently or reveal all interactions among tasks. e-components
What it supports. The strongest collision result uses ego status and detection. The strongest L2 result also includes mapping and occupancy, but excludes motion. The reported motion addition worsens both metrics relative to that row. The table supports testing which spatial information helps planning, rather than assuming that every additional prediction objective is beneficial.
Where the evidence stops. The caption does not name the ST-P3/UniAD protocol or explicitly state the L2 unit. No uncertainty is reported. The authors' limited-headroom explanation for weak mapping/motion gains is an interpretation, not an isolated causal result.
Table 7. Decoupling helps the tested understanding and planning metrics; mapping remains tied. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the table in three groups: understanding, perception and planning. General VQA summarizes several multimodal benchmarks, DriveBench measures driving-oriented understanding, and perception/planning are evaluated on nuScenes. Compare each column vertically. The largest visible understanding difference is general VQA, from 31.1% to 45.5%; DriveBench rises from 50.8% to 54.9%. Detection changes only from 0.437 to 0.439 NDS, and map mAP is exactly tied at 0.516. The last two columns decrease, which is favorable for L2 and collision. This is the most direct reported comparison for the expert-decoupling claim, but its configuration details need to be separated from the main benchmark tables. e-decouplinge-generale-diagnostice-vqa
What it supports. MoT reduces reported L2 from 0.641 to 0.533 m and collision from 0.175% to 0.140%, while improving the two understanding scores. These results support the architecture in the tested setup. They do not show complete preservation of the original VLM: Table 8 still reports lower general-benchmark scores than the listed Qwen3-VL model.
Where the evidence stops. Figure 2 instead labels collision 15.6% and L2 0.532; Table 6 reports DriveBench 51.97 rather than 54.9. These discrepancies remain unresolved. Parameter/compute matching and repeated-run uncertainty are not supplied, so the table alone cannot isolate reduced interference from added capacity.
7. Analysis & limitations
7.1 What the evidence leaves open
Preservation is relative: Table 8's 8B model trails Qwen3-VL on every listed general benchmark, including MMStar 43.3 versus 63.0. Table 7 supports better retention than a shared decoder, not unchanged foundation-model ability. e-generale-decoupling
Cross-display values remain unresolved: Figure 2 labels collision 15.6% and L2 0.532, while Table 7 gives 0.140% and 0.533 m. DriveBench is 51.97 in Table 6 but 54.9 in Table 7. These are retained as separate reported settings, without assuming identical checkpoints or correcting the source. e-diagnostice-vqae-decoupling
Bench2Drive leadership depends on metric and demonstration group. Table 2's mean ability is 51.53% versus Orion's 54.72%; no physical-driving or manipulation evaluation is supplied. Repeated-run uncertainty and deployment latency are absent. e-abilitiese-benche-conclusione-evaluation
7.2 Questions for discussion
- Does MoT still improve retention when total parameters, optimization and inference compute are matched? (e-decoupling, e-training)
- Does motion supervision help interactive closed-loop driving despite its unfavorable open-loop ablation? (e-components, e-abilities)
8. Reproducibility audit
8.1 Requirements and known gaps
Required ingredients include the described VLM variants, general/driving training mixture, nuScenes annotations, Bench2Drive demonstrations, sparse instance banks, and all three training stages. Preserve ego-input access, metric protocol and demonstration source when comparing results. e-traininge-planninge-sparsee-evaluation
The supplied method leaves hardware, training time, batch size, LoRA rank, EMA decay, loss weights, query counts, exact filtering thresholds and inference configuration unspecified. Epochs and learning rates alone do not determine a faithful implementation. e-traininge-sparsee-attentione-inputs
Proposed checks should match data and compute for shared versus separate experts, then independently toggle motion supervision and motion-token access while holding the training schedule fixed. Repeated-seed understanding, planning and collision measurements could distinguish interference reduction from capacity or auxiliary-task effects. e-decouplinge-componentse-training
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does expert separation help after matching capacity and training?
Reader-proposed, not run: initialize from the same VLM checkpoint and use identical data order, sparse queries, supervision, mask and three-stage schedule. Compare separate experts against a shared decoder, adding a shared-decoder control matched as closely as possible for trainable parameters and compute; disclose residual mismatches. Across repeated seeds, measure Table 7's VQA, DriveBench, detection, mapping, L2 and collision metrics, and the Figure 2 token-similarity diagnostic. The decoupling interpretation predicts better understanding retention and planning beyond the capacity-matched control. If those gains disappear after matching, the reported architecture advantage would not independently establish reduced interference. e-traininge-sparsee-attentione-decouplinge-diagnostic
Check 2: Separate the effect of motion supervision from motion-token access
Reader-proposed, not run: hold the stage-two checkpoint, frozen VLM, final-stage duration, occupancy and other spatial tasks fixed. Run a two-by-two comparison that independently enables motion supervision and lets the action expert attend to motion tokens. Keep query slots and computation present when masking their access, and report multiple seeds. First reproduce the adjacent occupancy-only and motion-added planning rows under an explicitly named nuScenes protocol. Then test the same variants on fixed Bench2Drive interactive scenarios, including merging and overtaking. Improved forecasting without improved planning would weaken the proposed dynamic-prior explanation; closed-loop gains despite worse open-loop error would expose a limitation of the open-loop proxy. e-traininge-attentione-componentse-perceptione-abilitiese-evaluation
8.3 Reading coverage
Visual audit: Visually inspected the title/authors/version on p. 1; paradigm and representation diagnostics on pp. 2 and 4; the system distinction on p. 3; architecture, equations, attention visibility and sparse perception on pp. 4–7; training configuration on p. 7; all eight result/ablation tables on pp. 8–10; and concluding qualifications on p. 11. All six final original-PDF crops were separately viewed. Figure 3's image-feature arrow and Figure 4's block visibility were checked against Sections 3.2–3.3 and Eqs. (3)–(8). No claim-relevant direction conflict was found; numerical discrepancies between Figure 2 and Tables 6–7 are disclosed in the report and relevant caution. Table 1 and Table 3 crops retain their compact protocol/marker notes because these are necessary to interpret the comparisons. Other crops omit captions and surrounding prose. The full text, including references on pp. 12–18, was read in all seven chunks; the references-only pages were not visually inspected. No appendix is present and separate supplements remain unverified.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- 1 Introduction (pp. 1–3)
- 2 Related Work, including 2.1–2.3 (pp. 3–4)
- 3 Methodology, including 3.1–3.4 (pp. 4–7)
- 4 Experiments, including 4.1–4.2 (pp. 7–11)
- 5 Conclusion and Acknowledgments (p. 11)
- References [1]–[124] (pp. 11–18)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This extraction limitation was addressed by visually inspecting PDF pages 1–11 and every final crop.
- Separate supplemental material availability has not been fully verified.
- All seven supplied text chunks were read individually, including the complete references. No appendix is present in the supplied PDF. References-only pages 12–18 were read as text rather than visually inspected.
- Identity/version note: the exact title and all 14 authors match the catalog. The inspected artifact is arXiv:2604.02190v1, stamped 2 Apr 2026, while its internal Date line says April 3, 2026. No other edition or revision chain was supplied or compared.
- Code and external project pages were not inspected, and no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author/affiliation block, Date line and arXiv margin stamp
The title and all 14 authors match the supplied catalog. The margin identifies arXiv:2604.02190v1, 2 Apr 2026; the internal Date line says April 3, 2026. Affiliations are Huazhong University of Science and Technology; Xiaomi EV; SKL-IOTSC, University of Macau. Project and code URLs are printed.
Go to primary source ↓e-problemPDF pp. 1–2, Abstract, Introduction and Figure 1
The authors attribute the spatial-perception/semantic-reasoning conflict to shared-parameter optimization and propose expert decoupling with controlled communication.
Go to primary source ↓e-architecturePDF p. 3, Section 2.1; pp. 5–6, Section 3.2 and Figure 3
Section 2.1 distinguishes fast–slow dual systems from unified trajectory predictors. UniDriveVLA uses understanding, perception and action experts with separate projections and feed-forward transformations inside jointly attending MoT layers.
Go to primary source ↓e-inputsPDF pp. 4–5, Section 3.1, Eq. (1), trajectory definition; p. 5, Section 3.2
The model maps multi-view images, trajectory history and navigation commands to future planar trajectory points. Understanding and sparse-perception tokens condition action tokens constructed by flow-matching interpolation between Gaussian noise and target velocity sequences.
Go to primary source ↓e-attentionPDF p. 6, Section 3.2, Eqs. (3)–(8), Figure 4 and adjacent masking paragraph
Expert-specific Q/K/V are concatenated for masked attention, then split for expert-specific updates. Understanding is causally masked from later perception/action tokens; perception reads understanding; actions aggregate semantic and spatial context. Figure 4 depicts dense within-perception and within-suffix visibility. Eq. (8) combines autoregressive, perception and action losses.
Go to primary source ↓e-sparsePDF pp. 6–7, Section 3.3
Dataset-level K-Means instance banks initialize sparse task queries. Temporal interaction, intra/inter-task reasoning, deformable image-feature aggregation and refinement support detection, mapping, ego-status and motion. Occupancy is an auxiliary latent branch. First-pass perception features enter the expert hidden space, receive semantic context, then return for subsequent sparse decoding.
Go to primary source ↓e-trainingPDF p. 7, Sections 3.4 and 4.1, Implementation Details
The paper describes Qwen3-VL with a SigLIP-2 encoder, MLP merger and Qwen3 LM, using 960×544 driving frames. Stage 1 fully fine-tunes for 3 epochs at 4×10^-5 with driving:general sampling 3:7, mainly FineVision for general data. Stage 2 uses AdamW, 30 epochs, base rate 2×10^-4, VLM multiplier 0.5, language-model LoRA and EMA. Stage 3 freezes the VLM, trains perception/action experts for 15 epochs at 10^-4, retains EMA and adds motion forecasting.
Go to primary source ↓e-evaluationPDF p. 8, Section 4.1, Dataset paragraph; Table 1 caption; p. 9, Tables 3–4 captions
nuScenes provides open-loop planning and perception evaluation; Table 4 specifies its validation set. Bench2Drive is CARLA-based closed-loop evaluation with six cameras and demonstrations attributed to Think2Drive. Bench2Drive open-loop L2 uses a 2-second horizon at 2 Hz. Table 3 separates ST-P3/UniAD metrics and ego-status access. Exact split sizes and simulator execution configuration are not supplied here.
Go to primary source ↓e-benchPDF p. 8, Table 1, caption and UniDriveVLA, Orion, DriveMOE, MomAD and dagger-marked rows
UniDriveVLA reports L2 0.72, Driving Score 78.37, Success Rate 51.82%, Efficiency 198.86 and Comfortness 11.78. Orion reports 77.74, 54.62%, 151.48 and 17.38 for the four closed-loop metrics. DriveMOE has L2 0.38; MomAD has Comfortness 51.20. Dagger rows use PDM-Lite; best-score/efficiency claims are restricted to methods without that demonstration source.
Go to primary source ↓e-abilitiesPDF p. 8, Table 2, UniDriveVLA and Orion rows
UniDriveVLA has merging 38.75%, overtaking 80.00%, emergency brake 50.00%, give way 30.00%, traffic sign 58.95% and mean 51.53%. Orion's mean is 54.72% and emergency brake is 78.33%.
Go to primary source ↓e-planningPDF p. 9, Table 3, protocol headers, ego-status groups, UniDriveVLA-Large, SparseDrive‡ and FSDrive rows
Without ego status, UniDriveVLA-Large reports ST-P3 average L2 0.51 m and collision 0.11%; UniAD averages are 0.90 m and 0.27%. SparseDrive‡ reports 0.55 m/0.08% and 0.99 m/0.21%; its results are re-evaluated using the GPT-Driver protocol. With ego status, Large reports L2 averages 0.42/0.74 m versus FSDrive's 0.28/0.45 m. Base/Large backbones are Qwen3-VL-2B/8B. FSDrive* also carries an unexplained dagger.
Go to primary source ↓e-perceptionPDF p. 9, Table 4, nuScenes validation, UniDriveVLA-Large and SparseDrive rows; pp. 9–10, perception-results discussion
Large achieves detection mAP 0.407/NDS 0.460, map mAP 0.535, motion minADE 1.264 m/minFDE 2.121 m. SparseDrive reports 0.418/0.525, 0.551 and 0.600 m/0.960 m. The authors acknowledge weaker motion forecasting.
Go to primary source ↓e-componentsPDF p. 10, Table 5, all six rows; Section 4.2, Ablation on Planning Components
Cumulative baseline→ego→detection→mapping→occupancy→motion rows report L2/CR(%): 0.75/0.27, 0.61/0.21, 0.58/0.10, 0.58/0.14, 0.53/0.14, 0.54/0.17. The table does not identify its ST-P3/UniAD protocol or L2 unit explicitly. Authors attribute absent map/motion gains to limited nuScenes open-loop headroom.
Go to primary source ↓e-decouplingPDF p. 10, Table 7, both architecture rows and caption
Shared decoder versus MoT: general VQA 31.1→45.5%, DriveBench 50.8→54.9%, detection NDS 0.437→0.439, map mAP 0.516→0.516, L2 0.641→0.533 m, CR 0.175→0.140%. Architecture-specific parameter counts, matched-compute details and repeated-run uncertainty are not reported with this comparison.
Go to primary source ↓e-vqaPDF p. 10, Table 6, caption and UniDriveVLA, GPT-4o, ReCogDrive† rows
UniDriveVLA's DriveBench average is 51.97 and behavior score 60.97. GPT-4o averages 51.96. ReCogDrive averages 56.71 but is explicitly pretrained only, without action training. The 51.97 average differs from Table 7's MoT DriveBench 54.9.
Go to primary source ↓e-generalPDF p. 10, Table 8, Qwen3-VL and UniDriveVLA 8B rows; p. 11, General Visual Capabilities continuation
UniDriveVLA trails the listed Qwen3-VL across all seven general benchmarks, including MMStar 43.3 vs 63.0 and RealWorldQA 49.9 vs 69.0. The prose claims retained meaningful capability, while acknowledging that stronger general-purpose VLMs outperform it.
Go to primary source ↓e-diagnosticPDF p. 4, Figure 2 panels (a)–(b) and caption; p. 10, Table 7
The shared-decoder semantic/perception cosine similarity rises toward one across plotted layers while MoT remains near zero. The caption interprets this as collapse versus decoupling. The radar labels collision 15.6% and L2 0.532, whereas Table 7 lists MoT collision 0.140% and L2 0.533 m. The source does not reconcile these values.
Go to primary source ↓e-conclusionPDF p. 11, Section 5, Conclusion
The conclusion reiterates the unified driving architecture and suggests extension to robotic manipulation; no manipulation experiment is presented.
Go to primary source ↓8.5 Primary sources
UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving ↗
PDF · 10,015 extracted words
Source fingerprint
8f66f7ce0c13bfc01898edcabb7d7c30e9c2b0f5c1fd43e6ba97823e33019249