SAMoE-VLA: A Scene Adaptive Mixture-of-Experts Vision-Language-Action Model for Autonomous Driving
1. Paper overview
In one sentence: SAMoE-VLA uses scene-conditioned parameter fusion to improve a world-informed driving planner, with accuracy gains that require separate scrutiny of collision behavior and implementation consistency. e02e04e05e07e09e11e12e14
| At a glance | What to know |
|---|---|
| Research problem | |
| Core mechanism | |
| A key reported result | LangAuto closed-loop driving: Full: 51.4/63.5/0.81; Short: 69.5/80.9/0.86; Tiny: 79.5/86.4/0.92 (DS/RC/IS). Driving Score (DS), Route Completion (RC, %), Infraction Score (IS); higher is better. CARLA-based LangAuto, Short and Tiny settings; Table 2. BEVDriver full: 48.9/59.7/0.82; SAMoE-VLA reports 3.6B total parameters versus BEVDriver's 7B label. Best listed DS and RC across all three settings. Full-setting IS is not best. The caption defines IS as Infraction Score; p. 10 calls it Instruction Score. e12 |
| Reading caution | Reader analysis Appendix A uses a Beta time distribution and noise-minus-action target; Algorithm 5 uses uniform time and action-minus-noise. Section 4.1 clamps Beta samples, whereas Appendix A rescales them. Its broad full-parameter fine-tuning wording also requires reconciliation with the frozen world expert. e07e08e09e10 |
Core contributions
Figure 1. World supervision trains context; the planning branch converts noise into a trajectory. Original paper, p. 2 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the inputs across the top before following the repeated transformer block downward. Camera features feed both the projected BEV tokens and DSE. Language and soft world prompts join the pink world-language stream, while ego history and noisy action tokens enter the blue planning stream. The horizontal CMCA bar connects their representations, but the actual visibility rule comes from Eq. (1): actions see conditioning and earlier action positions, while conditioning cannot read actions. At the bottom, the paper explicitly separates scene description and rendering during pretraining from trajectory generation in the final stage. The diagram's “Ruler Integration” label is a typo; Appendix A.3 specifies Euler integration. e02e03e07e09e10
What it supports. The architecture combines future-world supervision with a dedicated trajectory generator. Its two functional streams are meaningful architectural evidence for the dual-system part of the catalog entry. The rendering branch's stage label is essential: future point-cloud generation should not be described as an inference-time search over candidate driving futures.
Where the evidence stops. Figure 1 does not define an actuator controller or an inverse-dynamics decoder. The flow equations in Appendix A explain trajectory generation; Algorithm 5 disagrees about the velocity target's sign.
2. Motivation
2.1 The problem and the proposed response
Driving decisions depend on the whole traffic scene, whereas conventional MoE routes individual tokens. The authors argue that this mismatch harms coherent planning and propose scene-conditioned specialization while retaining language and spatial context. e01e02e14
2.2 What this reading follows
A driving planner must combine instructions, surrounding geometry and ego motion into one coherent trajectory. SAMoE-VLA makes the traffic scene determine how its planning experts are mixed. A world-language branch supplies representations learned with future-geometry supervision, and a flow-matching branch turns noisy trajectory tokens into planned positions. The useful distinction is between learning from future worlds and explicitly simulating them during deployment: the architecture marks rendering as pretraining-only. This reading follows the scene signal into the expert weights, separates logged-data accuracy from simulator driving, and uses the ablation tables to assess which claims survive the paper's internal inconsistencies. e02e04e05e07e09e11e12e14
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | IDM |
| Quadrant | Q4 · Dual-system × IDM |
3.1 Evidence-based assessment
Conflicts with the recorded classification
Separate world-language and planning parameter streams support the recorded dual-system aspect. However, the planner learns a conditional trajectory velocity field, not an explicit inverse mapping from current and predicted future states to actions. Rendering is pretraining-only, and geometry/language supervision does not establish JEPA. Thus the recorded Q4 × IDM categorization is not supported as a whole; the snapshot is preserved unchanged. e02e03e06e07e09
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Follow the world supervision into the planner
Begin with the world-language expert's training task. Multi-view images become BEV features; pooled features plus learned frame prompts form world tokens. The language model supplies future-aware representations that are expanded into future BEV features, lifted to voxels and supervised through rendered LiDAR geometry and language. Planning remains frozen during this pretraining stage. The final stage changes what is optimized: a trajectory planner is first trained densely, then its weights initialize the scene-adaptive experts. Appendix B says the world-language expert is frozen at this stage. My interpretation is that the planner benefits from a representation shaped by future-scene prediction. That is a different evidential claim from selecting actions by simulating candidate futures. Figure 1 supports the former and marks the renderer as pretraining-only. e02e06e07e10
Figure 2. One scene summary controls parameter fusion throughout the planning stack. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start on the right: the BEV and near-field distance prior feed the offset predictor, and deformable sampling produces scene features. Those features supply keys and values to the central cross-attention block; learned queries supply its queries. Follow H_BEV upward through mean pooling and a linear head to the colored expert weights. The arrows into the merged FFN mean parameter fusion, as established by Eq. (10), rather than averaging completed expert trajectories. On the left, CMCA output plus its residual becomes the planning hidden state that enters this FFN. The caption clarifies the repetition markers: DSE runs once, while routing heads differ across MoE layers. e04e05e19
What it supports. The scene signal chooses a shared FFN parameter mixture for the planning tokens in each layer. This changes both the routing granularity and where mixing happens. Reusing H_BEV amortizes scene encoding, while separate layer heads preserve depth-dependent expert preferences.
Where the evidence stops. Appendix I.5 instead writes a weighted sum of expert outputs. That formula conflicts with the main parameter-merging definition; Appendix E acknowledges these operations differ for nonlinear experts. This reading follows Eqs. (10)–(11).
5.2 Understand why the mixture is inside the FFN
The DSE does not assign a driving-skill label such as turning or overtaking. It uses BEV features and a distance prior to predict deformable sampling offsets, then summarizes those features with learned queries. Each MoE layer converts this common scene representation into its own softmax weights. Those weights combine the three matrices of each expert before the planning hidden states pass through the resulting nonlinear FFN. Thus all planning tokens in a sample share that layer's parameter mixture. Averaging expert outputs would generally compute a different function, which Appendix E explicitly recognizes. This distinction matters for reproduction because Appendix I later switches to an output-mixture formula. The main equations and Appendix C consistently identify parameter merging as the implemented design being proposed. e04e05e19
5.3 Separate action order from flow-integration time
Two different orderings coexist in the planner. CMCA limits an action position to earlier action positions and globally visible conditioning tokens; this is the trajectory-token visibility rule. Flow matching then refines the whole noisy trajectory over an artificial interpolation time. Appendix A defines the noisy point between a demonstrated trajectory and Gaussian noise, trains a velocity field toward noise as time increases, and integrates backward from noise with negative Euler increments at inference. The model predicts velocities for all six nuScenes trajectory positions at each evaluation. It does not execute one physical control at each ODE step. Algorithm 5 reverses the printed velocity target and changes the time distribution, so checking sign consistency is a prerequisite to a faithful implementation rather than a cosmetic notation fix. e03e08e09e10
5.4 Training and inference
During training
Pretraining proceeds through BEV-to-point-cloud learning, BEV-text alignment on NuInteract, refinement with OmniDrive-nuScenes descriptions, and joint language/future-geometry learning. Planning is frozen during this stage. The final stage first trains a dense planner, then initializes MoE sub-experts from its weights. e07
The paper reports eight NVIDIA A800 GPUs, InternVL2-2B planning initialization, MoE insertion every fourth transformer layer, and final learning rate 10^{-4}. Appendix B freezes the world-language expert during final training; Algorithm 5 updates the planner. e07e08e10
During inference
Initialize the trajectory from Gaussian noise, hold scene/language/state conditioning fixed, and apply ten reverse-time Euler steps. Each velocity prediction covers the full trajectory horizon. The output is a trajectory; the supplied algorithm does not specify the trajectory-to-actuator controller. e08e09
Figure 1 labels rendering as pretraining-only. The reported deployment path uses future-aware representations, without an explicit rendered-future search or inverse-dynamics stage. LangAuto measures closed-loop behavior in CARLA; this is separate from logged nuScenes trajectory prediction. e02e09e12
5.5 Implementation flow
- Encode the scene
OpenCLIP-ConvNext, CPFPN and single-frame BEVFormer produce 200 × 200 BEV features with 256 channels. Projected BEV and instruction tokens enter the world-language expert; state and action-time embeddings enter the planning expert. e02e08e09
- Build future-aware context
Pooled BEV features plus learned frame prompts become world tokens. LLM-conditioned future BEV features are lifted to voxel volumes and rendered with an SDF-based LiDAR model during pretraining. Depth, Chamfer, rendering and language losses supervise this branch. e06
- Constrain information flow
Every action token can attend to all conditioning tokens and action positions up to itself. Conditioning tokens cannot attend to actions and remain static through action refinement. This attention mask does not imply sequential execution of vehicle controls. e03e09
- Merge scene-specific experts
A near-field distance map and BEV features predict deformable offsets. Learned queries attend to encoded spatial features, producing reusable H_BEV. Each MoE layer has its own mean-pool/linear/softmax router, then merges three FFN matrices across experts. The resulting FFN processes planning hidden states. e04e05
6. Experiments & results
SAMoE-VLA conditions a flow-matching driving planner on world-language features and uses BEV scene context to merge expert parameters. Its strongest evidence is improved trajectory accuracy and LangAuto simulator driving scores; world rendering is pretraining-only, and several implementation and result descriptions conflict internally.
6.1 Read the original evidence
Table 1. The strongest average trajectory accuracy comes with horizon-specific tradeoffs. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The table has two metric groups, each split into three horizons and an average. Lower is better throughout; L2 uses meters, while collision values are already percentages. Read the final SAMoE-VLA row against both the traditional/world-model block and the VLA/VLM block. Its 0.29 m average L2 is the lowest listed, but its 0.24 m one-second error is worse than several competitors. Then move to collision: the 0.26% average equals Drive-WM's displayed value. The 3.6B label is explicitly a total parameter count for SAMoE-VLA. Dashes in the OpenEMMA and EMMA collision columns mean missing results, not zero collisions. e11
What it supports. SAMoE-VLA reports L2 errors of 0.24, 0.27 and 0.35 m at one, two and three seconds. Its best listed average supports improved logged-data planning accuracy. The table also shows that this advantage is not uniform across horizons and does not yield a uniquely best average collision rate.
Where the evidence stops. This is open-loop evaluation. The exact main split and uncertainty are unspecified here, and the relative horizon-improvement percentages in the adjacent prose do not consistently follow the displayed comparisons.
Table 2. Closed-loop driving scores improve, while infraction performance varies by setting. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Treat the three column groups as separate evaluations, not interchangeable samples. Within each group, DS is Driving Score, RC is route completion in percent, and the caption defines IS as Infraction Score; larger values are better. Compare the final row with BEVDriver in full LangAuto: DS rises from 48.9 to 51.4 and RC from 59.7 to 63.5, while IS decreases from 0.82 to 0.81. Next inspect the Short and Tiny groups, where SAMoE-VLA again leads the listed DS and RC columns. Its 3.6B total parameter label is smaller than the 7B labels on several comparison methods. e12
What it supports. This table supplies executed closed-loop evidence in the CARLA-based benchmark. SAMoE-VLA leads the listed driving-score and completion columns across all three settings, reaching 79.5 DS and 86.4% RC on Tiny. These results complement the open-loop table, but they measure a different evaluation regime.
Where the evidence stops. Page 10 calls IS “Instruction Score,” conflicting with the table caption's “Infraction Score.” The edition follows the caption and does not infer a separate instruction-grounding metric. No physical-road deployment is established.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| nuScenes open-loop trajectory planning Table 1; reported nuScenes protocol, 1/2/3-second horizons; exact evaluated split is not identified in this table. | L2: 0.24/0.27/0.35; average 0.29. Collision: 0.01/0.18/0.60%; average 0.26%. L2 error (m) and collision rate (%), lower is better | PreWorld: 0.31 m average L2; Drive-WM: 0.26% average collision. Lowest listed average L2, but average collision ties Drive-WM and several methods have better 1-second error. The prose's horizon-relative improvement percentages do not consistently match the table. e11 |
| LangAuto closed-loop driving CARLA-based LangAuto, Short and Tiny settings; Table 2. | Full: 51.4/63.5/0.81; Short: 69.5/80.9/0.86; Tiny: 79.5/86.4/0.92 (DS/RC/IS). Driving Score (DS), Route Completion (RC, %), Infraction Score (IS); higher is better | BEVDriver full: 48.9/59.7/0.82; SAMoE-VLA reports 3.6B total parameters versus BEVDriver's 7B label. Best listed DS and RC across all three settings. Full-setting IS is not best. The caption defines IS as Infraction Score; p. 10 calls it Instruction Score. e12 |
| Scene routing and MoE ablations nuScenes; Tables 3–4. | SAMoE: 0.29/0.26; prefix routing: 0.31/0.37; without DCN: 0.31/0.33; without near-field map: 0.30/0.29. Average L2 (m) / collision (%) | Dense without MoE: 0.32/0.25; sparse MoE: 0.30/0.36; SoftMoE: 0.46/0.57. Scene routing improves accuracy over these alternatives, but does not improve collision over the dense baseline. The prose's 38.4% sparse collision increase is inconsistent with Table 4's dense comparator. e13e14 |
| Attention and world-pretraining ablation Figure 8 and accompanying text; all variants omit SA-MoE. | Baseline 0.32; full attention 0.35; fully dense 0.35; without world modeling 0.33. Average L2 error (m) | Comparisons use the non-MoE baseline, not the 0.29 m final model. Supports modest benefits of the mask, functional specialization and world pretraining in this setup; does not establish inference-time world rollout planning. e15 |
| Single-GPU module efficiency Appendix I; FP16 module benchmark, approximately 276M-parameter MoE blocks. | SA-MoE 43.89 ms; DSE separately 2.883 ms. Average inference latency | Sparse MoE 46.17 ms; reported block speedup 1.05×. Module timing is not complete driving-stack latency. The benchmark GPU model and full timing configuration are unspecified; Orin appears as deployment discussion, not an identified measured platform. e18 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 3. BEV routing, deformable sampling and the near-field prior each contribute to the reported averages. Original paper, p. 13 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each row removes or replaces part of the scene router while retaining the planning task. Prefix Routing bases the merge on the first planning token instead of the DSE scene summary. The next two rows remove deformable convolution or the near-field map. Compare the average columns first: the complete model reports 0.29 m L2 and 0.26% collision, versus 0.31/0.37 for prefix routing and 0.31/0.33 without DCN. Removing only the distance prior gives 0.30/0.29. Then inspect individual horizons: the complete model is not best in every cell, and the strongest separation is often at the three-second horizon. e13
What it supports. Within these reported controls, using scene geometry to form the routing signal is associated with better average accuracy and collision performance. The near-field prior contributes a smaller observed change than replacing DSE with prefix routing. The table motivates controlled tests of which scene information the router actually uses.
Where the evidence stops. These single reported values include no uncertainty. Removing a component also changes its available computation or representation, so the table does not by itself isolate a universal causal explanation for each gain.
Table 4. Better L2 accuracy does not mean better collision performance than every baseline. Original paper, p. 29 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the top row as the dense reference, then compare each token-routing family with and without BEV bias. Sparse routing changes average L2/collision from the dense 0.32 m/0.25% to 0.30 m/0.36%; adding BEV bias yields 0.57 m/0.69%. SoftMoE changes from 0.46 m/0.57% without bias to 0.33 m/0.34% with bias. Finally, the scene-merged model reaches 0.29 m/0.26%. Appendix D.2 explains that BEV bias modifies token or slot routing logits; it does not convert these baselines into parameter-merging SAMoE. The table therefore compares more than the mere presence of scene features. e14e17
What it supports. The reported SAMoE configuration achieves the best average L2 in this comparison and avoids the large collision increases of the tested token-MoE variants. However, its 0.26% collision rate is slightly higher than the dense model's 0.25%. The evidence supports a particular accuracy–collision tradeoff, not blanket safety dominance.
Where the evidence stops. The prose's 38.4% sparse collision increase does not match the displayed dense comparator. The theoretical local-information assumption also cannot establish failure of all token routers, especially ones receiving global context.
7. Analysis & limitations
7.1 What the evidence leaves open
Appendix A uses a Beta time distribution and noise-minus-action target; Algorithm 5 uses uniform time and action-minus-noise. Section 4.1 clamps Beta samples, whereas Appendix A rescales them. Its broad full-parameter fine-tuning wording also requires reconciliation with the frozen world expert. e07e08e09e10
Appendix I.5 writes an output-weighted expert sum, conflicting with the parameter merging in Section 3.3 and Appendix C. Appendix E itself acknowledges that nonlinear parameter fusion generally differs from output mixing. e19
Challenging subsets contain only 38, 27 and 42 cases. Appendix G calls their trajectories closed-loop without specifying a nuScenes rollout environment. Figure 5's intersection baseline success is 51.09%, whereas p. 13 says 67.91%; those success comparisons remain ambiguous. e15e16
The theoretical guarantees need scrutiny: Eq. (53) replaces the squared norm of an integrated Jacobian action with an integral of squared norms; Eq. (118) reverses the general triangle inequality. The local-router assumption excludes global context. These arguments do not establish universal failure of token MoE. e17
The main result tables provide no repeated-seed uncertainty. Simulator and logged-data results do not establish physical-road safety, and selected qualitative overlays are not a deployment trial. e11e12e20
7.2 Questions for discussion
- Does correct scene routing outperform an equally sized constant or shuffled expert mixture?
- How much of the world-pretraining benefit survives matched training compute and data?
- Which flow-target convention and controller actually produced the reported simulator scores?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the stated pretrained modules, nuScenes, NuInteract and OmniDrive-nuScenes annotations, and LMDrive/LangAuto for simulator evaluation. The PDF says code will be released soon; release availability is unverified. Optimizer details, training duration, exact main evaluation split, controller and simulator configuration require clarification. e01e07e08e09e12
Reader-proposed checks: first reconcile velocity-target sign and time sampling against the reverse Euler solver; then hold expert capacity and training data fixed while comparing true scene routing with shuffled scene routing and a constant mixture. These are proposed tests, not reproduced results. e04e09e10e14
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Resolve the target sign before training
Reader-proposed check, not executed: use a fixed demonstrated trajectory and sampled Gaussian endpoint to construct Appendix A's interpolation. Substitute its exact constant target velocity into Algorithm 2's negative-step Euler solver and verify that integration returns the demonstrated endpoint. Repeat with Algorithm 5's opposite target as a control; retaining the same solver should fail the endpoint check. Then compare the documented clamped-Beta, rescaled-Beta and uniform time samplers in an otherwise matched small training run, logging endpoint error and flow loss separately. This distinguishes a sign error from a sampling choice before evaluating driving scores. e08e09e10
Check 2: Test whether scene correspondence causes the routing gain
Reader-proposed check, not executed: keep the pretrained context, expert count, initialization, mask, training budget and evaluation cases fixed. Compare learned DSE routing with a constant learned mixture and a version whose scene summaries are shuffled among samples before the per-layer router. Add an output-mixture control using the same experts to distinguish routing information from fusion location. Report per-horizon L2 and collision, paired scene-level uncertainty and router-weight variation. Correct scene correspondence should outperform shuffled or constant routing if the claimed adaptation is essential; equal performance would weaken that interpretation even if all MoE variants beat the dense planner. e04e05e13e14e19
8.3 Reading coverage
Visual audit: Visually inspected the title and author page, Figures 1–17, Tables 1–5, and the declared method, training, flow, proof and deployment pages supporting retained claims. All six final original crops were individually inspected; Table 1 was recropped to retain its metric headers. Diagram arrows, stage labels and fusion location were checked against the caption, equations and algorithms. All 64 pages were read through the 11 supplied text chunks; bibliography pages, remaining theory pages and notation Tables 6–8 were not part of the image pass. Separate supplemental availability is not fully verified. Code and experiments were not inspected or run.
PDF pages inspected for this edition: 1, 2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 36, 42, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; Sections 1–2: Introduction and Related Work
- Sections 3.1–3.4: Architecture, CMCA, scene-adaptive MoE, training
- Sections 4.1–4.3 and 5: Experiments, ablations, conclusion
- Bibliography
- Appendices A–D: Flow matching, world modeling, MoE comparisons, training details
- Appendix E.1–E.5: Representation, temporal stability, convergence and gradient analysis
- Appendix F and Algorithms 1–5
- Appendices G–I: Challenging subsets, qualitative cases, computational analysis
- Appendix J: Notation Tables 6–8
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All 11 supplied text chunks were read, covering PDF pages 1–64. Visual inspection is recorded separately in the illustrated edition; not every textual page was rendered.
- Identity notes: PDF p. 1 identifies arXiv:2603.08113v1 [cs.CV], 9 March 2026. The exact title and all seven authors match the supplied catalog. No different revision or edition was supplied or compared. Supplementary pagination restarts at 1 on PDF p. 21; all report page references use PDF page numbers.
- Code and external references were not inspected; experiments were not reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title block, arXiv margin and abstract
Exact catalog title and seven authors are present; arXiv:2603.08113v1 [cs.CV], 9 March 2026. Six affiliations are listed. The abstract motivates scene-level routing and says code will be released soon.
Go to primary source ↓e02PDF p. 2, Figure 1 and caption; PDF p. 5, Section 3.1
Two functional experts consume world/language/BEV versus state/action tokens and share CMCA. Figure 1 marks scene description/rendering as pretraining-only and trajectory generation as final-stage-only.
Go to primary source ↓e03PDF pp. 5–6, Section 3.2, Eqs. (1)–(2)
The binary mask permits all conditioning keys and action keys j ≤ i; conditioning cannot attend to action tokens. Masking precedes softmax over attention scores.
Go to primary source ↓e04PDF p. 6, Figure 2 and caption; PDF pp. 7–8, Section 3.3, Eqs. (3)–(8)
Near-field map and BEV features predict offsets; deformable features become keys/values for learned routing queries. H_BEV is computed once, with distinct linear routing heads per layer.
Go to primary source ↓e05PDF p. 8, Eqs. (9)–(11); PDF p. 28, Appendix C.3–C.4
Planning attention output plus residual enters one FFN formed by softmax-weighted sums of three expert weight matrices. Appendix C distinguishes parameter fusion from activation mixing.
Go to primary source ↓e06PDF pp. 23–26, Appendix B.1–B.6, Eqs. (13)–(37)
LLM-conditioned world prompts generate future BEV representations, which are lifted into voxels and rendered through SDF-based LiDAR rays. Losses cover depth, Chamfer geometry, rendering and optional language supervision.
Go to primary source ↓e07PDF p. 9, Section 3.4; PDF p. 23, Appendix B overview; PDF p. 29, Appendix D.1
Pretraining freezes planning and progresses through perception, text alignment and joint understanding/generation. NuInteract masking yields approximately 200K pairs; OmniDrive-nuScenes provides language annotations. Final training uses dense initialization followed by copied MoE experts; Appendix B says the world-language expert is frozen.
Go to primary source ↓e08PDF pp. 9–10, Section 4.1
OpenCLIP-ConvNext/CPFPN and BEVFormer supply 200 × 200 × 256 single-frame BEV features. Training uses eight NVIDIA A800 GPUs; planning initializes from InternVL2-2B. MoE replaces every fourth FFN; final learning rate is 1e-4 and inference uses ten ODE steps.
Go to primary source ↓e09PDF pp. 21–23, Appendix A.1–A.4, Eq. (12), Algorithms 1–2
Six two-dimensional nuScenes actions are interpolated as t epsilon + (1−t)a, with target epsilon−a. Conditioning includes BEV, language, world and ego context. Gaussian initialization and negative Euler increments yield a trajectory, without a specified actuator controller.
Go to primary source ↓e10PDF p. 10, Section 4.1; PDF p. 21, Appendix A.2; PDF p. 51, Algorithm 5, lines 7 and 20–22
Section 4.1 clamps Beta samples, Appendix A rescales Beta samples, and Algorithm 5 instead samples uniformly. Algorithm 5's velocity target is action−noise, opposite Appendix A; its update is explicitly to the planning expert.
Go to primary source ↓e11PDF p. 11, Table 1, all rows/columns; PDF p. 10, Section 4.2
SAMoE-VLA reports 0.24/0.27/0.35 m L2 and 0.01/0.18/0.60% collision, averages 0.29 m and 0.26%. PreWorld averages 0.31 m; Drive-WM also averages 0.26% collision. Table 1 lacks uncertainty and explicit split details; prose comparisons are not all consistent with its entries.
Go to primary source ↓e12PDF p. 9, Section 4.1; PDF p. 10, Section 4.2; PDF p. 12, Table 2 and caption
LangAuto is CARLA-based closed-loop evaluation. Table 2 reports SAMoE DS/RC/IS of 51.4/63.5/0.81, 69.5/80.9/0.86 and 79.5/86.4/0.92. Full BEVDriver scores are 48.9/59.7/0.82. IS is Infraction Score in the caption but Instruction Score in p. 10 prose.
Go to primary source ↓e13PDF pp. 11–12, Section 4.3 router discussion; PDF p. 13, Table 3
Prefix routing uses the first planning token. Average L2/collision are 0.31/0.37 for prefix, 0.31/0.33 without DCN, 0.30/0.29 without M_near, and 0.29/0.26 for the proposed router.
Go to primary source ↓e14PDF p. 29, Table 4; PDF p. 30, Appendix D.2; PDF p. 11, Section 4.3
Dense, sparse, BEV-biased sparse, SoftMoE, BEV-biased SoftMoE and SAMoE average L2/collision pairs are respectively 0.32/0.25, 0.30/0.36, 0.57/0.69, 0.46/0.57, 0.33/0.34 and 0.29/0.26. D.2 describes the BEV-biased baselines. The prose's 38.4% increase does not match its dense table comparator.
Go to primary source ↓e15PDF pp. 13–14, Section 4.3; PDF p. 14, Figures 5–6; PDF p. 15, Figure 8
CMCA/world ablations exclude SA-MoE; Figure 8 L2 bars are 0.35, 0.35, 0.33 and 0.32 for full attention, fully dense, without world modeling and baseline. Figure 5 prints intersection success 77.41 versus 51.09, while p. 13 gives baseline 67.91.
Go to primary source ↓e16PDF pp. 52–53, Appendix G.1–G.3 and Table 5
Thresholds are average agent count >40, yaw rate >0.05 rad/s and minimum inter-agent distance <8 m. Selected counts are 38/27/42 from a stated 34,149-sample split. Success requires L2 <0.4 m and zero collisions. G.3 uses closed-loop wording without identifying a nuScenes simulation environment.
Go to primary source ↓e17PDF pp. 32–34, Appendix E.1, Eqs. (49)–(59); PDF p. 36, local-routing assumption; PDF p. 42, Eq. (118)
The proof assumes upper Lipschitz continuity and a Jacobian lower bound. Eq. (53) equates a squared integrated output difference to an integrated quadratic form. Eq. (118) lower-bounds a norm of an integral by the integral of norms. Local token routing is assumed to have no global or long-range context.
Go to primary source ↓e18PDF pp. 55–61, Appendix I.1–I.5, especially pp. 56–58
Single-GPU FP16 block timings are 46.17 ms sparse versus 43.89 ms SA-MoE, approximately 276M parameters. DSE is separately 23.07M parameters and 2.883 ms. The text notes higher transient SA-MoE CUDA peak memory. The measured GPU model and complete timing protocol are unspecified; Orin is discussed illustratively.
Go to primary source ↓e19PDF p. 8, Eqs. (10)–(11); PDF p. 28, Appendix C.3–C.4; PDF p. 34, Eq. (58); PDF p. 61, Appendix I.5 unnumbered expert-mixture equation
The main method and Appendix C specify parameter merging. Appendix E defines its nonlinear residual relative to output mixing, while I.5 instead writes an output-weighted expert sum.
Go to primary source ↓e20PDF pp. 54–57, Figures 10–17 and captions
Selected multi-camera/BEV trajectory overlays illustrate intersections, turns and lane changes; Figures 16–17 compare MoE with non-MoE predictions. These are qualitative examples, not a physical deployment evaluation.
Go to primary source ↓8.5 Primary sources
SAMoE-VLA: A Scene Adaptive Mixture-of-Experts Vision-Language-Action Model for Autonomous Driving ↗
PDF · 18,260 extracted words
Source fingerprint
30b9a978b5dc7eb3b389b8889fd23e7c3215e2f39ad4903d7abcd7958bff8348