WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigation
1. Paper overview
In one sentence: WorldVLN decodes predicted latent futures into drone waypoints and refreshes its history after execution, improving short-range navigation while retaining a large server-side backbone. e-probleme-loope-sfte-grpoe-indoore-reale-limits
| At a glance | What to know |
|---|---|
| Research problem | Author claim Aerial VLN couples language understanding to viewpoint-dependent 3D motion. The authors argue that visual recognition priors alone inadequately represent movement consequences. Their concrete task maps partial egocentric observation history and an instruction to relative displacement/yaw actions, with new observations arriving after execution. e-problem |
| Core mechanism | |
| A key reported result | IndoorUAV-VLA short-horizon navigation: Full-set SR 41.76, NDTW 13.48; Hard SR 41.19. Success rate and NDTW (%). Full test set and Hard difficulty split; short sub-trajectories composed from local action types. pi0: full-set SR 27.16, NDTW 9.44; Hard SR 7.55. SR gains are 14.60 and 33.64 percentage points. The largest SR gain concerns local action composition. Easy NDTW declines from pi0's 14.52 to 13.04, so trajectory quality does not improve uniformly. e-indoore-protocol |
| Reading caution | Source description Long-horizon instructions, large-scale exploration, adverse weather, dynamic obstacles and broader real-world robustness remain insufficiently evaluated. Extensive multi-seed testing is absent; server-side inference remains necessary. e-limits |
Core contributions
- Author claim
Two-stage training combines navigation-video supervision, teacher-assisted latent action decoding, and online grouped rollout optimization. The claimed priority as the first aerial autoregressive WAM/GRPO method is an author claim, not independently established here. e-sfte-grpoe-priority
Figure 1. A predicted future becomes a waypoint segment, and the environment supplies the next context. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each vertical column from bottom to top: observed images enter the backbone, a colored latent block represents a predicted world transition, and the action decoder produces a waypoint segment. Then follow the blue environment arrows to understand how executing that segment yields the next real observation. The columns depict successive decision times. The dashed Video Decoder branches show that the latent can also be visualized; Section 4.1 and Appendix A.4 specify that this RGB branch is bypassed for action generation. Equations (3)–(6) make the crucial replacement explicit: subsequent context uses encoded observations from execution, rather than carrying the previous predicted segment forward as if it were observed. e-loope-sfte-decoder
What it supports. The world model is used at inference, and its output directly determines actions through a separate learned decoder. Feedback occurs after each segment, making the policy responsive to execution outcomes. This is the architectural basis for the report's Dual-system × IDM assessment, rather than an inference from joint training alone.
Where the evidence stops. The diagram does not establish calibrated physical prediction or an action-search planner. Its latent future is instruction-conditioned; the equations do not enumerate candidate controls or rank multiple predicted trajectories at inference.
2. Motivation
2.1 The problem and the proposed response
Aerial VLN couples language understanding to viewpoint-dependent 3D motion. The authors argue that visual recognition priors alone inadequately represent movement consequences. Their concrete task maps partial egocentric observation history and an instruction to relative displacement/yaw actions, with new observations arriving after execution. e-problem
2.2 What this reading follows
Following an aerial instruction requires more than recognizing a landmark: movement changes what the camera can see next. WorldVLN makes a video model's predicted state transition the input to a motion decoder, then corrects the next prediction with observations from actual execution. This reading follows that interface through supervised training, online reinforcement learning and the original benchmark tables. The strongest result is improved short-horizon simulated navigation, especially composed indoor actions. The real-flight images establish a narrower point: simulation-trained waypoint predictions can drive a physical platform with external stabilization and server-side inference. Neither attractive predicted frames nor selected successful flights establish general reliability. e-probleme-loope-sfte-grpoe-indoore-reale-limits
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | IDM |
| Quadrant | Q4 · Dual-system × IDM |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded Dual-system × IDM classification is supported by a world-prediction backbone feeding a distinct motion decoder trained to recover actions from visual/latent transitions. This is an inverse-dynamics-style latent-to-action route, not joint future/action token prediction in one backbone. Online GRPO does not remove the architectural separation. The representation is latent rather than an explicit reconstructed 3D world. e-loope-backbonee-decodere-sft
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 A latent future is an intermediate action representation
The useful unit is a short state transition. The backbone receives instruction tokens and actual observation history, predicts future token pyramids, and merges them into a latent segment. The motion decoder then converts that segment into relative translation and yaw. Its temporal attention can use changes across the latent frames, while spatial attention models within-frame structure; the regression head produces the executable waypoint representation. RGB rendering is available for visual supervision and probes but is absent from the navigation action path. After execution, newly encoded observations become the context for the next prediction. This separates prediction from reality at an explicit interface: a predicted segment proposes motion, while the environment determines what the policy conditions on next. Reader assessment: the architecture supports an inverse-dynamics-style decoder classification, even though the decoder operates on compressed future transitions rather than rendered frame pairs. e-loope-backbonee-decodere-sft
Figure 2. Supervision first connects video dynamics to actions; grouped online rollouts then optimize navigation consequences. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the two upper branches. The left branch trains instruction-conditioned future prediction while the snowflake marks the frozen video decoder; the right branch trains action recovery with a frozen video encoder. Appendix A.4 expands the right branch into teacher training, latent-to-token alignment and full decoder supervision. Below, the rollout rows accumulate actual observations as decisions progress; G denotes repeated rollouts of the same navigation case. The three reward boxes compare local trajectories, terminal positions and reference-policy alignment. Their scores return through the GRPO arrow to update the policy. Read the decay plot as the authors' intended emphasis on earlier segments, whose actual effect depends on the subsequent normalization. e-sfte-loope-grpoe-rewards
What it supports. The training design addresses two distinct requirements: a predicted latent must retain useful future visual structure, and its decoded motion must achieve the instruction. In the reported implementation, task reward has weight 0.7, trajectory consistency 0.2 and reference alignment 0.1. These are training signals, not additional sensors supplied to the deployed policy.
Where the evidence stops. Figure 2 labels the decoder output $a_{t+1:t+K}$, whereas Eqs. (4) and (8) associate that future observation segment with $a_{t:t+K-1}$. This source indexing discrepancy is preserved; the report follows the equations. The effect of decay after normalization is discussed below.
5.2 Why the action decoder needs its own grounding
A pretrained video's latent space does not automatically provide a reliable coordinate system for drone actions. The source addresses this by first training a video-to-action teacher from observed clips and expert trajectories. With the video encoder frozen, the teacher's embedding tokens supervise the latent decoder's vision embedding, after which the complete action decoder learns the expert action mapping. Meanwhile, the world backbone learns future prediction from ground-truth history using an initial frame followed by three 16-frame segments. Reader interpretation: these stages establish a shared interface between visual dynamics and motion recovery, but they leave a distribution bridge to cross, because the decoder initially sees ground-truth latents and later receives predicted ones. Online rollouts expose the composed system to its own decisions and their observations. The reported ablation supports an additional GRPO benefit, without separately measuring how much comes from repairing this latent mismatch. e-sfte-grpoe-ablation
5.3 Read the reward equations before accepting the credit-assignment story
The trajectory term rewards local agreement with expert actions, while the main task equation uses the rollout's final distance to the target. Thus the terminal task signal is shared by a rollout's segments; it is not an independently observed progress reward at each waypoint. Reference alignment is intended to preserve the supervised world prior. The authors multiply the combined reward by temporal decay before normalizing across rollouts separately at each segment. Reader analysis: if the group statistics are the usual mean and standard deviation, multiplying every member of a segment's group by the same positive factor cancels during normalization, apart from the stabilizing epsilon. Early segments therefore need not receive the stronger advantage signal suggested by the narrative. The source provides no isolated decay ablation and does not explain a later loss weighting that would resolve this. The overall GRPO result remains reported evidence, while this specific causal explanation requires verification. e-grpoe-rewardse-ablation
5.4 Training and inference
During training
Stage 1 trains the world backbone on instruction/video pairs using ground-truth history. Each sample has 49 frames: an initial frame and three K=16 future clips, each compressed to four latent temporal steps. The frozen video decoder remains a training-time visual interface. e-sft
A TSformer-VO-style video-to-action teacher first learns expert actions from observed videos. Its token representations supervise latent-to-token alignment using a frozen video encoder; the entire action decoder then trains on ground-truth latents and expert actions. Thus its initial input distribution differs from predicted inference latents. e-sft
Stage 2 samples G=4 simulator rollouts with actual feedback. Rewards combine trajectory consistency, terminal goal distance and reference alignment with weights 0.2/0.7/0.1. Trajectory distance uses translation/yaw/full-action MSE weights 0.45/0.45/0.1. The method applies gamma=0.9 before per-segment group normalization and clipped policy updates. e-grpoe-rewards
During inference
Instruction and observation history produce a latent future, then waypoint actions; the video decoder is bypassed. Real feedback supplies subsequent context. The problem formulation terminates at stop or a maximum horizon, but does not specify their implementation. e-loope-sfte-gaps
In physical deployment the server runs WorldVLN, the Orin forwards data, and PX4 handles low-level flight. Motion capture or GPS/LiDAR supports execution and recording; the policy itself receives RGB and language only. e-real
5.5 Implementation flow
- Encode observed history
The video VAE compresses actual observations; multiscale residual quantization creates image or historical-clip token pyramids. Instruction tokens condition the same spatiotemporal Transformer. e-backbone
- Predict and decode motion
The backbone predicts future token blocks coarse-to-fine and aggregates them into latents. A separate decoder reshapes, convolves, upsamples and projects these latents, applies temporal/spatial attention, and regresses continuous actions. e-backbonee-decoder
- Close the environmental loop
Execute the waypoint segment, encode the resulting real observation segment, and replace predicted context with those observed latents. This corrects the context at segment boundaries; it does not demonstrate exact physical dynamics or candidate-action search. e-loop
6. Experiments & results
WorldVLN turns an autoregressive video prior into a closed-loop aerial navigation policy: predict a short latent future, decode waypoint actions, execute them, and condition the next prediction on actual observations. Supervised grounding and online Action-aware GRPO improve reported outdoor and indoor success. Evidence is strongest for short-range simulated control; physical transfer is qualitative and server-assisted.
6.1 Read the original evidence
Table 1. Outdoor aggregate success improves, with substantial remaining differences across flight skills. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare models within one instruction condition before moving between Fixed and Open rows. The rightmost Average column gives the headline result: WorldVLN reaches 79.12% with fixed templates and 78.02% with open wording. The strongest baseline differs by condition, so a single baseline row cannot supply both comparisons. Then scan horizontally to see where the gains come from. Approach, Land and Move are strong WorldVLN columns, while Pass, Turn, Rotate and Surround leave clear room for improvement. A/D denotes ascending/descending in the benchmark description. Treat the displayed Average as the paper's reported statistic; the table does not explain its category weighting or supply category denominators. e-outdoore-protocole-limits
What it supports. Against the strongest corresponding baseline averages, WorldVLN gains 13.51 percentage points in Fixed and 12.24 in Open. However, Open Surround reaches only 41.67%, compared with 100% for OpenVLA-UAV. Thus the result supports higher overall benchmark success, with substantial skill-specific exceptions rather than uniform superiority.
Where the evidence stops. These are simulated test results under the benchmark's existing protocol. The table has no uncertainty intervals or per-category sample counts, and Appendix A.2 acknowledges limited multi-seed evaluation. Open instructions do not establish unrestricted language generalization.
Table 2. Composed indoor actions show the largest success-rate advantage. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read SR and NDTW as different outcomes: reaching the instructed terminal pose and matching a reference trajectory. Appendix A.5 says the NDTW evaluation incorporates both 3D position and yaw changes. Move across Easy, Medium and Hard to inspect increasing composition complexity; Figure 9 illustrates instructions combining one, two or three action types. The rightmost Average refers to the full test set, not a new fourth difficulty. Compare WorldVLN with pi0, the strongest baseline by full-set SR. The largest separation occurs on Hard, while the Easy NDTW column provides a useful counterexample to any claim that every aspect of navigation improves. e-indoore-protocole-limits
What it supports. WorldVLN raises full-set SR from 27.16% to 41.76% and NDTW from 9.44% to 13.48%. Hard SR increases from 7.55% to 41.19%, a 33.64-point gain. This supports improved local action composition; Easy NDTW nevertheless falls from 14.52% to 13.04%.
Where the evidence stops. Hard describes short sub-trajectories with more action types, not long-horizon exploration. The source does not report confidence intervals, and the full-set average should not be reconstructed as an unweighted average of the three difficulty columns.
Figure 5. Selected real flights demonstrate executable transfer with server inference and external stabilization. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow each instruction into its third-person flight composite, then inspect the egocentric observation sequence beneath it. The green-bordered initial view and blue-bordered final view correspond to colored markers in the flight illustration; intervening images sample the camera's changing perspective. The indoor example concerns an object-relative chair maneuver, while the outdoor example requires vertical motion toward a roof. These images show physical execution, but the deployment description is needed to understand what supports it. Figure 10 and Appendix A.9 place model inference on the ground station, communication on the Orin, and low-level control on PX4. Motion capture indoors and GPS/LiDAR outdoors support flight execution rather than policy perception. e-reale-limits
What it supports. The examples support the narrower claim that a policy trained in simulation can generate waypoints usable on this physical platform without additional real-world fine-tuning. The high-level model uses RGB and language; external state estimation remains part of the execution system. This is evidence of limited transfer, not fully onboard autonomous navigation.
Where the evidence stops. The figure supplies selected cases without a real-flight success-rate table, trial count or baseline comparison. Controlled arenas and enclosed outdoor conditions leave robustness to adverse weather, dynamic obstacles and communication delays unresolved.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| UAV-Flow-Sim navigation Test set; Fixed and Open instructions evaluated separately under the benchmark protocol. | Fixed 79.12; Open 78.02. Reported Average success rate (%) | Strongest corresponding baselines: OpenVLA-UAV 65.61 Fixed; pi0-0-UAV 65.78 Open. Gains: 13.51 and 12.24 percentage points. Broad aggregate gains do not imply every skill improves: Open Surround is 41.67%, versus 100% for OpenVLA-UAV. The reported Average is preserved without assuming a macro-average. e-outdoore-protocol |
| IndoorUAV-VLA short-horizon navigation Full test set and Hard difficulty split; short sub-trajectories composed from local action types. | Full-set SR 41.76, NDTW 13.48; Hard SR 41.19. Success rate and NDTW (%) | pi0: full-set SR 27.16, NDTW 9.44; Hard SR 7.55. SR gains are 14.60 and 33.64 percentage points. The largest SR gain concerns local action composition. Easy NDTW declines from pi0's 14.52 to 13.04, so trajectory quality does not improve uniformly. e-indoore-protocol |
| Autoregression and GRPO ablation Figure 4(b), UAV-Flow and IndoorUAV; full-sequence versus autoregressive SFT uses the same backbone and decoder. | UAV-Flow: 60.43 → 68.13 → 79.12. IndoorUAV: 31.49 → 37.19 → 41.76. Success rate (%) | Order: full-sequence SFT, autoregressive SFT, autoregressive SFT+GRPO. Feedback gains are 7.70/5.70 points; subsequent GRPO gains are 10.99/4.57 points. Both interventions help in this sequence of comparisons. There is no full-sequence+GRPO cell or componentwise reward ablation; feedback and training exposure are not independently isolated. e-ablation |
| Zero-shot physical drone navigation Simulation-trained policy; representative controlled indoor chair maneuver and outdoor roof approach. | Two illustrated real-world cases; no aggregate physical SR reported. Qualitative executed trajectories | No real-world baseline table or repeated-trial uncertainty is supplied. Supports waypoint executability and limited transfer, without establishing general deployment reliability or onboard inference. e-reale-limits |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 4(a–d). Feedback and GRPO both help, while the visual probes remain qualitative. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with panel b and its two configuration rows: the first group lacks autoregression and GRPO, the second enables autoregression, and the third enables both. Purple bars are UAV-Flow and yellow bars IndoorUAV. Panel a follows training steps rather than wall-clock cost; its dashed purple continuation denotes the GRPO stage. In panel c, the dashed arrows to RGB images are a diagnostic video-decoding branch, while the action-to-observation loop distinguishes autoregressive updating. Panel d compares the intended circular maneuver before and after reinforcement learning against the ground-truth row. Read the image panels as selected mechanism probes, then return to panel b for the quantitative evidence. e-ablatione-probese-learning
What it supports. Autoregressive SFT improves UAV-Flow SR from 60.43% to 68.13% and IndoorUAV from 31.49% to 37.19%. GRPO then raises them to 79.12% and 41.76%. The second increase is 10.99 points outdoors and 4.57 indoors, so the paper's over-ten-point statement should not be generalized to both benchmarks.
Where the evidence stops. There is no full-sequence+GRPO cell, reward-component ablation or latent-fidelity metric. Panel a preserves each model family's training configuration and does not equalize compute or pretraining. Visually coherent predictions alone are not evidence of accurate executed trajectories.
7. Analysis & limitations
7.1 What the evidence leaves open
Long-horizon instructions, large-scale exploration, adverse weather, dynamic obstacles and broader real-world robustness remain insufficiently evaluated. Extensive multi-seed testing is absent; server-side inference remains necessary. e-limits
Training-step efficiency is not compute-matched: WorldVLN and OpenVLA share data/protocols, but differ in priors, learning rates and training budgets. Figure 4(a)'s learning curve is a separate experiment from Table 1's baseline results. e-learninge-outdoor
Under the usual mean/standard-deviation interpretation of group normalization, a common positive gamma^(j−1) cancels from the normalized advantage except through epsilon. Thus the stated early-decision emphasis needs implementation clarification; the ablation does not isolate decay. e-grpoe-rewardse-ablation
7.2 Questions for discussion
- How much of the autoregressive gain comes from refreshed observations versus the training distribution induced by segment-wise supervision?
- Does temporal decay change the normalized learning signal, and how are action likelihoods implemented?
8. Reproducibility audit
8.1 Requirements and known gaps
Required assets include InfinityStar-8B, Wan VAE, decoder-teacher priors and both benchmark trajectory/video datasets. Reported hardware is eight A800 80GB GPUs plus RTX 4090 simulator rollouts. WorldVLN SFT uses AdamW/bf16, learning rate 1e-5, at most 26K steps and approximately eight packed clips per step. e-confige-learning
GRPO specifies at most 3000 iterations, learning rate 8e-7, KL coefficient 0.9, ratio clip 0.02, gradient clip 0.5, video batch size 1 and token limit 20480; the first five backbone chunks are frozen. e-config
Before reproducing, resolve action-likelihood computation through discrete latent sampling and continuous regression, off-expert action alignment, reference-cost sign, dense/binary task-reward composition and the undefined trajectory-level gate. Dataset counts, benchmark thresholds, software versions and stop details are not restated; inference latency is unreported. e-gapse-rewardse-real
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Hold the policy fixed and change only its observation refresh
Reader-proposed check: take one supervised autoregressive checkpoint and evaluate the same held-out UAV-Flow-Sim and IndoorUAV-VLA cases with paired random seeds. In one branch, replace each predicted latent segment with encoded observations after executing its waypoints; in the control, retain predicted history while still executing the same segment protocol. Keep decoder, segment length, action normalization and evaluation rules fixed. Record SR, endpoint error and per-segment trajectory divergence, with repeated seeds. A widening control-branch drift and a reproducible SR gap would support the feedback mechanism independently of separately trained full-sequence models. No gap would weaken that attribution. This check has not been run. e-loope-sfte-ablatione-protocole-learning
Check 2: Check whether temporal decay survives group normalization
Reader-proposed check: freeze one batch of grouped rollouts and its three raw reward components. Recompute advantages with gamma=1 and gamma=0.9 using the stated per-segment normalization and a fixed epsilon; compare advantage vectors and gradient contributions by segment. Add a diagnostic variant applying decay after normalization, while holding policy likelihoods, clipping and reference regularization fixed. Near-identical advantages for the first pair, but differences for the diagnostic variant, would contradict a strong early-weighting effect from reward scaling alone. Resolve the undocumented action-likelihood implementation before extending this to training and paired-seed SR comparisons. This is a proposed consistency test, not a reproduced result. e-grpoe-rewardse-confige-gaps
8.3 Reading coverage
Visual audit: The title/author/version page, all method and result pages, and all appendix pages were rendered and visually inspected. Figures 1–10 and Tables 1–2 were read visually; six final original crops were separately inspected. Appendix pages cover architecture, training configuration, evaluation context and real-platform details supporting both reports and proposed checks. Arrow directions and frozen-module markers were cross-checked against Eqs. (3)–(13) and Appendix A.4; the Figure 2 action-index discrepancy is disclosed. All seven text chunks, including references on pages 11–14, were read. Those reference-only pages were not visually inspected. Separate supplements, linked code and demonstrations remain outside the supplied-source review. Fine trajectory tick values and small token indices were not used to infer numerical results.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 15, 16, 17, 18, 19, 20, 21, 22, 23. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1 Introduction
- 2 Related work
- 3 Problem formulation
- 4 Method, including 4.1 and 4.2.1–4.2.2
- 5 Experiments, including 5.1–5.4
- 6 Conclusions, Limitations, and Future Works
- References
- Appendix A.1–A.9
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction's figure limitation was resolved by visually inspecting the supplied PDF: Figures 1–10 and Tables 1–2. All seven text chunks covering all 23 pages were read; reference-only pages 11–14 were not rendered.
- Identity/edition: the inspected title page identifies arXiv:2605.15964v1, 15 May 2026. The exact title and all 16 authors agree with the supplied identity. The five observed affiliations are Tsinghua University, Shandong University, Manifold AI, Beijing Institute of Technology and Northeastern University. No other revision or edition was supplied or compared.
- Code, linked project materials and external benchmark documentation were not inspected. No experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author/affiliation block and arXiv margin stamp
The title is WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigation. The title page credits 16 authors and identifies arXiv:2605.15964v1 [cs.RO], 15 May 2026. Its five affiliations are Tsinghua University, Shandong University, Manifold AI, Beijing Institute of Technology and Northeastern University.
Go to primary source ↓e-problemPDF pp. 2–3, Introduction and Section 3, Eqs. (1)–(2)
The task is partially observed instruction-following navigation. Actions are relative 3D translation and yaw, executed to obtain the next observation; termination uses a stop decision or maximum horizon.
Go to primary source ↓e-loopPDF p. 4, Figure 1 and Section 4.1, Eqs. (3)–(6)
The instruction-conditioned backbone predicts a latent segment, a separate decoder maps it to waypoint actions, and actual observations replace the predicted latents in subsequent context. The RGB video-decoding branch is not required for action generation.
Go to primary source ↓e-backbonePDF pp. 15–16, Appendix A.3 and Figure 6
A video VAE and multiscale residual quantization form visual token pyramids. The Transformer predicts coarse-to-fine target clip pyramids and aggregates them into future latents.
Go to primary source ↓e-decoderPDF pp. 16–17, Appendix A.3 and Figure 7
The action decoder uses reshaping, convolution, upsampling and projection, followed by factorized temporal/spatial attention and an MLP regression head for continuous displacement and yaw.
Go to primary source ↓e-sftPDF p. 5, Figure 2 and Eqs. (7)–(8); pp. 17–18, Appendix A.4, supervised-training paragraphs
Backbone supervision uses ground-truth history; samples have 49 frames, comprising an initial frame and three 16-frame clips, each compressed to four latent temporal steps. Decoder training uses a video-to-action teacher, latent-to-token alignment with a frozen video encoder, and final action supervision. Figure 2 shows frozen video modules but labels decoder output a_{t+1:t+K}, whereas Eq. (8) uses a*_{t:t+K-1}.
Go to primary source ↓e-grpoPDF pp. 5–6, Section 4.2.2, Eqs. (9)–(13) and advantage definition
Online grouped rollouts receive trajectory, terminal-distance and reference-log-probability rewards. A common gamma^(j-1) multiplies segment-j rewards before across-rollout normalization for that segment and a clipped likelihood-ratio objective.
Go to primary source ↓e-rewardsPDF pp. 18–19, Appendix A.4, Eqs. (14)–(15), reference reward and temporal-decay paragraphs
G=4; reward weights are 0.2, 0.7 and 0.1. Trajectory distance weights translation/yaw/full-action MSE by 0.45/0.45/0.1. Gamma is 0.9. Task reward discusses dense distance and binary success signals; reference alignment is described as a CE-style cost.
Go to primary source ↓e-configPDF p. 6, Section 5 experimental setup; p. 19, Appendix A.4 optimization settings and A.5 existing assets
The backbone is InfinityStar-8B with Wan VAE and TSformer-VO-style decoder priors. Training uses 8 A800 80GB GPUs and simulator rollouts an RTX 4090 workstation. GRPO lists at most 3000 iterations, learning rate 8e-7, KL coefficient 0.9, ratio clip 0.02, gradient clip 0.5, video batch size 1, token limit 20480, and first five model chunks frozen.
Go to primary source ↓e-protocolPDF p. 19, Appendix A.5; p. 20, Figure 8; p. 21, Figure 9 and Appendices A.6–A.7
UAV-Flow evaluates fixed/open instructions across flight skills. IndoorUAV-VLA segments longer trajectories into local sub-trajectories; difficulty corresponds to one, two or three action types. Indoor metrics are success rate and NDTW incorporating 3D position and yaw.
Go to primary source ↓e-outdoorPDF p. 7, Table 1, Average column and skill columns, Fixed/Open rows
WorldVLN reports Average SR 79.12/78.02%; the strongest corresponding baseline averages are OpenVLA-UAV 65.61% Fixed and pi0-0-UAV 65.78% Open. WorldVLN's Open Surround SR is 41.67%, versus 100% for Travel-UAV and OpenVLA-UAV.
Go to primary source ↓e-indoorPDF p. 7, Table 2, WorldVLN and pi0 rows, Easy/Medium/Hard/Average columns
WorldVLN full-set SR/NDTW are 41.76/13.48%, versus pi0 27.16/9.44%. Hard SR is 41.19 versus 7.55%; Medium SR is 37.72 versus 21.64%. Easy NDTW is 13.04 versus 14.52%.
Go to primary source ↓e-ablationPDF p. 8, Figure 4(b); p. 9, Section 5.3, autoregression and GRPO paragraphs
Full-sequence SFT, autoregressive SFT and autoregressive SFT+GRPO yield UAV-Flow SR 60.43, 68.13, 79.12% and IndoorUAV SR 31.49, 37.19, 41.76%. The authors specify the same backbone and decoder for the full-sequence/autoregressive comparison.
Go to primary source ↓e-probesPDF pp. 7–9, Sections 5.2–5.3, Figure 3 and Figure 4(c–d)
Selected cases illustrate landmark grounding, RGB renderings of predicted latents, and before/after-GRPO circular trajectories. The latent renderings are for visualization only; no quantitative latent-fidelity metric is supplied in these panels.
Go to primary source ↓e-learningPDF p. 8, Figure 4(a) and Section 5.3; pp. 21–22, Appendix A.8
OpenVLA and WorldVLN share UAV-Flow-Sim splits, action normalization and evaluation, with AdamW and bf16. WorldVLN uses learning rate 1e-5 and at most 26K steps; OpenVLA uses 5e-4 and 100K steps. Both use eight GPUs and global batch sizes of eight or approximately eight packed clips. The plotted baseline is a training-dynamics experiment.
Go to primary source ↓e-realPDF p. 9, Figure 5 and Section 5.4; pp. 22–23, Appendix A.9 and Figure 10
Two representative indoor/outdoor zero-shot cases use a 250 mm quadrotor, C270 RGB camera, Orin NX 16GB and PX4. Model inference is server-side. Indoor control uses a 10×15×3 m arena and 14-camera motion capture; outdoor control uses GPS and TFmini-S LiDAR. These pose sensors are not policy inputs.
Go to primary source ↓e-limitsPDF p. 10, Section 6; p. 15, Appendices A.1–A.2
The authors limit conclusions to short-range navigation and controlled real-world settings, acknowledge server dependence and lack of extensive multi-seed experiments, and leave long-horizon tasks and challenging real-world conditions insufficiently evaluated.
Go to primary source ↓e-gapsPDF p. 3, Section 3; p. 6, Eqs. (12)–(13); pp. 18–19, Appendix A.4 and A.5
The text invokes action likelihoods but does not specify how the continuous decoder and sampled latent tokens induce a tractable segment-action likelihood. Benchmark thresholds are deferred to the original protocols; dataset counts, software versions, stop implementation and off-expert action alignment are not specified here. The heading mentioning a trajectory-level gate is not followed by a gate definition.
Go to primary source ↓e-priorityPDF p. 1, Abstract; p. 2, Introduction, contribution bullets
The authors claim the first autoregressive world action model for aerial VLN and the first Action-aware GRPO method tailored to autoregressive WAMs; this report does not independently verify priority.
Go to primary source ↓8.5 Primary sources
WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigation ↗
PDF · 10,768 extracted words
Source fingerprint
8147db2629c1d686bb6ea56af71eb89ce993a6f86e5c528e9b56df742d922ccc