AeroAct: Action-Centered World-Action Models for Language-Conditioned Quadrotor Flight
1. Paper overview
In one sentence: AeroAct uses one Transformer to learn trajectory actions and their visual consequences, then removes future-video prediction for flight; temporal history is strongly supported in simulation, while physical evidence remains a short indoor demonstration. E02E04E05E13E16
| At a glance | What to know |
|---|---|
| Research problem | Source description A moving first-person camera makes goal grounding inseparable from ego-motion and control. A useful policy must infer motion from history and output locally executable trajectories while avoiding the latency of generating future video during flight. E02 |
| Core mechanism | |
| A key reported result | Closed-loop target tracking with temporal history: Nine-frame AeroAct: 100.0 / 0.0 / 3.436. View success / collision (%) / final distance (m). Isaac Lab; 20 saved target trajectories; matched scenes, camera, controller and server interface; 1 Hz policy queries, Table 3. One frame: 20.0 / 90.0 / 7.902. Nine-frame AeroAct-FT: 95.0 / 0.0 / 2.845. History gives an 80 percentage-point view-success gain. View success requires the target in view for at least 80% of a trajectory; it does not require collision-free flight. At 5 Hz, nine-frame AeroAct has 100% view success but 5% collision. E10E13 |
| Reading caution | Reader analysis Physical evidence is a short indoor, motion-capture-assisted target-reaching demonstration, including the instruction to fly to a yellow foam mat. No aggregate flight success rate is supplied. The authors identify offboard inference, limited memory, multistage instructions and recovery behaviors as current limitations; their first-WAM-flight priority claim is not independently established here. E15E16 |
Core contributions
Figure 2. The attention mask makes visual consequence learning compatible with action-only deployment. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the bottom encoders and follow their outputs into the shared DiT bar. Language enters through the left cross-attention arrow. Then read the matrix by query row and key column: Ts denotes state tokens, To reference observations, Ta actions and Tf future observations. Solid cells allow attention; hatched cells block it. The Ta row cannot read Tf, while the Tf row can read every group. This matches Section 3.2 on page 5. Snowflakes mark frozen text/VAE components; flames identify trained components. Finally, follow the action decoder upward and contrast its retained path with the dashed future-video branch disabled for inference. E03E04E05E13E14
What it supports. The architecture provides a verifiable basis for the One Model classification: modalities share a DiT, and action prediction does not require future-image tokens. The visual loss can still shape shared parameters during training. This mechanism explains why removing the future branch at deployment is consistent with the model's information dependencies.
Where the evidence stops. The mask establishes a dependency constraint, not the size of the visual loss's benefit. The reported experiments do not isolate auxiliary video supervision from temporal history and pretrained representations. The mask and prose agree; no graphic correction has been applied.
2. Motivation
2.1 The problem and the proposed response
A moving first-person camera makes goal grounding inseparable from ego-motion and control. A useful policy must infer motion from history and output locally executable trajectories while avoiding the latency of generating future video during flight. E02
2.2 What this reading follows
A quadrotor's camera moves with every action, so recognizing a requested object is only part of the problem: the policy must also infer motion and produce a trajectory the controller can follow. AeroAct addresses this through a video-model backbone trained on paired observations and flight actions. Its crucial design choice is directional information flow. Future-image prediction can learn from actions, but actions cannot read future-image tokens. That lets deployment keep action prediction and discard video generation. Read the architecture alongside the history ablations: the former explains what is possible, while the latter establishes which reported change most clearly improves closed-loop behavior. E02E04E05E13E16
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | One Model |
| Prediction paradigm | Joint prediction |
| Quadrant | Q1 · One Model × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded One Model × Joint prediction placement is supported by a single shared DiT sequence and joint action/visual losses, beyond co-training alone. Visual forecasting is action-conditioned in the same model. At deployment the visual branch is removed: this is direct action denoising with auxiliary visual training, not inverse dynamics from generated video or online imagined-rollout planning. E03E04E05
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Understand what the action decoder actually commands
Start with Appendix A.1 rather than interpreting the action decoder as a motor-command head. The input state contains velocity, acceleration and the body-z direction in a yaw-aligned local frame. Each nine-dimensional action gives a displacement through r, theta and psi, then endpoint velocity and acceleration. Those endpoint conditions, together with the starting state, determine a fifth-order position polynomial for a two-second segment in each spatial coordinate. The negative sign in the vertical displacement formula must be preserved. The policy predicts a chunk of these specifications; the controller subsequently tracks local trajectories. Reader interpretation: this interface places part of the execution structure outside the learned decoder. Smooth segment parameterization alone does not prove obstacle avoidance or guarantee that independently sampled neighboring chunks agree, which motivates the separate self-guidance procedure. E03E09E08E15
Figure 4. The next action chunk inherits an overlapping reference while the previous chunk remains in control. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read left to right along the time axis. Filled triangles mark inference starts and open triangles mark completion; the braces above indicate which result controls the vehicle. Result n still supplies commands while inference n+1 runs. The upward overwrite arrow transfers information from the older chunk into the new chunk's overlapping prefix. Equations (6a)–(6b) specify this operation during denoising: inference latency determines the prefix length, the inference interval selects the older suffix, and the remaining horizon follows the model's sampling update. The figure's arrow direction agrees with that formulation. The diagram is schematic and carries no measured time scale. E08E09E15
What it supports. Self-guidance addresses disagreement between consecutive samples without adding a Transformer expert or changing the action representation. The intended benefit is continuity when control switches to a newly completed chunk. It connects the policy's sampling procedure to the practical fact that an older prediction must remain usable during inference.
Where the evidence stops. The figure and equations describe a sampler operation; they do not report measured jerk reduction or a guidance-on/off ablation. The prefix update should not be interpreted as simply overwriting the quadrotor's measured physical velocity. Latency and action timestamps remain implementation-critical.
5.2 Separate shared training from future-video dependence
During training, state, reference-image, action and future-image tokens share one DiT, while language supplies cross-attention context. The action and visual prediction losses both train the shared system. But shared parameters do not imply unrestricted token access: the action-query row cannot read future-image keys, and context queries cannot read either predicted group. Future queries can read the action group, giving visual forecasting an action-conditioned role. At deployment, removing future tokens therefore preserves the action branch's declared inputs. The source reports a four-denoising-step time reduction from 0.296 to 0.184 seconds, but separately gives about 0.8 seconds for the flight setup. Reader interpretation: the architecture supports the proposed dependency separation; the size of the training benefit still requires a controlled loss ablation, and the timing numbers require distinct protocol labels. E04E05E17E15E13E14
5.3 Read history, frequency and success as separate variables
The appendix compares three checkpoints under shared scenes, camera, controller and inference-server conditions, sweeping policy queries at 1, 2 and 5 Hz. Begin with matched-frequency one-frame versus nine-frame rows. For tracking at 1 Hz, view success changes from 20% to 100%; however, tracking success only requires visibility during at least 80% of the trajectory. The nine-frame 5 Hz row can consequently show 100% view success alongside 5% collision. Searching uses a different success definition: entering a two-meter radius or receiving environment success, with no collision. This explains why search endpoints near two meters should not be read as precise docking. Reader interpretation: temporal context has the clearest reported controlled support, while general language competence, visual-loss necessity and physical-flight reliability remain separate questions requiring additional evidence. E10E13E14E16
5.4 Training and inference
During training
DiffAero supplies dynamics and expert controllers; Isaac contributes lighting realism and 3DGS contributes scene/object diversity, with weaker close-range geometry and lighting. The reported simulation set contains 900K clips and 320M frames: 500K tracking, 200K Isaac reaching and 200K 3DGS reaching clips. E06E07
A fisheye/T265 handheld device maps measured odometry to simulated quadrotor poses and crops matching first-person views. It supplies 858 trajectories, 332,429 frames, approximately three hours. AeroAct-FT denotes real-world finetuning; these handheld trajectories are not autonomous flight demonstrations. E06E11
Data are collected at 30 Hz and downsampled threefold; the implementation specifies one video frame per three actions, eight future frames, 24 actions and nine reference frames spanning 2.4 seconds. Pretraining takes 36 hours on eight NVIDIA A100 GPUs, using AdamW, learning rate 4.3×10^-5 and weight decay 0.01. The added MLP hidden sizes are 128 and 256. E03E07
During inference
Remove future tokens, denoise the action chunk and apply self-guidance across replanning calls. Equation (6a) guides the new prefix using the prior chunk's time-aligned suffix; Equation (6b) samples the remainder. The prefix length depends on inference latency divided by the data time step. This is a sampler constraint, not another learned module. E04E08
A workstation receives requests through ZeroMQ and returns actions for local trajectory conversion and on-manifold MPC tracking. The paper names a single NVIDIA RTX 5090 24G Laptop GPU and approximately 4,500 MB VRAM. The flight platform uses an OddityRC 35Pro frame, D435i camera and ROCK 5C computer, with motion capture for state measurement and safety monitoring. E15
5.5 Implementation flow
- Encode flight context
Wan2.1-1.3B supplies the video DiT, VAE and text encoder. Frozen visual encoding produces reference and future tokens with 3D RoPE. MLPs encode state and action; language enters through cross-attention. Figure 2 also marks the text encoder and VAE decoder as frozen. E03E04E07
- Restrict the information flow
The single sequence is [state; reference; action; future]. State/reference queries cannot read predicted tokens; action queries can read state, reference and action tokens; future queries can read every group. Figure 2's query rows and key columns agree with this rule, preventing future-token leakage into actions. E04
- Turn endpoint conditions into flight references
In a yaw-aligned local frame, each action specifies endpoint displacement through r, theta and psi, plus endpoint velocity and acceleration. Start and end position/velocity/acceleration determine fifth-order polynomial coefficients for a two-second segment. A chunk contains repeated segment specifications; it is not defined as 24 disjoint two-second maneuvers. E03E09
6. Experiments & results
AeroAct adapts a video diffusion Transformer into a language-conditioned quadrotor policy. It learns actions alongside their future visual consequences, then deploys only the action stream. Its strongest controlled evidence concerns temporal visual history; the physical demonstration establishes execution in a short indoor flight setting. Evidence: E04, E13, E16.
6.1 Read the original evidence
Table 2. The separate unseen-object comparison favors temporal context, although its split is incompletely specified. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the first two columns to identify the checkpoint and reference-frame count, then compare distance, success and collision across rows. The surrounding results text supplies meters for final distance; success and collision are percentages. The one-frame model ends at an average 3.819 meters, while the nine-frame model ends at 1.983 meters. Both nine-frame checkpoints reach the reported 100% success and zero-collision outcomes. Do not use boldface as a ranking rule: the source also bolds the poorer single-frame values. Keep this table separate from the frequency sweep in Appendix Table 4 because its table-specific frequency and episode count are not given. E12E14
What it supports. For the reported unseen targets, nine reference frames improve success from 75% to 100% and reduce average final distance from 3.819 to 1.983 meters. The finetuned checkpoint preserves the same success and collision percentages, with 1.988 meters average final distance. This supports transfer within the paper's reported object-search setting.
Where the evidence stops. The identities and construction of the unseen-object split are unspecified, as are this table's episode count and query frequency. The faithful crop preserves the inconsistent emphasis; the numerical values, column directions and surrounding text determine the interpretation.
Figure 6. The physical demonstration connects the learned trajectory interface to a quadrotor in a laboratory. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Locate the yellow mat and follow the red trajectory and overlaid quadrotor positions in each view. The accompanying text identifies the instruction as flying to the yellow foam mat; the figure itself has no metric axes or timing scale. Read it together with the hardware description on page 7 and Figure 5: observations and state feed a remote inference process, returned action chunks become local trajectories, and on-manifold MPC tracks them. The experiment uses motion capture for state measurement and safety monitoring. This is evidence of execution on a physical platform, while quantitative comparisons elsewhere in the paper come from simulation. E15E16E17
What it supports. The authors demonstrate that the learned action interface can guide a physical quadrotor toward a language-specified target. The figure makes the full perception-to-trajectory-to-controller connection concrete. It supports feasibility in the displayed indoor setting, alongside the source's explicit reliance on offboard inference and short trajectories.
Where the evidence stops. The montage supplies no repeated-trial success rate, calibrated tracking error or measured speed. The flight configuration reports roughly 0.8-second inference latency, distinct from the four-step action-only timing comparison. Outdoor, aggressive and multistage flight are not established by this demonstration.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Closed-loop target tracking with temporal history Isaac Lab; 20 saved target trajectories; matched scenes, camera, controller and server interface; 1 Hz policy queries, Table 3. | Nine-frame AeroAct: 100.0 / 0.0 / 3.436. View success / collision (%) / final distance (m) | One frame: 20.0 / 90.0 / 7.902. Nine-frame AeroAct-FT: 95.0 / 0.0 / 2.845. History gives an 80 percentage-point view-success gain. View success requires the target in view for at least 80% of a trajectory; it does not require collision-free flight. At 5 Hz, nine-frame AeroAct has 100% view success but 5% collision. E10E13 |
| Language-conditioned object search Isaac Lab; 20 episodes at each of 1, 2 and 5 Hz, Table 4. | Nine-frame AeroAct: 100 / 0 at every frequency; distances 1.982, 1.980 and 1.981 m, respectively. Success (%) / collision (%) / final distance (m) | One-frame success: 90, 80 and 75%; collision: 10, 20 and 20%. AeroAct-FT retains 100% success and zero collision at all frequencies. Search success requires environment success or entry into a 2.0 m radius, plus no collision. Near-two-meter endpoints therefore should not be described as precise object contact. Appendix collision formulas include termination events as well as collisions. E10E14 |
| Searching for unseen target objects Table 2's separate unseen-object simulation evaluation; object identities, episode count and query frequency are not stated for this table. | Nine-frame AeroAct: 100.0 / 0.0 / 1.983. Success (%) / collision (%) / average final distance (m) | One frame: 75.0 / 25.0 / 3.819; nine-frame AeroAct-FT: 100.0 / 0.0 / 1.988. The comparison supports transfer to the reported unseen objects, with incompletely specified split construction. Bold type also marks poorer one-frame numbers in the original table; it is not a reliable best-result indicator. E12 |
| Action-only inference timing Section 3.2 comparison with four denoising steps. | 0.184 seconds with video prediction disabled. Reported inference time per call (seconds) | 0.296 seconds with video prediction; reported time saving 37.8%. This comparison is distinct from the approximately 0.8-second latency reported for the flight hardware configuration. The source does not reconcile the timing protocols, so the faster figure is not a demonstrated flight-loop latency. E17E15 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 3. Visual history changes tracking outcomes more decisively than increasing query frequency in the single-frame policy. Original paper, p. 13 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First compare rows at the same frequency across the three checkpoint groups. Each setting uses 20 trajectories. View success is the percentage of trajectories with the target visible for at least 80% of their duration; In-view averages visibility, while Centered uses the stricter image-offset threshold of 0.25. Those columns and collision are percentages; distance is in meters. At 1 Hz, compare the first and fourth rows to isolate the reported history contrast. Next move within the nine-frame block to inspect frequency effects. AeroAct-FT is the checkpoint after real-world finetuning. The source holds scenes, camera, controller and inference-server interface fixed. E10E13E05E08
What it supports. At 1 Hz, nine-frame AeroAct raises view success from 20% to 100% and lowers collision from 90% to zero. Increasing its frequency from 1 to 5 Hz raises the centered-target rate from 64.2% to 78.7%, but the 5 Hz row also records 5% collision. Visibility and safety are distinct outcomes.
Where the evidence stops. A 100% view-success entry does not certify collision-free tracking. These are small reported evaluation sets without repeated-seed uncertainty. The history contrast does not separately identify the effects of the pretrained video model, visual loss or self-guidance.
Table 4. Nine-frame context sustains object-search performance across the tested query frequencies. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the same row-group structure as the tracking table, but change the meaning of success. Appendix A.2 on page 13 requires either environment success or entry within 2.0 meters of the object, together with no collision. Percentages are computed over 20 search episodes per setting, and final distance is measured in meters. The first block shows the single-frame policy; the next two show nine-frame AeroAct before and after real-world finetuning. Compare frequency-matched rows before comparing frequencies within one model. The near-two-meter distances should be read in the context of this stopping/success criterion rather than as object-contact accuracy. E10E11E14
What it supports. Nine-frame AeroAct and AeroAct-FT each achieve 100% success with zero collision at 1, 2 and 5 Hz. Single-frame success falls from 90% to 75% between 1 and 5 Hz. The reported results support the usefulness of history and show retained simulation search performance after real-world finetuning.
Where the evidence stops. The table concerns the specified simulation search protocol, not a physical-flight success rate or unrestricted language generalization. The source gives no confidence intervals. Appendix collision formulas include termination events, so exact reproduction requires matching the environment's termination semantics.
7. Analysis & limitations
7.1 What the evidence leaves open
Physical evidence is a short indoor, motion-capture-assisted target-reaching demonstration, including the instruction to fly to a yellow foam mat. No aggregate flight success rate is supplied. The authors identify offboard inference, limited memory, multistage instructions and recovery behaviors as current limitations; their first-WAM-flight priority claim is not independently established here. E15E16
Tables compare AeroAct history lengths, query frequencies and finetuning. They do not isolate the visual loss, pretrained video prior, renderer mixture or self-guidance, and report no repeated-seed uncertainty or competing-policy benchmark. Thus the large history benefit alone does not establish the causal value of every WAM component. E05E06E08E11E12E13E14
7.2 Questions for discussion
- How much of the history advantage remains without the auxiliary visual loss?
- When does reusing an older trajectory prefix trade smoothness for delayed response to a newly visible obstacle?
8. Reproducibility audit
8.1 Requirements and known gaps
Reconstruction requires synchronized RGB/state/language/actions, the specified local-frame endpoint conversion, matching controller/camera settings and explicit collision/visibility logging. The PDF leaves loss weight, exact noise/flow parameterization, batch size, training-step count, finetuning schedule, detailed split construction and software versions unspecified. These gaps prevent an exact training recipe despite the supplied optimizer and hardware details. E05E06E07E09E10E12E13
Proposed checks: hold history and training data fixed while removing visual consequence supervision; separately toggle self-guidance with the same checkpoint and controlled latency. Measure closed-loop success and collision for the first comparison, and boundary discontinuity plus tracking error for the second. Neither check was run for this report. E05E08E10E13
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does visual consequence supervision add value after history is fixed?
Reader-proposed comparison, not performed: start matched training runs from the same Wan checkpoint, using identical nine-frame histories, demonstrations, action representation, optimizer steps and random seeds. Compare the reported joint objective against an action-only objective with the visual-loss weight set to zero. Keep the action attention mask and deployment sampler unchanged, and report training compute separately. Evaluate both on identical saved tracking trajectories and object-search episodes at a fixed query frequency. Log visibility, search success and collision using the appendix definitions over repeated seeds. A reproducible closed-loop advantage for joint training would support the auxiliary-supervision mechanism; indistinguishable outcomes would leave the history explanation sufficient for these tasks. E04E05E07E10E13E14
Check 2: Does overlap guidance improve continuity without delaying recovery?
Reader-proposed comparison, not performed: use one trained checkpoint and identical simulation starts, controller settings, random draws and measured server latency, toggling only Equations (6a)–(6b)'s prefix guidance. Record each inference start/completion timestamp and compare the old and new trajectory references at handoff in position, velocity and acceleration; also record controller tracking error, target visibility and collision. Sweep imposed latency and include a repeatable target-direction change just before replanning. Guidance should reduce handoff discontinuity if the claimed mechanism works. If it preserves stale motion long enough to increase collisions or impair target reacquisition, continuity would reveal a responsiveness tradeoff rather than an unconditional control improvement. E08E09E10E13E15
8.3 Reading coverage
Visual audit: Actually inspected the title/author/version page; Figures 1–6; Tables 1–4; method, loss, sampler and local-polynomial equations; training/hardware/timing paragraphs; metric definitions; and limitations/conclusion on the declared PDF pages. Figure 2's query/key mask and Figure 4's overwrite direction were cross-checked against their captions and method equations. All six final original crops were viewed, including the corrected architecture crop. Table 2's boldface inconsistency is preserved and explained. Full text, including reference-only pages 10–11 and Appendix A.1–A.3, was read across all four supplied chunks. Separate supplemental availability remains unverified; code and experiments were not inspected or reproduced.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 13, 14. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, author block, version stamp and abstract, pp. 1–2
- 1 Introduction, pp. 2–3
- 2 Related Work, including 2.1–2.2, pp. 3–4
- 3 Methodology, including 3.1–3.4, pp. 4–6
- 4 Experiments, including 4.1–4.3, pp. 6–8
- 5 Limitations and 6 Conclusion, pp. 8–9
- Acknowledgments and References, pp. 9–11
- A.1 Observation and Action Spaces, p. 12
- A.2 Evaluation Metrics, pp. 12–13
- A.3 Additional Experimental Results, pp. 13–14
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Identity: the title page verifies arXiv:2607.14997v1 [cs.RO], 16 July 2026, the catalog title and all eleven authors (E01). AeroAct is typeset in small capitals. Only this supplied edition was inspected; no cross-version equivalence is asserted. The catalog affiliation string omits the title page's third affiliation, Independent researcher, assigned to Yu Hu and Xinjiang Wang.
- Acquisition omission preserved: Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This report supplements the complete four-chunk reading with visual inspection of Figures 1–6, Tables 1–4 and the relevant equations.
- Separate supplemental material availability has not been fully verified.
- No code, external project resources or separate supplements were inspected, and no experiments were reproduced. References were read as part of this paper; their cited works were not independently reviewed. Reference-only PDF pages 10–11 were read in text, without page-image inspection.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
E01PDF p. 1, title/author/affiliation block and arXiv margin stamp
Title and eleven authors match the catalog identity; the artifact is arXiv:2607.14997v1 [cs.RO], dated 16 July 2026. Affiliations include two Beijing Institute of Technology schools and Independent researcher. Xinhong Zhang, Qiyuan Zhu, Yubo Huang and Haolin Chen share equal contribution; Gang Wang is corresponding author.
Go to primary source ↓E02PDF pp. 1–3, Figure 1, Abstract and Section 1
The paper motivates local smooth control from rapidly changing first-person views, using future visual supervision in training and direct trajectory-action prediction in deployment.
Go to primary source ↓E03PDF p. 4, Section 3.1, Equations (1)–(3), and start of Section 3.2
History has h reference frames with stride Delta. The unified model predicts actions and an action latent that conditions future observations; Delta=3, p=24 and K=8. The backbone has 1.3B parameters.
Go to primary source ↓E04PDF p. 4, Figure 2 and caption; p. 5, Section 3.2, token construction and attention-mask paragraphs
State, reference, action and future tokens share one DiT; language provides cross-attention. The mask excludes future keys from action queries and predicted keys from context queries. Future queries access all groups. Figure freezing symbols identify the text encoder and VAE components; the text explicitly freezes video encoding. Future tokens are omitted at inference.
Go to primary source ↓E05PDF p. 5, Section 3.2, Equations (4)–(5) and following paragraph
The objective combines squared-error action and future-latent noise/velocity prediction losses with visual weight lambda_vis. The source allows diffusion or flow-matching wording without fixing the parameterization or giving the visual weight. Deployment uses the action branch.
Go to primary source ↓E06PDF p. 5, Figure 3 and Section 3.3; p. 6, Scenarios and Real-world data collection; p. 8, Figure 5(b)
DiffAero supplies dynamics/controllers to Isaac and 3DGS branches with complementary realism/diversity limitations. Tracking and object-reaching demonstrations supply aligned modalities. Fisheye and T265 handheld sensing generates cropped flight-like views from odometry: 858 trajectories, 332,429 frames, approximately three hours.
Go to primary source ↓E07PDF pp. 6–7, Section 4.1, implementation and pretraining paragraphs
Wan2.1-1.3B, MLP hidden sizes 128/256, 30 Hz collection and threefold downsampling, one video frame per three actions, eight future frames, 24 actions and nine reference frames spanning 2.4 s. Pretraining uses 900K clips/320M frames, split 500K/200K/200K across tracking/Isaac reaching/3DGS reaching, and 36 hours on eight A100s with AdamW at 4.3e-5 and weight decay 0.01.
Go to primary source ↓E08PDF p. 6, Figure 4, caption and Section 3.4, Equations (6a)–(6b)
The overlap diagram distinguishes inference start/completion and control intervals. During denoising, prior-chunk suffix values guide the new prefix over floor(t_infer/t_data) steps, offset by the inference interval; the model samples the remaining suffix. No standalone smoothness ablation is reported.
Go to primary source ↓E09PDF p. 12, Appendix A.1, Equations (7)–(11)
State is local velocity, acceleration and body-z direction, totaling nine dimensions. Nine-dimensional actions specify r/theta/psi displacement and endpoint velocity/acceleration for T=2 s quintic segments. Local z points upward and endpoint z equals -r sin(theta). Polynomial coefficients follow from start/end position, velocity and acceleration.
Go to primary source ↓E10PDF pp. 12–13, Appendix A.2, Equations (12)–(18); p. 13, Appendix A.3 before Table 3
Tracking success thresholds per-trajectory target visibility at 80%; centered rate uses normalized image offset at most 0.25. Search success requires environment success or distance at most 2.0 m and no collision. Collision formulas count termination or collision. Final distance is the last-step distance; it is not sufficient to characterize tracking.
Go to primary source ↓E11PDF p. 7, Table 1 and Section 4.2
Over 20 episodes, single-frame tracking/searching success is 20/90% and collision 90/10%; nine-frame AeroAct gives 100% success and zero collision for both. AeroAct-FT denotes real-world finetuning and gives tracking/search success 95/100%, with zero collision. Table 1 does not label query frequency.
Go to primary source ↓E12PDF p. 7, Table 2, all rows, and Section 4.2 final paragraph
Unseen-object search reports one-frame AeroAct distance/success/collision 3.819 m/75%/25%, nine-frame AeroAct 1.983 m/100%/0%, and nine-frame AeroAct-FT 1.988 m/100%/0%. The poorer first-row values are bold in the original. Object identities, split construction, table-specific episode count and frequency are not supplied.
Go to primary source ↓E13PDF p. 13, Appendix A.3, Equation (20), protocol paragraph and Table 3
Three checkpoints share Isaac scenes, camera, controller and server settings; tracking uses 20 trajectories at 1/2/5 Hz. At 1 Hz, single-/nine-frame AeroAct view success is 20/100%, collision 90/0%, distance 7.902/3.436 m. Nine-frame FT gives 95%, 0%, 2.845 m. At 5 Hz, nine-frame AeroAct retains 100% view success with 5% collision; its centered rate rises from 64.2% at 1 Hz to 78.7% at 5 Hz. No uncertainty is tabulated.
Go to primary source ↓E14PDF p. 13, Appendix A.3 protocol; p. 14, Table 4 and caption
Searching uses 20 episodes at each of 1/2/5 Hz. Single-frame success is 90/80/75%, collision 10/20/20%, final distance 2.616/3.421/4.064 m. Nine-frame AeroAct gives 100% success, 0% collision, distances 1.982/1.980/1.981 m; FT gives 100%, 0%, and 1.984/1.987/1.984 m. No competing-policy or repeated-seed statistics appear.
Go to primary source ↓E15PDF p. 7, Sections 4.1 and 4.3; p. 8, Figure 5(a) and caption
The paper names an NVIDIA RTX 5090 24G Laptop GPU, about 4,500 MB VRAM, ZeroMQ offboard requests, local trajectory conversion and on-manifold MPC. Flight hardware includes a modified OddityRC 35Pro frame, D435i camera, ROCK 5C computer and lab motion capture for state/safety. Section 4.3 reports approximately 0.8 s model inference latency in this configuration.
Go to primary source ↓E16PDF p. 8, Figure 6, Section 4.3 continuation and Section 5; p. 9, Section 6 continuation
A physical quadrotor approaches a yellow foam mat under a language instruction. The source presents a demonstration rather than an aggregate flight benchmark and claims priority with a to-the-best-of-our-knowledge qualification. Stated limitations include short indoor trajectories, offboard inference, inadequate multistage/recovery/long-memory capability; the conclusion identifies aggressive dynamics and onboard efficiency as future work.
Go to primary source ↓E17PDF p. 5, Section 3.2, inference timing sentence
With four denoising steps, disabling video prediction reduces reported inference time from 0.296 s to 0.184 s, a stated 37.8% saving. No reconciliation with the separate flight-configuration latency is supplied.
Go to primary source ↓8.5 Primary sources
AeroAct: Action-Centered World-Action Models for Language-Conditioned Quadrotor Flight ↗
PDF · 6,177 extracted words
Source fingerprint
feed93b669c5deb741b7296f2e373f7cfbb67d738ed90f33794905d4e2920255