EndoWAM: A Grounded World-Action Model for Generalizable Endoscopic Navigation
1. Paper overview
In one sentence: EndoWAM trains a video-model representation to reconstruct future target regions, then uses that representation for fast discrete endoscopic control while omitting the grounding branch at deployment. problemarchitectureheadsnavigationgeneralizationmissing-appendices
| At a glance | What to know |
|---|---|
| Research problem | Source description Endoscopic navigation must track instructed anatomical targets despite narrow viewpoints, deformable tissue, occlusions, and specular artifacts. The authors argue that observation-to-action policies lack useful predictive dynamics, while ordinary video-model inference is too slow. EndoWAM seeks target-aware prediction that supports responsive three-DoF control. problemarchitecture |
| Core mechanism | Source description A pretrained video DiT supplies shared predictive features to a discrete action expert and a future-target reconstruction branch, coupling navigation supervision with task-specific latent prediction. architectureheads |
| A key reported result | Three-stage navigation at unseen roll viewpoints: 77/96 successes; average 80.2%; ureteroscopy 90.6%, esophagoscopy 56.3%, ERCP 93.8%. Closed-loop success rate (%). Physical ureteroscopy, esophagoscopy, and ERCP phantoms; 32 trials each. Test rolls span 360 degrees and avoid training augmentation angles. Success requires reaching and retaining the instructed target in view. GR00T-N1.7 averages 27.1%: a reported 53.1 percentage-point gap. Qwen3AE scores 17.7%, versus Qwen3DiT 8.3%. Baselines share observations, instructions, augmented data, and optimization-step counts. Backbone differences prevent attributing the entire baseline gap to grounding alone; esophagoscopy remains weakest. protocolnavigation |
| Reading caution | Reader analysis Physical-phantom success does not establish patient safety, clinical tissue generalization, or reduced injury. Failures include lumen loss, wall impaction, and timeout, but the referenced failure breakdown is unavailable. protocolmissing-appendices |
Core contributions
- Source description
A pretrained video DiT supplies shared predictive features to a discrete action expert and a future-target reconstruction branch, coupling navigation supervision with task-specific latent prediction. architectureheads
- Source description
EndoMotion supplies aligned observations, target-derived actions, and future crops across three phantom procedures; rotation augmentation expands viewpoint coverage. data
- Author claim
The authors describe EndoWAM as the first WAM for generalizable robotic endoscopic navigation. This priority claim is reported, not independently established. problem
Figure 2. One predictive representation supports action decoding and auxiliary future-target reconstruction. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start in the center with the current frame and instruction. The Video DiT supplies hidden states along the two outward paths: left to the Grounding DiT, right to the action expert. The left branch also receives noisy future-target information during training, while the right branch receives learnable action queries. Outputs above the blocks are future target latents, future video latents, and an action chunk. The snowflake marks the frozen video backbone and the flame marks its trainable LoRA adapter; the two heads are trainable. Sections 3.2 and 4.1 specify that shared features come from a single pass at block 17. architectureheadsimplementation
What it supports. The action expert consumes predictive hidden features, so it need not wait for a future movie to finish denoising and decoding. Future-target reconstruction supplies an additional training signal to those same features. This is the architectural basis for retaining predictive information while reducing deployment work.
Where the evidence stops. The drawing contains training pathways. Section 3.2 explicitly removes the Grounding DiT and future-frame decoding at inference; the pictured future images are not observations available to the deployed controller or proof of accurate video forecasts.
2. Motivation
2.1 The problem and the proposed response
Endoscopic navigation must track instructed anatomical targets despite narrow viewpoints, deformable tissue, occlusions, and specular artifacts. The authors argue that observation-to-action policies lack useful predictive dynamics, while ordinary video-model inference is too slow. EndoWAM seeks target-aware prediction that supports responsive three-DoF control. problemarchitecture
2.2 What this reading follows
An endoscope must keep finding its target as the camera moves through a narrow lumen. EndoWAM uses a pretrained video model to supply predictive features, then teaches those features to emphasize future target regions. A separate action expert turns them into control chunks. The grounding branch is a training instrument: deployment retains one video-model pass and action decoding. The most useful evidence combines phantom navigation, grounding comparisons, and component ablations. Read the reported 80.2% three-stage success separately from the 90.1% two-stage shift result, and keep the absent implementation appendices in view when assessing reproducibility. problemarchitectureheadsnavigationgeneralizationmissing-appendices
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | IDM |
| Quadrant | Q4 · Dual-system × IDM |
3.1 Evidence-based assessment
Insufficient evidence to decide
The WAM and navigation labels are supported. Distinct video-model and action-expert modules support Dual-system. IDM is less explicit: Eq. (1) conditions actions on intermediate predicted-dynamics features, rather than a stated current/future-state inverse-dynamics pair; Eq. (2) describes joint latent-action-region training. Thus Q4 is plausible under a broad IDM definition but unverified under a narrow one. Joint training does not make this a single unified action/video generator. Efficient inference is supported by Table 4. architectureheadsefficiency
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Learn where the target will matter
Begin with the annotation pipeline rather than the network name. A detector and temporal filtering produce oriented target boxes along a demonstration. From the same track, EndoWAM derives target-relative action labels and crops from future frames. The video model is trained to predict latent dynamics, while the Grounding DiT must reconstruct those future crop latents from shared hidden features. Action queries also attend to those features. Reader interpretation: the reconstruction objective creates pressure to preserve target-specific information that a generic future-video loss might underweight. This is a plausible explanation of the ablation, not a measured causal decomposition. The binary target mask is also consequential: frames with missed detections do not teach the grounding branch, so the quality and coverage of the annotation track constrain this learning signal. dataheadsobjectivesablation
Figure 3. The grounding variants change where target information enters or supervises the model. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read panels (a) through (d) as different placements of target information. In (a), a grounding expert transforms the current image into an extra crop that enters EG. In (b), CG sends predictive features toward future oriented bounding boxes; the label CoT refers here to a box-regression grounding variant, not a displayed verbal reasoning trace. In (c), FG-Gen puts future frames and target regions together in the generative output. In (d), the separate Grounding DiT takes hidden features and noise to reconstruct future target latents. The arrows agree with Section 3.4's descriptions of input grounding, regression, generation, and reconstruction. variantsarchitecturegrounding-comparisonmissing-appendices
What it supports. This comparison asks whether giving the policy a current crop is enough, or whether supervising future target content is more useful. Under the reported matched navigation setting, reconstruction reaches 80.2% average success, compared with 61.5% for EG, 51.0% for FG-Gen, and 45.8% for CG.
Where the evidence stops. The panels summarize training concepts and do not specify full optimization settings. Future-frame output boxes should not be read as an inference requirement for the proposed method. The referenced configuration appendices are absent.
5.2 Keep prediction features in the control loop
At deployment, the model still encodes the current observation and instruction and performs one video DiT denoising pass. It extracts block-17 hidden states, then the action expert uses learnable queries to produce categorical actions across the chunk. Bidirectional self-attention connects positions inside that predicted chunk; it does not grant access to future camera observations. The Grounding DiT and future-frame decoder are removed. Figure 1 links the resulting controls to the physical endoscope and observation feedback. This separation explains how a training-time future-target objective can influence execution without reconstructing target crops online. Table 4 reports 7.5 Hz on one RTX 5090, faster than Cosmos Policy's 1.1 Hz but slower than the listed VLA baselines. The supplied source leaves chunk length and feedback timing unresolved. architectureheadsimplementationcontrol-overviewefficiencymissing-appendices
5.3 Separate evidence for the system from evidence for the mechanism
Use the experiments in a sequence. Table 1 first asks whether the complete policy navigates the physical phantoms successfully; its lower block compares alternative grounding placements. Table 2 then removes grounding or rotation augmentation to test component dependence. Figure 5 supplies a qualitative attention interpretation. Together these establish useful system-level and component-level evidence, but they leave temporal specificity open: no displayed comparison reconstructs current target crops with the same auxiliary head and loss budget. Finally, Table 3 broadens the tested conditions while shortening the task to two stages. Reader interpretation: success under selected shifts strengthens the transfer case within this setup, but it cannot close the clinical safety gap or repair missing implementation details. A reproduction should therefore distinguish benchmark replication from a targeted test of future grounding. protocolnavigationvariantsgrounding-comparisonablationattentiongeneralizationmissing-appendices
5.4 Training and inference
During training
EndoMotion contains 515 teleoperated phantom trajectories and 0.98M frames. Fine-tuned Grounding DINO detects targets; Kalman filtering produces oriented bounding-box tracks. Target-center offsets become discrete action labels, and future crops supervise grounding. Joint 45-degree rotations of trajectories and boxes yield 4,120 sequences and 7.84M frames. data
LoRA adapts attention and feed-forward projections; the text encoder and video VAE remain frozen. Training combines video rectified-flow velocity regression, validity-masked, class-weighted per-axis action cross-entropy, and cosine-schedule target-latent noise prediction. The binary target mask makes missed detections contribute no grounding gradient. implementationobjectives
During inference
The current observation and instruction produce H through one video-model denoising pass, then the action expert generates a chunk. Future-frame decoding and the Grounding DiT are omitted. Figure 1 connects the chunk to a physical endoscope controller and subsequent observations; chunk length and the executed prefix before feedback are unspecified in the supplied body. architectureheadscontrol-overviewmissing-appendices
5.5 Implementation flow
- Encode the current view and task
Cosmos-Predict2.5-2B provides the video DiT, text encoder, and video VAE. The VAE maps observations into tokens; the current frame stays clean, while future tokens are perturbed. Instruction features enter through cross-attention. architecture
- Extract predictive features
After one denoising pass, block-17 activations are flattened into shared tokens H. These are intermediate predictive features, not a fully decoded future video. headsimplementation
- Decode actions and supervise future grounding
Learnable action queries use bidirectional self-attention within the chunk and cross-attention to H; an MLP predicts per-axis categorical logits. Separately, a Grounding DiT conditions on projected, resampled H to reconstruct VAE latents of future target crops. heads
- Compare grounding routes
EG adds an expert crop of the current target to the input. CG regresses future oriented bounding boxes. FG-Gen makes future target regions part of the video generation target. EndoWAM instead uses a separate reconstruction branch. variants
6. Experiments & results
EndoWAM turns intermediate video-model features into discrete endoscope action chunks. During training, reconstructing future target crops makes those features task-aware; deployment drops that reconstruction branch and avoids decoding future frames. The paper reports 80.2% success on three-stage physical-phantom navigation, versus 27.1% for its strongest baseline, and 7.5 Hz control. Its key evidence is executed phantom navigation and component ablations, with clinical transfer and detailed reproducibility left unresolved (architecture, navigation, ablation, efficiency, missing-appendices).
6.1 Read the original evidence
Table 1. EndoWAM leads the three-stage phantom benchmark, with a pronounced procedure gap. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the three procedure columns before the average. Each procedure has 32 physical trials spanning unseen roll viewpoints, so the average summarizes equally sized groups. Success requires reaching the instructed target and keeping it visible through termination; the rollout includes entrance cannulation, lumen traversal, and target localization. The upper block compares five baselines with EndoWAM. The lower block holds the EndoWAM family closer while changing grounding strategy. The three checkmark columns describe discrete actions, visual grounding, and a dynamics prior. They are method attributes, not independent experimental interventions. Baselines use the same observations, instructions, augmented training dataset, and matched optimization-step counts. protocolnavigationgrounding-comparison
What it supports. EndoWAM succeeds in 77 of 96 trials, averaging 80.2%, versus 27.1% for GR00T-N1.7. Its 56.3% esophagoscopy success is much lower than 90.6% ureteroscopy and 93.8% ERCP. The aggregate lead therefore coexists with a substantial unresolved weakness in one procedure in this benchmark.
Where the evidence stops. Different backbones and action heads confound a causal reading of the upper block. The table supplies no confidence intervals or training-seed variability. Physical-phantom success does not establish clinical safety or patient generalization.
Table 3. Selected environment and target shifts are tested under a distinct two-stage protocol. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the grouped headers. Reflectance, geometry, and texture belong to cross-environment tests; shape, size, and color belong to cross-target tests. Each column contains 32 trials under a condition described as unseen during training. Read across each model row to distinguish a strong average from consistent performance: EndoWAM is above 80% in every displayed condition. Crucially, this table covers lumen traversal and target localization only. The source caption expressly separates it from Table 1's three-stage task. Figure 6 on the same page provides representative observations of these shifts, which help interpret the condition names but do not define their complete severity ranges. generalizationprotocol
What it supports. EndoWAM averages 90.1%, compared with 46.9% for GR00T-N1.7 and 36.5% for Qwen3AE. Its highest reported rate is 100.0% under texture shift, and its lowest is 81.3% under color shift. This supports transfer across the six tested condition changes under this two-stage protocol.
Where the evidence stops. The 90.1% result cannot be interpreted as an improvement over Table 1's 80.2% because the stages differ. The supplied source does not establish coverage of arbitrary anatomy, combined shifts, or clinical environments.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Three-stage navigation at unseen roll viewpoints Physical ureteroscopy, esophagoscopy, and ERCP phantoms; 32 trials each. Test rolls span 360 degrees and avoid training augmentation angles. Success requires reaching and retaining the instructed target in view. | 77/96 successes; average 80.2%; ureteroscopy 90.6%, esophagoscopy 56.3%, ERCP 93.8%. Closed-loop success rate (%) | GR00T-N1.7 averages 27.1%: a reported 53.1 percentage-point gap. Qwen3AE scores 17.7%, versus Qwen3DiT 8.3%. Baselines share observations, instructions, augmented data, and optimization-step counts. Backbone differences prevent attributing the entire baseline gap to grounding alone; esophagoscopy remains weakest. protocolnavigation |
| Alternative grounding strategies Matched EndoWAM variants under the same three-procedure, unseen-viewpoint protocol. | EndoWAM 80.2%. Average closed-loop success rate (%) | EG 61.5%, FG-Gen 51.0%, CG 45.8%; reported advantage over EG is 18.7 percentage points. The reconstruction design wins this comparison; the ranking does not establish which representation property causes the gain. variantsgrounding-comparisonprotocol |
| Grounding and viewpoint augmentation ablations Independent single-factor removals under the three-procedure navigation evaluation. | Full model 80.2%; without Grounding DiT 37.5%; without viewpoint augmentation 28.1%. Average closed-loop success rate (%) | Both removals reduce success across every procedure. Both components matter in this setup. Removing grounding also removes an auxiliary objective, so this does not isolate future timing from generic reconstruction supervision. ablationobjectives |
| Two-stage zero-shot environment and target shifts 32 trials per condition; lumen traversal and target localization, without entrance cannulation. Six unseen reflectance, geometry, texture, shape, size, and color shifts. | Average 90.1%; condition rates 93.8%, 87.5%, 100.0%, 84.4%, 93.8%, and 81.3%, respectively. Closed-loop success rate (%) | GR00T-N1.7 46.9%; Qwen3AE 36.5%; reported gap to GR00T is 43.2 percentage points. This tests selected unseen conditions under a shorter protocol. Its 90.1% must not be compared directly with the three-stage 80.2%. generalization |
| Closed-loop deployment efficiency Single NVIDIA RTX 5090; reported EndoWAM model size 2.1B parameters. | 7.5 Hz. Control frequency (Hz; higher is faster) | Cosmos Policy: 1.1 Hz, 2.0B; GR00T-N1.7: 8.9 Hz, 1.6B; Qwen3AE: 12.1 Hz, 2.2B. Authors report 6.8× over Cosmos Policy. One-pass features improve speed relative to iterative video denoising, but EndoWAM is slower than the two listed VLA baselines. Timing variance and detailed measurement boundaries are absent. efficiencyarchitecture |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2. Both future grounding and viewpoint augmentation contribute to navigation success. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the full-model row as the reference, then move upward one removal at a time. The first row deletes Grounding DiT supervision; the second deletes viewpoint augmentation. These are separate single-factor ablations, not cumulative removals. Compare within each procedure column to see whether the average hides an exception: both removals lower success in ureteroscopy, esophagoscopy, and ERCP. The augmentation matters to the evaluation design because test roll angles deliberately fall between the eight training rotations. The grounding removal matters to the learning objective because it eliminates the future-target reconstruction signal that would otherwise shape the shared predictive features. ablationobjectivesdataprotocol
What it supports. The full system averages 80.2%, while removing grounding yields 37.5% and removing augmentation yields 28.1%. The consistent direction across procedures supports retaining both components for this setting. These results provide stronger component evidence than the feature checkmarks in the main comparison table.
Where the evidence stops. Removing the Grounding DiT also removes its auxiliary reconstruction loss. This does not isolate future supervision from current-target reconstruction, and the table does not measure interactions when both components are removed.
Figure 5. The attention examples offer a qualitative interpretation of the grounding ablation. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read horizontally: each left image is the observation, the middle panel removes Grounding DiT, and the right panel uses the full model. Across the displayed rows, the middle maps spread colored activation over multiple regions, whereas the right maps concentrate more strongly near the region the authors identify as the target. Use the dark lumen openings and the surrounding anatomy as spatial references rather than treating bright reflections as targets. The caption interprets the unguided maps as distraction by specular highlights and mucosal texture. These are selected qualitative examples accompanying the success-rate ablation, not additional rollout statistics or a calibrated segmentation output. attentionablation
What it supports. The examples are consistent with the authors' explanation that future-region supervision makes predictive features more target-focused. They provide a possible representation-level account of the Table 2 improvement, while the executed navigation trials remain the evidence that the trained policy performs better.
Where the evidence stops. The original figure provides no numeric color scale, attention extraction recipe, or localization metric. Absolute values and cross-panel normalization cannot be recovered from the colors. Apparent concentration alone does not prove causal attention use or safety.
7. Analysis & limitations
7.1 What the evidence leaves open
Physical-phantom success does not establish patient safety, clinical tissue generalization, or reduced injury. Failures include lumen loss, wall impaction, and timeout, but the referenced failure breakdown is unavailable. protocolmissing-appendices
Tables report point estimates without confidence intervals or repeated-training variability. Figure 5 offers qualitative attention examples without a color scale or quantitative localization metric; it cannot by itself prove reliable target tracking. navigationablationattentiongeneralization
Detector-derived action labels and future crops share an annotation source. Undetected targets receive no grounding gradient, leaving performance under systematic detector errors unresolved. dataobjectives
7.2 Questions for discussion
- Would equally weighted reconstruction of current target crops retain the future-grounding benefit?
- How much does performance depend on detector-derived action labels and trajectory-level train/test separation?
- Which failure modes dominate esophagoscopy, and what measurements would support a clinical safety claim?
8. Reproducibility audit
8.1 Requirements and known gaps
Required ingredients include Cosmos-Predict2.5-2B, its frozen encoders, LoRA adaptation, block-17 feature extraction, action and grounding heads, EndoMotion annotations, and the three-DoF phantom platform. The deployment benchmark uses one RTX 5090; training hardware is not specified. architectureimplementationdataprotocolefficiency
The missing appendices prevent recovering action bins and offset thresholds, chunk and prediction horizons, LoRA rank, loss weights, optimizer settings, learning rate, batch size, step counts, training hardware, software versions, trajectory splits, and detailed timeout criteria. Dataset/code release and licensing are not established by the supplied PDF. missing-appendicesdataobjectivesprotocolimplementation
Proposed checks: compare matched future-crop and current-crop reconstruction to test temporal specificity, and split original trajectories before rotation to test whether unseen-angle gains survive strict trajectory separation. These are proposals, not reproduced results. dataobjectivesablation
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does future timing help beyond target reconstruction?
Reader-proposed experiment, not performed: train matched models with future-target reconstruction, current-target reconstruction using the same Grounding DiT capacity, and no reconstruction. Keep the original trajectory split, video/action losses, loss-weight budget, action head, training steps, and rotation augmentation fixed; normalize target availability so differing detector masks do not change supervision volume. Evaluate all models on identical held-out roll trials, reporting per-procedure success and uncertainty across training seeds. A repeatable future-over-current advantage would support temporal specificity; comparable current and future results would weaken that explanation while preserving evidence for target-aware auxiliary learning. Recover missing configurations before treating this as a replication. dataheadsobjectivesablationprotocolmissing-appendices
Check 2: Do viewpoint gains survive splitting before augmentation?
Reader-proposed experiment, not performed: assign all frames from each original teleoperation trajectory to one split before generating any rotated copies, and keep every copy of that trajectory in its original split. Compare otherwise matched models trained with and without the eight-angle augmentation, using identical optimization-step budgets and new physical rollouts at unseen roll offsets. Report success by procedure and by roll interval under the original three-stage criterion. If the augmentation advantage persists, it supports genuine viewpoint robustness under strict trajectory separation. If it shrinks markedly relative to the reported removal gap, shared trajectory content or the effective data budget deserves investigation. This proposal does not allege leakage; the supplied source does not document its trajectory split. dataprotocolablationmissing-appendices
8.3 Reading coverage
Visual audit: All nine supplied PDF pages were visually inspected, including the title/author/version block; Figures 1–6; Tables 1–4; Eqs. (1)–(6); method, data, training, evaluation, and deployment details; and the concluding references. Every final crop was inspected at its saved dimensions. Figure 2's hidden-state arrows and frozen/trainable markers and Figure 3's branch directions were checked against the captions and Sections 3.1–3.4. Their training pathways are distinguished from the branches removed at inference. Pages 4–6 refer to Appendices A–G, but the supplied PDF ends at page 9 without them. Separate supplements, code, data, and external resources remain outside this reading; no experiments were executed.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1 Introduction
- 2 Related Work
- 3 Method
- 3.1 Problem Formulation
- 3.2 Model Architecture
- 3.3 Training and Inference
- 3.4 Controlled Variants for Grounding WAM
- 4 Experiments
- 4.1 Implementation Details
- 4.2 Comparison with State-of-the-Art
- 4.3 Comparison with EndoWAM Variants
- 4.4 Ablation Study
- 4.5 Generalization Capability
- 4.6 Deployment Efficiency
- 5 Conclusion
- References
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Identity: the title and all 13 authors match the catalog after name-order normalization. The title page identifies arXiv:2608.01221v1 [cs.RO], 2 August 2026. Only this supplied revision was reviewed; no other edition was compared.
- All four supplied text chunks and all nine PDF pages were read. The acquisition's text extraction did not reconstruct figure images; the retained PDF was visually inspected for every figure, table, and equation layout.
- Appendices A–G are referenced but absent from the supplied nine-page PDF, which ends with references. Platform and detailed success criteria (A), annotation pipeline (B), per-procedure statistics (C), architecture and optimization (D), baseline configurations (E), extra rollouts (F), and failure breakdowns (G) remain unavailable.
- Separate supplemental material availability has not been fully verified.
- Code, datasets, checkpoints, and external resources were not inspected. No experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title, author/affiliation block, arXiv margin stamp
Observed title matches the catalog. The 13 named authors agree in order; the revision is arXiv:2608.01221v1 [cs.RO], 2 Aug 2026. Affiliations are printed as CUHK, SLAI, SYSU, Durham, NYU, and WFU.
Go to primary source ↓problemPDF pp. 1–2, Abstract and Section 1
The paper motivates target-directed endoscopic control under deformation, occlusion, limited data/viewpoints, and video-model latency; it claims first WAM coverage of generalizable robotic endoscopic navigation.
Go to primary source ↓control-overviewPDF p. 1, Figure 1 and caption
Current observation and instruction feed EndoWAM; an action chunk drives insertion and two bending axes through the endoscopy controller, with observation feedback.
Go to primary source ↓architecturePDF p. 3, Figure 2 and caption; Sections 3.1–3.2, Eqs. (1)–(2)
Video DiT hidden states feed separate Grounding DiT and action expert branches. Cosmos-Predict2.5-2B supplies the video prior; current observation is clean, future tokens are noised, and instructions use cross-attention. Inference drops grounding and future-frame decoding.
Go to primary source ↓headsPDF p. 4, Section 3.2 continuation, paragraphs above Section 3.3
Flattened intermediate features H condition action queries using bidirectional self-attention and cross-attention; an MLP produces per-axis categorical logits. Grounding reconstructs future target-crop VAE latents from projected, resampled H.
Go to primary source ↓dataPDF p. 2, Section 1 dataset paragraph; PDF p. 4, Section 3.3, Dataset
Teleoperation in three anatomical phantoms yields 515 trajectories and 0.98M frames. Fine-tuned Grounding DINO and Kalman-filtered OBB tracks supply offset-derived action labels and future crops. Rotations at 45-degree intervals yield 4,120 sequences and 7.84M frames.
Go to primary source ↓objectivesPDF p. 4, Section 3.3, Eqs. (3)–(6) and surrounding definitions
Training uses rectified-flow future-latent velocity regression, masked weighted per-axis action cross-entropy, and cosine-schedule future-target noise prediction. The target mask removes grounding gradients for missed detections; two coefficients balance the auxiliary losses.
Go to primary source ↓variantsPDF p. 4, Figure 3, caption, and Section 3.4
EG adds a current-target expert crop, CG regresses future OBB parameters, FG-Gen jointly generates future frames and target regions, and the proposed method reconstructs future target-region latents with a separate Grounding DiT.
Go to primary source ↓protocolPDF p. 5, Figure 4 and Section 4.1, Deployment and evaluation protocol / Baselines
Three-stage navigation covers entrance cannulation, lumen traversal, and target localization on a shared physical phantom platform. Each procedure uses 32 trials at roll viewpoints unaligned with training angles. Success requires reaching and retaining the target; failures are lumen loss, wall impaction, or timeout. Baselines share data, inputs, and optimization-step counts.
Go to primary source ↓implementationPDF p. 5, Section 4.1, Model and optimization
Attention and feed-forward projections use LoRA; text encoder and video VAE are frozen. Shared features come from block 17 after one denoising pass. Further configurations are referred to Appendix D.
Go to primary source ↓navigationPDF p. 5, Section 4.2 Results; PDF p. 6, Table 1 upper block, procedure and Average columns
EndoWAM succeeds in 77/96 trials: 90.6%, 56.3%, 93.8% by procedure and 80.2% average. GR00T-N1.7 averages 27.1%, Qwen3AE 17.7%, Qwen3DiT 8.3%, pi_0.5 15.6%, and Diffusion Policy 2.1%. The reported strongest-baseline gap is 53.1 percentage points.
Go to primary source ↓grounding-comparisonPDF p. 6, Table 1 lower block and Section 4.3
Average success is 45.8% for CG, 51.0% for FG-Gen, 61.5% for EG, and 80.2% for the proposed reconstruction-based grounding. The reported gap to EG is 18.7 percentage points.
Go to primary source ↓ablationPDF p. 6, Table 2 and Section 4.4
Full-model success is 90.6/56.3/93.8% across the three procedures, averaging 80.2%. Removing Grounding DiT gives 37.5/28.1/46.9%, averaging 37.5%; removing viewpoint augmentation gives 31.3/15.6/37.5%, averaging 28.1%.
Go to primary source ↓attentionPDF p. 6, Figure 5 and caption; Section 4.4 continuing onto PDF p. 7
Three observation/attention examples compare no Grounding DiT with the full model. The authors interpret the latter as more target-focused under darkness and occlusion. No numeric color scale or quantitative localization score accompanies the panels.
Go to primary source ↓generalizationPDF p. 7, Table 3 and caption, Figure 6, Section 4.5
Two-stage lumen traversal and target localization use 32 trials per unseen shift. EndoWAM scores 93.8/87.5/100.0/84.4/93.8/81.3% for reflectance/geometry/texture/shape/size/color, averaging 90.1%; GR00T averages 46.9%, Qwen3AE 36.5%. The caption warns against direct comparison with three-stage Table 1.
Go to primary source ↓efficiencyPDF p. 7, Table 4 and Section 4.6
A single RTX 5090 yields EndoWAM 7.5 Hz at 2.1B parameters, Cosmos Policy 1.1 Hz at 2.0B, GR00T-N1.7 8.9 Hz at 1.6B, and Qwen3AE 12.1 Hz at 2.2B. Authors attribute their reported 6.8× Cosmos speedup to one-pass intermediate features.
Go to primary source ↓missing-appendicesPDF p. 4, Section 3.3 appendix references; p. 5, Section 4.1 appendix references; p. 6, Section 4.2 continuation; pp. 7–9, terminal Conclusion/References
Body references point to A for platform/evaluation, B for annotation, C for procedure statistics, D for architecture/optimization, E for baseline configurations, F for demonstrations, and G for failure breakdowns. The supplied PDF ends with references and contains none of these appendices.
Go to primary source ↓8.5 Primary sources
EndoWAM: A Grounded World-Action Model for Generalizable Endoscopic Navigation ↗
PDF · 4,425 extracted words
Source fingerprint
cc3a0e1d4ab6fb8e097a10064907fe68b1339c6e074c70a3eb40f8a922de83a7