Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning
1. Paper overview
In one sentence: A shared video-action diffusion model converts surgical video pretraining into better simulated manipulation under fixed action supervision, while joint future sampling and incomplete evaluation details leave efficiency and real-world transfer unresolved. e-probleme-architecturee-pretraine-controle-maine-stepse-horizone-curvee-real
| At a glance | What to know |
|---|---|
| Research problem | Reader analysis Synchronized surgical video and kinematics are expensive to collect, whereas endoscopic recordings are comparatively abundant. The paper asks whether action-free video improves closed-loop manipulation without increasing action supervision. Its central comparison fixes the demonstrations and contrasts the same WAM with and without surgical video pretraining. This tests initialization, rather than directly measuring how many demonstrations can be removed. e-probleme-setup |
| Core mechanism | Source description The contribution is a surgical instantiation and two-stage training recipe for an existing Cosmos Policy architecture, with joint video/action generation directly driving receding-horizon control. e-architecturee-pretraine-control |
| A key reported result | SurRoL four-task mean success: 77.8% with surgical video pretraining Closed-loop task success rate (%). Same 10,000 demonstrations; defaults 80k fine-tuning steps, H_c=16, H_e=4. Section 4.1 states 100 evaluation episodes per result; held-out split and seed aggregation unspecified. 63.5% without pretraining; reported improvement 14.3 percentage points. Rounded mean of simulated task execution; no clinical performance or video-for-label substitution rate is established. e-setupe-main |
| Reading caution |
Core contributions
- Source description
The contribution is a surgical instantiation and two-stage training recipe for an existing Cosmos Policy architecture, with joint video/action generation directly driving receding-horizon control. e-architecturee-pretraine-control
- Author claim
The authors claim the first WAM application to surgical robot learning and argue that action-free visual dynamics transfer into action prediction. Priority is their claim, not independently established here. e-probleme-architecturee-main
Figure 2. One shared denoiser links video learning to action prediction; the lower panel separates the two sources of supervision. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the upper blue block: current proprioception and the endoscopic image condition the model. The orange block contains the action chunk, future proprioception and future image. The line and downward arrows connect these slots under one unified-denoising label; they do not depict a separate inverse-dynamics pipeline. The caption additionally names goal and task text, which are not separate boxes. Then read the lower arrows left to right: surgical video adapts the world model before action-labeled demonstrations adapt it for control. Section 3.2 specifies that the full model is fine-tuned jointly in the second stage. e-architecturee-pretraine-finetunee-control
What it supports. The architectural basis for One Model × Joint prediction is the shared sequence processed by one diffusion transformer. Video-derived weights are reused when actions enter training. The diagram establishes the intended information flow, while the measured value of pretraining must come from the executed-task comparisons.
Where the evidence stops. The caption says Stage 1 action frames are masked/zeroed and labels losses L_vid/L_joint; Section 3.2 excludes action and state slots and uses L_pre/L_ft. These descriptions are not fully reconciled. Follow Equations (2)–(3) for supervision, without inferring a specific mask implementation.
2. Motivation
2.1 The problem and the proposed response
Synchronized surgical video and kinematics are expensive to collect, whereas endoscopic recordings are comparatively abundant. The paper asks whether action-free video improves closed-loop manipulation without increasing action supervision. Its central comparison fixes the demonstrations and contrasts the same WAM with and without surgical video pretraining. This tests initialization, rather than directly measuring how many demonstrations can be removed. e-probleme-setup
2.2 What this reading follows
Surgical recordings can show how tools and objects move without recording the commands that caused the motion. Surgical WAM uses that asymmetry: first adapt a video model to surgical dynamics, then teach the same generative backbone to produce robot actions from synchronized demonstrations. The useful test is executed task success after repeatedly observing and replanning. Under its fixed-label SurRoL comparison, the paper reports gains across four tasks. This reading follows the shared latent architecture, the task-level numbers and two control/training diagnostics, while separating those measurements from a qualitative real-video claim and a learning curve whose caption does not match its visible endpoints. e-probleme-architecturee-pretraine-controle-maine-stepse-horizone-curvee-real
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | One Model |
| Prediction paradigm | Joint prediction |
| Quadrant | Q1 · One Model × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
One Model × Joint prediction is supported by shared visual/state/action slots in one transformer and joint diffusion at inference. Actions originate inside that model, rather than a separate inverse-dynamics network. This architectural evidence goes beyond joint losses. Data efficiency is studied; real-time speed is not benchmarked. e-architecturee-finetunee-controle-horizon
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 1. Spend video and action labels on different learning stages
The starting point is the cost asymmetry between recordings and synchronized robot commands. In Stage 1, the future-video denoiser can learn from an observation history without receiving kinematic labels. Equation (2) asks it to recover added noise from future-video latents, not to infer missing actions. The implementation description initializes this stage from the surgical video world model attributed to He et al. Its visual and future-prediction weights transfer to Cosmos Policy, while action and state slots are initialized fresh. Stage 2 then supplies observations, robot states, task embeddings and commands together. Equation (3) jointly denoises action and future tokens through the full shared model. The source’s masked-versus-excluded slot descriptions remain unresolved, but both formulations make the absence of Stage 1 action supervision explicit. e-probleme-architecturee-pretraine-finetune
5.2 2. Follow a predicted chunk through one feedback cycle
Consider the default SurRoL cycle. The current endoscopic observation, robot proprioception and task embedding condition a diffusion sampler that generates 16 actions together with future-prediction slots. Each action specifies absolute Cartesian end-effector position, orientation and gripper command per arm. During denoising, action tokens attend to denoised future tokens; the predicted future therefore participates in producing the command sequence. Only four actions are executed before observing again. The remaining actions and future outputs are discarded, and a new sample starts from updated context. The reader’s interpretation is that feedback limits how long a prediction error can accumulate before replanning, although the paper does not isolate that causal benefit. Table 3 measures success across execution prefixes, while leaving the associated wall-clock latency unreported. e-controle-setupe-horizon
5.3 3. Distinguish a useful initialization from a proven dynamics mechanism
The most interpretable empirical contrast is the pair of WAM rows in Table 1: architecture, action representation and downstream data are matched, with additional surgical video pretraining providing the main difference. Improvements on all four tasks support the practical usefulness of that initialization under this protocol. They do not isolate which visual features or temporal dependencies cause the gains. Table 2 adds another constraint: a pretrained model can improve sooner and still deteriorate with more fine-tuning, so checkpoint selection belongs in any replication plan. Table 3 shows a benefit across execution horizons without proving an optimal latency–success tradeoff. Finally, Figure 3 concerns recorded real scenes, not a quantified autonomous deployment. These distinctions keep the central simulation result useful while leaving mechanism and real-world transfer as separate tests. e-setupe-maine-stepse-horizone-real
5.4 Training and inference
During training
Stage 1 uses the surgical video world model attributed to He et al. (2026), adapted from a general video diffusion backbone. Section 3.2 excludes action/state slots from input and supervision, transfers visual/future weights, and initializes action/state slots fresh. Figure 2 instead describes action slots as masked or zeroed; that implementation difference remains unresolved. e-pretraine-architecture
Stage 2 fine-tunes the full model on 10,000 SurRoL demonstrations shared across methods and ablations. Defaults are 80,000 optimization steps, batch size 1, learning rate 10^{-4}, and five inference denoising steps. Optimizer, image resolution and schedule are said to be shared, but their identities or values are omitted. e-finetunee-setup
During inference
Sample future/action slots jointly, execute the first H_e actions, then re-observe and replan. Defaults are H_c=16 predicted actions and H_e=4 executed actions. Action tokens attend to denoised future tokens at every diffusion step; only actions reach actuators. Future sampling therefore contributes inference computation, beyond an auxiliary training loss. No separate rollout scoring or search is described. e-controle-setup
5.5 Implementation flow
- Represent observations and actions together
Dedicated visual, robot-state, future-prediction and action slots occupy one latent sequence processed by a single diffusion transformer. Compatibility with video-only backbones permits visual-weight transfer; no separate inverse-dynamics action decoder is used. e-architecture
- Ground predictions in commands
Stage 2 pairs observation/state/task context with action chunks and corresponding future targets. Jointly noised tokens are denoised through shared attention and visual-prediction parameters, coupling video-derived representations to action prediction. e-finetune
6. Experiments & results
Surgical WAM adapts Cosmos Policy so one diffusion transformer jointly predicts future endoscopic observations and robot actions. Surgical video pretraining precedes fine-tuning on a fixed action-labeled dataset. Receding-horizon execution converts these predictions into simulated dVRK control. The reported four-task mean improves from 63.5% to 77.8%, but incomplete protocols and inconsistent learning-efficiency presentations limit reproducibility; real-video evidence remains qualitative.
6.1 Read the original evidence
Table 1. Additional surgical video pretraining improves every WAM task result under the reported fixed-label protocol. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the bottom two rows first, keeping the column order fixed: Needle Pick, Peg Transfer, Needle Regrasp and BiPeg Transfer. They compare the same WAM without and with surgical video pretraining, denoted PT. Their shared protocol uses 10,000 labeled demonstrations and the default 80k fine-tuning steps; Section 4.1 states 100 evaluation episodes per result. Then compare each column with the baseline rows above. Bold identifies the best reported entry, so the BET value in Needle Regrasp matters. Dashes in the BiPeg Transfer column indicate absent baseline results and cannot be included as zero-success measurements. e-maine-setup
What it supports. Success changes from 96/66/50/42% to 99/86/62/64%. The reported four-task mean rises from 63.5% to 77.8%; the largest task gain is 22 percentage points on BiPeg Transfer. Surgical WAM with PT leads three columns, while BET reaches 63% versus its 62% on Needle Regrasp.
Where the evidence stops. These are simulated closed-loop outcomes. The paper reports matched data and fine-tuning budgets but omits splits, seed aggregation and uncertainty. The architecture-matched WAM contrast supports the pretraining comparison more directly than attributing every cross-baseline gap to visual dynamics.
Figure 3. Real dVRK teleoperation recordings illustrate the data domain, while quantitative transfer remains unspecified. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow each row from left to right. The upper sequence depicts knot tying and the lower depicts suturing, as identified by the source caption. Tools, thread and the suturing surface make the visual domain concrete, but there are no generated-versus-observed labels, policy-comparison columns or numerical axes. These are representative JIGSAWS teleoperation frames used in the real-data experiments. Section 4.4 says that applying the same two-stage protocol produces qualitatively stronger downstream policies from pretrained initializations. The frames themselves illustrate the recordings; they do not show a controlled performance comparison between those policies. e-reale-conclusion
What it supports. The paper extends its discussion beyond simulator imagery by using real-hardware dVRK recordings. What this visual directly establishes is the type of surgical footage considered. A qualitative transfer claim accompanies it, but no numerical improvement, task-success comparison or autonomous execution outcome can be read from these frames.
Where the evidence stops. These are teleoperation recordings, not demonstrated autonomous Surgical WAM rollouts. Section 4.4 gives no quantitative metric, split or real-robot evaluation protocol. The conclusion that gains are not a simulator artifact therefore remains an author claim rather than independently checkable transfer evidence.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| SurRoL four-task mean success Same 10,000 demonstrations; defaults 80k fine-tuning steps, H_c=16, H_e=4. Section 4.1 states 100 evaluation episodes per result; held-out split and seed aggregation unspecified. | 77.8% with surgical video pretraining Closed-loop task success rate (%) | 63.5% without pretraining; reported improvement 14.3 percentage points. Rounded mean of simulated task execution; no clinical performance or video-for-label substitution rate is established. e-setupe-main |
| SurRoL Peg Transfer Table 1; fixed labeled data and fine-tuning budget. | 86% Closed-loop task success rate (%) | 66% without PT; DEX 77%, BET 73%, ALOHA 14%, Diffusion 2%, pi_0.5 36%. Matched WAM gain: 20 percentage points. PT means additional surgical video pretraining. e-setupe-main |
| SurRoL Needle Pick Table 1 protocol. | 99% Closed-loop task success rate (%) | 96% without PT; DEX 98%. Three-point within-WAM gain near the success ceiling; no uncertainty supplied. e-setupe-main |
| SurRoL Needle Regrasp Table 1 protocol; bimanual task. | 62% Closed-loop task success rate (%) | 50% without PT; BET 63%. PT adds 12 points but does not lead this task. e-setupe-main |
| SurRoL BiPeg Transfer Table 1 protocol; bimanual task. | 64% Closed-loop task success rate (%) | 42% without PT; pi_0.5 33%; other baseline entries unreported. Largest within-WAM gain: 22 points. Missing entries are not zeros. e-setupe-main |
| Fine-tuning-step ablation Table 2; labeled dataset and pretraining corpus fixed. Task identity or aggregation unspecified. | PT: 52%, 86%, 34%, 56.5% at 40k, 80k, 120k, 160k steps. Closed-loop success rate (%) | No PT: 41.5%, 65.5%, 71.5%, 79%, respectively. PT peaks with half the fine-tuning steps, excluding pretraining cost. Authors attribute later degradation to overfitting without a diagnostic establishing its cause. e-steps |
| Execution-horizon ablation Table 3; H_c=16 and model parameters fixed; task identity or aggregation unspecified. | PT: 86%, 70%, 86%, 80% for H_e=1,2,4,8. Closed-loop success rate (%) | No PT: 50%, 52%, 66%, 60%. PT improves all tested horizons, nonmonotonically. No measured latency establishes the optimal real-time operating point. e-horizone-control |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2. The pretrained model peaks earlier, then loses much of its measured success at later checkpoints. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each row horizontally to compare equal fine-tuning steps, and then read down each model column to assess checkpoint sensitivity. At 80k steps, PT gives 86% while no PT gives 65.5%. The final Best row instead compares maxima attained at different budgets: 86% at 80k versus 79% at 160k. That distinction separates a fixed-budget comparison from a best-checkpoint comparison. The dataset and video corpus are held fixed according to Section 4.3. Unlike the main-results table, this table does not identify a task or explain whether its numbers aggregate multiple tasks or runs. e-stepse-setupe-curvee-pretrain
What it supports. The reported pretrained peak takes half as many fine-tuning steps as the unpretrained peak. However, PT falls to 34% at 120k and recovers only to 56.5% at 160k. This supports substantial checkpoint sensitivity; the authors’ overfitting explanation is a proposed cause, not an isolated mechanism test.
Where the evidence stops. Neither aggregation nor uncertainty is specified, and half-point values need clarification against the stated 100-episode protocol. Do not equate this table with the four-task mean or Figure 1’s curve. Stage 1 computation is excluded from the fine-tuning-step comparison.
Table 3. Changing the number of actions executed between observations reveals a nonmonotonic feedback–computation tradeoff. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The row variable H_e counts actions actually executed before the controller observes again. It does not change H_c, the predicted chunk length, which remains 16; model parameters also stay fixed. Compare the two columns within each row, then inspect how each column changes with H_e. The pretrained results are 86%, 70%, 86% and 80%, whereas the unpretrained results are 50%, 52%, 66% and 60%. The default H_e=4 uses only the beginning of each sampled chunk. Section 3.3 explains why shorter execution intervals require sampling more often and longer intervals postpone feedback. e-horizone-controle-setup
What it supports. PT improves all four tested horizons and stays at or above 70%, but more frequent replanning does not monotonically improve success: H_e=2 underperforms H_e=4 here. The table supports a benefit across the tested settings, rather than a universal rule that the shortest execution prefix is best.
Where the evidence stops. The task or aggregation is unspecified. No measured latency, sampling throughput or control frequency accompanies these success rates. Thus the source’s choice of H_e=4 as a balance cannot establish an optimal real-time operating point.
Figure 1(ii). The visible curve favors pretraining at its final point, but its caption describes a different baseline checkpoint. Original paper, p. 2 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the horizontal axis in thousands of optimization steps and the vertical axis as success percentage. Orange squares denote pretraining; blue circles denote no pretraining. At the rightmost displayed point, 50k steps, the orange endpoint is labeled 62% and the blue endpoint 38%. Both shaded envelopes are retained, but the source does not define their statistical meaning. The nearby caption calls the curve an average over the SurRoL suite and compares 62% at 50k against 50% at 60k. There is no displayed 60k point, so the caption’s comparison cannot be checked against this plotted range. e-curvee-stepse-main
What it supports. The visible 50k endpoints favor the pretrained model and motivate examining learning efficiency. They cannot independently support the caption’s stated 50%-at-60k baseline peak. Table 2 provides a separate budget study, but its different checkpoints and missing task aggregation prevent joining the two into a single quantitative learning trajectory.
Where the evidence stops. The plot–caption discrepancy remains unresolved in this revision. Neither shaded region can be labeled a confidence interval or standard deviation. Use the directly legible endpoints as figure observations, without replacing the main benchmark or treating the caption’s extra point as plotted evidence.
7. Analysis & limitations
7.1 What the evidence leaves open
Figure 1(ii) labels PT 62% and no-PT 38% at 50k steps; its caption compares 62% at 50k with 50% at 60k, beyond the axis. Table 2 uses another schedule and unspecified aggregation. These cannot be merged into one verified learning curve; shaded bands are undefined. e-curvee-steps
JIGSAWS evidence comprises teleoperation frames and a qualitative claim of stronger policies, without numerical results, splits, metrics or documented autonomous real-robot evaluation. The conclusion that gains are not a simulator artifact exceeds what can be independently checked. e-reale-conclusion
No video-volume or action-label-budget sweep is presented, despite introductory language about varying video volume. Task allocation, seed counts and uncertainty are missing. Tables 2–3 omit aggregation; fractional percentages in Table 2 remain unexplained alongside the stated 100-episode protocol. e-probleme-setupe-maine-stepse-horizon
Figure 2 mentions goal/task-text conditioning, Section 3 uses a task embedding, and Section 4.2 claims no language supervision. Encoding and supervision details do not resolve this distinction. e-architecturee-main
7.2 Questions for discussion
- Would an equal-compute pretraining control preserve the matched-label improvement? [e-pretrain, e-setup]
- Does interrupting future-token information flow reduce executed task success? [e-control]
8. Reproducibility audit
8.1 Requirements and known gaps
A faithful attempt needs exact Cosmos Policy and surgical-video checkpoints, corpus composition/overlap checks, tokenizer and latent layout, video horizon, state/action normalization and coordinate conventions. The source omits checkpoint identifiers, corpus scale, Stage 1 schedule and detailed preprocessing. e-architecturee-pretraine-setup
Recover splits, per-task counts, success thresholds, episode limits, simulator version, optimizer/schedule and image size. GPU models/counts, training duration, software versions and control latency are absent. Proposed checks should distinguish matched-step from validation-selected success and test dependence on future tokens under controlled execution horizons. e-setupe-stepse-horizone-control
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate pretraining gains from checkpoint choice and extra optimization
Reader-proposed check, not performed: hold the same 10,000 demonstrations, per-task evaluation initializations, H_c=16, H_e=4 and sampler fixed. Compare no additional surgical PT, intact surgical-video PT, and a control with the same Stage 1 clips/updates but temporally shuffled future frames. Evaluate all three at 40k, 80k, 120k and 160k Stage 2 steps across multiple training seeds. Report per-task success and uncertainty, both at prespecified budgets and at checkpoints chosen on a separate validation split; disclose total pretraining compute. An intact-video advantage over shuffled video would support a role for temporal structure. Equal performance would weaken that attribution even if either pretraining condition still beats no PT. e-pretraine-setupe-maine-steps
Check 2: Test whether future-token coupling matters during execution
Reader-proposed check, not performed: start from the same pretrained checkpoint and compare standard joint sampling with an intervention that blocks action-token attention to future-prediction tokens. Retain current-observation/state conditioning, token count, five denoising steps and random seeds, so this first comparison does not claim a speed benefit. Evaluate the same task episodes at H_e=1,2,4,8 with H_c=16 and measure success plus end-to-end sampling latency. A reproducible success loss after blocking attention would support dependence on future-token information at inference; unchanged success would weaken its necessity for these tasks. Because the attention intervention changes the inference distribution, add a separately adapted masked-attention control before attributing any loss specifically to useful imagined dynamics. e-controle-setupe-horizon
8.3 Reading coverage
Visual audit: All ten supplied PDF pages were rendered and visually inspected, including the title/authors/version, Figure 1 and its caption, Figure 2 with Equations (1)–(3) and training/inference text, Table 1 with evaluation settings, Tables 2–3, Figure 3 and the real-data discussion, conclusion and references. Six final original crops were viewed. The learning-curve crop was widened and viewed again to retain the full 38% endpoint label. Architecture arrows and conditioning/output labels were cross-checked with the caption and method; the Stage 1 slot-handling and loss-label differences are disclosed. The Figure 1 plot–caption discrepancy remains disclosed. The supplied PDF has no appendix; no separate supplement or code was inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, author block and Abstract (p. 1)
- 1 Introduction (pp. 1–3)
- 2 Related Work, including 2.1–2.3 (pp. 3–4)
- 3 Method, including 3.1 Surgical World-Action Model, 3.2 Two-Stage Training and 3.3 Closed-Loop Policy Execution (pp. 4–5)
- 4 Experiments, including 4.1–4.4 (pp. 5–7)
- 5 Conclusion (pp. 7–8)
- References (pp. 8–10)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Identity: the inspected title and all six authors match the catalog. The title page identifies arXiv:2608.11204v1 [cs.RO], 11 August 2026. Only this supplied revision was reviewed; no comparison with other editions was possible.
- The title page lists University of Central Florida and AdventHealth, resolving the catalog affiliation placeholder. Tianyun Jiang and Zhiben Chen have no affiliation superscripts; their individual affiliations remain unassigned.
- All four supplied text chunks were read in full. All ten PDF pages and the six final crops were visually inspected, resolving the acquisition warning that text extraction alone does not reconstruct figure images.
- Separate supplemental material availability has not been fully verified; no separate supplement was supplied. The PDF contains no appendix. The apparent extracted heading beginning ‘A Large Vision-Language Model’ belongs to a reference title on p. 10.
- Code, external cited papers and repositories were not inspected; no experiments were reproduced. The preliminary report was not used as scientific evidence.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title/author/affiliation block and arXiv margin
Exact title matches the catalog; authors are Wenrui Bao, Tianyun Jiang, Zhiben Chen, Ser-Nam Lim, Peter D. Peng and Yuzhang Shang. Margin: 2608.11204v1, 11 Aug 2026. Institutions: University of Central Florida and AdventHealth; Jiang and Chen lack affiliation markers.
Go to primary source ↓e-problemPDF pp. 1–3, Abstract and Section 1; p. 2, central question and contributions
The question fixes action labels while adding video pretraining. Introduction describes varying video volume and claims first surgical WAM application; presented results later compare PT/no-PT.
Go to primary source ↓e-architecturePDF p. 4, Figure 2 and caption; Section 3.1, Eq. (1), continuing p. 5
One Cosmos Policy transformer processes shared visual/state/action slots. Figure shows current proprioception/image conditioning and action/future-proprioception/future-image outputs. Caption adds goal/task text and masked action frames with L_vid/L_joint; Eq. (1) uses histories and task embedding.
Go to primary source ↓e-pretrainPDF p. 5, Section 3.2, Stage 1 and Eq. (2)
Video-only denoising excludes action/state input and supervision. He et al.’s surgical video model supplies visual/future weights; action/state slots start fresh. Corpus scale, checkpoint identifiers and Stage 1 schedule are unspecified.
Go to primary source ↓e-finetunePDF p. 5, Section 3.2, Stage 2 and Eq. (3)
Full-model fine-tuning uses synchronized observations, states, actions and task embeddings. Jointly corrupted action/future tokens share attention and visual-prediction parameters; epsilon is added noise.
Go to primary source ↓e-controlPDF p. 5, Section 3.3, Joint sampling of video and action slots; pp. 5–6, Section 4.1 action representation
Execute H_e<=H_c actions then reobserve. Action tokens attend to denoised future tokens each diffusion step; future outputs are discarded. Commands specify absolute Cartesian position, orientation and gripper state per arm.
Go to primary source ↓e-setupPDF pp. 5–6, Section 4.1, tasks, demonstrations, compared variants, evaluation metric and implementation details
Four SurRoL tasks use 10k demonstrations, default H_c=16/H_e=4, 80k fine-tuning steps, batch 1, learning rate 1e-4 and five denoising steps. Each result reportedly uses 100 episodes. WAM variants share architecture, baseline initialization and optimization except additional PT. Split, aggregation, detailed success definitions and hardware are unspecified.
Go to primary source ↓e-mainPDF p. 6, Table 1, all rows/columns; pp. 6–7, Section 4.2
Needle Pick/Peg Transfer/Needle Regrasp/BiPeg Transfer: PT 99/86/62/64%; no PT 96/66/50/42%; BET 78/73/63/unreported; DEX 98/77/56/unreported; ALOHA 95/14/11/unreported; Diffusion 21/2/2/unreported; pi_0.5 8/36/12/33%. Text reports means 77.8 vs 63.5%, 14.3-point gain and no language supervision.
Go to primary source ↓e-stepsPDF p. 7, Table 2 and Section 4.3, Effect of Fine-Tuning Steps
40k/80k/120k/160k: no PT 41.5/65.5/71.5/79%; PT 52/86/34/56.5%. Best: 79 and 86%. Authors attribute late degradation to overfitting. Task/suite aggregation and uncertainty unspecified.
Go to primary source ↓e-horizonPDF p. 7, Table 3 and Section 4.3, Effect of the Execution Horizon
Fixed H_c=16 and model parameters. H_e=1/2/4/8: no PT 50/52/66/60%; PT 86/70/86/80%. Default H_e=4. Timing and task aggregation absent.
Go to primary source ↓e-curvePDF p. 2, Figure 1(ii), endpoint annotations, axes, legend and caption
Plot ends at 50k: PT 62%, no PT 38%. Caption compares PT 62% at 50k with no PT 50% at 60k, calling it a suite average. No 60k point appears; shaded bands are undefined.
Go to primary source ↓e-realPDF p. 7, Figure 3 and caption; Section 4.4
Five frames per row show JIGSAWS dVRK teleoperation: knot tying above, suturing below. Text claims qualitative benefit under the same two-stage protocol but supplies no numerical metric, split or autonomous execution protocol.
Go to primary source ↓e-conclusionPDF pp. 7–8, Section 5
Conclusion repeats simulation gains, claims JIGSAWS confirms benefit beyond simulation, and proposes scaling unlabeled video to reduce dependence on action labels.
Go to primary source ↓8.5 Primary sources
Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning ↗
PDF · 5,925 extracted words
Source fingerprint
ca3d4b054276606d74572b8aed63550377ef5b49377548ffb2591f3f85919e54