Spatially Aware World Action Model via Geometric Latent Diffusion
1. Paper overview
In one sentence: SA-WAM fits metric depth into a frozen video tokenizer and jointly predicts geometry and actions, improving tested manipulation while adding a geometric input and prediction burden. e02e04e08e09e10e14e18
| At a glance | What to know |
|---|---|
| Research problem | Source description The authors target the lack of explicit metric geometry in RGB-based world action models. The technical challenge is to preserve manipulation-relevant depth precision while mapping depth into a frozen video tokenizer's bounded input domain, avoiding a separately trained geometry encoder. e01e04 |
| Core mechanism | |
| A key reported result | LIBERO-Plus robustness: 86.6 Weighted-average success rate, %. Zero-shot evaluation after training only on original LIBERO demonstrations; episode-weighted aggregation across seven perturbation axes Cosmos-Policy: 81.4; π0.5: 84.6 Reported gains are 5.2 and 2.0 percentage points, respectively. This aggregate does not establish improvement on every perturbation axis. Sensor-noise evaluation applies the same perturbation to RGB and depth. e06e11 |
| Reading caution | Author claim The authors acknowledge that predicted futures and actions can remain inconsistent and that inference efficiency needs improvement. Shared generation therefore does not guarantee geometrically consistent outcomes. e19 |
Core contributions
Figure 2. Geometry enters the same generative sequence as actions and future appearance. Original paper, p. 2 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the bottom: wrist, left and right RGB observations are paired with depth images. Pink boxes normalize metric depth before the frozen VAE encodes it; RGB and depth then occupy separate slots. The black dashed group contains observed conditioning, including robot state q_t. The red dashed group contains noisy targets. Follow those targets through the shared DiT to the action chunk a_t and horizon-end RGB-D latents. The top decoder turns predicted latents into images only when needed. Text conditioning enters separately from the left. Snowflakes mark frozen components; the flame marks the trainable backbone. e02e03e05
What it supports. The architecture makes depth both an observed input and a predicted future modality. Actions are generated directly in the shared sequence, so this figure supports joint prediction within one backbone. It does not show a separate inverse-dynamics model extracting actions from a completed video.
Where the evidence stops. Some right-camera latent boxes repeat the word “left” in the original artwork. Their r superscripts and the lower camera labels identify the right view; Section III-A uses w, l and r.
2. Motivation
2.1 The problem and the proposed response
The authors target the lack of explicit metric geometry in RGB-based world action models. The technical challenge is to preserve manipulation-relevant depth precision while mapping depth into a frozen video tokenizer's bounded input domain, avoiding a separately trained geometry encoder. e01e04
2.2 What this reading follows
SA-WAM asks a practical question: can a video-based policy use metric geometry without learning a new geometry encoder? Its answer is to transform depth into the input format of the existing frozen tokenizer and place the resulting latents beside RGB and action slots. The important choice is not simply to add depth, but to decide which distances deserve precision after compression. Read the normalization ablation before the headline benchmark: it connects that interface choice to both reconstructed depth and executed tasks. The later figures distinguish physical partial-credit results from a simulator-only diagnostic that links prediction mismatch to failure. e02e04e08e09e10e14e18
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | One Model |
| Prediction paradigm | Joint prediction |
| Quadrant | Q1 · One Model × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded Q1 classification is supported by architecture evidence: one DiT jointly denoises action and future RGB/depth slots, without a separate action head or inverse-dynamics stage. Future observations participate in inference-time generation even when decoding is skipped. Multiple camera-aligned metric-depth streams support the 3D multiview label, although the method does not establish an explicit fused 3D scene representation or enforce cross-view geometric consistency. e02e03e04e05e19
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Make metric depth fit a frozen visual tokenizer
A pretrained video tokenizer expects bounded three-channel images, while a depth sensor produces positive metric distances whose useful range depends on camera placement. SA-WAM resolves this interface mismatch before the neural network sees depth. It estimates the second and 98th training-depth percentiles separately for each camera, clamps observations to that interval, applies a monotone normalization, rescales to minus one through one, and replicates the result across three channels. Log normalization has sensitivity proportional to the inverse distance, between linear encoding and the stronger near-field emphasis of inverse depth. My interpretation is that this allocates more representational precision to manipulation distances while preserving useful context farther away. The round-trip and policy ablations test different consequences of that choice; neither requires retraining the VAE. e03e04e08e09
5.2 Denoise one sequence, execute only the action prefix
The clean target sequence contains an action chunk together with the future RGB and depth latents. The EDM objective corrupts this sequence with Gaussian noise and trains the DiT to reconstruct it, conditioned on current RGB-D, proprioception and instruction. Thus future depth contributes through the same denoising problem as future appearance and control, rather than through an isolated auxiliary head. At deployment the model still jointly predicts those latent slots, but decoding predicted images is optional. In simulation it predicts 32 actions and executes the first 16 before replanning. The UR5 setting instead predicts three semantic key-step commands and executes two; each command specifies an absolute end-effector pose and gripper aperture. These execution schedules should be reproduced separately. e02e05e07e15e16
5.3 Keep representation quality, control and diagnosis separate
The evidence forms three linked tests with different endpoints. Tokenizer reconstruction measures whether geometry survives encoding; the manipulation benchmarks measure whether the resulting policy completes tasks; the post-grasp diagnostic asks whether prediction mismatch is associated with failure. The 0.63% depth reconstruction error therefore cannot be interpreted as a control error, and the approximately 0.88 diagnostic AUC cannot be interpreted as a policy success rate. My reading is that the alignment among these results makes the proposed representation plausible, but does not identify every causal path behind the task gains. Depth also changes the information available to the policy. A stronger causal analysis would separate observed geometry from future-depth supervision and test each under the same data and optimization budget. e08e09e10e14
5.4 Training and inference
During training
Simulation training fine-tunes the 2B-parameter DiT while freezing the VAE. RoboCasa/LIBERO use 40 H100 GPUs, effective batches of 800/960, and 45,000/40,000 iterations. AdamW peaks at 10⁻⁴; after warm-up, learning rate decays until step 30,000, drops fivefold, then remains constant. Noise comes from the base log-normal distribution with probability 0.7 and U(1, 85) with probability 0.3. e07
The UR5 dataset has 20 demonstrations for each of 10 tasks, with 3–11 semantic key steps per episode. SA-WAM and Cosmos-Policy train for 2,400 iterations on eight H100 GPUs with batch size 192, a 300-step warm-up, and linear learning-rate decay to zero. e15e16
During inference
Actions and future RGB-D latents undergo joint denoising; decoding future observations is optional. Simulation predicts 32 actions and executes the first 16 open-loop. e02e07
UR5 inference predicts three semantic key-step actions and executes two. Each action specifies an absolute end-effector position, quaternion orientation, and gripper aperture in an eight-dimensional vector; these are waypoint commands rather than dense control-rate deltas. e15e16
The described action path uses directly generated action slots. The supplied method does not introduce inverse dynamics, candidate-trajectory search, or an inference-time controller driven by the later geometric-error diagnostic. e02e05e14
5.5 Implementation flow
- Convert depth into tokenizer-compatible frames
For each camera, collect valid training depths, estimate the second and 98th percentiles, and clamp observations to those bounds. Normalize depth, rescale it to [−1, 1], and replicate the scalar into three channels. The retained log encoding emphasizes nearby geometry while preserving more distant resolution than inverse depth. e04
- Assemble the shared representation
Encode RGB and normalized depth with the same frozen Wan2.1 VAE. Current RGB/depth pairs, proprioception, and instruction provide conditioning. Actions occupy latent-sequence positions directly; target slots contain the action chunk and horizon-end RGB/depth latents. e02e03e05
- Jointly denoise actions and future observations
Fine-tune the Cosmos-Predict2 diffusion transformer to reconstruct all target slots under one EDM objective. Future geometry participates in the generative target rather than being supplied only through a separate training loss. e05
6. Experiments & results
SA-WAM adds metric depth to a pretrained video diffusion policy by encoding log-normalized depth through the same frozen tokenizer used for RGB. A shared transformer jointly predicts actions and future RGB-D observations. Its strongest evidence combines a controlled depth-normalization ablation, improved simulated task success, and physical UR5 evaluations scored with partial credit. The prediction-error analysis establishes an association with failures, without demonstrating a deployed failure detector.
6.1 Read the original evidence
Table II. The closest data-budget comparison is the final pair of rows. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First inspect the training-demonstration column. Most earlier methods use 300 or 1,000 demonstrations per task, whereas Cosmos-Policy and SA-WAM both use 50. Then compare those final two rows across pick-and-place, open/close, turn/toggle and coffee tasks. The overall column averages 24 tasks, with three evaluation seeds stated in the protocol. Category averages are not equally weighted substitutes for the overall result: the categories contain different numbers of tasks. Dashes indicate missing category-level results for some baselines, so the table supports fewer detailed cross-method comparisons than its overall ranking might suggest. e06e10
What it supports. SA-WAM reaches 76.6% overall versus 67.1% for Cosmos-Policy, a 9.5-point gain at the same demonstration count. Pick-and-place improves from 51.8% to 68.5%, and turn/toggle from 66.3% to 77.5%. These are executed simulated-task results, separate from image quality. The displayed gains extend across all four task categories.
Where the evidence stops. Equal demonstration counts do not imply equal input information: SA-WAM receives metric depth in addition to RGB. The table lacks uncertainty intervals and does not establish equal pretraining exposure or compute across all listed baselines.
Figure 5(a)-(b). Physical robustness is measured with subgoal credit, not a binary success rate. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare each green SA-WAM bar with the yellow Cosmos-Policy bar within the same category, then compare clean and randomized panels. The vertical scale is completion score out of ten trials. A partially completed multistage trial contributes fractional credit: one completed subgoal is worth 0.5 in two-stage tasks or 0.25 in four-stage tasks. This explains totals such as 6.5. The four categories are put-in-plates, put-in-box, mug-to-hanger and stack-cups. Randomization adds visual changes and distractors to the changed object positions already present in clean evaluation. The same colors identify policies in both panels. e15e16e17e18
What it supports. SA-WAM totals 36 out of 40 completion credits in clean scenes and 31 out of 40 under randomization, corresponding to 90.0% and 77.5%. Cosmos-Policy totals 30 and 19.5. The preserved advantage under randomization supports this physical test of geometry-assisted control.
Where the evidence stops. Each condition has only ten trials per category. These fractions include partial progress and must not be reported as the proportion of fully successful episodes. The physical controller also executes semantic waypoints rather than the simulation action sequence.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Depth reconstruction through the frozen tokenizer Eight-task RoboCasa study; encode–decode reconstruction, aggregated across camera views | Log: 0.63 Absolute Relative Error, %, lower is better | Linear: 1.47; inverse depth: 1.41 Reported reconstruction measurements support log scaling's overall fidelity. It is not best in every depth range: inverse depth is strongest below 0.3 m and linear scaling beyond 1 m in the wrist-view breakdown. These are tokenizer reconstruction results, not future-prediction or control scores. e08 |
| Depth-normalization policy ablation Eight RoboCasa tasks; 50 demonstrations per task; 6,000 gradient steps; three evaluation seeds with 50 trials per task per seed | RGB-D with log normalization: 61.7 Task success rate, % | RGB-only: 52.7; linear depth: 56.5; inverse depth: 59.5 The matched training protocol supports choosing log normalization among these encodings. It does not separately identify the effects of depth conditioning and future-depth supervision. e09e02 |
| RoboCasa manipulation 24 atomic tasks; 50 demonstrations per task for SA-WAM and Cosmos-Policy; average across three seeds | 76.6 Average task success rate, % | Cosmos-Policy: 67.1; reported improvement: 9.5 percentage points This is simulated execution success. Category results favor SA-WAM throughout, with the largest reported gains in pick-and-place and turn/toggle tasks. Other table entries use different demonstration budgets. e06e10 |
| Standard LIBERO Evaluation across the four standard suites | 98.4 Average success rate, % | The authors describe performance as comparable to the strongest policies; no accompanying per-suite numerical comparison is supplied here. The reported aggregate indicates strong performance on the standard benchmark, but does not establish a suite-specific advantage. e06e12 |
| LIBERO-Plus robustness Zero-shot evaluation after training only on original LIBERO demonstrations; episode-weighted aggregation across seven perturbation axes | 86.6 Weighted-average success rate, % | Cosmos-Policy: 81.4; π0.5: 84.6 Reported gains are 5.2 and 2.0 percentage points, respectively. This aggregate does not establish improvement on every perturbation axis. Sensor-noise evaluation applies the same perturbation to RGB and depth. e06e11 |
| Future RGB prediction 24 RoboCasa tasks; wrist-camera predictions compared with simulator renders; masked evaluation covers robot, gripper, and target object | Wrist PSNR: 15.16; wrist LPIPS: 0.354; masked wrist LPIPS: 0.321 PSNR, higher is better; LPIPS, lower is better | Cosmos-Policy: 14.64, 0.417, and 0.381, respectively These reported image-prediction metrics improve, including in manipulation-relevant regions. They are separate from executed-task success and do not establish that prediction improvements cause better control. e13 |
| Diagnosing pick-and-place rollout failures Object-centric wrist-depth error after a grasp attempt, using ground-truth simulator masks and realized future depth | AUC ≈ 0.88; 80% failure detection at 13% false-positive rate ROC AUC and failure-detection operating point | Successful episodes have lower geometric prediction error than failed episodes. These reported values are legible in the body text. The analysis measures retrospective outcome separability with simulator information; deployment as a prospective physical-robot detector is untested. e14 |
| Physical UR5 manipulation in clean environments Four composite categories; 10 trials per category; object positions differ from demonstrations | 90.0 Aggregate completion score with subgoal partial credit, % | Cosmos-Policy: 75.0; π0: 21.3 This is measured physical execution, but the score credits partially completed tasks and should not be reported as binary episode success. e17e18 |
| Physical UR5 manipulation under randomization Four composite categories; 10 trials per category; added distractors and visual perturbations | 77.5 Aggregate completion score with subgoal partial credit, % | Cosmos-Policy: 48.8; π0: 11.3 The reported physical results support robustness in this tabletop waypoint-control setting. They do not establish performance for dense control or broader robot platforms. e15e17e18 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table I(a)-(b). The tokenizer test and the policy test favor a balanced depth encoding; the dagger marks the retained configuration. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the upper table as a frozen-tokenizer round trip, rather than a future-prediction experiment. Values are absolute relative depth errors in percent, so lower is better. Overall combines cameras, while the distance bins belong only to the wrist view. Linear encoding loses near-field detail; inverse depth preserves that detail but performs poorly beyond one metre. The lower table changes the question to executed task success. Its RGB row is the baseline, and the three RGB-D rows vary the depth normalization. These policies share the eight-task setup, 50 demonstrations per task and 6,000 training steps. e04e08e09
What it supports. Log encoding gives the lowest overall reconstruction error, 0.63%, and the highest ablation success, 61.7%. Relative to RGB alone at 52.7%, that is a 9.0 percentage-point increase. Log is not best in every distance bin: inverse depth remains more accurate below 0.3 m.
Where the evidence stops. The eight-task ablation is smaller and trained for fewer steps than the main RoboCasa experiment. Its success scores should not be compared directly with the 24-task results as if they used one protocol.
Figure 4(a)-(b). Future geometry can diagnose a failed interaction after reality has become observable. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the top panel vertically: the logarithmic metre scale spans orders of magnitude, and failed episodes tend to have larger object-centric 3D mismatch. Overlap remains, so error is not a perfect classifier. The lower panel sweeps a threshold on that mismatch; moving along the ROC trades detected failures against false alarms. The black point marks TPR 0.80 and FPR 0.13. To construct the score, the authors compare predicted and realized wrist depth at a short horizon after grasping, within a simulator-provided target-object mask, and back-project those pixels into 3D. e14
What it supports. The paper reports an AUC of approximately 0.88, consistent with useful discrimination in this pick-and-place diagnostic. At the labeled operating point, 80% of failures are detected with a 13% false-alarm rate. This connects geometric prediction error with outcome, without making prediction consistency equivalent to success.
Where the evidence stops. The diagnostic uses realized future observations and ground-truth simulator masks. It is neither a pre-action score available at the original decision nor a demonstrated physical failure detector; learned masks and intervention timing remain open.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors acknowledge that predicted futures and actions can remain inconsistent and that inference efficiency needs improvement. Shared generation therefore does not guarantee geometrically consistent outcomes. e19
RGB versus RGB-D changes both conditioning information and prediction targets. The normalization ablation isolates encoding choice more directly than it isolates the benefit of predicting future geometry. Equal demonstration counts also do not establish equal total pretraining or compute across methods. e02e09e10
The failure diagnostic depends on realized simulator depth and target-object masks. The exact short horizon, error aggregation, and threshold-selection split are not specified in the supplied description, limiting reproducibility and interpretation of the reported AUC. e14
The physical evaluation uses few trials per category, semantic waypoints, and partial-credit scoring. The reported aggregate scores lack uncertainty intervals, so the size and reliability of gains should be tested over additional trials. e15e17e18
7.2 Questions for discussion
- How much of the improvement comes from observing depth versus jointly predicting future depth? [e02, e09]
- How robust are fixed training-percentile depth bounds when deployment changes camera distance or workspace scale? [e04]
- Can a failure signal based on observed physical depth and estimated object masks retain the simulator diagnostic's discrimination under a held-out threshold protocol? [e14]
- Would explicit geometric consistency constraints improve action–future agreement without making inference substantially slower? [e19]
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the pretrained Cosmos-Predict2/Wan2.1 components, RGB-D demonstrations, camera-specific training-depth statistics, and the stated training schedules. Exact checkpoint revisions, numeric depth bounds, action-slot scaling, inference sampler settings, and latency are not established by the supplied main text. e02e04e05e07e16
A focused reproduction would first repeat the frozen-tokenizer round trip for all three normalizers, then reproduce the eight-task, 6,000-step policy ablation. Additional depth-input-only and future-depth-target ablations would test which geometric pathway drives the control gain. e08e09e02
Code was not inspected and experiments were not reproduced in this review. The project link and named software components identify potential starting points; they do not establish availability of runnable SA-WAM code, checkpoints, or datasets. e01e05e17
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate depth observation from depth prediction
Reader-proposed: keep the backbone, camera inputs, data, action horizon and optimization fixed, then compare RGB-only, current-depth-only and current-plus-future-depth variants. Match token count with masked placeholders where necessary. Report task success and masked future-image error over shared seeds. If future-depth supervision adds no repeatable gain beyond observed depth, the evidence would support geometric perception more strongly than a benefit from predicting geometry. e02e05e09e13
Check 2: Test the diagnostic without privileged masks
Reader-proposed: freeze the trained policy and compare the original simulator-mask diagnostic with estimated object masks on held-out rollouts. Choose thresholds on a separate calibration split and preserve the after-grasp timing. Measure AUC, false alarms and detection delay before testing any recovery action. A large collapse under estimated masks would identify a perception dependency that the published ROC does not resolve. e14e19
8.3 Reading coverage
Visual audit: Original PDF pages 2, 4, 5, 6 and 7 were visually inspected. Final crops contain Figure 2, Table I(a)-(b), Table II, Figure 4(a)-(b) and Figure 5(a)-(b), each re-opened after cropping at 200 DPI. Captions and surrounding methods text were read separately; no charts were reconstructed. Figure 1, supplementary videos, code and experiments are outside this illustration pass.
PDF pages inspected for this edition: 2, 4, 5, 6, 7. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and I. Introduction, pp. 1–2
- II. Related Work, pp. 2–3
- III-A. 3D Modality Injection for World Action Modeling, p. 3
- III-B. Training Objective, pp. 3–4
- IV-A. Experimental setup, p. 4
- IV-B. Ablations on the depth normalization approach, pp. 4–5
- IV-C. Comparison to the State of the Art, p. 5
- IV-D. World Modeling Quality Analysis, pp. 5–6
- IV-E. Prediction Quality versus Rollout Success, p. 6
- V. Real-World Experiments, pp. 7–8
- VI. Conclusion and Limitations, p. 8
- Acknowledgment and References, pp. 8–9
- Figure captions as supplied text
- Independent PDF visual audit: pp. 2–8, including architecture Figure 2, Equations (1)–(3), the EDM objective, Tables I–IV, and Figures 3–6
Outside the original text pass
- The full main text was reviewed; an independent visual audit inspected PDF pp. 2–8. Figure 1 was read as extracted text and caption only.
- No appendix appears in the supplied main paper. Separate supplementary material availability was not verified.
- No supplementary videos, source code, datasets or model executions were inspected.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/authors, Abstract, Introduction, and arXiv version line
The title and three authors match the catalog. The text identifies arXiv:2609.02531v1, dated 2 September 2026, introduces geometry-aware world action modeling, and supplies a project link.
Go to primary source ↓e02PDF p. 2, Figure 2 caption; PDF p. 3, Section III-A, opening paragraphs
Current instruction, proprioception, and RGB-D condition joint denoising of action chunks and future observation latents at t + H. Action/proprioception slots are inserted directly. The architecture uses no dedicated action head or geometry encoder, and future-frame decoding is optional. The shared DiT and target-slot layout were visually verified in Figure 2.
Go to primary source ↓e03PDF p. 3, Section III-A, Equation (1)
RGB and three-channel depth representations are encoded by the same frozen VAE into separate latent frames; the camera notation names wrist, left, and right views.
Go to primary source ↓e04PDF p. 3, Section III-A, Depth Injection, Equations (2)–(3) and sensitivity discussion
Per-camera training-depth percentiles P₂ and P₉₈ define clipping bounds. Linear, inverse-depth, and logarithmic mappings produce bounded three-channel inputs. Their sensitivities are constant, proportional to 1/z², and proportional to 1/z, respectively; mappings are invertible within the retained interval. Equation (2) parentheses and channel replication were verified in the rendered PDF, resolving damaged extraction glyphs.
Go to primary source ↓e05PDF pp. 3–4, Section III-B, Training Objective
Cosmos-Predict2 initializes the DiT and Wan2.1 supplies the frozen tokenizer. A Gaussian-corruption EDM reconstruction objective jointly supervises action, future RGB, and future depth slots without dedicated prediction heads or modality-specific losses.
Go to primary source ↓e06PDF p. 4, Section IV-A(a), Simulation Benchmarks
RoboCasa evaluation covers 24 atomic tasks and averages three seeds. LIBERO covers four standard suites. LIBERO-Plus is evaluated without adaptation after original-LIBERO training, across seven perturbation dimensions; sensor-noise perturbations are applied to depth as well as RGB.
Go to primary source ↓e07PDF p. 4, Section IV-A(b), Training Details
The 2B DiT is fine-tuned using 40 H100 GPUs, batches 800/960, and 45,000/40,000 iterations for RoboCasa/LIBERO. AdamW peaks at 10⁻⁴; the schedule changes at 30,000 steps. Noise mixes base log-normal and U(1,85) distributions with probabilities 0.7/0.3. Chunks have 32 actions, with 16 executed.
Go to primary source ↓e08PDF p. 4, Table I(a), Overall and wrist depth-range columns; Section IV-B(a)
Overall reconstruction AbsRel percentages are 1.47 for linear, 1.41 for inverse, and 0.63 for log. Below 0.3 m they are 6.68, 0.62, and 1.25; beyond 1 m they are 1.78, 13.19, and 3.19. Evaluation is a tokenizer encode–decode round trip. The cited table cells were visually verified in the retained PDF.
Go to primary source ↓e09PDF p. 4, Table I(b), caption, and Section IV-B(b)
The eight-task ablation uses 50 demonstrations per task and 6,000 gradient steps. Success averages three seeds with 50 trials per task per seed. RGB, linear RGB-D, inverse RGB-D, and log RGB-D score 52.7%, 56.5%, 59.5%, and 61.7%. The cited table cells were visually verified in the retained PDF.
Go to primary source ↓e10PDF p. 5, Table II, SA-WAM and Cosmos-Policy rows; Section IV-C(a)
With 50 demonstrations per task, SA-WAM averages 76.6% versus 67.1% for Cosmos-Policy. Pick-and-place scores are 68.5% versus 51.8%; turn/toggle scores are 77.5% versus 66.3%. Other compared methods use 300 or 1,000 demonstrations per task. The cited table cells were visually verified in the retained PDF.
Go to primary source ↓e11PDF p. 5, Table III, caption and Avg. SR row; Section IV-C(b)
LIBERO-Plus success is weighted by episode counts across seven perturbation axes. SA-WAM scores 86.6%, Cosmos-Policy 81.4%, and π0.5 84.6%, yielding reported advantages of 5.2 and 2.0 percentage points. The cited table cells were visually verified in the retained PDF.
Go to primary source ↓e12PDF p. 5, Section IV-C(b), opening sentence
The authors report 98.4% average success on standard LIBERO and describe it as comparable to the strongest policies on that benchmark.
Go to primary source ↓e13PDF p. 5, Section IV-D; PDF p. 6, Table IV, wrist and masked-wrist columns
RGB prediction metrics average 24 RoboCasa tasks; masks cover robot arm, gripper, and target object. SA-WAM/Cosmos-Policy wrist PSNR is 15.16/14.64, wrist LPIPS 0.354/0.417, and masked-wrist LPIPS 0.321/0.381. The cited table cells were visually verified in the retained PDF.
Go to primary source ↓e14PDF p. 6, Section IV-E and rendered Figure 4(a)–(b), including the labeled ROC operating point
The diagnostic back-projects predicted and realized wrist depth within ground-truth target-object masks after grasp attempts. The body reports lower error for successful pick-and-place rollouts, AUC approximately 0.88, and 80% failure detection at 13% false alarms. Figure 4 visually confirms the 0.80 true-positive / 0.13 false-positive operating point.
Go to primary source ↓e15PDF p. 7, Section V, Real-world UR5 setup
The physical platform uses a six-DoF UR5, RG6 gripper, and fixed Orbbec Femto Mega RGB-D camera. State/action vectors contain position, quaternion, and gripper aperture. The dataset provides 20 demonstrations for each of 10 tasks, with 3–11 semantic key steps per episode.
Go to primary source ↓e16PDF p. 7, Section V, Training details
SA-WAM and Cosmos-Policy train for 2,400 iterations on eight H100 GPUs with batch size 192, a 300-step warm-up, and decay to zero. They predict three key-step actions and execute the first two.
Go to primary source ↓e17PDF p. 7, Section V, UR5 results, evaluation protocol paragraphs
Four composite categories receive 10 trials each in clean and randomized settings. Randomization adds distractors and visual perturbations. Completed subgoals earn 0.5 in two-stage tasks and 0.25 in four-stage tasks. π0 is fully fine-tuned from official weights on the same key-step dataset with horizon three using LeRobot.
Go to primary source ↓e18PDF p. 7, Section V, final UR5 results paragraph and rendered Figure 5 category-score bars
The body reports aggregate clean completion scores of 90.0%, 75.0%, and 21.3% for SA-WAM, Cosmos-Policy, and π0. Randomized scores are 77.5%, 48.8%, and 11.3%, respectively. Visually checked Figure 5 shows SA-WAM category totals of 9/10, 10/10, 9/10 and 8/10 in clean scenes, and 6.5/10, 8/10, 8/10 and 8.5/10 under randomization; these retain partial credit.
Go to primary source ↓e19PDF p. 8, Section VI, Limitations paragraph
The authors acknowledge remaining inconsistencies between predicted futures and actions, suggest geometric training and verification as future work, and identify inference efficiency as an unresolved issue.
Go to primary source ↓8.5 Primary sources
Spatially Aware World Action Model via Geometric Latent Diffusion ↗
PDF · 6,094 extracted words
Source fingerprint
1f8731822b7960f7007cff9be8dd73dbab4a12257d829f83106317ee9e7c4569