Geometric Action Model for Robot Policy Learning
1. Paper overview
In one sentence: GAM makes geometric prediction part of action decoding inside a shared backbone, improving reported camera robustness while leaving the causal role of future supervision and several implementation details unresolved. e02e03e04e05e11e13e14e16
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors argue that policies built on image or video representations leave depth, scale and occlusion implicit. GAM instead reuses a geometric foundation model throughout perception, temporal prediction and action decoding, aiming to preserve manipulation performance when viewpoints change. e02 |
| Core mechanism | |
| A key reported result | LIBERO and zero-shot LIBERO-Plus manipulation: GAM: original 97.6; Plus 85.5; camera perturbations 83.1. Task success rate (%). Separate policies for four ten-task suites; original evaluation uses 50 trials/task, Plus one rollout per perturbed instance. Baselines mix re-evaluated and published results. Cosmos-Policy: 98.5 / 82.4 / 73.4; π0.5: 96.9 / 84.6 / 72.0. Camera gain over Cosmos is 9.7 percentage points. GAM is not best on nominal performance or every perturbation; no uncertainty accompanies Table 1. e10e11 |
| Reading caution |
Core contributions
Figure 3. Predicted features return to the geometric backbone before actions are decoded. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the left panel from RGB images through GFM shallow layers into each temporal block U. Each block contains robot state, previous action and geometric features; language tokens enter separately above. The purple predictor emits future geometry and an action token, which continue through the gold deep layers. The upper geometry head is marked optional, whereas the action head supplies executable chunks. In the right panel, keys run horizontally and queries vertically. Yellow means attention is allowed; gray means blocked. The temporal rows form a block-causal pattern, but the language row is entirely yellow. Sections 4.2–4.3 specify causal prediction and extend masking into the deep global-attention layers. e03e04e05e23
What it supports. The method does more than attach a policy head to observed geometric features: deep pretrained blocks process predicted future features together with action tokens. This supports the shared-backbone interpretation. The diagram also makes clear that geometry decoding can be optional while latent geometric computation still participates in action generation.
Where the evidence stops. The language-query row conflicts with the prose's no-future-leakage guarantee: updated language tokens could relay later blocks to earlier queries across layers. The crop is preserved exactly. Without an implementation or a specified language-token update exception, the effective mask remains unresolved.
2. Motivation
2.1 The problem and the proposed response
The authors argue that policies built on image or video representations leave depth, scale and occlusion implicit. GAM instead reuses a geometric foundation model throughout perception, temporal prediction and action decoding, aiming to preserve manipulation performance when viewpoints change. e02
2.2 What this reading follows
A robot can recognize a bowl yet fail when a camera moves, because recognition alone does not specify the geometry of reaching and contact. GAM addresses this by adapting a model pretrained to recover geometry from images. Its central move is to interrupt the geometric backbone, predict what comes next in its feature space, and resume that same backbone with both future features and an action token. The six visuals below follow this information flow into simulation, physical execution, ablations and deployment cost. Read the gains alongside their conditions: future losses behave differently before and after pretraining, and the fastest timing uses an extra runtime optimization. e02e03e04e05e11e13e14e16
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | One Model |
| Prediction paradigm | IDM |
| Quadrant | Q2 · One Model × IDM |
3.1 Evidence-based assessment
Conflicts with the recorded classification
The shared GFM encoder/decoder supports One Model, and predicted geometric futures support WAM and multiview modeling. The IDM label is not supported by the described information flow: action and future tokens are jointly predicted, then jointly refined, rather than actions being inferred from a separately specified current/future state pair. No planning search is described. The recorded Q2 snapshot is retained unchanged. e03e04e05e20
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Why insert prediction before the deep geometric blocks?
A pretrained geometric model does not treat every layer as interchangeable. Its shallow blocks convert image patches into features, and its deeper blocks combine information that the DPT head needs for dense geometry. GAM therefore splits the model at L_s and requires L_s < m_1, where m_1 is the earliest feature layer consumed by DPT. The predictor must output features early enough for the remaining geometric computation to remain useful. The separate split-layer study supports this design choice: at layer 12 it reports 99.6% original and 70.1% Plus success; very early or late insertion collapses performance. This study excludes future-depth loss and reinitializes the predictor, so its numbers must not be substituted for the full component-ablation result. Reader interpretation: the insertion point balances feature maturity against remaining decoder capacity. e03e07e15
Figure 10(a). Future depth illustrates what the geometric prediction target represents. Original paper, p. 17 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read down a column before moving along the progress arrow. The current RGB and current-depth rows describe the observed scene, while GT Future and Pred Future refer to the next labeled timestep. For example, current T = 0 is paired with future T = 1; later displayed columns similarly pair 2 with 3, 4 with 5, 7 with 8, and 9 with 10. Each cell juxtaposes the two camera views. Compare the gripper, bowl boundary and scene silhouettes between the last two rows. The columns sample progress through a bowl-transfer task; they are not evidence of a free-running prediction made from only the first frame. e19e06
What it supports. The predicted maps reproduce recognizable aspects of the upcoming two-view scene and provide a visual interpretation of future-depth supervision. The correspondence is qualitative: shapes and boundaries can be compared with the ground-truth row, but the figure does not turn those similarities into a measured geometric accuracy result.
Where the evidence stops. No calibrated depth color bar, units or numerical prediction error accompanies this panel. Do not infer metric depth accuracy, physical task success or long-horizon open-loop world-model reliability from these selected examples. Appendix B.3 offers visualizations, not a quantitative depth benchmark.
5.2 Separate supervision, latent prediction and executed feedback
During training, the future is available as a target. GAM compares predicted tokens with next-frame tokens extracted from a frozen GFM and supervises decoded future depth with simulator measurements or teacher pseudo-depth. Action imitation supplies the executable-command target. During deployment, the next observed frame is unavailable: current observations, robot state, language and previous actions produce the predicted features and action token. The deep backbone then converts these into an action chunk, with depth decoding optional. The benchmark recipe executes all eight commands before taking another observation. This makes the overall policy feedback-driven between chunks, while each chunk itself is open-loop. The loss ablation adds a useful boundary: keeping future losses during post-training hardly changes the pretrained model's Object robustness, which does not establish that removing future latent computation at inference would also be harmless. e03e05e06e07e08e09e12e14
5.3 Test what the shared action decoder actually contributes
The causal predictor already contains a predicted action token, so why not decode the command immediately? Appendix C.1 tries exactly that: direct action supervision gives 84.1% Plus Object success, compared with 89.7% when the token traverses the remaining geometric blocks. This supports retaining the decoder, but the comparison also removes substantial computation, so it does not uniquely attribute the gain to geometric knowledge. The attention images show saliency near task objects at several layers; this is compatible with object-aware refinement, without proving a causal contact model. Architecturally, future and action tokens are predicted together and refined together. Reader assessment: this supports a single shared model, while the catalog's IDM label is questionable because the paper does not specify a separate inverse mapping from a chosen future state to an action. e04e05e20e21
5.4 Training and inference
During training
Pretraining uses 784K trajectories, sampled 72% OXE, 18% MimicGen and 10% RoboCasa365. Future depth targets are teacher pseudo-depth for OXE and simulator depth for simulation data. Real-robot post-training also uses pseudo-depth. Context changes from H = 4 during pretraining to H = 1 during benchmark post-training. e07e08e12
AdamW uses constant learning rates; Table 5 gives 5.16 × 10⁻⁵ for the backbone and 5.16 × 10⁻⁴ for predictor/action head. Table 9 freezes blocks 0–12 and DPT, trains blocks 13–39, predictor and action head, and reports about 983.2M trainable parameters within 1.4B. T5 remains frozen. e07e09e24
During inference
Online key-value caching supports a single feed-forward update from new observations and previous actions. The deployed benchmark configuration uses single-step observations, predicts eight commands and executes all eight open-loop before observing again. Future latent computation remains in the action pathway; optional depth decoding is not a search over candidate plans. e03e05e06e09
5.5 Implementation flow
- Encode observations at an intermediate layer
DA3-Giant, fine-tuned on Track4World, supplies patch and camera tokens. Shallow blocks encode each timestep independently. The split L_s = 12 must precede the earliest DPT feature tap so predicted tokens can traverse every required geometry-decoding layer. e03e07
- Predict action and future features together
A 12-layer, width-1024 transformer receives frozen-T5 language tokens followed by blocks containing projected robot state, previous action and geometric tokens. Geometry slots predict the next latent state; the previous-action slot predicts an action token. The prose specifies block-causal attention, with a diagram discrepancy discussed below. e04e07e23
- Propagate before regressing commands
The predicted action token is replicated across views, appended to their predicted geometry tokens, and processed by the remaining GFM blocks. Global attention also receives a causal mask. The action head aggregates action tokens to regress commands; the original depth head decodes future geometry. e05
6. Experiments & results
GAM turns a pretrained geometric transformer into a language-conditioned manipulation policy by inserting temporal prediction between its shallow and deep blocks. Future geometry and action tokens share the decoder. Camera robustness is its clearest reported gain; ablations, runtime settings and internal reporting discrepancies qualify the broader claims.
6.1 Read the original evidence
Table 1. The strongest reported advantage is camera robustness, not uniform dominance. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the bottom GAM row and compare Orig., Plus and Cam. separately. Orig. is nominal LIBERO; Plus is the reported aggregate under perturbations, and Cam. isolates viewpoint changes. Every success entry is a percentage; parenthetical arrows denote drops in percentage points. The retained color key identifies ranks within columns. Compare GAM's 83.1 camera score with Cosmos-Policy's 73.4, then check other columns: π0.5 exceeds GAM on robot-state and layout perturbations. Appendix A.2 describes separate suite policies and zero-shot Plus evaluation. It also distinguishes re-evaluated checkpoints from baseline results taken from earlier benchmark reports, so this is not a uniformly retrained comparison. e10e11
What it supports. GAM reports 85.5% overall Plus success versus 84.6% for π0.5 and 82.4% for Cosmos-Policy. Its camera score is 9.7 percentage points above Cosmos. Nominal success is 97.6%, below Cosmos's 98.5%; the evidence therefore favors robust adaptation to certain shifts more strongly than universal manipulation superiority.
Where the evidence stops. Some printed drops are arithmetically inconsistent: Spatial Forcing's 94.0 to 25.7 is 68.3 points, not 58.3; ROCKET's 95.3 to 47.5 is 47.8, not 46.6; Fast-WAM differs by 0.1 point. The original cells remain intact. No confidence intervals are supplied.
Figure 4. Physical trials test whether the geometric policy tolerates a moved external camera. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the four task groups from left to right, using the photographs to understand the object transfers and multi-stage tasks. The pale full-height bar is in-distribution performance; the darker overlaid bar is out-of-distribution performance, as specified by the original caption. These are not stacked quantities to add. For stacking milk and a cube, GAM's pale/dark pair is 80/60, compared with π0.5's 70/10. Appendix A.3 states that each condition has ten trials per task. The perturbation moves the external camera by 85 cm and rotates it by 45 degrees while retaining the wrist view. e12e13e09
What it supports. GAM retains higher success under the tested camera change in all four task groups. The stacking example shows a particularly large gap: 60% OOD success versus 10% for each baseline. The low absolute success on the pot-and-pan task also matters: GAM's 20% OOD result leaves substantial physical manipulation failures.
Where the evidence stops. Each condition has only ten trials/task, and baseline chunks have ten commands versus GAM's eight. Figure 1 reports GAM's average OOD success as 52.5%, whereas these four dark bars average 50%. The discrepancy is unresolved; this edition uses the visible task-level values.
Table 8. Separate architecture speed from the extra gain obtained through CUDA Graphs. Original paper, p. 13 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across the runtime flags before comparing the final latency column. Every method uses an official PyTorch path, bf16 precision and Torch Compile. Only the second GAM row enables CUDA Graphs. Thus the first GAM row, at 17.5 ms, is the direct comparison with the three baseline rows under the reported common runtime settings. The second row measures GAM's deployment optimization at 6.9 ms. Appendix A.5 says the measurement uses a single GH200 and excludes model loading and input preprocessing. It does not enumerate software versions or exact warmup and measurement repetition counts. e16
What it supports. GAM is faster in the common setting: 17.5 ms versus 29.2 for π0.5 and 382.4 for Cosmos-Policy. CUDA Graphs further reduce GAM's measured latency to 6.9 ms. The reported 55.4-fold speedup over Cosmos uses this optimized GAM row, so it includes both model and deployment differences.
Where the evidence stops. These are model-only action-prediction timings, not measured sensor-to-actuator latency or physical control rate. Baselines were not reported with CUDA Graphs enabled. Table 4 gives OpenVLA-OFT as 77.8 ms, while this appendix table gives 70.1 ms; both source values are retained.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO and zero-shot LIBERO-Plus manipulation Separate policies for four ten-task suites; original evaluation uses 50 trials/task, Plus one rollout per perturbed instance. Baselines mix re-evaluated and published results. | GAM: original 97.6; Plus 85.5; camera perturbations 83.1. Task success rate (%) | Cosmos-Policy: 98.5 / 82.4 / 73.4; π0.5: 96.9 / 84.6 / 72.0. Camera gain over Cosmos is 9.7 percentage points. GAM is not best on nominal performance or every perturbation; no uncertainty accompanies Table 1. e10e11 |
| Physical stack milk and cube Four tasks jointly trained; stacking has 202 demonstrations. Ten ID and ten OOD trials/task; external camera translated 85 cm and rotated 45°. | GAM 80 / 60. Task success rate, ID / OOD (%) | π0.5 70 / 10; Spatial Forcing 30 / 10, from Figure 4. These are executed physical tasks. Small trial counts and different action-chunk lengths limit precision and strict architectural isolation. e12e13 |
| LIBERO-Object component ablation Post-training ablation; H = 1 except history comparisons. | Full pretrained GAM 89.7; pretrained without both future losses 89.5; no pretraining with both 73.4; no pretraining without both 50.0. LIBERO-Plus Object success rate (%) | Without pretraining, depth-only reaches 80.0 and feature-only 66.5; pretrained H = 2 / 4 yields 84.4 / 85.1. Future supervision helps without policy pretraining, but the losses are not uniformly additive. Their marginal post-training benefit is small after pretraining. e14 |
| LIBERO-Object action-token decoding Same setting as the component ablation; action loss applied before versus after deep GFM propagation. | Full GAM 99.6 / 89.7. Original / Plus success rate (%) | Direct-action supervision 98.4 / 84.1. The 5.6-point Plus gain supports retaining deep geometric decoding; it does not isolate geometry from added decoder computation. e20 |
| Model-only inference latency Single GH200, bf16, official PyTorch paths and Torch Compile; loading and preprocessing excluded. | GAM 17.5 without CUDA Graphs; 6.9 with CUDA Graphs. Milliseconds per action prediction | Without CUDA Graphs: π0.5 29.2, OpenVLA-OFT 70.1, Cosmos-Policy 382.4. The 55.4× headline compares GAM's graph-enabled deployment with Cosmos without graphs. Table 4 instead lists OpenVLA-OFT at 77.8 ms. Neither timing establishes end-to-end physical control frequency. e16 |
| RoboCasa-Kitchen manipulation 24 tasks; three views, 16-step chunks, successful trajectories from 300 demonstrations/task. | 69.4. Average success rate (%) | Cosmos Policy 67.1; FLARE 66.4. A distinct adaptation protocol, not the two-view LIBERO setting. Evaluation rollout counts and uncertainty are not specified in B.1. e17 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2. Future supervision matters most when policy pretraining is absent. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the first three columns as switches and H as the observation-context length. Green checks enable pretraining or a loss; red crosses disable it. Compare rows within the same pretraining group before drawing conclusions. In the upper group, removing both future losses changes Plus success from 89.7 to 89.5, while increasing H to two or four lowers success. In the lower group, retaining both losses raises Plus success from 50.0 to 73.4 relative to removing both. However, depth-only reaches 80.0, above the two-loss configuration. The table concerns LIBERO-Object and LIBERO-Plus Object, not the aggregate benchmark of Table 1. e14e07e18
What it supports. The ablation supports a conditional conclusion: future supervision can materially improve robustness without the robot-policy pretraining stage, but its marginal benefit during post-training is small with that stage present. Depth and feature losses are not consistently additive. The best displayed history choice is a single observation step.
Where the evidence stops. 'No pretraining' here concerns GAM's policy pretraining, not removal of the geometric foundation model's prior. No seed variation is reported. Table 12 separately gives Object Plus as 90.6%, whereas this ablation's full configuration is 89.7%; these are distinct reported results.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors identify frozen-text-encoder limits on language reasoning and commonsense. Future-depth and attention examples are qualitative; they provide neither calibrated depth error nor causal proof of contact reasoning. e19e21e22
Figure 3(b) lets language queries attend to every temporal block, while earlier blocks attend back to language. If updated this way across layers, language could relay future information, conflicting with the no-leakage prose. The implementation is unresolved. e23
Internal summaries disagree: Figure 1 gives GAM OOD physical success as 52.5%, while Figure 4's four OOD bars average 50%; Tables 12 and 13 give original Object success as 99.6% and 99.0%. These values are preserved separately, not reconciled. e13e18
Table 1 also contains inconsistent parenthetical drops: Spatial Forcing prints 58.3 despite 94.0 − 25.7 = 68.3; ROCKET prints 46.6 despite 95.3 − 47.5 = 47.8. Comparisons here use the explicit success cells. e11
7.2 Questions for discussion
- Does blocking temporal information from flowing back into language tokens change multi-step pretraining or downstream robustness?
- Does the deep decoder help because of geometric pretraining, extra trainable computation, or both?
8. Reproducibility audit
8.1 Requirements and known gaps
Reported pretraining requires 64 GH200 GPUs, batch 1024 and about 96 hours. Appendix A reports simulation post-training on 16 GH200s for about 48 hours, whereas Table 5 lists eight GH200s. Table 9's frozen blocks also qualify Appendix A's loose 'entire model' wording. e09e24
Reproduction needs the Track4World-adapted DA3 checkpoint, dataset/action conversion, filtered successful LIBERO demonstrations and depth targets. Exact action normalization, no-op threshold, T5 variant, optimizer betas/weight decay, software versions, seed counts and timing repetition counts are not specified in the supplied recipes. e07e08e09e10e16
Proposed checks: perturb future input blocks while holding the prefix fixed to audit leakage, then compare deep geometric decoding with direct-action decoding under matched training and compute controls. These are proposed experiments, not reproduced findings. e23e20e14e15
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Audit the language-token route for future leakage
Reader-proposed check, not performed: use a fixed H = 4 sequence and identical language, state, action and image prefix. Change only later temporal blocks, then compare earlier predicted features and action tokens with stochastic layers disabled. Test the mask drawn in Figure 3(b) against a control in which language queries cannot read temporal blocks, while temporal queries can still read language. Trace differences through predictor layers and deep global-attention layers; include a prefix-only forward pass as a reference. Earlier outputs changing beyond numerical tolerance would falsify the claimed no-future-leakage behavior for that implementation. Invariance would support causality under the tested route. Record whether language representations are updated or held fixed, since the supplied formulation does not resolve that distinction. e23e04e05e07
Check 2: Separate geometric initialization from extra decoder computation
Reader-proposed check, not performed: reproduce the LIBERO-Object direct-action versus deep-decoder comparison, and add a same-depth decoder initialized randomly as a parameter-matched control for the pretrained deep blocks. Hold the shallow encoder, predictor initialization, data split, H = 1, eight-command execution, training budget and loss recipe fixed; report multiple seeds and paired evaluation instances. Measure nominal and Plus success, including camera perturbations, and record future-depth error separately. The direct head tests whether the extra path helps; the random deep decoder tests whether its pretrained geometry helps beyond added capacity. If pretrained and random decoders perform alike while both beat direct decoding, geometric initialization is not supported as the specific explanation. A reproducible pretrained-decoder advantage would strengthen that mechanism claim. e03e05e09e10e14e20e24
8.3 Reading coverage
Visual audit: All six supplied text chunks were read individually, including references on pages 19–22. PDF pages 1–18 were rendered and visually inspected: title/authors/version; all method equations; Figures 1–11; Tables 1–14; simulation, physical, training, hardware and runtime protocols. Every final crop was inspected, including the revised Table 1 crop retaining its color key. The six crops are faithful excerpts; Figure 10 is represented by panel (a), with its other panels inspected on the full page. Figure 3's language-row attention discrepancy, physical-result summary disagreement, differing Object scores, hardware counts and latency entries remain disclosed. Separate supplements and implementation code were not supplied or inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title page and abstract, PDF p. 1: arXiv:2606.17046v2 [cs.RO], 22 June 2026
- Sections 1–3: Introduction, Related Work, Geometric Foundation Models, pp. 2–4
- Sections 4.1–4.4: observation encoding, causal prediction, decoding, training and inference, pp. 4–6
- Sections 5.1–5.5 and 6: implementation, evaluation, results, ablations, analysis and limitations, pp. 6–9
- Acknowledgments, p. 9
- Appendix A.1–A.6: data, training, simulation, physical setup, baselines, runtime and parameters, pp. 10–14
- Appendix B.1–B.3: RoboCasa-Kitchen, LIBERO breakdowns and future depth, pp. 14–17
- Appendix C.1–C.2: direct action-token ablation and attention analysis, p. 18
- References, pp. 19–22; all six supplied text chunks read individually
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The supplied PDF is v2; its exact title and all ten authors match the catalog after name-order normalization. The catalog URL is versionless. Earlier revisions were not supplied, so revision-to-revision changes cannot be established.
- Text extraction does not reconstruct figure images; this limitation was addressed by visually inspecting PDF pages 1–18 and all six final original crops. Reference pages 19–22 were read as text.
- Separate supplemental material availability has not been fully verified; none was supplied.
- Code, project website and external references were not inspected; no experiments were reproduced.
- The appendix roadmap mentions backbone variants, but the supplied Appendix C contains direct action-token supervision and attention analyses, without a corresponding backbone-variant results table.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/author block and arXiv margin
Exact title; all ten catalog authors; KAIST AI, ETH Zurich and ETH AI Center; arXiv:2606.17046v2, 22 June 2026.
Go to primary source ↓e02PDF pp. 2–3, Section 1 and Figure 2
Motivates explicit geometric priors and distinguishes GAM's shared geometric backbone from video WAMs and feature-distilled VLAs.
Go to primary source ↓e03PDF pp. 4–5, Sections 3–4.1, Eqs. (1)–(5)
Multi-view camera/patch tokens; policy context and open-loop action chunks; shallow/deep split before the first DPT feature tap.
Go to primary source ↓e04PDF pp. 5–6, Section 4.2, Eq. (6) and prediction-slot paragraphs
Language, robot-state and previous-action conditioning; geometric slots predict future tokens while the previous-action slot predicts an action token.
Go to primary source ↓e05PDF p. 6, Section 4.3, Eq. (7)
Replicate action token per view, propagate with predicted future features through deep GFM blocks, extend causal masking, then decode action and depth.
Go to primary source ↓e06PDF p. 6, Section 4.4, Eqs. (8)–(9) and inference paragraph
Weighted action/feature/depth objectives; frozen-GFM future targets; scale-invariant and gradient-matching depth supervision; cached feed-forward inference.
Go to primary source ↓e07PDF pp. 6–7, Section 5.1
DA3-Giant adapted on Track4World; 12-layer width-1024 predictor at split 12; frozen T5; H = 4/1, C = 8, seven-dimensional state/action; AdamW; loss weights 3/1/3.
Go to primary source ↓e08PDF pp. 10–11, Appendix A.1 and Figure 6
784K-trajectory mixture, source weights 72/18/10; common action compatibility filtering, original instructions, two views when available, depth/pseudo-depth targets and image augmentation.
Go to primary source ↓e09PDF pp. 10–11, Appendix A opening paragraphs and Table 5
Pretraining 64 GH200s, batch 1024, approximately 96 hours; prose post-training 16 GH200s, batch 160, approximately 48 hours; Table 5 instead lists eight GH200s and gives learning rates, eight executed commands and augmentation settings.
Go to primary source ↓e10PDF pp. 10–11, Appendix A.2 and paragraph after Table 5
Four ten-task LIBERO suites, separate policies, successful/no-op-filtered training; 50 original trials/task; one Plus rollout/instance; up to 110k training steps; selected baselines re-evaluated, others imported.
Go to primary source ↓e11PDF p. 7, Table 1, GAM, Cosmos-Policy, π0.5 and geometry-aware VLA rows
GAM original/Plus/camera 97.6/85.5/83.1; Cosmos 98.5/82.4/73.4; π0.5 96.9/84.6/72.0. Parenthetical Spatial Forcing and ROCKET drops do not match their success cells; Fast-WAM's 47.5 drop also differs from 97.6 − 50.0 = 47.6.
Go to primary source ↓e12PDF p. 7, Section 5.2; p. 12, Appendix A.3 and Figures 7–8; p. 13, Tables 6–7 and A.4
Real targets use GFM pseudo-depth; wrist ZED/external RealSense; task demonstrations 284/202/184/169; joint training; 10 ID and 10 OOD trials/task with 85 cm/45° external-camera change. Baselines execute 10-step chunks versus GAM's eight.
Go to primary source ↓e13PDF p. 8, Figure 4, four task groups and caption; p. 1, Figure 1(b), physical-results bars
Figure 4 GAM ID/OOD values: 100/80, 80/60, 30/20, 50/40. Stacking baselines are π0.5 70/10 and Spatial Forcing 30/10. Light bars denote ID, dark OOD. Figure 1 prints GAM ID/OOD averages 65/52.5, although Figure 4's OOD bars average 50.
Go to primary source ↓e14PDF p. 8, Section 5.4, Table 2, all rows
Pretraining/loss/history factorial comparisons on Object. Pretrained full and no-future-loss Plus rates 89.7/89.5; unpretrained full/none/depth-only/feature-only 73.4/50.0/80.0/66.5; H = 2/4 pretrained 84.4/85.1.
Go to primary source ↓e15PDF p. 8, Table 3 and Split Layer Selection paragraph
Predictor reinitialized at different splits; depth loss excluded. Original/Plus: split 0 5.4/1.8; 12 99.6/70.1; 19 95.6/63.4; 27 1.2/1.6; 33 and 39 0/0.
Go to primary source ↓e16PDF p. 8, Table 4; p. 9, Section 5.5; p. 13, Table 8 and Appendix A.5
Single-GH200 model-only latency with bf16/Torch Compile: GAM 17.5 ms without graphs, 6.9 with; π0.5 29.2, OpenVLA-OFT 70.1, Cosmos 382.4 without graphs. Main Table 4 instead gives OpenVLA-OFT 77.8. Loading and preprocessing excluded.
Go to primary source ↓e17PDF p. 14, Appendix B.1 and Tables 10–11
RoboCasa-Kitchen: 24 tasks, three-view adaptation, 16-step chunks, successful training trajectories from 300 demos/task. Average success GAM 69.4, Cosmos 67.1, FLARE 66.4.
Go to primary source ↓e18PDF p. 15, Table 12(b,d) and Figure 9; p. 16, Table 13, suite headers
Table 12 reports GAM Object original 99.6 and Plus 90.6; Table 13 reports Object original 99.0. Long-suite Plus is GAM 78.0 versus Cosmos 81.0, showing a boundary to aggregate dominance.
Go to primary source ↓e19PDF pp. 15–17, Appendix B.3 and Figure 10(a–d)
Current RGB/current depth/GT future/predicted future grids across four suites. Panel (a) follows bowl transfer, with current T = 0/2/4/7/9 paired to future T = 1/3/5/8/10. No quantitative depth-error table or calibrated color scale is provided.
Go to primary source ↓e20PDF p. 18, Appendix C.1 and Table 14
Action supervision directly at predictor output versus after deep GFM blocks: original/Plus 98.4/84.1 versus 99.6/89.7; same Object setting as component ablation.
Go to primary source ↓e21PDF p. 18, Figure 11 and Appendix C.2
Action-token attention visualizations at layers 13, 26, 33 and 39, with author interpretation of saliency near task-relevant objects/contact regions.
Go to primary source ↓e22PDF p. 9, Section 6, final paragraph
Frozen text encoder bounds language reasoning and commonsense; LLM or external reasoning module proposed as future work.
Go to primary source ↓e23PDF p. 4, Figure 3(b), Key/Query axes and yellow/gray legend; pp. 5–6, Sections 4.2–4.3
The diagram's language-query row permits attention to all temporal blocks; temporal rows can attend to language. Temporal-to-temporal cells are block-causal. Prose says there is no future leakage; no language-row exception is explained.
Go to primary source ↓e24PDF pp. 13–14, Appendix A.6 and Table 9; p. 10, Appendix A
Table 9: backbone blocks 0–12 frozen, 13–39 trainable; DPT frozen; predictor 210.2M and action head 8.0M trainable; about 1404.8M total and 983.2M trainable. T5 is not itemized in the parameter breakdown; Appendix A's 'entire model' wording is broader.
Go to primary source ↓8.5 Primary sources
Geometric Action Model for Robot Policy Learning ↗
PDF · 11,141 extracted words
Source fingerprint
19b79e58aac413f80c3eac7d34c7c570abc921ace4ba79a42b12aeda5ff61a42