A0: An Affordance-Aware Hierarchical Model for General Robotic Manipulation
1. Paper overview
In one sentence: A0 makes manipulation transferable through sparse object-contact trajectories, while relying on a separate geometric execution stack to turn image coordinates into robot motion. e-probleme-representatione-traininge-executione-robot-results
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors target the “where” and “how” of manipulation: identifying a useful contact location and specifying subsequent movement. A sparse object-centric interface is intended to reuse localization and interaction data across robot embodiments, while leaving geometric execution to a separate module. e-probleme-representation |
| Core mechanism | Source description A common image, language, contact-point and trajectory representation pools Internet localization, human interaction, real-robot and simulated data. e-representatione-data |
| A key reported result | Four household tasks on Franka Emika: A0: 60, 75, 70, 45; mean 62.50%. Task success rate (%) and four-task mean. Place Object, Open Drawer, Press Button and Wipe Board; 20 trials per task, consistent placements and camera settings across methods. Molmo: mean 43.75%; Magma: 16.25%, both using the A0 execution module. A0 exceeds Molmo by 18.75 percentage points in this setup; no uncertainty intervals are reported. e-protocole-robot-resultse-baselines |
| Reading caution | Reader analysis Twenty trials per task, different baseline sets across platforms and no reported confidence intervals limit broad rankings. The VLA comparison uses only five fine-tuning episodes per task; the retained text does not establish equal adaptation data or compute for every method. e-protocole-robot-resultse-vla |
Core contributions
- Source description
A common image, language, contact-point and trajectory representation pools Internet localization, human interaction, real-robot and simulated data. e-representatione-data
- Source description
A diffusion transformer combines temporal token differences, alternating language/image conditioning and an MLP coordinate decoder; a separate action expert converts its output into executable motion. e-architecturee-execution
Figure 3. Language and image features condition waypoint generation before a separate action expert executes the result. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read from the two encoder branches toward the DiT block. Snowflakes mark the frozen Qwen2.5 language encoder and SigLiP image encoder; flames identify trainable components, including the noise/timestep embeddings and coordinate decoder. The image branch receives consecutive observations and constructs motion tokens. Section 3.3 says image and text conditions enter alternating cross-attention layers, matching the two conditioning arrows. After denoising, the Spatial Information Aggregation Layer maps features to sparse coordinates: the red first point identifies contact, and green points describe subsequent movement. Follow the final arrow into the Action Expert, whose depth projection and grasp-selection operations are specified in Section 4. e-architecturee-representatione-executione-count
What it supports. The learned interface is a sparse trajectory in image coordinates. This makes the representation independent of a particular robot’s joint vector, but executable behavior still depends on the downstream action expert. The figure therefore supports a hierarchical predictor/executor interpretation, rather than demonstrating a single network that learns every stage of control.
Where the evidence stops. The POA output labels retained tokens as previous-frame features, whereas Section 3.3 concatenates current-frame features with motion tokens. This crop preserves that unresolved discrepancy; the explanation follows the text. The figure’s one-million pretraining label also conflicts with Section 3.1’s 100,000.
2. Motivation
2.1 The problem and the proposed response
The authors target the “where” and “how” of manipulation: identifying a useful contact location and specifying subsequent movement. A sparse object-centric interface is intended to reuse localization and interaction data across robot embodiments, while leaving geometric execution to a separate module. e-probleme-representation
2.2 What this reading follows
Opening a drawer requires both choosing the handle and deciding how it should move. A0 places that distinction at the center of a hierarchical manipulation system: a diffusion transformer predicts a contact point and subsequent 2D waypoints, then an action expert supplies depth, grasping and executable motion. The paper trains localization before trajectories and tests both pixel-space prediction and physical task success. This reading follows that chain through the architecture, transfer ablations and robot tables. The evidence supports useful performance on the evaluated Franka and Kinova tasks, while leaving several implementation details, source inconsistencies and broader transfer claims unresolved. e-probleme-representatione-traininge-executione-robot-results
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Conflicts with the recorded classification
Dual-system, Other mechanisms and Outside quadrants fit the explicit predictor/executor split. However, the detailed tags overreach: the model uses temporal external-camera frames, not demonstrated multiview modeling, and depth lifting/grasp sampling is not a learned inverse-dynamics model. It predicts desired sparse tracks rather than action-conditioned future observations. WAM membership is therefore a broad catalog interpretation, not evidence of a learned world-transition model. e-representatione-architecturee-execution
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Learn where to touch before learning how to move
A0’s supervision is organized around a reusable spatial interface. Each interaction entry associates an image and language instruction with a contact point and a post-contact track. The localization stage makes a simpler demand: using a single image and an object label, predict the first point. Fine-tuning then extends the output to a trajectory and the language to a manipulation instruction. Its diffusion objective corrupts ground-truth waypoints and trains the model to recover clean coordinates with mean-squared error. This explains why an Internet pointing dataset can contribute to manipulation without containing robot joint actions. Figure 4 tests that pretraining choice through offline domain transfer. The data-count inconsistency remains explicit: the dataset section reports one million localization annotations, while Section 3.1 says 100,000. e-representatione-traininge-datae-pretraininge-count
5.2 Follow a predicted pixel through the execution boundary
The transformer’s output is still only a set of image coordinates. To execute a drawer interaction, the system must turn the selected contact location into a reachable 3D grasp and give later points physically meaningful heights. Section 4 uses depth and camera intrinsics for deprojection, then chooses a grasp candidate nearest the lifted contact location. For free-space motion, a VLM selects a discrete height category, such as target level or above the target. The resulting grasp and waypoints feed SE(3) trajectory generation. Consequently, embodiment independence applies to the shared affordance representation; an implementation still needs an appropriate geometric execution stack. The source describes single-shot deployment, so consecutive input frames should not be taken as proof that the robot continuously replans after observing its own actions. e-representatione-executione-inference
5.3 Separate localization evidence, controller evidence and transfer claims
Reader analysis: the experiments answer different questions and should remain separate. Figure 4 asks whether localization pretraining improves waypoint prediction across data domains. Table 1 asks whether two network components reduce pixel error under the offline dataset-split protocol. Table 2 asks whether the assembled system completes four physical tasks on Franka and Kinova. Finally, Table 3 compares that system with VLA policies adapted using five episodes per task. Agreement across these results makes the overall design plausible, but does not isolate every causal link: the paper supplies no robot-success ablation for POA or SIAL, and its execution stack contributes grasping and height decisions. A stronger demonstration of embodiment transfer would control adaptation data and report the same baselines, success criteria and uncertainty across robots. e-pretraininge-ablatione-confige-protocole-robot-resultse-vlae-execution
5.4 Training and inference
During training
Pretraining supervises only the first point using one image, an object label and coordinate MSE. Fine-tuning extends the label to an instruction and the output to a waypoint sequence, learning clean coordinates from noisy trajectories with MSE. This is supervised affordance prediction. e-training
The dataset section lists one million PixMo-One-Point annotations, 22,000 HOI4D trajectories, 3,056 DROID trajectories and 4,965 ManiSkill trajectories. Section 3.1 instead says 100,000 pretraining samples; Figure 3 and Section 5.1.2 support one million, but the conflict cannot be resolved from this PDF. e-datae-count
A0-1B uses 28 transformer layers, 1,000 forward diffusion steps and five denoising steps. The reported training uses four A100 80GB GPUs, batch size 200 and 73GB per card: 80,000 pretraining steps over five days, then 30,000 fine-tuning steps over 50 hours. e-config
During inference
Sample Gaussian waypoint noise and denoise with a fast ODE solver conditioned on images and instruction. The reported deployments predict four to five waypoints at the beginning and execute through the action module. Temporal conditioning does not itself establish repeated perception or corrective replanning during a rollout. e-inferencee-execution
5.5 Implementation flow
- Represent object interaction
Each training entry contains an object-centric image, instruction, contact point and post-contact track. The first predicted waypoint is the contact position. The experimental chunk size is five; these are sparse image coordinates, not joint commands or generated future images. e-representatione-config
- Condition the diffusion transformer
Figure 3 marks Qwen2.5-7B and SigLiP-400M frozen and the diffusion embeddings, DiT and decoder trainable. Section 3.3 forms motion tokens by subtracting corresponding frame tokens and concatenates them with current-frame tokens. Figure 3 instead labels the retained tokens as previous-frame tokens; this discrepancy is unresolved. e-architecture
- Decode points and lift them into actions
The Spatial Information Aggregation Layer is a nonlinear MLP mapping latent features to coordinates. Depth deprojection lifts points into 3D. The system selects a sampled grasp nearest the projected contact location; a VLM chooses discrete free-space height categories before SE(3) trajectory generation. e-architecturee-execution
6. Experiments & results
A0 predicts an object contact point and a short 2D post-contact trajectory from an image and instruction. A separate depth, grasp and motion pipeline executes those points. Its strongest evidence combines lower offline waypoint error with real Franka and Kinova task success; embodiment independence remains conditional on the execution stack and tested settings.
6.1 Read the original evidence
Figure 5. Predicted image-space affordances guide physical robot execution in four illustrated tasks. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow each row from left to right: the current scene, the predicted 2D affordance overlay, then four physical execution snapshots. The drawer row shows a handle contact and pulling direction; the object-placement row links the object to its destination; the button row concentrates the prediction near the pressing location; the wiping row depicts a path over the marked board. The overlay is the learned output, while the arm motion belongs to the downstream execution pipeline. Section 4 supplies the missing geometric link through depth, grasp selection and height categories. These example sequences help interpret what a waypoint prediction is intended to accomplish. e-rolloutse-executione-robot-results
What it supports. The figure establishes that the presented interface is used for real Franka execution, with qualitatively different spatial demands across tasks. It is especially useful for understanding why contact localization alone is insufficient for wiping: post-contact movement specifies how the tool should traverse the target region.
Where the evidence stops. These are selected example snapshots, not a success-rate estimate or a failure analysis. They do not reveal controller frequency, elapsed time or corrective feedback. Aggregate success belongs to Table 2.
Table 2. Average success improves on both tested platforms, although A0 is weaker at Kinova object placement. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First separate the Kinova block from the Franka block: they contain different baseline sets and should not be treated as a single common leaderboard. Within each block, read Place Object, Open Drawer, Press Button and Wipe Board before the average. The protocol uses twenty trials per task and consistent placements and camera configurations across methods. Section 5.2.2 states that MOKA and ReKep were evaluated on Kinova, while Magma and Molmo supplied points to the authors’ execution module on Franka. Grey rows identify A0; boldface emphasizes task or mean winners, including ReKep’s Kinova placement result. e-protocole-robot-resultse-baselinese-scope
What it supports. Franka average success is 62.50% for A0 versus 43.75% for Molmo, an 18.75-percentage-point advantage. Kinova average success is 53.75% versus MOKA’s 45.00%, an 8.75-point advantage. The Kinova placement column limits any universal superiority claim: A0 reaches 60%, compared with MOKA’s 70% and ReKep’s 75%.
Where the evidence stops. Twenty trials per task and no reported confidence intervals leave ranking uncertainty unquantified. The two robot blocks do not establish identical transfer conditions, and the paper does not supply corresponding quantitative Realman or Dobot tables.
Table 3. Sparse waypoint execution accompanies higher success in the reported small-data Kinova comparison. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the task columns before the Steps column. Although the original caption singles out Wipe Board, the table actually reports all four Kinova tasks and their mean. Section 5.2.3 describes collecting five image-action episodes per task, including third-person and wrist images, to fine-tune the VLA baselines. A0’s grey row repeats its Kinova success values from Table 2. The final column compares the authors’ four-to-five initial waypoints with the VLA methods’ twenty-five-to-fifty iterative steps. That difference describes the organization of a rollout; it does not provide a common unit of compute or a wall-clock timing measurement. e-vlae-robot-resultse-protocol
What it supports. A0 reaches 53.75% mean task success against π0’s 20.00%, a 33.75-percentage-point difference. For Wipe Board, the corresponding values are 50% and 10%, a 40-point difference. The other reported VLA means are 11.25% for RDT-1B and 18.75% for π0 plus FAST.
Where the evidence stops. The retained text does not establish equal adaptation data or compute for A0 and every baseline. The small five-episode fine-tuning regime constrains generalization of this ranking, and fewer listed steps alone cannot establish a measured speedup.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Four household tasks on Franka Emika Place Object, Open Drawer, Press Button and Wipe Board; 20 trials per task, consistent placements and camera settings across methods. | A0: 60, 75, 70, 45; mean 62.50%. Task success rate (%) and four-task mean | Molmo: mean 43.75%; Magma: 16.25%, both using the A0 execution module. A0 exceeds Molmo by 18.75 percentage points in this setup; no uncertainty intervals are reported. e-protocole-robot-resultse-baselines |
| Four household tasks on Kinova Gen3 Same four named tasks and 20-trial protocol; MOKA and ReKep evaluated on Kinova. | A0: 60, 65, 40, 50; mean 53.75%. Task success rate (%) and four-task mean | MOKA: mean 45.00%; ReKep: 33.75%. Place Object: MOKA 70%, ReKep 75%, A0 60%. The mean advantage over the strongest listed Kinova affordance baseline is 8.75 percentage points. A0 does not win every task. e-protocole-robot-resultse-baselines |
| Kinova comparison with VLA policies Four tasks; authors collected five image-action episodes per task with third-person and wrist views to fine-tune the compared VLA models. | A0: 53.75% mean, 50% Wipe Board, 4–5 steps. Mean success (%) and reported execution steps | π0: 20.00% mean, 10% Wipe Board; RDT-1B: 11.25% mean; π0 + FAST: 18.75% mean; all list 25–50 steps. The mean difference against π0 is 33.75 percentage points. Steps compare sparse waypoints with iterative inference, not measured wall-clock speed. Table 3 contains four tasks despite its Wipe Board caption. e-vlae-protocol |
| Cross-domain waypoint prediction after localization pretraining Real-to-Sim: train on HOI4D/DROID, test on ManiSkill. Sim-to-Real reverses these domains. | With pretraining: ManiSkill 43.9, HOI4D 125.2, DROID 29.1. Image-pixel MAE; lower is better | Without pretraining: 50.4, 172.2 and 35.1 respectively. All three plotted transfers improve; these are offline errors, not robot success rates. e-pretraining |
| POA and SIAL architecture ablation PixMo-pretrained A0-1B; offline datasets split 80:20 for training/testing. | Full model: HOI4D 47.5, ManiSkill 5.5, DROID 17.5. Image-pixel MAE; lower is better | Without POA: 47.9, 6.3, 18.5. Without SIAL: 61.1, 10.2, 19.6. SIAL removal has the larger penalty. Table 1 implies a 13.6-pixel HOI4D increase; adjacent prose says 13.2. These are separate protocols from Figure 4. e-ablatione-config |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 4. Localization pretraining reduces waypoint error in each plotted transfer direction. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the panel titles: these are cross-domain tests, not three interchangeable test sets from a single training run. Section 5.1.2 defines Real-to-Sim as training on HOI4D and DROID and testing on ManiSkill; Sim-to-Real trains on ManiSkill and tests on HOI4D and DROID. Blue bars denote A0-1B without the pretraining suffix, while green bars denote the pretrained model. The vertical quantity is mean absolute waypoint error in image pixels, so shorter bars are preferable. Compare the paired bars within each dataset before comparing absolute error magnitudes across datasets, whose visual content and trajectories differ. e-pretraininge-ablation
What it supports. Pretraining changes ManiSkill MAE from 50.4 to 43.9, HOI4D from 172.2 to 125.2, and DROID from 35.1 to 29.1. The consistent direction supports transfer of localization knowledge to downstream waypoint prediction; the especially large HOI4D change remains an offline prediction result.
Where the evidence stops. No uncertainty bars or repeated-seed statistics are supplied here. Pixel-error reductions do not establish robot success gains, and these cross-domain results must remain separate from the ordinary dataset-split ablations in Table 1.
Table 1. The coordinate decoder has a larger reported ablation penalty than the temporal-token module. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each column downward. The first row is the complete PixMo-pretrained A0-1B; the next two remove Position Offset Attention (POA) or the Spatial Information Aggregation Layer (SIAL). Section 5.1.1 describes 80:20 train/test splits for the trajectory datasets, and the table uses image-pixel MAE with lower values preferred. On ManiSkill, removing POA changes 5.5 to 6.3, while removing SIAL changes 5.5 to 10.2. HOI4D and DROID also worsen under both removals. Keep the full model as the reference for each subtraction rather than comparing one ablated row against the other. e-ablatione-confige-architecture
What it supports. The table supports contributions from both components, with a larger degradation when the nonlinear coordinate decoder is removed. On HOI4D, 61.1 minus 47.5 is 13.6 pixels. The adjacent paragraph reports 13.2 instead, so the numerical interpretation here follows the visible table and explicitly preserves the disagreement.
Where the evidence stops. These ablations measure waypoint error, without a corresponding controlled robot-success table or reported uncertainty. Removing a component can change model capacity as well as its proposed mechanism; a matched-capacity replacement would test that alternative explanation.
7. Analysis & limitations
7.1 What the evidence leaves open
Twenty trials per task, different baseline sets across platforms and no reported confidence intervals limit broad rankings. The VLA comparison uses only five fine-tuning episodes per task; the retained text does not establish equal adaptation data or compute for every method. e-protocole-robot-resultse-vla
Quantitative tables cover Franka and Kinova; Realman and Dobot deployment is claimed without corresponding main-paper success tables. Long-horizon and orientation-sensitive tasks require VLM decomposition; dynamic, unstructured environments are left to future work. e-scopee-robot-resultse-vla
7.2 Questions for discussion
- Does correct temporal information improve executed manipulation beyond the modest POA pixel-error gains?
- How much of the transfer advantage survives when grasping, height selection and adaptation data are controlled?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the annotated datasets, pretrained encoders, diffusion model and calibrated RGB-D/grasp/motion stack. The offline protocol specifies 80:20 trajectory-dataset splits and a 3,600-image PixMo test set, but not split identifiers, optimizer settings, learning rates, image resolution, noise-schedule values or full annotation procedures. e-datae-confige-traininge-execution
Resolve temporal-token labels, the 100,000-versus-one-million sample count and waypoint indexing before implementation. Eq. (5) starts at t+1 although the contact point is x_t; Section 4.3 switches to x_2 through x_T. The VLM height prompt, grasp-distance convention and motion-controller details are also underspecified. e-architecturee-counte-execution
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test whether POA uses meaningful temporal information
Reader-proposed check; not performed. On a fixed trajectory dataset split, compare the full model, removal of POA, and a duplicate-current-frame control that preserves the temporal branch but provides no motion. Also compare current-frame-plus-difference and previous-frame-plus-difference variants to resolve Figure 3 versus Section 3.3. Keep initialization, training budget, decoder and evaluation resolution fixed; report first-contact and post-contact MAE separately over repeated seeds. If correct temporal pairing improves later waypoints beyond the duplicate-frame control, that supports the motion-information explanation. Similar results across controls would weaken that interpretation even if an extra branch improves aggregate MAE. e-architecturee-ablatione-confige-training
Check 2: Separate affordance prediction failures from execution failures
Reader-proposed check; not performed. On Kinova, repeat Place Object and Wipe Board with the same calibrated camera, grasp sampler, height-selection prompt, controller and initial scene distribution. Compare A0’s predicted points with verified annotated points supplied to the same executor; record contact error, grasp failures, path failures and task success. The annotated-point condition estimates the execution stack’s ceiling. For both conditions, measure wall-clock model time, total rollout time and controller updates separately from waypoint count. Persistently low success with annotated points would implicate execution; a large gain would implicate prediction. This directly tests the interface behind Table 2 and the efficiency interpretation of Table 3. e-executione-protocole-robot-resultse-vla
8.3 Reading coverage
Visual audit: All eleven pages of the supplied CVF paper were rendered and visually inspected, including the title/authors and edition notice (p. 1), Figures 1–6, all three tables, method equations and training/execution text (pp. 3–6), evaluation protocol and comparisons (pp. 6–8), and the concluding references (pp. 9–11). All six final original crops were inspected at their returned dimensions. Figure 3 arrows and frozen/trainable symbols were cross-checked against Sections 3.3–3.4; its temporal-token label disagreement is disclosed. Figure 4 labels and Table 1–3 cells were checked directly, preserving the sample-count, SIAL-difference and Table 3 caption discrepancies. The retained PDF has no appendix; separate supplementary experiments, annotation details, code, datasets and videos remain outside this reading.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; 1. Introduction (PDF pp. 1–3)
- 2. Related Works, including 2.1–2.2 (p. 3)
- 3. The A0 Model, including 3.1–3.4 and Eqs. (1)–(4) (pp. 3–5)
- 4. Action Execution, including 4.1–4.3 and Eqs. (5)–(6) (p. 5)
- 5. Experiment, including 5.1.1–5.1.3, 5.2.1–5.2.4 and Eq. (7) (pp. 6–8)
- 6. Conclusion (p. 8)
- 7. Acknowledgments and complete References (pp. 9–11)
Outside the original text pass
- Text extraction does not reconstruct figure pixels or preserve equation/table layout; downstream reading must inspect the retained original PDF.
- PDF font mapping and reading order can affect equation glyphs and spacing. This extraction is not a verified mathematical transcription; equations and tables still require direct PDF inspection.
- Separate supplementary material is not included in the retained 11-page proceedings PDF.
- Availability of external supplements, code, datasets and videos beyond the retained PDF has not been fully verified.
- All five supplied text chunks were read. All eleven PDF pages and all six final crops were visually inspected; this visual pass addresses extraction layout limitations without treating the text as a verified mathematical transcription.
- Identity and edition: the title and all fourteen author identities match the catalog. The supplied CVF ICCV 2025 open-access paper spans printed pp. 13491–13501; its title-page notice states that only the watermark differs from the accepted version. No arXiv revision identifier or separate edition comparison is established.
- No appendix is present in this PDF, although the dataset section refers to one. Separate supplements, detailed annotation procedures, RoboPoint comparisons and additional demonstrations were not supplied or read.
- External code, datasets, videos and project links were not accessed; no implementation was inspected and no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1 (printed 13491), title, author block and CVF notice; p. 11 footer
Title and all fourteen authors match; affiliations are Spatialtemporal AI, MBZUAI, Sun Yat-sen University and Southern University of Science and Technology. The notice identifies the accepted-paper open-access version.
Go to primary source ↓e-problemPDF pp. 1–2, Abstract, Introduction and Figure 2
Spatial affordance understanding is separated from low-level action execution through contact points and post-contact trajectories.
Go to primary source ↓e-representationPDF pp. 3–4, Sections 3.2–3.3, Eqs. (1)–(2)
Representation entries contain images, language, contact points and tracks; outputs are normalized 2D coordinates with an initial contact point.
Go to primary source ↓e-dataPDF p. 4, Section 3.2, Dataset paragraph
Lists one million PixMo points, 22,000 HOI4D, 3,056 DROID and 4,965 ManiSkill trajectories; detailed annotation is deferred to an appendix.
Go to primary source ↓e-countPDF p. 3, Section 3.1; p. 4, Figure 3 and Dataset; p. 6, Section 5.1.2
Section 3.1 says 100,000 pretraining samples, whereas the figure, dataset description and experiment section say one million.
Go to primary source ↓e-architecturePDF p. 4, Figure 3 and Section 3.3; p. 5, Position Offset Attention and Spatial Information Aggregation Layer paragraphs
Frozen encoders condition trainable diffusion blocks; an MLP decodes points. Figure 3 labels retained POA tokens as previous-frame tokens, whereas the text concatenates current-frame and motion tokens.
Go to primary source ↓e-trainingPDF p. 5, Section 3.4, Eq. (4), forward-noising expression and unnumbered fine-tuning loss
First-point MSE pretraining becomes clean-trajectory MSE fine-tuning with language instructions and optional temporal observations.
Go to primary source ↓e-inferencePDF p. 5, Section 3.3, inference paragraph and Eq. (3); p. 6, Section 5.1.1; pp. 7–8, Figure 6 caption and Sections 5.2.3–5.2.4
An ODE solver denoises waypoints in five steps; deployment is described as single-shot prediction followed by execution.
Go to primary source ↓e-executionPDF p. 5, Sections 4.1–4.3, Eqs. (5)–(6)
Depth and camera intrinsics lift points; grasp sampling, nearest-contact selection, VLM height categories and SE(3) trajectory generation complete execution.
Go to primary source ↓e-configPDF p. 6, Section 5.1.1
Specifies 28 layers, 1B parameters, five waypoints, 1,000/5 diffusion steps, 80:20 splits, 3,600 PixMo test images, training durations and four A100 80GB GPUs.
Go to primary source ↓e-pretrainingPDF p. 6, Section 5.1.2 and Figure 4, all labeled bars
Real-to-Sim MAE changes 50.4 to 43.9; Sim-to-Real changes 172.2 to 125.2 on HOI4D and 35.1 to 29.1 on DROID.
Go to primary source ↓e-ablationPDF p. 6, Table 1, all rows/columns, and Section 5.1.3
Full/without-POA/without-SIAL errors are 47.5/47.9/61.1, 5.5/6.3/10.2 and 17.5/18.5/19.6. The prose states a 13.2-pixel SIAL penalty inconsistent with the table.
Go to primary source ↓e-protocolPDF pp. 6–7, Section 5.2.1
Four household tasks, consistent object placements/camera settings, RGB-D perception and 20 trials per task define the real-robot evaluation.
Go to primary source ↓e-rolloutsPDF p. 7, Figures 5–6 and captions
Figure 5 shows input, predicted affordances and four execution snapshots for Franka; Figure 6 gives Kinova examples and describes single-shot inference.
Go to primary source ↓e-robot-resultsPDF p. 7, Table 2, all task and average-success columns
Franka means: A0 62.50, Molmo 43.75, Magma 16.25. Kinova means: A0 53.75, MOKA 45.00, ReKep 33.75. A0 loses Kinova Place Object.
Go to primary source ↓e-baselinesPDF pp. 7–8, Section 5.2.2
MOKA/ReKep are evaluated on Kinova; Magma/Molmo point outputs use the authors’ execution module. RoboPoint details are deferred to supporting material.
Go to primary source ↓e-vlaPDF p. 8, Table 3, all columns, caption and Section 5.2.3
The four-task Kinova table reports means 11.25/20.00/18.75/53.75 for RDT/π0/π0+FAST/A0 and 25–50 versus 4–5 steps. VLA adaptation uses five episodes per task.
Go to primary source ↓e-scopePDF p. 8, Sections 5.2.4 and 6; pp. 9–11, Acknowledgments and References
The conclusion claims multiple platforms, delegates long-horizon/orientation-sensitive decomposition to VLMs and leaves dynamic environments to future work. The retained PDF ends with references, without an appendix.
Go to primary source ↓8.5 Primary sources
A0: An Affordance-Aware Hierarchical Model for General Robotic Manipulation ↗
PDF · 7,377 extracted words
Source fingerprint
2b478032438ac2f551db26676fa94b9180564851b8745a4b4e646409a6fdab7e