GIFT: Guided Intermediate Feature Training via Action-Oriented Structural Supervision for Robotic Manipulation
1. Paper overview
In one sentence: Training visual features to retain geometry, interactions and goals improves matched robot policies without auxiliary inference inputs, at the cost of structured training labels and with uneven robustness gains. e-gape-overviewe-wame-injectione-guidancee-real-results
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors identify an action-sufficiency gap: semantic pretraining and visual prediction can preserve appearance while omitting spatial and interaction information needed for manipulation. They ask whether explicit intermediate supervision helps across policy families without adding auxiliary action conditions. e-gape-overview |
| Core mechanism | Source description GIFT combines geometric teacher alignment, object-centric affordance regression and goal-mask prediction; it tests VLA, direct-action WAM and inverse-dynamics WAM instantiations, with injection and same-head controls. e-overviewe-injectione-design |
| A key reported result | LIBERO-Plus zero-shot robustness: VLA 79.6; Fast 72.6; IDM 87.8. Policy success rate (%). Original LIBERO training only; full perturbed evaluation set; each instance tested once. StarVLA-OFT 75.0; Fast-WAM 60.0; Fast-WAM-IDM 82.6. Gains: 4.6, 12.6 and 5.2 percentage points. IDM leads the reported overall table, but robot-initialization success falls from 86.0 to 80.5; aggregate improvement does not mean universal improvement. e-sim-protocole-plus |
| Reading caution | Source description Guidance fails under severe noise, subtle object depth, confused object identities and goal masks spilling into irrelevant regions. Structured supervision does not eliminate perceptual ambiguity. e-failures |
Core contributions
- Source description
GIFT combines geometric teacher alignment, object-centric affordance regression and goal-mask prediction; it tests VLA, direct-action WAM and inverse-dynamics WAM instantiations, with injection and same-head controls. e-overviewe-injectione-design
Figure 2. Three supervision signals shape shared visual features while each policy retains its action pathway. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow observations, instruction and proprioception toward the policy backbone, then follow the KV arrow to the action head. The upper path uses a vision-language model; the lower path couples video and action transformers. The central band contains geometry distillation and affordance/goal predictions. Snowflakes mark frozen modules, including the geometric teacher, while flames mark trainable modules. Crucially, the central predictions have no default arrow into the action head. Section 3.3.1 explicitly masks affordance and goal tokens from the VLA head’s raw-token cross-attention; Section 3.4 separately defines the optional injection route. Read these interface rules alongside the schematic. e-overviewe-geometrye-vlae-wame-injection
What it supports. GIFT adds objectives to features that the action generator already uses. Its shared contribution is the geometry–affordance–goal training signal, while direct VLA regression, direct WAM action denoising and future-conditioned inverse dynamics remain distinct implementations. The frozen teacher supplies training targets and is unnecessary during default deployment.
Where the evidence stops. The lower drawing includes future observations as part of the training overview. It does not mean Fast generates future frames at deployment: Section 3.3.2 specifies current-frame caching, whereas Section 3.3.3 gives IDM its separate future-generation stage.
2. Motivation
2.1 The problem and the proposed response
The authors identify an action-sufficiency gap: semantic pretraining and visual prediction can preserve appearance while omitting spatial and interaction information needed for manipulation. They ask whether explicit intermediate supervision helps across policy families without adding auxiliary action conditions. e-gape-overview
2.2 What this reading follows
A robot can recognize a bowl yet fail to place an object into it when the camera or layout changes. GIFT addresses this problem by supervising what intermediate visual features preserve: geometry, object–end-effector relations and instruction-relevant regions. Its central experiment carries the same three objectives across a VLA and two world-action policies while keeping their different action formulations. The default model learns from these extra targets but does not feed their predictions into the action generator. Read the architecture first, then compare paired success rates and ablations. The feature visualizations and physical trials help explain the proposal while exposing distinct evidence limits. e-gape-overviewe-wame-injectione-guidancee-real-results
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is WAMs. View the current classification.
3.1 Evidence-based assessment
Insufficient evidence to decide
The recorded catalog is entirely unassigned. GIFT is a supervision framework with several architectures: VLA directly predicts actions; WAM-Fast has video/action experts with joint training but direct inference; WAM-IDM conditions inverse dynamics on generated futures. Joint training alone does not justify One Model, and a single paper-wide quadrant would obscure these differences. e-overviewe-vlae-wam
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Translate scene structure into training targets
Begin with one demonstration frame and its instruction. Geometry supervision asks selected shallow visual tokens to match a frozen VGGT teacher, separating feature direction from log magnitude. Affordance supervision asks a different question: which entities matter, and how are their associated end effectors situated relative to them? Its 20-D slot combines an entity role, two poses and closure state, using the first interacted object as the anchor. Goal supervision then adds dense image-space localization through an instruction-relevant binary mask. All three losses act on shared features, but only the native conditions reach the default action generator. In simulation, privileged poses and instance masks construct these targets. In the physical data, human-verified tracking and estimated object poses make the same supervision possible; that annotation pipeline is a substantive reproduction requirement. e-overviewe-geometrye-affordancee-goale-targetse-real-data
5.2 Follow the action pathway after the auxiliary heads disappear
The deployment sequence depends on the host policy. GIFT-VLA converts Qwen3-VL features and proprioceptive conditioning into a continuous action chunk through BridgeAttention. Its training objective is direct L1 regression, and affordance/goal tokens are masked from raw-token action cross-attention in the default design. GIFT-WAM-Fast trains a video expert alongside an action expert, but action tokens cannot access noisy future-video tokens. At inference it caches current-frame features and performs two action-denoising steps. GIFT-WAM-IDM instead learns actions with teacher-forced future context; deployment first denoises a future trajectory in two steps, freezes its features as context, then denoises actions in two more steps. Thus, removing GIFT’s auxiliary heads preserves both direct control and future-conditioned inverse dynamics. Feedback comes from executing part of a chunk and replanning from new observations at the benchmark-specific interval. e-vlae-wame-injectione-traininge-real-training
5.3 Separate representation evidence from control evidence
Reader analysis: the most useful evidence forms a sequence of controls. Matched baseline comparisons establish a success-rate difference; the same-head VLA control narrows how much comes from supervision rather than BridgeAttention. The injection experiment then shows that supplying auxiliary hidden features is unnecessary for the reported gain: VLA performs better without injection, while WAM differences are small. Single-signal ablations make the supervision account plausible, yet missing pairwise combinations leave interactions unresolved. Figure 4 shows interpretable auxiliary predictions, but its depth maps retain earlier teacher features, and attention maps are observational diagnostics. Finally, physical trials demonstrate executed actions on two trained platforms, with only ten attempts per condition. Together these results support structured feature training as a useful recipe; they do not prove that each learned representation is sufficient for control under every distribution shift. e-designe-inject-resultse-guidancee-diagnostice-attentione-real-protocole-real-results
5.4 Training and inference
During training
GIFT is benchmark-specific post-training. Simulator replay supplies privileged object poses, instance masks and successful actions. RoboCasa uses 1,000 demonstrations for each of 24 tasks. Real data comprise 400 teleoperated trajectories, half with glove inpainting; object masks are human-verified and poses combine masked depth with orientation estimates. e-traininge-targetse-sim-protocole-real-data
Simulation VLA training uses 32 GPUs, global batch 256 and 60,000 steps. WAM uses 64 GPUs: 50 LIBERO epochs or at most 60,000 RoboCasa steps. Both use AdamW. Real-world training uses 64 Zhenwu 810E PPUs, 96 GB each, for 50,000 iterations; per-device batches are four for WAM and two for VLA. e-traininge-real-training
During inference
Default deployment discards the VGGT teacher and need not execute auxiliary heads. Fast caches current-frame features and denoises actions in two steps; IDM first denoises future video in two steps, then actions in two. Optional injection instead appends internally predicted affordance/goal hidden features, never ground-truth labels. e-injectione-wame-training
All variants predict 32-step chunks. Replanning occurs every 10 control steps on LIBERO/Plus, 12 on RoboCasa and 32 on the real robots. LIBERO executes 7-D relative end-effector commands; RoboCasa executes 29 joint targets from a 47-channel training target; real robots execute relative joint commands. e-traininge-targetse-real-training
5.5 Implementation flow
- Preserve the native policy interface
Current observations become shared visual tokens. Geometry supervises layer 6; affordance and goal supervise final-layer current-frame tokens. Auxiliary predictions are excluded from default action conditions. e-overview
- Align geometry
Resample frozen VGGT patch features to the student grid. An MLP and scale head separately match feature direction and log magnitude, avoiding direct regression across incompatible feature scales. This is feature distillation, not depth-target training. e-geometry
- Encode interactions
Each 20-D slot contains a scalar entity role, two 9-D poses and closure state. Poses describe the entity relative to the first-interacted anchor and its associated end effector relative to that entity. Valid slots receive Smooth L1 loss. e-affordance
- Localize the goal
Predict a low-resolution binary mask for instruction-relevant objects or action regions in each view using BCE and soft Dice. This goal denotes where interaction occurs, rather than a generated future scene. e-goal
- Retain three action formulations
GIFT-VLA uses Qwen3-VL features, BridgeAttention and L1 action regression. WAM-Fast jointly trains video/action flow matching but restricts action attention to current-frame video tokens. WAM-IDM uses a teacher-forced future branch for action training and an imagine-then-act inference sequence. e-vlae-wam
6. Experiments & results
GIFT trains robot-policy features to retain geometry, object–end-effector relations and instruction-relevant regions. The same auxiliary objectives improve three different action formulations while their default deployment uses no auxiliary predictions. Evidence is strongest for matched-baseline robustness gains, with substantial annotation requirements and uneven benefits across shifts.
6.1 Read the original evidence
Table 2. Zero-shot robustness improves overall, with an informative exception in the IDM robot-initialization column. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the three bold GIFT rows, then locate each matched baseline: StarVLA-OFT, Fast-WAM and Fast-WAM-IDM. Compare the Total column before inspecting the seven perturbation columns. All methods are described as trained on original LIBERO trajectories without LIBERO-Plus fine-tuning; every perturbed instance is evaluated once. Parameter counts are in billions, and success entries are percentages. The strongest causal comparison is within a matched family, with the VLA head change checked separately in Table 7. The broad leaderboard contains models with different architectures and pretraining, so ranking across every row is a different question from measuring the effect of GIFT. e-sim-protocole-pluse-design
What it supports. The reported totals increase from 75.0 to 79.6 for VLA, 60.0 to 72.6 for Fast and 82.6 to 87.8 for IDM. These are gains of 4.6, 12.6 and 5.2 percentage points. IDM is strongest overall in this table; Fast shows the largest improvement over its baseline.
Where the evidence stops. The IDM Robot column falls from 86.0 to 80.5, despite the overall gain. Fast still reaches only 37.4 under camera shifts. No confidence intervals or training-seed variation are reported here; overall robustness should not be read as uniform reliability.
Table 3. The largest WAM gains occur on articulated interactions in the separately trained GR1 benchmark. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read Art. as the six articulated-object tasks and P&P as the eighteen pick-and-place tasks. Each task has 1,000 demonstrations and is evaluated with 50 rollouts using the GR1 humanoid setting. Avg. summarizes all 24 tasks; the two task groups contain different numbers of tasks and should not be averaged with equal weight. Compare Fast-WAM against GIFT-WAM-Fast, and Fast-WAM-IDM against GIFT-WAM-IDM. Those pairs preserve their action-generation mechanism. The VLA comparison remains useful but has the separate BridgeAttention control discussed elsewhere. Missing group values shown as dashes remain unreported. e-sim-protocole-traininge-roboe-design
What it supports. GIFT-WAM-Fast reaches 83.6 overall versus 74.6 for Fast-WAM; GIFT-WAM-IDM reaches 82.3 versus 73.9. Their articulated-task scores increase by 21.3 and 24.6 points respectively. VLA also improves from 48.8 to 61.4 overall. These patterns support testing structured supervision beyond the near-saturated original LIBERO tasks.
Where the evidence stops. These models receive RoboCasa-specific training. This table therefore does not demonstrate zero-shot transfer from a Panda arm to GR1. The articulated-task gains combine all three guidance losses, so they do not individually identify geometry or affordance as the cause.
Table 9. Physical robustness improves in small trial sets, while compounded pose changes still cause failures. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Every cell is a count of successes out of ten trials. Task 2 is xArm7 layer-conditioned placement: Level 1 adds time-varying colored illumination and Level 2 also changes the tablecloth. Task 4 is ARX X5 bimanual test-tube insertion: Level 1 replaces background objects and Level 2 additionally rotates the tubes and rack by approximately 15–20 degrees. These conditions are absent from the training demonstrations. Compare each GIFT row to its baseline within the same policy family. Level 2 retains the earlier perturbation, so its effect cannot be read as an isolated lighting or pose intervention. e-real-datae-real-traininge-real-protocole-real-results
What it supports. GIFT-WAM-IDM maintains 7/10 on both Task 2 levels and achieves 8/10 then 5/10 on Task 4, compared with 3/10, 2/10, 1/10 and 0/10 for its baseline. GIFT-VLA improves too, but its Task 4 Level 2 score remains 1/10. These are executed-robot outcomes, distinct from the paper’s feature visualizations.
Where the evidence stops. Ten trials per condition give a coarse success estimate, and no repeated-training uncertainty is supplied. The design tests four task/perturbation conditions on two separately trained platforms; it does not establish unrestricted physical generalization or isolate each component of the compounded shifts.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO in-distribution manipulation Four ten-task suites; 50 rollouts per task. | VLA 97.9; Fast 97.7; IDM 98.5. Policy success rate (%) | StarVLA-OFT 96.6; Fast-WAM 97.6; Fast-WAM-IDM 98.0. Reported gains of 1.3, 0.1 and 0.5 percentage points leave little headroom on this benchmark. e-sim-protocole-libero |
| LIBERO-Plus zero-shot robustness Original LIBERO training only; full perturbed evaluation set; each instance tested once. | VLA 79.6; Fast 72.6; IDM 87.8. Policy success rate (%) | StarVLA-OFT 75.0; Fast-WAM 60.0; Fast-WAM-IDM 82.6. Gains: 4.6, 12.6 and 5.2 percentage points. IDM leads the reported overall table, but robot-initialization success falls from 86.0 to 80.5; aggregate improvement does not mean universal improvement. e-sim-protocole-plus |
| RoboCasa GR1 tabletop manipulation 24 tasks; 50 rollouts per task; separately trained on RoboCasa. | VLA 61.4; Fast 83.6; IDM 82.3. Policy success rate (%) | StarVLA-OFT 48.8; Fast-WAM 74.6; Fast-WAM-IDM 73.9. Articulated-task Fast/IDM success rises 62.0→83.3 and 59.7→84.3. This tests another trained embodiment, not zero-shot cross-robot deployment. e-sim-protocole-traininge-robo |
| Guidance and injection ablations LIBERO-Plus full evaluation set; outer weights of removed losses set to zero. | Fast geometry/affordance/goal/all: 70.4/70.5/71.7/72.6. Policy success rate (%) | Fast baseline 60.0. VLA injection/no-injection: 76.7/79.6; same-head no-guidance VLA: 75.7. Full guidance has the highest reported total in each family. VLA gains 3.9 points over its same-head control; tiny WAM injection differences lack uncertainty estimates. e-guidancee-inject-resultse-designe-objectivee-sim-protocol |
| Real-robot original and perturbed manipulation Four original tasks and four perturbation conditions; ten trials per task/condition and method. | Original averages: VLA 57.5, IDM 87.5. Perturbed averages: VLA 27.5, IDM 67.5. Policy success rate (%) | StarVLA-OFT/Fast-WAM-IDM: 35.0/52.5 originally and 5.0/15.0 under perturbations. IDM test-tube insertion improves 2/10→8/10 originally and 0/10→5/10 with novel distractors plus rotated tubes/rack. These are small executed-robot samples. e-real-datae-real-protocole-real-results |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 4. Full guidance has the best reported total, but different signals favor different shifts. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Within each policy block, the first row is its named baseline, the next three activate one guidance signal each, and the last activates all three. The checkmarks identify training objectives, not inference-time inputs. Removed losses receive zero outer weight while remaining weights stay fixed. Read across a row to see which distribution shifts benefit, then down Total to compare aggregate performance. For Fast, geometry, affordance and goal alone yield reported totals of 70.4, 70.5 and 71.7 against 60.0; all three reach 72.6. For VLA, consult the separate 75.7 same-head control before attributing the full baseline difference solely to supervision. e-guidancee-objectivee-designe-sim-protocol
What it supports. All-guidance totals are 79.6, 72.6 and 87.8 for VLA, Fast and IDM, each above its single-guidance totals. Nevertheless, IDM affordance-only attains 91.0 on Robot while all-guidance falls to 80.5. The combined model improves the overall objective while sacrificing one form of robustness.
Where the evidence stops. No two-signal combinations or seed dispersion appear in this table. The rows support benefits from each signal and a higher combined total, but do not establish additive effects, identify pairwise interference, or show that the full combination dominates every shift.
Figure 4. Structured predictions provide qualitative diagnostics of what the supervised features retain. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each row supplies an instruction and two groups of images ordered by the Timestamp arrow. Within each group, read current observation, geometry prediction, affordance prediction and goal prediction. The top three rows are VLA examples and the bottom three are IDM examples, as identified by the caption. Look for correspondence between object extent in the scene, the overlaid interaction configuration and highlighted goal regions. The geometry maps require special care: the authors reconstruct student feature magnitude and direction, substitute that feature at VGGT’s final layer, then use the frozen depth head together with earlier image-derived VGGT features. e-diagnostice-geometrye-affordancee-goale-failures
What it supports. The chosen examples show that guided features can produce scene-layout maps, object-linked end-effector configurations and localized instruction-relevant regions. This supports the intended interpretation of the auxiliary objectives and complements the success ablations. It supplies qualitative evidence about representation content, rather than a measured estimate of geometric accuracy or task success.
Where the evidence stops. The hybrid decoder retains teacher information beyond the student feature. No metric depth scale, error distribution or standalone student-depth evaluation is shown. Figure 7 also documents failures under noise, object ambiguity and irrelevant-region masks; these selected examples do not characterize their frequency.
7. Analysis & limitations
7.1 What the evidence leaves open
Guidance fails under severe noise, subtle object depth, confused object identities and goal masks spilling into irrelevant regions. Structured supervision does not eliminate perceptual ambiguity. e-failures
The combined objectives need not help each shift: IDM robot-initialization performance regresses despite broad robustness language in the conclusion. Single-signal and full-signal rows do not isolate pairwise interactions. Reported success tables lack seed dispersion or confidence intervals. e-pluse-guidancee-conclusione-real-results
Geometry visualizations retain earlier image-derived VGGT features, so they cannot establish standalone student depth accuracy. Selected attention maps offer qualitative association, not a causal intervention proving why actions improve. e-diagnostice-attention
7.2 Questions for discussion
- Would matched multi-seed training preserve the small no-injection advantage in WAMs?
- Which pair of guidance signals explains the IDM robot-initialization regression, and does it persist with controlled annotation noise?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires benchmark trajectories, exact pretrained backbones, VGGT targets and successful simulator replay with privileged poses/segmentations. Preserve four simulation slots, valid-slot masking, action padding masks, image views, action channels and replanning intervals; changing these would alter the comparison. e-targetse-traininge-sim-protocol
Real-world replication additionally requires calibrated RGB-D observations, corrected mask tracks, orientation labels and arm/object assignments. The paper specifies hardware and trial counts but leaves simulator/software versions, seed lists, inpainting implementation and the IDM future-corruption schedule unspecified. A project-page link alone does not establish released executable artifacts. e-real-datae-real-traininge-traininge-wame-identity
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Repeat the injection comparison with a verified information boundary
Reader-proposed, not executed: train no-guidance, full-guidance/no-injection and full-guidance/injection models using the same BridgeAttention VLA head, trajectories, update budget and at least three paired seeds. Keep image views, 32-step chunks and ten-step LIBERO replanning fixed. Test the full LIBERO-Plus set and report per-shift success with seed variation. Inspect the action cross-attention mask, then verify that skipping auxiliary decoders leaves no-injection outputs unchanged for fixed inputs. A persistent no-injection gain over the same-head control would support feature supervision; disappearance across seeds or dependence on decoder outputs would weaken that explanation. e-vlae-injectione-designe-inject-resultse-traininge-sim-protocol
Check 2: Test which objective combination hurts IDM robot-initialization robustness
Reader-proposed, not executed: complete an eight-condition factorial study of geometry, affordance and goal losses for WAM-IDM, adding the missing two-signal combinations. Use identical demonstrations, target construction, native video/action losses, remaining loss weights and paired training seeds. Keep two future-video plus two action-denoising steps fixed. Evaluate the same full LIBERO-Plus instances and separately track Robot, Camera and overall success. The paper reports Robot success of 91.0 with affordance alone but 80.5 with all signals. Repeated degradation after adding a specific objective would identify an interaction; disappearance with paired seeds would instead implicate experimental variability. e-guidancee-objectivee-wame-targetse-traininge-sim-protocol
8.3 Reading coverage
Visual audit: Actually inspected the title/author/version page and PDF pages 2–16, including method equations and attention descriptions, all ten original figures, all nine tables, simulator target construction, training compute and real-world protocols. Cross-checked Figure 2 with Eqs. (1)–(2) and Sections 3.3–3.4, and the Figure 4 geometry display with Eq. (8) and Section 4.4 Q3. All six final original crops were viewed; narrow tables were rendered at higher DPI, with no upscaling or redrawing. Pages 17–18 contain references read in the complete text pass; they were not visually inspected. No appendix is present, separate supplement availability remains unverified, and external code/project videos were not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, authors, affiliations, arXiv version and abstract (p. 1)
- 1 Introduction (pp. 1–2)
- 2 Related Work, including distinction from prior work (pp. 2–3)
- 3 Methodology, Sections 3.1–3.5 and Eqs. (1)–(21) (pp. 3–7)
- 4 Simulation Evaluation, Sections 4.1–4.5 (pp. 7–13)
- 5 Real-World Experiments, Sections 5.1–5.3 (pp. 13–16)
- 6 Conclusion (p. 16)
- References [1]–[69] (pp. 16–18)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction-only figure limitation was addressed by inspecting PDF pages 1–16, every figure and table, and all six final crops. Reference-only pages 17–18 were read as text.
- No appendix is present in this 18-page PDF. Separate supplemental material was not supplied.
- The supplied v1 title and complete author list match the catalog. No other revision was supplied, so no cross-version comparison is claimed.
- Code, linked project materials and videos were not inspected; experiments were not reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author/affiliation block and arXiv margin
The exact catalog title and all fourteen authors match the title page. The artifact is arXiv:2609.04193v1 [cs.RO], dated 3 September 2026. Five affiliations are explicitly listed.
Go to primary source ↓e-gapPDF pp. 1–3, Abstract, Section 1 and Section 2.3
The authors call the mismatch between rich visual features and control-relevant information the action-sufficiency gap; GIFT extends their preliminary PokeVLA approach with affordance guidance and three policy formulations.
Go to primary source ↓e-overviewPDF pp. 3–4, Section 3.1, Eqs. (1)–(2), Figure 2 and caption
The policy maps observations, language and state to intermediate features and an action chunk. Geometry supervises layer 6; affordance and goal use final-layer current-frame features. Default action conditions exclude auxiliary predictions.
Go to primary source ↓e-geometryPDF pp. 4–5, Section 3.2.1, Eqs. (3)–(8)
Frozen VGGT patch features are resampled to the student grid. Projected features learn direction by cosine alignment and log magnitude by squared error, with weights 0.2 and 0.05. The teacher is unnecessary at deployment.
Go to primary source ↓e-affordancePDF p. 5, Section 3.2.2, Eqs. (9)–(13)
Each entity has a 20-dimensional role/pose/closure target anchored to the first interacted object. Each pose comprises translation and continuous 6-D rotation. Valid slots receive element-averaged Smooth L1 supervision; labels are training-only.
Go to primary source ↓e-goalPDF p. 5, Section 3.2.3, Eqs. (14)–(15)
The sole goal target is a per-view binary mask of instruction-relevant objects or regions, downsampled to the decoder grid and supervised with binary cross entropy and soft Dice loss.
Go to primary source ↓e-vlaPDF p. 6, Section 3.3.1, Eq. (16)
Qwen3-VL supplies features to a layer-aligned BridgeAttention action head trained by L1 regression. There is one action query per horizon step; auxiliary affordance and goal tokens are masked from raw-token action cross-attention in the default design.
Go to primary source ↓e-wamPDF p. 6, Sections 3.3.2–3.3.3, Eqs. (17)–(19)
Separate video and action experts retain flow-matching losses. Fast action tokens can access current-frame tokens but not noisy future-video tokens. IDM trains with a teacher-forced future branch and generates future-video context before action denoising at inference.
Go to primary source ↓e-injectionPDF pp. 6–7, Section 3.4, Eq. (20), Figure 3 and caption
Optional injection projects affordance and goal hidden features into action context during training and inference. Geometry remains alignment-only. Default no-injection heads and the geometry teacher need not run at deployment.
Go to primary source ↓e-objectivePDF p. 7, Section 3.5, Eq. (21); p. 9, Section 4.2, Loss weights
GIFT adds geometry, affordance and goal losses to the native action/video objective with outer weights 1.0, 0.5 and 1.0. BCE and Dice each use 0.2; video and action flow losses each use 1.0. Removed objectives have zero weight in ablations.
Go to primary source ↓e-sim-protocolPDF p. 7, Section 4.1
LIBERO has four ten-task suites evaluated with 50 rollouts per task. LIBERO-Plus tests each perturbed instance once, with no fine-tuning beyond original LIBERO trajectories. RoboCasa GR1 has 24 tasks, 1,000 demonstrations each, and 50 evaluation rollouts each, split into 18 pick-and-place and six articulated tasks.
Go to primary source ↓e-trainingPDF pp. 7–9, Section 4.2, implementation and action-space paragraphs
Benchmark-specific post-training uses Qwen3-VL-4B-Instruct for VLA and the Fast-WAM experts for WAM. Images are 224×224 and action chunks have 32 steps. VLA uses 32 GPUs for 60,000 steps; WAM uses 64 GPUs, 50 epochs on LIBERO and at most 60,000 steps on RoboCasa. Replanning intervals are 10 and 12 control steps respectively.
Go to primary source ↓e-targetsPDF pp. 8–9, Section 4.2, Action spaces and Simulation structured-target construction
Successful expert replay supplies executable actions, simulator poses, object segmentations and four-slot affordance targets. LIBERO actions are 7-D relative end-effector commands. RoboCasa trains on 47 channels but executes only its 29 joint targets; extra end-effector pose channels supervise training.
Go to primary source ↓e-liberoPDF p. 8, Table 1, matched baseline and GIFT rows, Total column
LIBERO totals are 96.6/97.9 for StarVLA-OFT/GIFT-VLA, 97.6/97.7 for Fast-WAM/GIFT-WAM-Fast and 98.0/98.5 for Fast-WAM-IDM/GIFT-WAM-IDM.
Go to primary source ↓e-plusPDF p. 8, Table 2, matched baseline and GIFT rows, Total and perturbation columns
LIBERO-Plus totals are 75.0/79.6, 60.0/72.6 and 82.6/87.8 for the three baseline/GIFT pairs. IDM robot-initialization success drops from 86.0 to 80.5. GIFT-WAM-Fast camera success remains 37.4.
Go to primary source ↓e-roboPDF p. 9, Table 3, matched baseline and GIFT rows, Art., P&P and Avg. columns
RoboCasa averages are 48.8/61.4, 74.6/83.6 and 73.9/82.3 for the three pairs. Articulated-task WAM scores increase from 62.0 to 83.3 and 59.7 to 84.3; Fast has 83.7 pick-and-place success.
Go to primary source ↓e-guidancePDF p. 10, Table 4, all three policy blocks and all guidance rows
Reported no-guidance/geometry/affordance/goal/all totals are 75.0/77.1/79.0/79.1/79.6 for VLA, 60.0/70.4/70.5/71.7/72.6 for Fast and 82.6/84.8/85.4/86.9/87.8 for IDM. No two-signal ablation rows appear; all-signals is not best in every column.
Go to primary source ↓e-inject-resultsPDF p. 10, Table 5 and Section 4.4, Q2
VLA injection/no-injection scores are 76.7/79.6 on LIBERO-Plus and 58.6/61.4 on RoboCasa. WAM differences favor no injection by only 0.2–0.3 points. The table supplies no uncertainty estimates.
Go to primary source ↓e-diagnosticPDF p. 10, Section 4.4, Q3; p. 11, Figure 4 and caption; p. 5, Eq. (8)
Geometry images replace only final-layer VGGT features with scale-reconstructed student features while retaining image-derived earlier VGGT features and its frozen depth head. Figure 4 shows selected VLA and IDM geometry, affordance and goal predictions without quantitative error metrics or a calibrated depth scale.
Go to primary source ↓e-attentionPDF p. 11, Section 4.4, Q4; p. 12, Figures 5–6 and captions
Selected action-to-vision attention maps compare GIFT-VLA and GIFT-WAM-IDM with matched baselines on LIBERO-Plus and RoboCasa. The authors interpret more concentrated object/interaction attention as supporting their representation account.
Go to primary source ↓e-failuresPDF pp. 11–12, Section 4.4, Q5; p. 13, Figure 7 and caption
Examples show geometry degradation under severe noise or subtle object depth, affordance confusion over object identity or roles, and goal masks covering irrelevant regions under visual shifts.
Go to primary source ↓e-designPDF pp. 12–13, Section 4.5, Tables 6–7; p. 10, Table 4, full GIFT-VLA row
Two denoising steps per applicable modality attain 87.8 for IDM and 72.6 for Fast. VLA MLP/action-query attention/action-query-plus-vision attention controls score 75.0/75.4/75.7, versus 79.6 with full guidance.
Go to primary source ↓e-real-dataPDF p. 13, Sections 5.1.1–5.1.3; p. 14, Figure 8 and caption
Four tasks use xArm7 with two D435 cameras or dual-arm ARX X5 with three D405 cameras. There are 100 teleoperated demonstrations per task; glove inpainting is applied to half. Verified Grounding DINO/SAM2 masks, masked depth and Orient Anything V2 construct labels. Single/dual-arm settings use four/five valid slots.
Go to primary source ↓e-real-trainingPDF pp. 13–14, Section 5.1.4
Each platform is trained separately with matched family-specific settings: 50,000 iterations on 64 Zhenwu 810E PPUs of 96 GB each; per-device batches are four for WAM and two for VLA. Relative joint commands are predicted and executed in 32-step chunks with replanning every 32 steps.
Go to primary source ↓e-real-protocolPDF p. 14, Section 5.2 and Table 9 caption; p. 15, Figures 9–10 and captions
Each original task and each perturbation condition uses ten trials per method. Task 2 adds colored illumination, then a new tablecloth. Task 4 replaces background objects, then rotates tubes and rack approximately 15–20 degrees. These conditions are absent from training.
Go to primary source ↓e-real-resultsPDF p. 14, Tables 8–9 and Section 5.3; p. 16, continuation of Section 5.3
Original four-task averages are 35.0/57.5 for baseline/GIFT VLA and 52.5/87.5 for baseline/GIFT IDM. Task 4 original success is 2/10 versus 8/10 for IDM; at Level 2 it is 0/10 versus 5/10. Perturbed-condition averages are 5.0/27.5 and 15.0/67.5 respectively.
Go to primary source ↓e-conclusionPDF p. 16, Section 6
The conclusion claims reusable structured supervision and proposes additional guidance, scaling training and improving reliability under severe perceptual shifts. Its broad robustness wording does not enumerate the per-shift exceptions in Table 2.
Go to primary source ↓8.5 Primary sources
GIFT: Guided Intermediate Feature Training via Action-Oriented Structural Supervision for Robotic Manipulation ↗
PDF · 13,329 extracted words
Source fingerprint
adffef4f4bc7266729d930dc2309fe2b4b742668cae1a82de4fb16281714906d