Self-Correcting VLA: Online Action Refinement via Sparse World Imagination
1. Paper overview
In one sentence: Shared progress and end-effector predictions improve a flow policy and guide a separate residual controller, but the demonstrated online gains rely on simulation interaction and sparse environment reward. e02e04e05e07e08e10e14
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors argue that imitation-trained VLAs lack explicit short-term physical constraints, while external reward models can be disconnected from policy representations. Their question is whether small, interpretable predictions inside the action backbone can improve imitation and subsequently guide local corrections during environment interaction. e02e04e07 |
| Core mechanism | |
| A key reported result | ManiSkill3 four-task manipulation success: SPI+OAR: 1.00 / 1.00 / 0.88 / 0.56; reported average 0.86. Task success fraction; higher is better. 100 demonstrations per task; 50 evaluation episodes per task; StackCube, PlaceSphere, LiftPegUpright and PegInsertion. SPI alone averages 0.82; GR00T N1.5 0.72; specialist DP† 0.77; multitask DP‡ 0.36. The full method exceeds specialist DP by 9 percentage points. OAR adds 4 average percentage points over SPI. DP† uses separate task specialists; OAR additionally receives online interactions, so equal demonstration budgets do not imply equal total experience. e09e10e18 |
| Reading caution | Reader analysis The reward is only partly endogenous. Figure 2 includes environment feedback while its caption says 'without external supervision'; Equation (15) explicitly includes sparse reward. Predicted end-effector motion alone does not establish learned object dynamics or general physical understanding. e04e05e08 |
Core contributions
Figure 2. Prediction heads share the action backbone; online correction is learned in a separate residual module. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the upper half from left to right. Vision and text enter the VLM; proprioception enters the action backbone. The branch after Layer M feeds the ΔState and Progress heads, while the later Layer N feeds the Action Head. Equations (9)–(10) identify these as intermediate prediction heads trained alongside flow matching, rather than an image-generation cascade. In the lower half, the snowflake marks the frozen predictive base and the flame marks the trainable residual module. The base-action arrow reaches Final Action, while the residual supplies a correction. Progress modulates the dense guidance; the environment loop supplies actual state feedback for learning. e04e05e06e07e08
What it supports. The same base representation supports action generation and sparse prediction, but the complete controller contains an additional learned policy. This architecture explains why SPI can be evaluated alone and why OAR can improve it without updating the base network. The pictured ΔState is a compact end-effector target, not a simulated future scene.
Where the evidence stops. The caption's 'without external supervision' wording is broader than the formulation. The graphic itself includes Env Reward, and Equation (15) explicitly retains sparse environment reward and a time penalty. It supports endogenous dense shaping, not reward-free learning.
2. Motivation
2.1 The problem and the proposed response
The authors argue that imitation-trained VLAs lack explicit short-term physical constraints, while external reward models can be disconnected from policy representations. Their question is whether small, interpretable predictions inside the action backbone can improve imitation and subsequently guide local corrections during environment interaction. e02e04e07
2.2 What this reading follows
A manipulation policy may choose a plausible action yet still need small corrections near contact. SC-VLA gives its base policy two additional prediction tasks: estimate task progress and forecast a short-horizon end-effector change. Those predictions first act as supervised constraints inside the action backbone. A second stage freezes that backbone and learns residual actions, using predicted motion to shape directional feedback. Read the architecture together with the reward equations: the dense guidance comes from the model, but sparse environment reward remains. The experiments then separate three questions—whether prediction helps, whether residual refinement helps, and whether the predictive base works on a physical arm. e02e04e05e07e08e10e14
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is WAMs. View the current classification.
3.1 Evidence-based assessment
Insufficient evidence to decide
The recorded taxonomy is entirely unassigned, so there is no substantive label to affirm or reject. Architecture evidence supports joint action and sparse future-state prediction within one base DiT, followed by a separate residual controller. It is not inverse dynamics or video rollout. A whole-system 'One Model' label would need to distinguish that shared base from the two-stage control pipeline. e04e05e07
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Why supervise predictions inside the action backbone?
The first stage changes what the policy must represent during supervised learning. A conventional flow objective teaches a vector field that transforms noise into demonstrated actions. SC-VLA augments the DiT query sequence with progress and relative-state targets while retaining the action queries. Intermediate features feed two lightweight prediction heads, and the final block feeds the action head. The relative-state target combines local-frame translation, relative rotation and gripper change at a jittered future horizon. Its supervision therefore asks the shared features to describe an aspect of short-term evolution as well as an action distribution. The paper adds MSE losses for these predictions to flow matching. Table 3 tests this design by removing the predictive queries; its average pattern favors both signals, although individual tasks do not all improve. e03e04e05e06e12
5.2 How does a predicted change become useful feedback?
After the base is frozen, its forecasts play a different role. The residual MLP observes the current robot state together with predicted progress and state change, and adjusts the base action by a scaled additive term. The guidance reward then compares actual end-effector displacement with the predicted translation direction. Reader interpretation: this rewards moving consistently with the base policy's intended local trend, but it is not a complete test of whether an object was successfully manipulated. Sparse environment reward remains the task feedback. The progress schedule reduces reliance on the directional prior later in a task. An unresolved implementation boundary is the frame convention: Equation (8) produces local-frame translation, while Equations (13)–(14) do not show its conversion before comparison with measured displacement. e05e07e08e19
5.3 Separate learning phases from the evidence they produce
The online stage has its own initialization. First the frozen base populates the replay buffer with residual scale zero; then the residual contribution is introduced gradually before main training. This action-scale schedule is distinct from the progress-dependent coefficient on the guidance reward. Appendix D additionally uses smaller residual scales during evaluation than training, with larger scales and a longer interaction budget for PegInsertion. Reader interpretation: comparing all methods only by demonstration count misses a substantial part of the full system's learning resources. Figure 4 intentionally omits the initialization phases, whereas Tables 1 and 2 summarize final success and successful-episode lengths. The real-robot table answers a narrower question still: it tests the SPI base trained on physical demonstrations, without measuring online residual improvement on the robot. e09e10e11e13e14e16e18
5.4 Training and inference
During training
Stage I jointly minimizes flow-matching error and two weighted MSE prediction losses. Appendix D reports AdamW, 50,000 iterations, batch size 32, learning rate 0.0001, initialization seed 42 and one NVIDIA L40 GPU. e03e06e17
Stage II freezes the base. Buffer collection first uses zero residual weight, followed by linear residual injection and main training. Table 7 gives 500,000 steps for StackCube/PlaceSphere, 600,000 for LiftPegUpright and 3,000,000 for PegInsertion. PegInsertion uses train/evaluation residual scales 0.1/0.03; the other tasks use 0.01/0.005. e16e17e18
During inference
Flow matching generates base actions by numerically integrating a learned vector field from Gaussian noise. With OAR, the frozen base also supplies predictions, and the residual policy adjusts its action before execution. SAC optimization belongs to online training; the paper does not establish continual weight updates during evaluation. e03e07e18
The dense term rewards alignment with predicted translation after executing n steps, with n smaller than the execution horizon. It ignores predicted rotation and gripper change in that reward. Equation (15) retains sparse environment reward and a time penalty despite broader endogenous-reward language. e05e08
5.5 Implementation flow
- Fuse observations
Section 4.1 names SigLIP-2 for images and Eagle-2 for multimodal processing. An intermediate VLM representation conditions the DiT. The backbone citation is ambiguous, as noted under reproduction. e04e20
- Share action and prediction features
The query sequence contains current state, one progress query, one relative-state query and 16 action queries. Prediction heads read an intermediate DiT block; the action head reads the final block. This is shared-feature supervision, not a demonstrated sequential rollout of imagined scenes. e04e05e06
- Predict local physical change
The target covers translation, relative Euler rotation and gripper-opening change at a future time offset by the execution horizon plus random temporal jitter. It describes the end effector, not a complete object/contact state. e05
- Correct and collect feedback
A Gaussian MLP residual policy receives current state, predicted progress and predicted state change. The summed action is executed; observed displacement supplies directional feedback, and transitions enter a replay buffer for SAC updates. e07e08
6. Experiments & results
SC-VLA adds progress and end-effector-change predictions to a GR00T N1.5-based flow policy, then freezes it and trains a SAC residual controller. Predicted motion supplies a directional reward whose influence decreases with predicted progress. Simulation supports both stages; physical ARX5 trials test only the predictive base policy. The evidence supports improved executed manipulation under the reported protocols, with unresolved reward and evaluation details.
6.1 Read the original evidence
Table 1. Both predictive supervision and residual refinement improve the reported simulation average. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the two shaded SC-VLA rows, which separate SPI from SPI plus OAR. Read each task column before the Avg column: PlaceSphere is already at 1.00 for SPI, whereas the other tasks still improve. The green bottom row records absolute changes in success fractions expressed as percentages; these are percentage-point gains. For the baselines, the original caption defines ‡ as one multitask policy and † as separate task specialists, reflecting the lack of language guidance in DP and ACT. Section 5.1 supplies the common budget of 100 demonstrations and 50 evaluation episodes per task. OAR additionally uses online environment experience. e09e10e15e18
What it supports. The full method reports average success 0.86, versus 0.82 for SPI and 0.72 for GR00T N1.5. Specialist DP reaches 0.77, so the full method's margin over that baseline is 9 percentage points. The OAR increment is smaller—4 points on average—and includes no gain on the already-solved PlaceSphere task.
Where the evidence stops. These comparisons share demonstration counts but not total training experience or all baseline configurations. OAR receives additional interactions. The table reports no confidence intervals or repeated-seed variability, so small task-level differences should not be treated as established statistical effects.
Table 2. The reported efficiency gain measures successful-episode steps, with a notable benefit on PegInsertion. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read lower values as shorter completion trajectories, and keep the successful-episode qualifier in the table title. As in Table 1, ‡ denotes a multitask policy and † denotes separate specialists. Comparing the two shaded rows isolates the reported change after OAR: PegInsertion falls from 262 to 173 steps, while LiftPegUpright barely changes, from 190 to 189. The green row prints relative reductions, unlike Table 1's absolute success-rate gains. The rightmost 16.0% compares full SC-VLA with SPI alone. It does not describe the reduction relative to the fastest external baseline, whose reported average is specialist DP's 172. e10e11
What it supports. SC-VLA with OAR has the lowest reported average, 157 steps, compared with 187 for SPI and 172 for specialist DP. The evidence supports shorter successful trajectories in this protocol. It does not measure inference latency, robot control frequency, reset overhead or completed tasks per wall-clock hour.
Where the evidence stops. Table 1 gives multitask DP zero PegInsertion successes, yet this success-conditioned table assigns it 800 steps. That aggregation convention is unexplained. Preserve the printed values and use the success table alongside lengths rather than treating every entry as an unambiguous conditional mean.
Table 4. Physical trials evaluate the predictive base policy, with no OAR row. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the task headers: PushCube replaces the simulation suite's LiftPegUpright, so the averages are not the same benchmark. Section 5.3 reports an ARX5 arm, 60 demonstrations per task and 20 execution trials per task; Figure 3 shows wrist and third-person cameras. The SC-VLA row is explicitly SPI. The DP row carries the ‡ multitask marker defined in Table 1. Compare task columns as well as averages: SPI is highest on all four displayed tasks, but PegInsertion remains difficult at 0.40. The absence of an OAR row is scientifically consequential because the experiment does not evaluate online residual adaptation on the real arm. e10e14
What it supports. Table 4 reports SPI at 0.71 on average, versus 0.57 for GR00T N1.5 and 0.28 for DP. The task values are 0.85, 0.60, 1.00 and 0.40. This supports the predictive base under the physical training-and-evaluation setup; it does not establish the complete self-correcting loop on hardware.
Where the evidence stops. The adjacent paragraph says 70%, while the table prints 0.71 and its task values average 0.7125. This edition preserves that discrepancy. Twenty trials per task and no reported confidence intervals also limit precision and generalization claims.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| ManiSkill3 four-task manipulation success 100 demonstrations per task; 50 evaluation episodes per task; StackCube, PlaceSphere, LiftPegUpright and PegInsertion. | SPI+OAR: 1.00 / 1.00 / 0.88 / 0.56; reported average 0.86. Task success fraction; higher is better | SPI alone averages 0.82; GR00T N1.5 0.72; specialist DP† 0.77; multitask DP‡ 0.36. The full method exceeds specialist DP by 9 percentage points. OAR adds 4 average percentage points over SPI. DP† uses separate task specialists; OAR additionally receives online interactions, so equal demonstration budgets do not imply equal total experience. e09e10e18 |
| ManiSkill3 successful-episode completion length Table 2 labels this as completion length over successful episodes on the same four tasks. | SPI+OAR: 158 / 110 / 189 / 173; reported average 157. Environment steps; lower is better | Reported averages: SPI 187, specialist DP† 172, GR00T N1.5 195, π0 276. Table 2 reports a 16% reduction from SPI. These are step counts, not wall-clock throughput. The headline 16% compares with SPI, not the strongest external length baseline. Table 2 reports 800 for multitask DP on PegInsertion although Table 1 reports zero successes; that cell's aggregation is unresolved. e10e11 |
| SPI prediction-component ablation Same simulation demonstration data and evaluation protocol; SPI base without OAR. | Both predictions: 0.82; without state: 0.78; without progress: 0.80; without both: 0.72. Average success fraction | Removing state lowers PegInsertion from 0.50 to 0.42 and StackCube from 0.96 to 0.88. The pattern supports complementary auxiliary signals, but is not uniform: removing state raises LiftPegUpright from 0.82 to 0.84. No uncertainty is given for these table entries. e12 |
| ARX5 four-task physical manipulation success SPI only; 60 demonstrations and 20 execution trials per task; StackCube, PlaceSphere, PushCube and PegInsertion. | 0.85 / 0.60 / 1.00 / 0.40; Table 4 reports average 0.71. Task success fraction | GR00T N1.5 reports 0.57; multitask DP‡ reports 0.28. Rounded table averages differ by 14 and 43 percentage points. Adjacent prose says 70%, conflicting with 0.71; the displayed task values average 0.7125 (reader calculation). These trials support SPI on physical hardware, not physical online residual learning or zero-shot simulation transfer. e14 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 3. Removing both predictive queries causes the largest average loss in the reported ablation. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The bottom row is SPI without the online residual stage. The three rows above remove the relative-state query, the progress query, or both. 'Without state' refers to the auxiliary relative-state prediction query, not removal of the robot's current proprioceptive observation. Compare the Avg column first, then inspect where the differences arise. Removing state hurts StackCube and PegInsertion most visibly. Removing progress mainly changes StackCube and LiftPegUpright. PlaceSphere stays at 1.00 throughout, so its ceiling contributes no evidence for either head. This table probes the predictive training design; the reward-scheduling experiment is a separate ablation in Figure 4. e05e06e10e12e13
What it supports. Full SPI averages 0.82, compared with 0.78 without state, 0.80 without progress and 0.72 without both. That pattern is consistent with complementary supervision. The joint removal recovers the GR00T baseline's reported average, but the table alone cannot show that the learned features encode accurate physical dynamics.
Where the evidence stops. The effect is not uniformly positive: LiftPegUpright reaches 0.84 without state versus 0.82 with full SPI. No uncertainty or prediction-accuracy measurements accompany this table, limiting claims about mechanism and statistical reliability.
Figure 4. Directional guidance and its progress-dependent decay affect different parts of residual learning. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the legend before comparing panels: green is full SC-VLA, orange removes dynamic weighting, and blue removes imagination reward. The horizontal axis counts environment interactions; the vertical axis is average length, with different ranges across tasks. Section 5.2 describes the orange variant as fixed-weight guidance and the blue variant as learning from sparse environment reward without the guiding term. PegInsertion separates the green curve from both controls, while PlaceSphere mainly reveals late deterioration under fixed weighting. The caption and Appendix C say that these curves begin in the main training phase, after buffer collection and residual injection, so the visible starts are not training from scratch. e08e13e14e16
What it supports. The plot is consistent with two distinct benefits: directional shaping helps residual exploration in difficult tasks, and decreasing its weight can prevent a stale prior from dominating later behavior. On PlaceSphere, removing imagination reward remains relatively competitive, whereas keeping a fixed guidance weight deteriorates late. The guidance mechanism is therefore task-dependent.
Where the evidence stops. The shaded bands have no stated statistical definition. These are main-training average-length curves, not Table 2's explicitly successful-episode completion lengths. They exclude initialization and do not establish end-to-end wall-clock learning cost or physical-robot residual performance.
7. Analysis & limitations
7.1 What the evidence leaves open
The reward is only partly endogenous. Figure 2 includes environment feedback while its caption says 'without external supervision'; Equation (15) explicitly includes sparse reward. Predicted end-effector motion alone does not establish learned object dynamics or general physical understanding. e04e05e08
Equation (8) predicts local-frame translation, whereas Equation (13) adds it directly to P_t and Equation (14) compares it with measured displacement. The necessary common-frame conversion is not specified; silently inserting one would change the published formulation. e05e08
Figure 4 shows main-training average-length curves only, excluding buffer collection and residual warm-up. The shaded bands lack a defined statistical meaning. The paper supplies no dedicated unseen-task or controlled perturbation evaluation, and OAR gains are demonstrated only in simulation. e09e13e14e16
7.2 Questions for discussion
- Does guidance retain its advantage when the coordinate frame is varied systematically?
- Does predicted progress outperform an elapsed-time schedule under matched interaction budgets?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs the named GR00T N1.5 initialization, demonstrations, ManiSkill3 success criteria and task-specific SAC configuration. Table 5 caps PlaceSphere at 500 steps and the other tasks at 800. Baseline horizons, iterations and hardware differ; matching demonstration counts alone is insufficient. e09e15e17e18
Missing specifications include progress-label construction, auxiliary-loss weights, chosen feature layers, temporal-jitter bound, exact guidance schedule, time penalty, reward interval n and coordinate conversion. Software/checkpoint versions and residual-stage hardware/runtime are not pinned. Section 4.1's Eagle-2 citation points to a bibliography entry about language-model draft trees, leaving the intended multimodal checkpoint unresolved. e19e20
Keep action-scale warm-up separate from progress-dependent reward decay. Figure 5 is schematic, rising to 1 despite smaller task scales in Table 7; its jump at learning start also differs from the stated linear injection from zero. Section 3.2 describes automatically tuned entropy, while Table 7 lists 0.2 without clarifying whether it is fixed or initial. e16e18e19
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test whether directional reward is invariant to coordinate conventions
Reader-proposed check, not performed: use recorded end-effector positions and orientations with fixed local-frame prediction outputs. Compare the literal addition in Equation (13) against an explicitly documented conversion of predicted translation into the measurement frame. Apply passive rotations to the global coordinate description while preserving the same physical motion. A geometrically consistent reward should remain unchanged. If the literal implementation changes reward sign or alignment under this coordinate change, the missing conversion affects behavior rather than notation alone. Report both implementations and resolve the intended convention before claiming a reproduction. e05e08e19
Check 2: Separate progress information from generic reward decay
Reader-proposed check, not performed: freeze one SPI checkpoint and compare predicted-progress decay, elapsed-time decay, constant guidance and no guidance in PegInsertion and PlaceSphere. Hold sparse reward, time penalty, residual scales, warm-up, seeds and interaction budgets fixed. Since the paper omits the exact schedule, preregister the chosen reconstruction and its sensitivity range. Log all phases, final success, all-episode lengths and successful-episode lengths separately. If progress-based decay consistently improves late-stage success over a matched time schedule, it supports the proposed progress signal; equal performance would weaken that specific explanation even if decaying guidance remains useful. e08e09e11e13e16e18e19
8.3 Reading coverage
Visual audit: All 15 original PDF pages were rendered at 200 DPI and actually inspected, including the title/byline, Equations (1)–(15), Algorithm 1, Figures 1–7, Tables 1–7, references and Appendices A–D. Six final original crops were inspected; narrow tables and Figure 4 were cropped from 400-DPI renders. Architecture branches, frozen/trainable markers and reward feedback were checked against Equations (9)–(15) and Algorithm 1. Table 1's long caption is excluded from its crop; its essential †/‡ protocol definitions are retained in the reading guide and supported by the inspected full page. The Figure 2 caption/reward-equation discrepancy, local-frame ambiguity, Table 2 zero-success aggregation issue, real-world 70%/0.71 discrepancy, schematic Figure 5 limitations and undefined Figure 4 bands remain explicit. Figures 6–7 were read as selected task images, not continuous video evidence. No external linked material or code was inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; 1. Introduction
- 2. Related Works
- 3. Preliminary: 3.1 Basic Robot Policy; 3.2 Soft Actor-Critic
- 4. Self-Correcting VLA: 4.1 Sparse World Imagination; 4.2 Online Action Refinement; Algorithm 1
- 5. Experiments: 5.1 Simulation Setup and Baselines; 5.2 Ablation Study; 5.3 Real World Experiments
- 6. Conclusion; Impact Statement; References
- Appendix A: Task Setup and Evaluation Details (A.1–A.2)
- Appendix B: Baseline Details and Settings (B.1–B.4)
- Appendix C: Experiment Details
- Appendix D: Implementation Details; final simulation and real-world visualizations
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All five supplied text chunks and all 15 PDF pages were read; the extraction's image limitation was addressed by inspecting the original PDF pages and six final crops.
- Identity: the title page shows arXiv:2602.21633v1 [cs.RO], 25 February 2026, and Preprint Version. Its title and seven authors match the supplied catalog. No different revision was supplied or compared.
- The paper links code, but code, external references, datasets and separate supplements were not inspected; no experiments were reproduced.
- No dedicated limitations section is present; the limitations below identify reader assessments and unresolved reporting details.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/byline, affiliation footnote and arXiv margin stamp
Exact title and seven catalog authors; Tongji University, University of Technology Sydney, University of Electronic Science and Technology of China, Advanced Institute of Big Data; v1, 25 Feb 2026.
Go to primary source ↓e02PDF pp. 1–2, Abstract and Section 1
Motivates explicit sparse predictions and self-correction through progress-dependent residual reinforcement learning.
Go to primary source ↓e03PDF p. 3, Section 3.1, Eqs. (1)–(3)
Conditional flow matching learns noise-to-action velocity with MSE and generates actions through numerical ODE integration.
Go to primary source ↓e04PDF p. 4, Section 4.1, Eqs. (6)–(7); p. 5, Figure 2 and caption
Names SigLIP-2/Eagle-2 conditioning; shared DiT has intermediate prediction heads and final action head. Figure 2 marks a frozen SPI base, trainable residual and environment reward; caption claims no external supervision.
Go to primary source ↓e05PDF p. 4, Query Sequence Construction, Eqs. (7)–(9)
State, progress, relative-state and 16 action queries; future target at t+H+δ; seven components encode local translation, relative Euler rotation and gripper change.
Go to primary source ↓e06PDF p. 4, Joint Optimization, Eqs. (9)–(10)
Intermediate DiT features feed separate MLP heads; total objective adds weighted progress and state-change MSE to flow matching.
Go to primary source ↓e07PDF p. 5, Section 4.2, Eqs. (11)–(12); p. 6, Algorithm 1
Residual Gaussian MLP observes state and predictions; base is frozen; scaled actions are summed, executed and stored for SAC updates.
Go to primary source ↓e08PDF p. 5, Eq. (13); p. 6, Eqs. (14)–(15), Algorithm 1 and Dynamic Weight Scheduling
Adds predicted translation to current position; cosine-style alignment with observed n-step displacement shapes reward, with n<H. Progress-decayed guidance is added to sparse environment reward minus a time penalty.
Go to primary source ↓e09PDF p. 6, Section 5.1; p. 11, Appendix A.1–A.2 and Table 5
Four ManiSkill3 tasks use 100 demonstrations and 50 evaluation episodes each. Official reward criteria are referenced; PlaceSphere cap is 500 steps, others 800.
Go to primary source ↓e10PDF p. 7, Table 1, all rows and caption
SPI+OAR task successes 1.00/1.00/0.88/0.56 average 0.86; SPI 0.82; GR00T 0.72; specialist DP† 0.77; multitask DP‡ 0.36. Caption defines † specialists and ‡ multitask training.
Go to primary source ↓e11PDF p. 7, Table 2 and following completion-length paragraph
Successful-episode length: full method 158/110/189/173, reported average 157; SPI 187, DP† 172, GR00T 195, π0 276. OAR average reduction is printed 16%. DP‡ PegInsertion cell is 800.
Go to primary source ↓e12PDF p. 7, Section 5.2, Table 3 and component-ablation paragraphs
Average success is 0.78 without state, 0.80 without progress, 0.72 without both, 0.82 with SPI. Without-state LiftPegUpright is 0.84 versus SPI 0.82; no table uncertainty is reported.
Go to primary source ↓e13PDF p. 8, Figure 4, caption and reward/scheduling ablation paragraphs; p. 13, Visualization Rationale
Average-length curves compare full SC-VLA, fixed guidance weight and no imagination reward. Caption excludes initialization phases; band meaning is unspecified. PegInsertion benefits from guidance and PlaceSphere fixed weighting degrades late.
Go to primary source ↓e14PDF p. 8, Section 5.3, Figure 3 and Table 4
ARX5 wrist/third-person setup uses SPI only, 60 demonstrations and 20 trials per task. Task success 0.85/0.60/1.00/0.40; table average 0.71 versus GR00T 0.57 and DP‡ 0.28; adjacent prose instead says 70%.
Go to primary source ↓e15PDF p. 12, Appendix B.1–B.4
DP uses prediction/execution horizons 16/8, ACT 50/20; both train 200,000 iterations on one RTX 5090. π0 and GR00T train 50,000 iterations on RTX PRO 6000 and L40 respectively.
Go to primary source ↓e16PDF pp. 12–13, Appendix C, stages 1–3, Figure 5 and Visualization Rationale
Zero-residual buffer collection precedes linear residual injection and main training. Figure 4 omits initialization. Figure 5 is explicitly schematic, with a jump at learning start and a plateau at 1.
Go to primary source ↓e17PDF p. 13, Appendix D, Stage I/II and Table 6
Base training: GR00T N1.5 DiT, AdamW, one NVIDIA L40, 50,000 steps, batch 32, learning rate 0.0001, seed 42. Base is fully frozen for residual training.
Go to primary source ↓e18PDF p. 14, Appendix D and Table 7, shared and task-specific columns
Seed 0; guidance weight 0.6; entropy coefficient 0.2. Task budgets 500k/500k/600k/3M. Residual train/eval scales 0.01/0.005 except PegInsertion 0.1/0.03; warm-up parameters vary by task.
Go to primary source ↓e19PDF pp. 4–6, Sections 3.2 and 4.1–4.2; pp. 13–14, Appendix D, Tables 6–7
Provides symbolic losses, jitter and reward schedule but no exact progress labels, layer choices, loss weights, η, c, n or frame conversion. Implementation tables do not pin software/checkpoints or residual hardware/runtime. SAC preliminaries call α automatically tuned; Table 7 lists 0.2.
Go to primary source ↓e20PDF p. 4, Conditional Information Processing; p. 10, References, Li, Y. et al. (2024)
Multimodal backbone is called Eagle-2 and cites Li et al. (2024); that bibliography entry is 'Eagle-2: Faster inference of language models with dynamic draft trees.'
Go to primary source ↓8.5 Primary sources
Self-Correcting VLA: Online Action Refinement via Sparse World Imagination ↗
PDF · 8,523 extracted words
Source fingerprint
49d491f99cd63466c5b7e08ef037496a294c4f634348964280e97b9d457f4bed