Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents
1. Paper overview
In one sentence: Harness VLA improves simulated manipulation by combining memory-guided staging and recovery with a frozen contact policy, at the cost of reference-task exploration and a larger execution system. problemexecution-loopbootstrap-memorymemory-detailpro-resultsoverview-figure
| At a glance | What to know |
|---|---|
| Research problem | Source description A policy can grasp an object reliably yet pursue the wrong object or destination after an instruction or layout changes. Conversely, an agent that reasons correctly can fail at delicate contact. The paper asks whether explicit grounding, staging and recovery can extend existing contact skills without changing their weights during deployment. problem |
| Core mechanism | Source description A fixed primitive interface exposes VLA_ACT alongside analytic control, allowing a planner to select and retry contact operations without generating new skills. primitive-contractexecution-loop |
| A key reported result | LIBERO-Pro perturbed manipulation: CC: 82.4%; Codex: 72.1%. Benchmark task success (%). Few-shot memory; eight instruction-redirection/position-swap cells, 100 trials each, 800 total. Same frozen πRLinf: 50.0%. The authors advertise 38.6 percentage points over RATS's 43.8%. The matched-backend improvement is 32.4 points for CC, calculated from rounded reported values. The RATS headline compares eight-cell and six-cell averages, since RATS omits both LIBERO-10 cells. pro-resultslibero-protocol |
| Reading caution | Reader analysis All reported harness evaluations are simulated. Deployment on physical robots is unestablished. The principal tables and invocation curves provide no confidence intervals, and the memory comparison removes both memories together. execution-looplibero-protocolcasa-protocoltwin-protocolstandard-resultspro-resultscasa-resultsmemory-comparisontwin-resultsinvocation-diagnostic |
Core contributions
- Source description
A fixed primitive interface exposes VLA_ACT alongside analytic control, allowing a planner to select and retry contact operations without generating new skills. primitive-contractexecution-loop
- Source description
Two memories separate reusable task structure from cross-task success and failure rules; current observations supply geometry. bootstrap-memorymemory-detail
Figure 1, left system panel. A planner selects fixed operations while memory records how those operations can be composed. Original paper, p. 1 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the left: task language, RGB/depth observations and robot state enter the planner. Above it, the library contains a learned VLA_ACT branch and analytic control operations. The lower action box represents the selected primitive, not raw actions generated by the planner. Follow the blue memory arrow toward the planner for retrieval; the gold arrows and few-shot connection depict experience returning toward memory during exploration. Section 2.2 and Appendix A clarify that the task store includes both procedural commands and a semantic summary, whereas global memory contains reusable operating rules. Actual simulator execution produces the next observation before another command is selected. overview-figureexecution-loopbootstrap-memoryprimitive-contractmemory-detailcasa-results
What it supports. The architectural intervention changes when a frozen VLA acts and what surrounds each call. A grasp or fixture interaction can be delegated to VLA_ACT while analytic motion positions the robot and carries the task onward. The interface permits the planner to diagnose and retry a local failure without changing the policy's weights during evaluation.
Where the evidence stops. ROTATE is a schematic label; Table 1 distinguishes wrist and pitch operations. The uncropped RoboCasa summary on this page shows 55.4%, inconsistent with Table 4's 57.1% for Codex and 48.6% for CC. This edition uses the tabulated results.
2. Motivation
2.1 The problem and the proposed response
A policy can grasp an object reliably yet pursue the wrong object or destination after an instruction or layout changes. Conversely, an agent that reasons correctly can fail at delicate contact. The paper asks whether explicit grounding, staging and recovery can extend existing contact skills without changing their weights during deployment. problem
2.2 What this reading follows
A robot may know how to grasp a bowl yet still move it to the wrong place after the scene changes. Harness VLA separates those problems: a planner identifies the current target, analytic controllers arrange the approach and transport, and a frozen VLA handles local contact. Its memories preserve successful command structure and failure knowledge while requiring fresh geometric grounding. This reading follows that information flow, then tests the interpretation against the original result tables and invocation diagnostic. The main qualifications are the evaluation's memory privileges, unequal coverage in an advertised baseline comparison, and an unreconciled summary-figure number. problemexecution-loopbootstrap-memorymemory-detailpro-resultsoverview-figure
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | VLA |
| Architecture | Dual-system |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded hierarchical/dual-system VLA classification fits a separate semantic planner orchestrating a frozen action policy and analytic controllers. One shared primitive interface does not make them one learned model. The harness does not learn future observations or recover actions through inverse dynamics, supporting the recorded not-applicable prediction paradigm and quadrant. architecture-assessmentexecution-loopprimitive-contractbackends
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Turn a task instruction into a local contact attempt
The source's black-bowl example illustrates the separation of decisions. The planner first identifies the bowl and destination from current perception, rather than treating the reference scene as current state. RGB provides identity; corresponding world-map samples provide metric targets. Analytic movement can arrange a useful pre-contact pose. VLA_ACT then supplies the learned interaction, conditioned on language and live observations. Its early-return predicate only hands control back to the planner; it does not certify completion of the whole task. After checking whether contact is stable, the planner can issue transport and release commands, or diagnose an empty grasp and re-stage the next attempt. The learned output is an action chunk executed by the worker, while the high-level output remains a structured primitive call. execution-loopprimitive-contractmemory-detailperception
5.2 Build memory on a reference seed and ground it again
During bootstrapping, reset and a larger budget let the planner discover a workable primitive sequence. The stored result has two complementary parts: a JSONL procedure recording what was executed and a semantic summary explaining strategy, recovery and failure. Global Memory adds cross-task operating rules, such as checking that an object actually follows the closed gripper. This is adaptation of the planner's available context, not an update to the frozen VLA weights. Although Section 2.2 describes replacing concrete coordinates with symbolic queries, the appendix shows literal reference coordinates and explicitly prohibits replaying them. The common requirement is therefore procedural reuse with fresh geometry. At evaluation, reset is disabled and the reference seed is excluded from scoring; current observations still determine command arguments and recovery decisions. bootstrap-memorymemory-detailperceptionlibero-protocol
5.3 Separate mechanism evidence from aggregate improvement
Reader analysis: keeping the same VLA backend makes the direct-policy comparison informative, but it does not identify which harness component produced the gain. The Goal experiment helps by withdrawing memory: position-swap success falls much more than instruction-redirection success. Because both memory sources are removed, it cannot assign that effect to task traces alone. Likewise, the invocation curve shows that permitting further local attempts helps, but additional calls also permit more physical execution. Usage counts describe the resulting division of work, and terminal-primitive attribution describes when completion occurs; neither measures what would happen without analytic control. A strong causal reading consequently needs independent memory controls and matched-budget retry comparisons. RoboTwin adds another boundary: its zero-shot evaluation transfers a clean-setting trace, so it is not the same information regime as memory-free Goal evaluation. pro-resultsmemory-comparisoninvocation-diagnosticusage-statisticscompletion-attributiontwin-protocol
5.4 Training and inference
During training
There is no joint harness/VLA optimization in these experiments. LIBERO uses the released LIBERO-130 π0.5-SFT checkpoint; RoboCasa uses released RLDX-1. LingBot-VLA is first post-trained on RoboTwin trajectories, then frozen for both baseline and harness evaluation. Thus, no deployment fine-tuning does not mean no benchmark-specific policy training. backendsposttrainingsource-limitations
The harness acquires task knowledge through reference-seed interaction and memory refinement rather than gradient updates to the VLA. Successful shorter traces can replace earlier traces while failure observations remain useful. bootstrap-memorymemory-detail
During inference
Evaluation disables reset and uses a shorter budget. Each primitive returns fresh perceptual and diagnostic evidence before the next decision. The benchmark predicate, rather than apparent visual proximity or an empty closed gripper, determines completion. RoboCasa may continue a capped but still-progressing VLA call. bootstrap-memorymemory-detaillibero-protocolprompt-policy
5.5 Implementation flow
- Ground the current target
The planner identifies task entities in RGB and samples corresponding world-map coordinates, typically aggregating stable pixels with a median. It receives object names and proprioception but no oracle object coordinates. Localization is refreshed after scene, robot, camera or grasp changes. execution-loopperception
- Assign control to a primitive
Analytic controllers handle movement, pose adjustment and gripper operations; RoboCasa adds base navigation, and RoboTwin binds commands to arms. Availability differs across embodiments despite shared names. The planner emits JSON arguments, while a simulator worker performs low-level execution. primitive-contractembodimentsmemory-detail
- Stage, invoke and diagnose
VLA_ACT runs the benchmark's frozen policy with a prompt and early-return predicate. It emits action chunks until that predicate or a chunk budget ends the call. The planner then inspects the outcome, continues analytically, or re-stages a failed contact attempt. A local return condition does not establish task success. primitive-contractinvocation-diagnosticlibero-protocol
- Reuse procedure with new geometry
Bootstrapping on a reference seed allows reset and repeated exploration. Task Specific Memory stores a command trace plus a strategy/failure summary; Global Memory stores reusable rules. Appendix examples retain concrete reference coordinates but explicitly require re-grounding at deployment, clarifying the main text's symbolic-parameterization description. bootstrap-memorymemory-detailperception
6. Experiments & results
Harness VLA makes a frozen visuomotor policy a callable contact primitive inside a memory-guided planner. RGB-D grounding and analytic movement connect local VLA attempts; stored strategies guide recovery across changed scenes. Its strongest evidence is improved simulated task success with the same frozen backend. Interpretation depends on memory access, unequal external-baseline coverage, and a figure/table discrepancy.
6.1 Read the original evidence
Table 3. The matched frozen-backend comparison is the clearest aggregate test of the harness on LIBERO-Pro. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each family as a pair: T redirects the instruction, while S swaps object positions. Every harness cell contains ten tasks evaluated on ten held-out seeds, after reference-seed memory construction. The last three rows compare direct πRLinf with the two planners using that same frozen policy inside VLA_ACT. The slash symbols in Cap-X and RATS are missing results, not zero successes. Their Overall columns therefore summarize only six cells; the harness summarizes all eight. The CC and Codex rows also show that planner choice changes the distribution of strengths: Codex leads the Object columns, while CC leads most other cells. pro-resultslibero-protocolbackends
What it supports. CC reaches 82.4% overall compared with 50.0% for the direct checkpoint, a 32.4-percentage-point difference calculated from the reported rounded values. Improvements extend across both perturbation types, supporting the value of the complete planner, memory and primitive combination when the VLA backend is held fixed.
Where the evidence stops. The headline 38.6-point gap to RATS subtracts averages with different cell coverage. Treat it as the authors' summary comparison, not a matched eight-cell estimate. No uncertainty intervals or equal-compute comparison are supplied in this table.
Table 4. Kitchen-task gains are large, but unseen composite templates remain a weak point. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across a row before comparing Overall. Atomic-Seen contains eighteen tasks; each composite group contains sixteen. Seen and unseen refer to templates in pretraining, while all three groups still receive reference-seed harness bootstrapping. Atomic tasks use ten held-out seeds and composite tasks use five. Reader calculation: the displayed overall rates align with weighting these split rates by task counts, rather than by the unequal rollout counts. Compare the harness primarily with its direct RLDX-1 backend. Other rows come from prior reports and provide context, but the supplied caption does not establish that every external result shares the harness's complete protocol. casa-resultscasa-protocoloverview-figure
What it supports. Codex improves the reported overall score from RLDX-1's 30.0% to 57.1%, with strong Atomic-Seen and Composite-Seen results. The 13.8% Composite-Unseen score shows that the same orchestration does not reliably solve novel composite templates. CC is slightly higher on that split at 15.0%, despite lower overall performance.
Where the evidence stops. The 27.1-point improvement specifically uses RLDX-1; WorldDreamer's imported overall is 35.3%. Figure 1's 55.4% summary conflicts with this table, and the source does not explain that difference. The table supplies no uncertainty intervals.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Standard LIBERO manipulation Few-shot memory; four suites, 40 tasks × ten held-out seeds; seed 0 excluded. | CC: 96.0% (384/400). Benchmark task success (%) | Direct πRLinf: 95.3%; cited AtomVLA: 97.0%. Competitive standard-suite performance; this table does not establish the highest overall score or a statistically significant improvement. standard-resultslibero-protocol |
| LIBERO-Pro perturbed manipulation Few-shot memory; eight instruction-redirection/position-swap cells, 100 trials each, 800 total. | CC: 82.4%; Codex: 72.1%. Benchmark task success (%) | Same frozen πRLinf: 50.0%. The authors advertise 38.6 percentage points over RATS's 43.8%. The matched-backend improvement is 32.4 points for CC, calculated from rounded reported values. The RATS headline compares eight-cell and six-cell averages, since RATS omits both LIBERO-10 cells. pro-resultslibero-protocol |
| RoboCasa365 target50 kitchen manipulation Few-shot memory; 18 atomic-seen and 16 tasks in each composite split; ten/five/five held-out seeds, 340 rollouts. | Codex: 92.0/61.0/13.8% by split, 57.1% overall; CC: 48.6% overall. Reported split and overall task success (%) | Direct RLDX-1: 60.0/21.3/5.0%, 30.0% overall; Codex gains 27.1 points. Composite-unseen remains difficult despite per-task bootstrapping. Reader calculation: the overall is consistent with weighting split rates by 18/16/16 tasks, not 180/80/80 rollouts. The imported WorldDreamer row is 35.3%, so 27.1 points specifically refers to RLDX-1. casa-resultscasa-protocol |
| LIBERO-Pro Goal without target-setting memory CC with neither target-setting Task Specific Memory nor corresponding Global Memory; ten tasks × ten seeds per perturbation. | Position-swap: 31.0%; instruction-redirection: 79.0%. Benchmark task success (%) | Few-shot CC: 87.0% in both settings; reported Cap-X: 25.6% and 16.8%. Removing memory hurts spatial transfer much more here. This joint removal does not isolate either memory module's contribution. memory-comparisonpro-results |
| RoboTwin clean-to-randomized bimanual transfer 50 tasks × five expert-verified randomized seeds; transfer from one expert-verified clean seed per task. | CC: 58.4%; Codex: 58.0%. Benchmark task success (%) | Same frozen post-trained LingBot-VLA: 50.4%; CC gain: 8.0 percentage points. Zero-shot refers to absence of randomized-setting adaptation. A clean-setting task trace is available, unlike the memory-free LIBERO-Pro experiment. twin-resultstwin-protocol |
| LIBERO-Pro VLA invocation-budget diagnostic CC mechanism analysis; cap the permitted number of VLA primitive calls. | 52% at at most one call; 80% at at most six calls. Cumulative task success (%) shown in Figure 4 | Plot labels: frozen-policy baseline 50%; full harness 82%. Repeated calls support higher success under the plotted protocol. Their effect combines contact opportunities, staging and extra execution; the plot alone does not isolate recovery quality at equal compute. invocation-diagnostic |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 4. More permitted contact calls improve success, with different saturation behavior across benchmarks. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The horizontal axis is the maximum permitted number k of VLA primitive invocations, and the vertical axis is cumulative task success. Each invocation can execute multiple low-level action chunks, so k is neither a timestep count nor a duration. Read the orange labeled points against the blue frozen-policy reference and gray full-harness reference within each panel. The mechanism analyses use CC unless stated otherwise. Those reference labels are rounded: for example, the table's 82.4% LIBERO-Pro result appears as 82% here. The three panels use different axis ranges, making their apparent slopes unsuitable for direct visual comparison. invocation-diagnosticprimitive-contractpro-resultscasa-resultstwin-results
What it supports. LIBERO-Pro rises from 52% with at most one call to 80% with six, approaching its plotted 82% full-harness reference. RoboCasa is still at 39% with ten calls against a 48% reference, so its displayed curve leaves a substantial gap. The graphic supports useful additional invocation opportunities, not uniform rapid saturation across tasks.
Where the evidence stops. The cap changes the opportunity to act as well as to recover. Without controls for total action steps, staging and contact attempts, these curves cannot isolate reasoning quality from extra execution. No error bars are shown.
Table 19. The share of VLA calls varies substantially with the benchmark's contact demands. Original paper, p. 39 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each cell gives an aggregate count followed by its percentage of that environment's manipulation calls. Read down a column to divide the recorded activity into analytic and VLA-backed operations. Analytic includes both goal-reaching composite controllers and atomic set-point commands. Appendix F excludes rendering, reset, notes and no-ops from these totals. It describes the LIBERO data as Pro-family runs even though the column header uses the shorter LIBERO name. These are CC usage statistics, not a fresh success-rate comparison. Keep them separate from Figure 6, which instead classifies successful episodes by the primitive executing when the terminal predicate fires. usage-statisticscompletion-attributionprimitive-contract
What it supports. VLA_ACT accounts for 15.8% of LIBERO calls, 35.3% of RoboCasa calls and 47.4% of RoboTwin calls. The method therefore uses different control mixtures across embodiments. Even in RoboTwin, analytic primitives make slightly more than half of the recorded calls, while the learned policy supplies the contact-oriented component.
Where the evidence stops. Calls have unequal durations and chunk counts. These proportions cannot establish a corresponding fraction of compute, motion time, or causal contribution to success. Successful use of both families also does not substitute for removing one in a controlled ablation.
7. Analysis & limitations
7.1 What the evidence leaves open
All reported harness evaluations are simulated. Deployment on physical robots is unestablished. The principal tables and invocation curves provide no confidence intervals, and the memory comparison removes both memories together. execution-looplibero-protocolcasa-protocoltwin-protocolstandard-resultspro-resultscasa-resultsmemory-comparisontwin-resultsinvocation-diagnostic
The authors cite an open planner–VLA feedback loop, absent joint fine-tuning, and limited captioning for cluttered reasoning. Their open-loop wording remains unresolved alongside the explicit observation-after-each-primitive protocol; it should not be read as absence of environment feedback. source-limitationsmemory-detail
Figure 1's RoboCasa summary shows 55.4%, whereas Table 4 gives Codex 57.1% and CC 48.6%, without reconciliation. This report retains the tabulated results. Figure 4 uses rounded reference labels. Primitive-call shares and final-primitive attribution establish neither execution-time savings nor causal necessity. overview-figurecasa-resultsinvocation-diagnosticusage-statisticscompletion-attribution
7.2 Questions for discussion
- Which memory component explains the Goal-S gap when interaction budgets and task order are controlled? [memory-comparison]
- How much of the retry gain survives equal low-level execution time and matched contact attempts? [invocation-diagnostic]
8. Reproducibility audit
8.1 Requirements and known gaps
A faithful attempt needs the exact frozen backends, benchmark-specific controllers, perception/world-map construction, prompts, reference traces, memory snapshots and held-out seed lists. Fix task order and memory contents before evaluation; enforce no reset and predicate-based scoring. backendsembodimentsperceptionbootstrap-memorylibero-protocolcasa-protocoltwin-protocolprompt-policy
For LingBot post-training, Table 14 specifies AdamW, learning rates 1e-4 and 1e-6 for the vision encoder, L1 Flow Matching, chunk size 50, ten flow steps, batch 256, 224×224 three-camera input, mixed bf16/fp32 and FSDP2. GPU model/count, training duration, fully numerical rollout budgets and immutable planner build identifiers remain unspecified; Figure 1's model names do not fill those gaps. posttrainingbootstrap-memoryembodimentsprompt-policyoverview-figure
Proposed checks should independently remove each memory and compare informed re-staging against retries with equal action and call budgets. Success gains that disappear under these controls would weaken the respective mechanism claims. memory-comparisoninvocation-diagnosticmemory-detail
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Independently remove task memory and global rules
Reader-proposed experiment, not performed: run a 2×2 Task Specific Memory / Global Memory availability comparison on LIBERO-Pro Goal-T and Goal-S, using the same frozen πRLinf checkpoint, pinned planner, ten tasks and ten held-out seeds per setting. Construct seed-0 material with an identical exploration budget, then freeze every memory snapshot before testing; prevent held-out experience from entering any condition. Hold prompts, reset prohibition and execution budgets fixed. Report paired task-success differences, wrong-target events and failed-contact retries. A selectively large Goal-S drop when task traces are removed would support their spatial-organization role; an unchanged result would weaken that explanation. The factorial interaction would reveal whether task traces depend on global recovery rules. memory-comparisonlibero-protocolbootstrap-memorymemory-detail
Check 2: Test informed re-staging against equal-budget retry
Reader-proposed experiment, not performed: on fixed LIBERO-Pro held-out episodes, compare normal recovery with a control that retries the VLA without planner-selected analytic re-staging after failed contact. Keep the frozen policy, target binding, available observations, memory, maximum VLA calls and total low-level execution budget identical; count analytic movements against that budget. Sweep small call caps and log contact success after each retry, final predicate success, VLA action steps and wall-clock cost. If informed staging improves recovery at matched resources, that supports the proposed local-state mechanism. If it only wins with more action time, Figure 4's gain should be attributed partly to execution budget rather than staging quality. invocation-diagnosticprimitive-contractmemory-detaillibero-protocol
8.3 Reading coverage
Visual audit: Visually inspected the title/author block, all body and appendix pages (1–16 and 23–39), Figures 1–8, Tables 1–19, interface notation, evaluation protocols, post-training configuration and prompt specifications. All five final original crops were viewed after extraction. The architecture crop selects the left system panel; the full page's inconsistent RoboCasa summary is disclosed alongside Table 4. Figure 4 retains its original rounded reference labels. Reference pages 17–22 were read in the complete text pass but not rendered. Separate supplements, code and external references remain uninspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title and Abstract
- 1 Introduction
- 2 The Harness VLA Framework (2.1–2.3)
- 3 Experiments (3.1–3.3)
- 4 Related Work
- 5 Conclusion and Limitations
- References
- A File-Mediated REPL Protocol
- B Primitive Vocabulary and Environment-Specific Extensions
- C Details about the Evaluation Benchmark (C.1–C.5)
- D VLA Model Instantiations (D.1–D.4)
- E Agent Prompt Specification (E.1–E.6, including complete prompt skeleton)
- F Primitive Usage Statistics
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Identity/edition note: the supplied PDF is arXiv:2607.08448v4, dated 2 September 2026, with the exact catalog title. Compared with the catalog author string, it adds Jiaji Rao and moves Yi Nie earlier in the ordering. Earlier revisions were not supplied or compared; the catalog submission date is not this revision's date.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This acquisition limitation was addressed by visually inspecting all body and appendix pages, all eight figures and all nineteen tables.
- Separate supplemental material availability has not been fully verified.
- The complete supplied text, including references and appendices, was read. Reference pages 17–22 were read as text; their page images were not inspected.
- Code, project website and external references were not accessed. No implementation, checkpoint installation or experiment was reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identity-v4PDF p. 1, title, author/affiliation block and arXiv margin stamp
The exact catalog title appears on arXiv:2607.08448v4, dated 2 September 2026. The observed list contains all sixteen catalog authors, adds Jiaji Rao, and places Yi Nie sixth. Seven affiliations are printed; Yixian Zhang and Huanming Zhang share equal contribution, and Wenbo Ding and Chao Yu are corresponding authors.
Go to primary source ↓problemPDF pp. 2–3, Abstract, Section 1 and Figure 2/caption
The paper motivates separating semantic retargeting, scene changes and long-horizon organization from locally learned contact control. Figure 2 is a conceptual distribution schematic, not a measured state-space map.
Go to primary source ↓execution-loopPDF p. 4, Sections 2 and 2.1; p. 5, Section 2.3 opening
The planner consumes RGB, aligned metric depth, robot state, language and memory, emits a JSON primitive invocation, and receives refreshed observations after simulator execution. Primitive selection and low-level action generation are separate operations.
Go to primary source ↓bootstrap-memoryPDF p. 5, Section 2.2, Exploratory Bootstrapping Phase and Deployment Evaluation Phase
A single reference task instance allows RESET and a generous wall-clock budget. Successful traces and generalized rules populate two memories. Evaluation disables reset, shortens the budget, and grounds the stored structure in live observations; exact budget values are not supplied here.
Go to primary source ↓primitive-contractPDF pp. 6–7, Table 1 and Section 2.3, VLA-backed contact primitive
Six shared analytic primitive names accompany VLA_ACT, with two mobile-base names for RoboCasa. The frozen policy emits action chunks until a planner-selected early-return predicate or chunk budget ends the call.
Go to primary source ↓memory-detailPDF pp. 23–24, Appendix A, Table 7 and Task Specific Memory / Global Memory / Iterative memory construction
The file-mediated worker owns simulator state. Memory pairs a JSONL command sequence with a semantic JSON summary. Literal reference coordinates are re-grounded; failed attempts and recoverable failures inform diagnosis, including empty grasps and false visual success.
Go to primary source ↓perceptionPDF p. 33, Appendix E.2; p. 34, Appendices E.2–E.3
The planner receives object names and proprioception but cannot access oracle object coordinates. It selects RGB pixels, reads corresponding world-map coordinates, aggregates stable samples, and re-localizes after changes. Task memory transfers procedural structure, not reference geometry.
Go to primary source ↓embodimentsPDF pp. 25–26, Appendix B and Table 8
Primitive availability varies by embodiment: RoboCasa adds navigation/base motion, while RoboTwin binds calls to arms without adding a handover primitive. Example tolerances are shown, but the complete numerical tolerance and stop-predicate configuration remains benchmark-specific.
Go to primary source ↓backendsPDF p. 7, Section 3.1; pp. 29–31, Appendices D.1–D.4
LIBERO-family experiments use the frozen RLinf pi05_libero130_fullshot π0.5-SFT checkpoint trained on LIBERO-130. RoboCasa uses the released RLDX-1 checkpoint without additional adaptation. RoboTwin uses a LingBot-VLA checkpoint post-trained on RoboTwin trajectories and then frozen for both direct and harness evaluations.
Go to primary source ↓posttrainingPDF pp. 31–32, Appendix D.3 and Table 14
LingBot-VLA post-training uses AdamW, learning rate 1e-4, vision-encoder learning rate 1e-6, zero weight decay, L1 Flow Matching, chunk size 50, maximum sequence length 2048, ten flow steps, maximum action/state dimensions 75, global batch 256, 224×224 images, top/left-wrist/right-wrist views, mixed bf16/fp32 precision and FSDP2. Training duration and GPU model/count are not listed.
Go to primary source ↓standard-resultsPDF p. 7, Section 3.2, Standard LIBERO; p. 8, Table 2, Overall column and caption
Harness VLA (CC) reports 96.0% (384/400), direct πRLinf 95.3%, and the cited AtomVLA row 97.0%. The authors evaluate the harness and direct checkpoint on 100 trials per suite.
Go to primary source ↓pro-resultsPDF p. 8, Table 3, all columns and caption; p. 9, opening paragraph
LIBERO-Pro overall success is 82.4% for CC, 72.1% for Codex, and 50.0% for direct πRLinf. CC Goal-T and Goal-S are both 87.0%. RATS reports 43.8% averaged over six cells, with both LIBERO-10 cells unavailable; Harness covers all eight. The text advertises a 38.6-point gap to RATS.
Go to primary source ↓casa-resultsPDF p. 9, Section 3.2, RoboCasa365 and Table 4, all rows/columns and caption
Codex reports 92.0%, 61.0%, 13.8% and 57.1% on Atomic-Seen, Composite-Seen, Composite-Unseen and Overall. CC reports 79.4%, 47.5%, 15.0% and 48.6%. Direct RLDX-1 reports 60.0%, 21.3%, 5.0% and 30.0%; the imported WorldDreamer row reports 35.3% overall. The text's 27.1-point comparison uses RLDX-1.
Go to primary source ↓memory-comparisonPDF p. 9, Zero-shot evaluation paragraph; p. 10, Table 5, Average column and caption; p. 8, Table 3, CC Goal cells
Memory-free CC reports 31.0% for position swaps and 79.0% for instruction redirection on LIBERO-Pro Goal, using ten seeds per task. Cap-X's reported averages are 25.6% and 16.8%. The text excludes target-setting task and global memory; the corresponding few-shot CC averages are 87.0% in each setting.
Go to primary source ↓twin-resultsPDF pp. 9–10, RoboTwin C2R discussion and Table 6/caption
On 50 tasks with five randomized seeds each, direct post-trained LingBot-VLA reports 50.4%; Harness reports 58.0% with Codex and 58.4% with CC. External VLA rows are contextual comparisons, distinct from the matched frozen backend.
Go to primary source ↓invocation-diagnosticPDF p. 10, Section 3.3 setup; p. 11, Key Finding 2 and Figure 4/caption; p. 12, Key Finding 2 continuation and Figure 5/caption
Mechanism analyses use CC unless stated otherwise. Figure 4 labels LIBERO-Pro success as 52% at at most one VLA call and 80% at six, versus a 50% direct-policy line and an 82% full-harness line. RoboCasa reaches 39% at ten calls versus a 48% full-harness line; RoboTwin reaches 58% at eight. The authors interpret the curves and illustrative retries as evidence for sparse staged invocation.
Go to primary source ↓completion-attributionPDF p. 13, Figure 6/caption and Key Finding 3
The chart classifies successful rollouts by the primitive executing when the final benchmark predicate fires. It does not count every primitive's contribution or measure success after removing either primitive family.
Go to primary source ↓source-limitationsPDF p. 16, Section 5, Limitations and future work
The authors identify an open feedback loop between planner and VLA, absent joint fine-tuning from rewards/preferences, and missing fine-grained image captioning for cluttered long-horizon reasoning. The phrase open feedback loop is not further operationally defined here.
Go to primary source ↓libero-protocolPDF pp. 26–27, Appendices C, C.1–C.2 and Tables 9–10
Success requires the benchmark predicate before the rollout horizon/budget, not a primitive's local post-condition. Standard LIBERO has 40 tasks and 400 evaluation rollouts. LIBERO-Pro has eight cells of ten tasks and ten held-out seeds, totaling 800. Seed 0 constructs memory and is excluded from evaluation. T redirects instructions; S swaps positions.
Go to primary source ↓casa-protocolPDF p. 28, Appendix C.3 and Table 11
The target50 split has 18 atomic-seen, 16 composite-seen and 16 composite-unseen tasks. Held-out seed counts are ten, five and five respectively, totaling 180/80/80 rollouts. Seen/unseen describes pretraining task templates; every split still uses seed-0 harness bootstrapping.
Go to primary source ↓twin-protocolPDF pp. 28–29, Appendices C.4–C.5 and Tables 12–13
RoboTwin transfers a trace from one scripted-expert-verified demo_clean seed to five scripted-expert-verified demo_randomized seeds per task, totaling 250 rollouts. Expert filtering checks feasibility independently of Harness rollouts. No randomized-setting search, fine-tuning or task-level adaptation occurs.
Go to primary source ↓prompt-policyPDF pp. 35–38, Appendices E.4–E.6, Tables 16–17 and compact prompt skeleton
Global Memory provides cross-task guidance rather than an executable trace. Benchmark prompts specify success channels, camera/embodiment context, recovery and reset restrictions. RoboCasa passes full task language to VLA_ACT and may continue a capped but progressing call. The compact skeleton contains configuration slots rather than a complete numerical runtime configuration.
Go to primary source ↓usage-statisticsPDF pp. 38–39, Appendix F, Tables 18–19 and captions
CC manipulation-call counts exclude helpers such as rendering, reset and no-ops. Table 19 gives analytic/VLA counts of 8536/1598 for LIBERO, 881/794 for RoboTwin C2R and 5026/2746 for RoboCasa365. VLA shares are 15.8%, 47.4% and 35.3%. Appendix F describes the LIBERO collection as Pro-family despite the shorter table header.
Go to primary source ↓overview-figurePDF p. 1, Figure 1, system panel, right-hand summary charts and caption; p. 9, Table 4
The system panel shows observation-to-planner, memory retrieval, exploration write-back and primitive-action connections. Its generic ROTATE label is expanded into wrist/pitch primitives in Table 1. The right-hand RoboCasa chart labels Ours as 55.4%, differing from Table 4's Codex 57.1% and CC 48.6%; the source does not reconcile this.
Go to primary source ↓architecture-assessmentPDF p. 4, Sections 2–2.1; p. 15, Sections 4–5; p. 31, Appendix D.4
The paper explicitly describes an asymmetric hierarchical framework: a planner orchestrates a frozen VLA and analytic controllers through a fixed interface. It does not propose learned future-observation prediction or an inverse-dynamics stage for its harness.
Go to primary source ↓8.5 Primary sources
Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents ↗
PDF · 16,007 extracted words
Source fingerprint
717c1717e6cfb6842ef646750f289ac957ed8fa4901376d50997ed1ab916d269