RoboHarness: Memory-Driven Orchestration of Heterogeneous Robot Policies for Long-Horizon Planning
1. Paper overview
In one sentence: RoboHarness improves long-horizon execution by selecting complementary controllers and preparing their handoff states, while depending on accumulated memory and adaptive orchestration. e02e03e06e07e11e13e15
| At a glance | What to know |
|---|---|
| Research problem | Source description A policy can understand a subtask yet fail from the state left by its predecessor. RoboHarness therefore jointly addresses task decomposition, policy assignment and state preparation. Valid assignment requires both an achievable instruction and an in-distribution observation. The source defines zero-shot as unseen long-horizon compositions without task-specific joint training, not absence of prior demonstrations or online adaptation. e02e07e08 |
| Core mechanism | Source description A common callable-skill wrapper and policy card allow heterogeneous controllers to retain native interfaces while exposing capabilities, constraints and execution history to the planner. e03 |
| A key reported result | LIBERO-LoHo long-horizon manipulation: 97.5 / 95.2 Average progress score / success rate (%). Five benchmark tasks; unseen compositions under the paper's adaptive zero-shot protocol. Baseline results are attributed to prior benchmark evaluations. H-WM-π0.5: 84.9 / 64.8; π0.5: 55.3 / 6.4. Success exceeds H-WM-π0.5 by 30.4 percentage points (reader subtraction). Strong full-system completion evidence. Controller diversity, routing, bridging and adaptation vary together relative to these baselines, so this comparison does not isolate bridging. e07e09e11 |
| Reading caution | Source description The library bounds achievable behavior; sparse relevant memory weakens capability estimates and bridges. Sensor degradation harms routing evidence, and unreachable layouts cannot be repaired by orchestration. Navigation, MPC, world-action-model policies and tactile feedback remain future extensions. e12e16 |
Core contributions
- Source description
A common callable-skill wrapper and policy card allow heterogeneous controllers to retain native interfaces while exposing capabilities, constraints and execution history to the planner. e03
Figure 1. The harness coordinates native controllers through assessment, memory and execution feedback. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the four input boxes at the left, then follow the input arrow into Understanding Skills. Its five assessments supply evidence about the current scene and policy compatibility. The orange Memory Skills box stores linked experience and constructs bridges; the gray policy box retains separate VLA, TAMP and RL controllers. The output panel shows a sequence of policy assignments with bridge modules between them. Finally, follow the update/retrieve and improve/feedback arrows around the loop. These are information and adaptation exchanges, not shared neural layers. The caption and Sections 4 and 4.4 explain that the coding agent invokes these modules while controllers retain their native interfaces. e03e04e06e07
What it supports. The design makes two decisions explicit: which controller should handle each subtask, and how to prepare the state it receives. Its modular interfaces support independently developed policies without imposing a common action representation. This architecture evidence places the paper in orchestration and planning rather than a joint future/action prediction quadrant.
Where the evidence stops. The feedback loop includes changes to policy metadata, parameters and orchestration. Section 4.3 enables these evolution mechanisms in simulation and retains validated updates across episodes, so the diagram should not be read as a fixed system throughout evaluation.
2. Motivation
2.1 The problem and the proposed response
A policy can understand a subtask yet fail from the state left by its predecessor. RoboHarness therefore jointly addresses task decomposition, policy assignment and state preparation. Valid assignment requires both an achievable instruction and an in-distribution observation. The source defines zero-shot as unseen long-horizon compositions without task-specific joint training, not absence of prior demonstrations or online adaptation. e02e07e08
2.2 What this reading follows
Opening a cabinet and precisely stacking its contents place different demands on a robot. RoboHarness lets existing controllers divide that work, but treats the transfer of control as a planning problem of its own. A controller that can perform the next subtask may still fail from the pose left by its predecessor. The Memory Bridge retrieves relevant execution states, estimates local progress and moves the robot toward a supported handoff. Read the architecture first, then distinguish the benchmark gains from the component ablation. The physical trials show executed behavior; the reported zero-shot setting still permits prior policy training and updates retained across evaluation episodes. e02e03e06e07e11e13e15
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Foundational work |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
This table preserves the labels recorded at reading time. The current major category is Related resources. View the current classification.
3.1 Evidence-based assessment
Supports the recorded classification
The recorded not-applicable architecture/prediction/quadrant is supported: RoboHarness orchestrates separate policies with memory and planning, without a joint future/action architecture or inverse-dynamics pipeline. Theory and planning is appropriate. Its inclusion under surveys/technical resources is broader than its actual type: this is an empirical method paper. Memory use does not itself establish a world-action model. e03e06e08e16
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Separate task capability from starting-state compatibility
Consider the physical construction sequence. A planner can determine which blocks are needed and how they should be stacked, while a VLA can handle the cabinet interaction needed to expose hidden blocks. Those complementary abilities do not guarantee that either controller can start from the state left by the other. Section 3 formalizes both requirements: the assigned subtask must belong to the controller's achievable instruction space, and its observation must lie in its supported input space. RoboHarness uses policy cards and understanding skills to reason about the first requirement and current compatibility; the Memory Bridge supplies an explicit state-preparation step. Figure 6 makes the division concrete, assigning drawer operations to the VLA and block retrieval/assembly to TAMP. The output is an executed sequence of native skills with feedback, rather than a single policy trained to absorb every capability. e02e03e04e06e15
Figure 2. Retrieved temporal neighborhoods provide the reference states for a handoff. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read from the frozen language and visual encoders toward memory. Although the drawing places their similarity arrows in parallel, Section 4.2.1 specifies text retrieval followed by visual reranking. The central trajectory marks an anchor at t and neighboring states before and after it. Section 4.2.2 labels those neighbors by signed temporal offset, fits a progress estimator and separately limits target states to a neighborhood of retrieved samples. The estimator-to-bridge arrow therefore hides a constrained choice: the target must be motion-feasible, supported by memory and have nonnegative predicted progress. The motion planner then connects the current robot state to that target before control passes onward. e05e06e17e18
What it supports. Memory supplies more than a nearest pose: temporal ordering supplies a direction of progress. The bridge chooses among locally supported, reachable states while penalizing motion cost. In the implementation appendix, the estimator is a pairwise-ranking SVM, while the selected policy remains independently trained and executes its own actions.
Where the evidence stops. The caption uses distribution-confidence language, but the equations distinguish progress score from support membership; neither is a calibrated success probability. The diagram also omits the support, feasibility and nonnegative-score constraints. This explanation follows the explicit formulation on page 6.
5.2 Turn retrieved trajectories into a constrained handoff target
Retrieval begins with the upcoming instruction and current image, selecting semantically relevant nodes and then reranking them visually. A retrieved node is an anchor, not necessarily the final target. Temporal links provide earlier and later robot states, labeled with negative and positive offsets around zero. These labels train the local progress estimator; the appendix identifies pairwise-ranking SVM. Section 4.2.2 then adds a separate nearest-sample support region. A candidate must lie within that region, be reachable by the motion planner and have nonnegative predicted progress. Among eligible states, the objective balances progress against movement cost. Reader interpretation: separating support from progress discourages selecting an attractive extrapolated score in an unfamiliar state. It still depends on meaningful state distances and relevant retrieved experience; the equations do not establish a universal guarantee of downstream task success. e05e06e16e18
5.3 Read the gains as adaptive system evidence
The long-horizon table and custom-task ablation answer different questions. Table 2 asks whether the assembled system completes benchmark compositions better than listed alternatives. Figure 4 asks how much performance changes when major modules are removed on tasks designed to require heterogeneous capabilities. Neither alone separates every possible cause. In particular, Evolution includes TAMP adaptation, code refinement, parameter tuning and metadata updates, with validated changes retained across simulated evaluation episodes. Reader interpretation: the results describe an adaptive execution process, so episode order and accumulated memory are part of the experimental state. The bridge ablation's larger loss in success than progress is consistent with a handoff bottleneck, but a sharper test would hold routing and experience fixed while changing only target selection. That would distinguish the bridge's geometric contribution from improvements in policy choice over time. e06e07e09e11e13
5.4 Training and inference
During training
Simulation combines pi05_libero, trained on LIBERO-Spatial/Goal/Long/Object; GRPO-post-trained OpenVLA-OFT on LIBERO-90; and a predefined TAMP domain. Frozen encoders are SigLIP2-base-patch16-224, DINOv2-base and BGE-large-en-v1.5. The bridge fits a local estimator without joint policy retraining. e08e17e18
All four evolution mechanisms are enabled in simulation: TAMP adaptation through SIMPACT/PDDLLM, orchestration-code refinement, discretized parameter tuning and capability-metadata updates. Persistent failures trigger changes; validated changes persist across evaluation episodes. Thus evaluation is adaptive. Physical drawer control additionally uses 50 demonstrations each for opening and closing. e07e15
During inference
Codex powered by GPT-5.5 serves as planner/router. It repeatedly assesses observations, assigns native controllers, bridges incompatible transitions and uses execution feedback to update plans and memory. The learned score estimates local progress; a motion planner and the selected robot policy execute actions. No future-video rollout or inverse-dynamics action extraction is specified. e03e06e07e09
5.5 Implementation flow
- Assess and route
The coding agent invokes pose-uncertainty, semantic-context, visual-context, input-quality and state–policy compatibility skills. Their structured outputs guide decomposition and routing; the underlying policies perform control. Policy cards summarize interfaces, assumptions, training tasks and outcomes. e03e04
- Retrieve execution context
Memory nodes store instructions, observations, joint configurations, end-effector poses and embeddings. Retrieval first selects instruction matches by cosine similarity, then visually reranks that subset. Temporal links recover states before and after each retrieved anchor. Historical outcome statistics separately inform policy selection. e05
- Build a supported progress model
For the incoming policy, retrieved anchors receive zero progress; neighboring states receive signed temporal-offset labels. A lightweight estimator predicts relative progress. A nearest-memory-state distance threshold separately restricts admissible targets. The appendix names pairwise-ranking SVM as the practical estimator. e06e18
- Prepare and execute the handoff
Select a motion-feasible, memory-supported target with nonnegative predicted progress, trading progress against movement cost. An off-the-shelf motion planner executes the bridge before the next policy starts. Unsupported states may be traversed but cannot be selected as handoff targets; recorded transitions become reusable memory. e06
6. Experiments & results
RoboHarness routes subtasks among independently developed robot policies, then uses retrieved execution states to prepare compatible handoffs. It combines a coding-agent planner, capability assessments and online adaptation. The central empirical evidence is improved executed long-horizon manipulation and a substantial completion loss when bridging is removed; this is an orchestration method, not a new joint future/action predictor (e03, e06, e11, e13).
6.1 Read the original evidence
Table 1. Strong perturbation results come with an unresolved average definition. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Separate Original, which evaluates LIBERO task following, from the seven perturbation columns to its right. All entries are success percentages; bold and underlining identify the source's best and second-best values. Compare RoboHarness with π0.5 within the same column before looking at Average. The robot-state column shows 90.4 versus 75.4, whereas Original shows 98.7 versus 96.9. Sensor noise is a useful counterexample to uniform dominance: Cosmos-Policy reaches 92.7 against RoboHarness's 90.3. Section 6 connects robustness to routing and state preparation, but this table compares complete systems. It does not independently manipulate the bridge while holding every other component fixed. e09e10e12
What it supports. The category-level results support improved executed robustness in the displayed benchmark conditions, including a 15.0-percentage-point advantage over π0.5 under robot-state perturbations. The separate routing diagnostic on page 9 uses only π0.5 and TAMP and identifies sensor noise as an assessment outlier; it should not be conflated with this full-system table.
Where the evidence stops. The printed 93.2 Average is preserved. The seven displayed RoboHarness perturbation values have a simple mean of about 92.4%, and the source does not explain the weighting. Section 5.2 attributes baseline results to prior benchmark evaluations; uncertainty is absent.
Table 2. Progress and complete-task success reveal different long-horizon capabilities. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each cell contains progress score followed by success rate, both expressed as percentages in the table caption. Read those two quantities separately: a controller can complete meaningful task segments while rarely finishing the full composition. The π0.5 row illustrates this with 55.3 average progress and 6.4 success. Move upward through LLM-guided, logic-guided and H-WM-guided π0.5 to see the improvements associated with hierarchical guidance. RoboHarness reaches 97.5 progress and 95.2 success. The source gives Task 1–5 labels here without reproducing their benchmark instructions; keep these labels rather than assigning descriptions from the separate custom-task appendix. e02e07e08e09e11e13
What it supports. Against the strongest listed hierarchical baseline, H-WM-π0.5, RoboHarness raises average success from 64.8% to 95.2%, a reader-computed increase of 30.4 percentage points. This supports effective long-horizon execution by the complete orchestration system; it is stronger evidence of completion than reporting progress alone.
Where the evidence stops. Zero-shot concerns new task compositions without task-specific joint training. It coexists with trained constituent policies and retained online evolution. Baseline evaluations come from prior work, and complete episode/reset details and uncertainty are not supplied, limiting causal and statistical interpretation.
Figure 5(b–c). Physical execution succeeds across structures but repeated retrieval can exhaust the time budget. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The upper panel compares five structure-construction tasks; the lower panel tests four disturbances on the Bridge task only. There are fifteen trials per setting, totaling 135. Compare each orange bar with the blue Bridge bar at 86.7%, not with the upper panel's average or tallest bar. Re-hiding blocks yields 66.7%, dismantling progress 80.0%, distractors 86.7% and perception noise 73.3%. The appendix specifies a UR5e, ArUco pose estimation and a π0.5 policy trained with fifty demonstrations each for drawer opening and closing. Figure 6 on page 17 shows VLA drawer interaction alternating with TAMP block retrieval and assembly. e14e15
What it supports. These measurements concern executed physical manipulation. The system can recover after some changes to task state, but repeated exploration and replanning remain costly: the authors attribute the largest disturbance drop to timeouts after blocks are hidden again. Taller Bridge also exposes structural complexity and physical instability.
Where the evidence stops. Only fifteen trials support each bar, with no confidence intervals or comparative physical baseline. The perception test uses 5–10% relative pose noise in a marker-based setup; it does not establish robustness to arbitrary visual corruption or unseen robot embodiments.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO-LoHo long-horizon manipulation Five benchmark tasks; unseen compositions under the paper's adaptive zero-shot protocol. Baseline results are attributed to prior benchmark evaluations. | 97.5 / 95.2 Average progress score / success rate (%) | H-WM-π0.5: 84.9 / 64.8; π0.5: 55.3 / 6.4. Success exceeds H-WM-π0.5 by 30.4 percentage points (reader subtraction). Strong full-system completion evidence. Controller diversity, routing, bridging and adaptation vary together relative to these baselines, so this comparison does not isolate bridging. e07e09e11 |
| LIBERO task following and LIBERO-Plus robustness Original LIBERO and seven perturbation categories in Table 1. | Original: 98.7; reported Average: 93.2; robot-state perturbation: 90.4 Success rate (%) | π0.5: 96.9 original, 85.7 reported Average, 75.4 robot-state. Cosmos-Policy leads sensor noise: 92.7 versus 90.3. Preserve the printed averages: RoboHarness's seven perturbation cells have a simple mean of about 92.4%, not 93.2%. The aggregation is unclear; no corrected benchmark average is asserted. e10 |
| Component ablation on custom long-horizon tasks 500 simulated instances: 10 task classes with 50 random configurations each. | Full system: 86.0 / 93.1 Success rate / progress score (%) | No Memory Bridge: 60.4 / 82.9; no Evolution: 54.6 / 74.9; no Understanding: 37.6 / 68.7. Removing bridging costs 25.6 success points versus 10.2 progress points (reader subtraction), consistent with handoff failures preventing completion. Module removal does not isolate each internal algorithm. e13 |
| Physical structure construction and Bridge disturbances UR5e with TAMP and π0.5; 135 trials, 15 per structure or disturbance setting. Disturbances are tested on Bridge. | Bridge 86.7; Taller Bridge 66.7; Tower 93.3; Chinese Character 80.0; Boat 86.7 Success rate (%) | Bridge disturbances: re-hidden blocks 66.7; dismantled progress 80.0; distracting blocks 86.7; pose noise 73.3. Actual robot execution evidence with a small sample per condition. Pose noise is 5–10%; no physical baseline or confidence intervals are reported. Re-hiding failures are attributed to exploration/replanning timeouts. e14e15 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 4. Without a bridge, substantial partial progress fails to become complete execution. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the legend before comparing bars: blue is success and orange is progress, reversing the progress/success ordering used in Table 2. The left pair is the full system at 86.0 success and 93.1 progress. The next three pairs remove Memory Bridge, Evolution and Understanding respectively. These are 500 custom instances, with ten classes and fifty random configurations per class in Section 11.1.1, not the five LIBERO-LoHo tasks. Compare each removal with the full system, then inspect the single-policy variants on the right. The large separation between progress and success shows why an apparently productive rollout can still fail the requested composition. e06e07e13
What it supports. Removing the Memory Bridge lowers success from 86.0% to 60.4%, while progress falls from 93.1% to 82.9%. The respective drops are 25.6 and 10.2 percentage points. This pattern supports the authors' handoff interpretation: much high-level progress remains, but more executions fail before completion.
Where the evidence stops. This removes an entire bridge module. It does not separately test retrieval, support constraints, ranking or motion cost. Likewise, No-Evolution disables several adaptation mechanisms together. No error bars or multi-seed uncertainty are shown.
7. Analysis & limitations
7.1 What the evidence leaves open
The library bounds achievable behavior; sparse relevant memory weakens capability estimates and bridges. Sensor degradation harms routing evidence, and unreachable layouts cannot be repaired by orchestration. Navigation, MPC, world-action-model policies and tactile feedback remain future extensions. e12e16
The routing diagnostic uses only π0.5 and TAMP. Its usage/performance association is observational, with sensor noise an outlier. Benchmark tables provide no uncertainty or complete rollout/reset protocol; the OpenVLA-OFT baseline row is not explicitly equated to the internal GRPO checkpoint. e08e09e10e11e12
7.2 Questions for discussion
- How much of the long-horizon gain survives resetting all learned updates between evaluation episodes (e07, e11)?
- Does support-constrained progress ranking outperform simply moving to the nearest retrieved feasible state (e06, e13)?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the named checkpoints, frozen encoders, linked memory, pairwise-ranking estimator, planner/router and TAMP domain. Physical replication needs UR5e control, PDDLStream, ArUco pose estimation and drawer demonstrations. The simulation planner uses PDDL and Fast-Forward with continuous sampling. e08e15e17e18
The supplied text leaves initial memory size/provenance, retrieval counts, expansion horizon, support threshold, state normalization, motion weight, SVM kernel/loss settings, encoder weights, tuning grids and empty-feasible-set behavior unspecified. It also omits complete fine-tuning settings, GPU/compute costs, exact software versions, episode order and adaptation-reset rules. These must be resolved before a faithful numerical replication. e06e07e09e15e17e18
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Isolate supported progress ranking at identical handoffs
Reader-proposed check, not an experiment performed here: save identical post-policy states from the ten custom task classes and freeze routing, controller checkpoints and memory. Compare direct handoff, motion to the nearest feasible retrieved state, the full support-constrained progress objective, and the same objective without its support constraint. Give all motion-based variants the same planner budget. Measure incoming-policy success, complete-task success, support distance and bridge motion cost. If full bridging improves completion over direct transfer but does not outperform nearest-state transfer, the evidence favors state preparation while leaving the added ranking mechanism unsupported. Vary memory density to test the paper's sparse-experience limitation; preregister an explicit empty-feasible-set rule because the source does not supply one. e05e06e13e16e18
Check 2: Measure how retained evolution changes zero-shot performance
Reader-proposed check, not a reproduction claim: initialize identical policies and memory, then evaluate the same LIBERO-LoHo task instances under three conditions: frozen evolution and memory; memory accumulation with evolution disabled; and the full retained-update system. Repeat with shuffled episode orders, keeping controller execution limits and planner budgets equal. Record first-encounter and later success separately, along with every code, parameter, policy and metadata change. Report paired task success, progress, uncertainty and planning cost. If the headline gain appears mainly after repeated exposure or changes substantially with ordering, retained adaptation is a major contributor; if first-encounter gains remain stable under frozen settings, that supports composition and bridging independently of accumulated evolution. e02e05e07e09e11
8.3 Reading coverage
Visual audit: Visually inspected the title/authors/version on page 1; problem and method definitions on pages 2–6; policy interfaces, evaluation protocols and both tables on pages 7–8; routing diagnostic on page 9; ablation, physical results and limitations on pages 10–11; all appendix task, hardware, encoder, memory, estimator, adaptation and controller details on pages 16–21. Figures 1–8 and Tables 1–2 were viewed, as were all six final crops. Figure 2's arrows and temporal labels were checked against the hierarchical retrieval equations and page-6 target constraints; its confidence wording is qualified in the reading guide. Figure 8 was treated only as a representative score visualization, not a numerical validation. Reference pages 12–15 were read in the complete text but were not part of the visual pass. No separately supplied supplement was available.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 16, 17, 18, 19, 20, 21. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1 Introduction
- 2 Related Work
- 3 Problem Definition
- 4 Methodology, including 4.1–4.4
- 5 Experimental Setup and Baselines, including 5.1–5.2
- 6 Results, RQ1–RQ4
- 7 Ablation Study
- 8 Real Robot Experiments
- 9 Limitations and Future Work
- 10 Conclusion
- References
- 11 Appendix: 11.1.1 Simulated Tasks; 11.1.2 Real Robot Experiment; 11.2.1–11.2.4 auxiliary skills and underlying policies
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The supplied 21-page artifact is arXiv:2607.18060v2 [cs.RO], dated 28 July 2026 on its title page. Its title and nine authors match the catalog. No earlier edition or revision history was supplied, so changes from v1 cannot be established (e01).
- Separate supplemental material availability has not been fully verified; no separate supplement was supplied.
- Text extraction alone does not reconstruct figures; this was addressed by inspecting original PDF page renders and all six final crops. All eight figures and both tables were visually reviewed; reference-only pages 12–15 were read as text.
- Code, linked checkpoints, external benchmark definitions and project resources were not inspected. No experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, author/affiliation block and arXiv margin stamp
Exact RoboHarness title; authors Jinbang Huang, Yuanzhao Hu, Zhiyuan Li, Ran Qi, Yixin Xiao, Zhanguang Zhang, Mark Coates, Tongtong Cao and Yingxue Zhang; arXiv:2607.18060v2 [cs.RO], 28 Jul 2026. Five numbered affiliations and an internship footnote are shown.
Go to primary source ↓e02PDF pp. 1–3, Sections 1 and 3
Motivates capability boundaries and cross-policy mismatch; defines zero-shot compositions, policy-specific achievable instructions and in-distribution observations, and joint decomposition/assignment/bridging.
Go to primary source ↓e03PDF pp. 3–4, Section 4 and Figure 1; pp. 6–7, Section 4.4
Coding agent orchestrates native policy interfaces using Understanding, Memory and Evolution skills. Policy cards contain capabilities, requirements, assumptions, training tasks and execution statistics.
Go to primary source ↓e04PDF p. 4, Section 4.1; p. 18, Section 11.2.1
Five assessments cover temporal pose uncertainty, visual and semantic similarity, observation quality and state–policy compatibility; image-quality metrics include exposure, Laplacian sharpness, contrast and entropy.
Go to primary source ↓e05PDF p. 5, Figure 2 and Section 4.2.1 retrieval equations; pp. 18–19, Section 11.2.2 and Figure 7
Linked nodes contain observation, prompt, robot state and embeddings. Text top-K precedes visual top-K; temporal links and rollout statistics support memory updates and bridging.
Go to primary source ↓e06PDF pp. 5–6, Section 4.2.2, state–score pairs, support-region definition and constrained target objective
Expanded anchors receive jΔt labels; f_score,t measures progress. An ε-neighborhood of retrieved states defines admissible support. Target maximizes progress minus motion cost within support and motion feasibility, with nonnegative score. An external planner executes the bridge.
Go to primary source ↓e07PDF p. 6, Section 4.3; pp. 19–20, Section 11.2.3
Persistent simulated failures trigger four enabled evolution mechanisms. Validated updates persist across episodes. Appendix identifies SIMPACT/PDDLLM for TAMP adaptation, code refinement with GPT-5.5, grid-direction tuning and metadata updates.
Go to primary source ↓e08PDF p. 7, Section 5.1; pp. 20–21, Section 11.2.4
Default simulated library uses pi05_libero trained on four LIBERO suites, RLinf/RLinf-OpenVLAOFT-GRPO-LIBERO-90 and TAMP. TAMP uses PDDL, Fast-Forward and PDDLStream-based continuous sampling.
Go to primary source ↓e09PDF pp. 7–8, Sections 5, 5.1 and 5.2
Planner/router is Codex with GPT-5.5. Evaluations cover LIBERO, LIBERO-Plus, LIBERO-LoHo and custom tasks. Section 5.2 attributes comparison results to prior benchmark evaluations; complete rollout counts, uncertainty and reset protocols are not given here.
Go to primary source ↓e10PDF p. 7, Table 1, all headers and RoboHarness, π0.5 and Cosmos-Policy rows
RoboHarness reports Original 98.7, perturbations 90.4/97.0/86.8/97.1/90.3/87.6/97.4 and Average 93.2. π0.5 reports 96.9/85.7 for Original/Average and 75.4 for Robot State; Cosmos-Policy has Sensor 92.7. Average weighting is not defined.
Go to primary source ↓e11PDF p. 8, Table 2, Task 1–5 and Average columns
Entries are progress/success percentages. RoboHarness average is 97.5/95.2; H-WM-π0.5 84.9/64.8; logic-guided π0.5 73.2/48.4; LLM-guided π0.5 66.8/26.8; π0.5 55.3/6.4. No uncertainty is displayed.
Go to primary source ↓e12PDF pp. 9–10, Figure 3 and Section 6, RQ3–RQ4
Two-policy routing diagnostic compares π0.5 invocation frequency with standalone success. Sensor noise is an outlier; authors attribute assessment difficulty to degraded images and note unreachable object layouts.
Go to primary source ↓e13PDF p. 10, Figure 4 and Section 7; p. 16, Section 11.1.1
500 custom instances comprise ten classes with fifty configurations each. Success/progress: full 86.0/93.1, no bridge 60.4/82.9, no evolution 54.6/74.9, no understanding 37.6/68.7, motion planner 36.4/71.8, π0.5 0.0/28.5, RL-Open-VLA 0.0/4.6.
Go to primary source ↓e14PDF p. 10, Section 8; p. 11, Figure 5(b–c)
Fifteen trials per setting, 135 total. Five structure success rates are 86.7/66.7/93.3/80.0/86.7. Bridge disturbances yield 66.7/80.0/86.7/73.3 for hidden blocks/broken progress/distraction/pose noise. Re-hiding timeouts and taller-bridge instability are described.
Go to primary source ↓e15PDF pp. 16–18, Section 11.1.2 and Figure 6
UR5e physical tasks use TAMP/PDDLStream, ArUco poses and π0.5 initialized from pi05_libero, fine-tuned on 50 opening and 50 closing demonstrations. Figure 6 assigns block retrieval to TAMP and drawer operation to VLA. Disturbance noise is 5–10% relative pose error.
Go to primary source ↓e16PDF pp. 10–11, Section 9
Authors limit capability to the policy library and warn about sparse memory. Navigation, MPC, world-action models, new skill integration, online policy training and tactile feedback are future directions.
Go to primary source ↓e17PDF pp. 18–21, Sections 11.2.1–11.2.4
Names frozen google/siglip2-base-patch16-224, facebook/dinov2-base and BAAI/bge-large-en-v1.5; combines visual similarities by a weighted sum. Gives modules and checkpoints but not numerical retrieval/tuning settings, complete training/compute configuration or exact software versions.
Go to primary source ↓e18PDF p. 19, Section 11.2.2, Memory Bridge Example; p. 20, Figure 8
Appendix identifies pairwise-ranking SVM as practical progress estimator. Figure 8 is a representative projected score visualization, not an aggregate calibration or handoff-success benchmark; SVM training details are not specified.
Go to primary source ↓8.5 Primary sources
RoboHarness: Memory-Driven Orchestration of Heterogeneous Robot Policies for Long-Horizon Planning ↗
PDF · 10,371 extracted words
Source fingerprint
5ba0aaf36ed8e76e7dcc26f5d71308c0d382cb9959eb1df9e4ffa6e00c635cbd