Awesome-WAMs
GitHub
PAPER REPORTENAll readings ↗

HiMem-WAM: Hierarchical Memory-Gated World Action Models for Robotic Manipulation

English reading report: Method, equations, original figures, experiments and reproducibility.

Authors: Xiaoquan Sun; Ruijian Zhang; Chen Cao; Yihan Sun; Jiahui Chen; Zetian Xu; Bo Chen; Haijier Chen; Zhen Yang; Jiarun Zhu; Yijun Hong; JingZhe Xu; Jingrui Pang; Mingqi Yuan; Jiayu Chen

Affiliations: The University of Hong Kong; INFIFORCE; Huazhong University of Science and Technology; Tsinghua University; Wuhan University; Southern University of Science and Technology

Source: 2606.10363 ↗ · Catalog record

Reading: 175 / 558 · 6 original figures & tables · ~21 min ·

1. Paper overview

In one sentence: Flow-supervised motion latents and discovered skill boundaries support a causal, memory-augmented robot policy, with stronger evidence for Stage II pretraining than for the isolated value of memory gating. e02e03e08e09e11e15e16e21

At a glanceWhat to know
Research problem
Source description

Long manipulation sequences require remembering objects or completed subtasks after they leave view. The authors seek both transferable motion structure and persistent task state, without generating future video during deployment. Their proposed division of labor is motion tokenization, skill prediction, and selective storage of task history. e02e03e09e11

Core mechanism
Source description

A continuous two-level action representation combines flow-supervised local motion with variable-length skill segments; a planner and executor translate current multimodal inputs into robot-action chunks. e03e05e07e08

A key reported resultZero-shot robustness on LIBERO-PLUS: 76.0 average.

Success rate (%). Training only on standard LIBERO; seven deployment perturbation categories; 50 rollouts per task.

Without Stage II: 72.2; HoloBrain-0: 75.3. The 3.8-point Stage II gain exceeds the standard-benchmark gain. Robustness remains uneven: initial-state success is 38.1 versus HoloBrain-0's 58.2. e13e14e15

Reading caution
Author claim

The authors acknowledge substantial offline computation and engineering complexity, dependence on latent/boundary/memory quality, and evaluation on only one dual-arm platform with ten tasks. e20

Core contributions

  • Source description

    A continuous two-level action representation combines flow-supervised local motion with variable-length skill segments; a planner and executor translate current multimodal inputs into robot-action chunks. e03e05e07e08

  • Source description

    A learned write gate receives a boundary prediction and is supervised by discovered skill boundaries. External memory is retrieved through a separate read gate, rather than being equated with the planner's internal state. e08e09e10

Figure 1, upper three-stage framework. Motion and skill supervision precede the addition of external memory. Original paper, p. 2 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Follow the blue panel downward from raw RGB through DPFlow to low-level tokens. Yellow diamonds mark skill boundaries, and orange hexagons represent pooled high-level latents. The purple middle panel points a supervision arrow from latent targets toward the predicted chunk. The right panel shows proprioception joining visual-language features, a dashed path into the write gate, and a dashed return from the read gate to action prediction. For temporal order, use Appendix A.7: read existing memory, predict skill and motion, decode actions, then conditionally write. The schematic does not itself specify the gate threshold or the planner/executor interfaces. e02e03e08e09e10e11e21

What it supports. The organizing idea is to reuse future-motion information as training targets and retain selected past states at deployment. The memory bank serves the causal policy rather than requiring a generated future at each decision. The method text supplies the explicit skill-to-latent-to-control factorization that this compressed diagram does not fully expose.

Where the evidence stops. The crop faithfully retains Wan2.2-TI2V-5B labels, whereas the method and Appendix A.5 name Qwen3-VL-4B-Instruct. Figure 1 also places skill extraction in Stage I; Appendix A.6 assigns it to Stage II. Their exact architectural integration and stage grouping are unresolved.

2. Motivation

2.1 The problem and the proposed response

Source description

Long manipulation sequences require remembering objects or completed subtasks after they leave view. The authors seek both transferable motion structure and persistent task state, without generating future video during deployment. Their proposed division of labor is motion tokenization, skill prediction, and selective storage of task history. e02e03e09e11

2.2 What this reading follows

A robot placing two plates must distinguish the current scene from the history of what it has already completed. HiMem-WAM addresses that distinction through two temporal representations: local motion latents and longer skill latents. The described planner chooses a skill using current inputs and external memory; an executor unfolds it into motion, and a decoder produces robot controls. Optical flow supplies training supervision, while deployment uses current observations and stored history. The figures below explain that separation and examine the reported robustness and execution results. They also expose unresolved documentation differences that matter when reconstructing the system. e02e03e08e09e11e15e16e21

3. Research context

We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.

Catalog dimensionRecorded classification
Major categoryVLA
ArchitectureDual-system
Prediction paradigmNot applicable
QuadrantNot applicable

3.1 Evidence-based assessment

Supports the recorded classification

Reader analysis

The described planner → latent executor → action decoder supports the recorded hierarchical/dual-system VLA and latent-action-pretraining labels. The external memory bank is an adapter, not itself a second planner. Future-motion supervision is offline; deployment neither jointly generates future observations/actions nor infers controls from a generated future, supporting the recorded not-applicable prediction quadrant. This assesses the textual architecture; the inconsistent backbone diagrams prevent a stronger implementation-level conclusion. e03e08e09e11e12e21

This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.

4. Problem formulation

4.1 Inputs and outputs

InputsOutputs
  • Current multi-view RGB observations o_t
  • Proprioception p_t and language instruction ℓ
  • External memory bank M_t
  • Training supervision: future observations, optical flow, offline latent/skill/boundary targets, and available action labels
  • High-level skill and boundary predictions
  • Low-level latent-action chunk of length K
  • Decoded executable robot-action chunk and conditional memory update

4.2 Equations and their role

Ll=∥Φ^t−Φt∥1+λaItact∥a^t−at∥22+βDKL ⁣(qϕ(ztl∣ct) ∥ N(0,I))\mathcal{L}_l=\|\hat{\Phi}_t-\Phi_t\|_1+\lambda_a I_t^{\mathrm{act}}\|\hat a_t-a_t\|_2^2+\beta D_{\mathrm{KL}}\!\left(q_\phi(z_t^l\mid c_t)\,\|\,\mathcal N(0,I)\right)
Equation (3): Φ_t is optical flow, its hat is reconstruction, z_t^l is the low-level latent, and c_t contains current/future observations, proprioception, instruction and flow. I_t^act masks missing action labels; â_t is auxiliary action prediction. λ_a and β weight action alignment and Gaussian-prior regularization. e03
ri(s)=12 ⁣(1−(q^i−1(s))⊤k^i(s)),bi(s)=I[ri(s)≥δs](i>1),b1(s)=1r_i^{(s)}=\tfrac12\!\left(1-(\hat q_{i-1}^{(s)})^\top\hat k_i^{(s)}\right),\qquad b_i^{(s)}=\mathbb I[r_i^{(s)}\geq\delta_s]\quad(i>1),\qquad b_1^{(s)}=1
Equations (14)–(15): s indexes hierarchy stages; normalized query/key features compare adjacent encoded tokens. Dissimilarity r reaching threshold δ_s starts a segment; the first token always starts one. This offline ≥ comparison differs from the strict > comparison used for online memory writing. e05e09
Mt+1={Uψ(Mt,γt),αtw>η,Mt,otherwise.\mathcal M_{t+1}=\begin{cases}U_\psi(\mathcal M_t,\gamma_t),&\alpha_t^w>\eta,\\\mathcal M_t,&\text{otherwise}.\end{cases}
Equation (32): M_t is the memory bank, α_t^w the sigmoid write gate, η its threshold, and γ_t the candidate token from current adapted state, predicted skill and pooled latent chunk. U_ψ appends the token and compresses overflow beyond N_max; neither budget nor compression implementation is specified. e09e24

5. Method in detail

5.1 Turn future motion into supervision for a current-state policy

Source description

Begin with a demonstration transition, not an imagined rollout at deployment. Stage I estimates multi-view optical flow and uses it to train a variational low-level tokenizer. The reconstruction loss makes the latent retain motion information; the Gaussian-prior penalty regularizes that representation; action labels, when available, provide additional control alignment. After training, the tokenizer is frozen and its outputs become offline targets. Stage II learns to predict those targets and their skill-level organization from policy inputs. Its skill-discovery objective also reconstructs motion through a frozen flow decoder. Stage III finally grounds the representation in executable action labels while activating memory. This sequence explains Table 5's distinction between supervision and inference: the robot predicts the useful latent representation without observing the future transition that originally taught it. e03e04e06e10e12

Table 5. The deployed policy predicts the representations that offline processing supplied during training. Original paper, p. 11 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read each row across the four stage columns. RGB, proprioception and instruction persist throughout, but action annotations are optional in Stage I and required for Stage III grounding. Low-level latents change from tokenizer output to supervision and finally to predictions. High-level skills and boundary labels likewise move from targets to predictions. The last row is especially informative: external memory is disabled during Stage II and active during Stage III and inference. The optical-flow row's Stage II 'supervision only' entry should be read with Appendix A.2: flow extraction is offline, and DPFlow is not called in the deployed policy. e04e10e11e12e24

What it supports. This table makes the causal-input claim concrete. Future-dependent motion targets can shape a policy without becoming its online inputs. It also shows why Stage II performance gains cannot directly demonstrate the effectiveness of the memory gate: the latent pretraining stage is explicitly conducted with external memory disabled.

Where the evidence stops. The matrix specifies which signal is used, not a complete data interface. The source does not explain how missing proprioception or instructions are handled for action-free videos, and it provides no measured inference-latency comparison.

5.2 Convert variable-length segments into timestep-level teaching signals

Source description

The hierarchy first encodes the low-level latent sequence. At each stage it compares adjacent normalized query/key features: larger dissimilarity can mark a new segment, with the first position always starting one. Attention pooling summarizes each segment, allowing the next hierarchy stage to operate on a shorter sequence. Training combines next-motion prediction with penalties on boundary frequency and within-segment inconsistency. The policy still needs a target at each control timestep, however. Appendix A.4 therefore carries each pooled token's start index back to the original timeline, unfolds the final skill across its segment, and marks the segment start as a boundary label. Those discovered labels supervise both boundary prediction and memory writing. They are learned motion-segmentation targets; the source does not establish that each one matches a human-defined subtask. e05e06e07e10

5.3 Separate remembering a plan from verifying that it succeeded

Reader analysis

At a decision step, the policy first queries the existing external memory bank. A read gate controls how much retrieved context enters the adapted state; the Qwen planner then predicts a skill and boundary score. The executor generates a low-level latent chunk, and the decoder produces robot actions. The write gate can store a token containing the adapted state, predicted skill and pooled predicted chunk. Reader interpretation: this token records what the policy currently represents and plans, rather than certifying that the physical skill finished successfully. Appendix A.7 supplies no explicit post-execution verification step before writing. Consequently, missed boundaries, failed grasps or stale retrieved context could affect later decisions; the report treats that as a testable concern, not an observed failure mechanism. A gate-specific controlled experiment is needed to distinguish selective retention from simply adding memory capacity. e08e09e11e16

5.4 Training and inference

During training

Source description

Stage I trains only the tokenizer. Stage II discovers high-level targets and trains planner/executor with skill MSE, latent-chunk MSE and boundary BCE, with memory disabled. Stage III activates memory and action supervision while retaining auxiliary latent/boundary losses; write-gate BCE and read/write sparsity penalties train the memory interface. e03e10e12

Source description

The appendix permits teacher-forced memory warmup using discovered boundaries and target latents, then predicted writing; it does not establish whether or how long warmup was used. Real-world training reports 400 demonstrations per task and five SFT epochs, without a complete pretraining data or optimization recipe. e11e17e24

During inference

Reader analysis

Each decision uses current RGB, proprioception, instruction and stored memory: retrieve context, predict skill/boundary, expand latents, decode controls, and conditionally write memory. Future video and DPFlow are absent from this online sequence. The memory candidate includes planned motion, not verified action completion. Execution frequency and how many actions precede fresh feedback are unspecified. e09e11e12e24

5.5 Implementation flow

  1. Learn local motion offline

    DPFlow estimates inter-frame flow separately for each view; components are normalized by image width and height. A variational tokenizer combines motion, visual, proprioceptive and instruction context. Flow reconstruction and KL regularization use unlabeled videos; auxiliary action alignment is masked to labeled transitions. The trained tokenizer is frozen to generate cached low-level targets. e03e04

  2. Discover and unfold skills

    At each hierarchy stage, adjacent normalized query/key dissimilarity determines segment starts; attention pooling creates one token per segment. Next-latent prediction, frozen-decoder flow reconstruction, boundary-ratio regularization and within-segment consistency train the hierarchy. Index maps unfold final skills and boundaries to control timesteps; invalid chunk-tail positions are masked. e05e06e07

  3. Read memory and choose a skill

    Attention retrieves continuous tokens from the external bank, returning zero context when empty. A scalar read gate adds projected context to the current state. Appendix A.5 specifies Qwen3-VL-4B-Instruct with projected proprioception and memory inputs, plus continuous skill and sigmoid boundary heads. e08e09

  4. Expand, ground and store

    The executor expands the predicted skill into a latent chunk; the action decoder maps it and the adapted state to controls. A candidate memory token combines state, predicted skill and pooled predicted latents. Writing requires the write gate to exceed its threshold; overflow triggers compression under a fixed memory budget. e09e11

6. Experiments & results

HiMem-WAM trains motion latents from optical flow, groups them into skill latents, and uses learned skill transitions to supervise sparse external-memory writes. Its described deployed policy selects a skill, predicts a latent action chunk, and decodes robot controls using current observations and memory. Stage II ablations support improved manipulation robustness; comparisons on memory tasks do not isolate the gate's contribution. Backbone inconsistencies and missing implementation settings limit reproducibility.

Source and visual limitations
Reader analysis

The source supplies Stage II removal ablations in Tables 2–4, but no isolated memory-gate, boundary-quality, memory-budget or inference-latency experiment. Figure 5 contains qualitative rollouts and flow images rather than measurements of memory fidelity. This edition therefore treats selective memory's mechanism as described but not causally isolated by the reported evaluations. e14e15e16e19e23

6.1 Read the original evidence

Tables 2 and 3, with shared footnote. Stage II produces a larger average gain under deployment perturbations. Original paper, p. 7 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. The left table reports standard LIBERO's Spatial, Object, Goal and Long suites. The right table reports zero-shot LIBERO-PLUS: its caption on the source page specifies training only on standard LIBERO. Read the shared footnote for camera, initialization, language, lighting, background, noise and layout abbreviations. Start with the two bottom rows in each table to compare the same named method with and without Stage II, then inspect external baselines. Appendix Figure 6 illustrates the seven perturbation categories. Compare averages within a benchmark; the standard and perturbed suites test different conditions and their percentages should not be pooled. e13e14e15e23e24

What it supports. On standard LIBERO, Stage II raises the average from 96.6% to 97.7%, a 1.1-point gain. Under LIBERO-PLUS perturbations, it raises 72.2% to 76.0%, a 3.8-point gain. The right table also exposes a weakness: initialization perturbations remain at 38.1%, below HoloBrain-0's 58.2%.

Where the evidence stops. Removing Stage II changes a training bundle, so the result does not isolate optical flow, hierarchy or training duration. The tables provide no seed uncertainty or evidence of equal pretraining compute; the narrow lead over an external baseline needs that context.

Table 1. Memory-task gains remain far from reliable long-horizon manipulation. Original paper, p. 6 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read the Task Memory Complexity column before interpreting the average. The upper block contains five M(1) tasks and the lower block four M(n) tasks; their printed averages summarize different task sets. The bottom line summarizes all nine tasks rather than giving equal weight to the two group averages. Move across a task row to compare methods under that task's success criterion, then down the HiMem-WAM column to see the uneven difficulty. 'Press Button' remains particularly weak at 8%. Section 4.1 specifies 100 rollouts per task. The table is an execution comparison, not a direct measurement of what the memory bank stores. e13e16e21

What it supports. HiMem-WAM reaches 26.3% overall versus 10.8% for π0.5 and 9.8% for X-VLA in this table. Its M(1) mean is 31.6%, dropping to 19.8% on M(n). Those gains support usefulness of the complete policy on memory-dependent tasks while also showing that repeated state updates remain difficult.

Where the evidence stops. There is no memory-disabled HiMem-WAM row. Mem-0 is described as stronger in the adjacent prose but has no tabulated score. Figure 1's π0.5 mini-chart says 10.4%; this reading retains Table 1's 10.8% without silently reconciling them.

Figure 3(a)–(e). Physical execution gains increase on the two hard tasks, but success remains limited. Original paper, p. 6 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read each task's paired ST and GE groups, using white for π0.5 and blue for HiMem-WAM. The upper panels divide tasks into four easy, four medium and two hard tasks. The lower-left examples clarify the perturbed conditions: target height, lighting, unseen objects, object position and paraphrased language. The lower-right drawing locates the front, head and wrist cameras around the dual-arm workspace. Appendix B defines the hard tasks as sequential plate placement and bread–ham–bread assembly. Keep the ST/GE comparison separate from the action-representation experiment in Table 4, whose footnote restricts it to ST. e17e18e22

What it supports. For the hard tasks, the blue bars are 40% and 30% in ST, and 30% and 20% in GE. Averaging those two tasks gives 35% and 25%, respectively. The corresponding π0.5 means are 10% and 5%, matching the text's 25- and 20-percentage-point advantages. These are completed robot-task outcomes.

Where the evidence stops. The protocol reports 20 trials per task, without uncertainty bars. Section 4.3 says GE trials span four perturbation types, while this figure and Appendix B show five. Perturbation magnitudes and exact trial allocation are not specified.

6.2 Results and evaluation conditions

Task & protocolReported resultComparison & interpretation
Standard manipulation on LIBERO

Four suites: Spatial, Object, Goal and Long; 50 rollouts per task.

97.7 average; suites 98.2 / 99.8 / 98.4 / 94.5.

Success rate (%)

Without Stage II: 96.6 average; Fast-WAM: 97.6.

Stage II adds 1.1 percentage points. The 0.1-point lead over Fast-WAM has no reported uncertainty and does not establish a reliable ranking. e13e14

Zero-shot robustness on LIBERO-PLUS

Training only on standard LIBERO; seven deployment perturbation categories; 50 rollouts per task.

76.0 average.

Success rate (%)

Without Stage II: 72.2; HoloBrain-0: 75.3.

The 3.8-point Stage II gain exceeds the standard-benchmark gain. Robustness remains uneven: initial-state success is 38.1 versus HoloBrain-0's 58.2. e13e14e15

Memory-dependent manipulation on RMBench

Nine tasks; five M(1) and four M(n) tasks; 100 rollouts per task.

26.3 total average; 31.6 on M(1), 19.8 on M(n).

Success rate (%)

Table 1: π0.5 10.8 total average and X-VLA 9.8.

Improvement over listed baselines coexists with low absolute success. The authors say performance remains below Mem-0, whose scores are not tabulated. No memory-removal comparison isolates causality. e13e16

Hard real-world manipulation versus π0.5

Place two plates and make breakfast on the dual-arm platform; standard (ST) and perturbed generalization (GE) settings; 20 trials per task are stated.

+25.0 ST; +20.0 GE.

Reported success-rate advantage (percentage points)

π0.5 under the corresponding settings in Figure 3 and Section 4.4.

These are physical-execution results, not video-generation scores. The two-task category and perturbation-protocol ambiguity constrain generalization. e17e18e22

Stage II ablation across real-world action representations

Hard tasks, ST only; joint-position versus end-effector-pose controls.

Joint positions: 15.0 without Stage II → 35.0 with; end-effector pose: 10.0 → 30.0.

Success rate (%)

Within-representation Stage II removal in Table 4.

Both hard-task gains are 20 percentage points. The ablation tests the bundled Stage II procedure, not hierarchy, flow priors and memory separately. e17e19

6.3 Ablations and diagnostic examples

Read component removals and qualitative examples within their stated evaluation conditions.

Table 4. Stage II benefits both action interfaces on hard tasks. Original paper, p. 7 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Use the grouped column headers to make two separate comparisons: without versus with Stage II for joint positions, then the same comparison for end-effector pose. The rows share the Easy, Medium and Hard task grouping described in Appendix B. On easy tasks, joint-position control already reaches 100%, leaving no room for an absolute gain, while the end-effector-pose condition rises from 90% to 100%. The hard row therefore gives a more discriminating comparison of pretraining. Retain the footnote's restriction: every entry is from ST, so these numbers are not a separate GE robustness measurement. e17e19e22

What it supports. The hard-task success rate rises from 15% to 35% with joint positions and from 10% to 30% with end-effector poses. Both improvements are 20 percentage points. Joint-position control ends higher in this setting, but the result also supports a benefit of Stage II under the second action representation.

Where the evidence stops. These are bundled Stage II ablations, not tests of individual losses or the write gate. The source's discussion of a larger end-effector-pose benefit should not be read as a larger absolute hard-task gain; those gains are equal.

7. Analysis & limitations

7.1 What the evidence leaves open

Author claim

The authors acknowledge substantial offline computation and engineering complexity, dependence on latent/boundary/memory quality, and evaluation on only one dual-arm platform with ten tasks. e20

Reader analysis

Architecture documentation is inconsistent: Figure 1 labels Wan2.2-TI2V-5B and puts skill extraction in Stage I, while the method and Appendix A specify a Qwen3-VL-4B planner and Stage II skill discovery. Figure 2's DPFlow path also lacks the explicit offline/online separation supplied by Table 5. e21e08e10e12

Reader analysis

Figure 1 prints a 10.4% RMBench π0.5 score whereas Table 1 prints 10.8%; this report retains Table 1. Section 4.3 mentions four GE perturbation types, but Figure 3 and Appendix B enumerate five. These source differences remain unresolved. e21e16e17e18e22

Reader analysis

The supplied comparisons report no confidence intervals or seed variation. Stage II removal changes a training bundle; neither the memory gate nor learned boundaries receive an isolated ablation. Qualitative flow/rollout images do not measure boundary correctness or memory fidelity. e14e15e16e18e19e23

7.2 Questions for discussion

  1. Does boundary timing improve memory usefulness after controlling write count and capacity?
  2. How much robustness comes from flow supervision versus additional Stage II training?
  3. How are planned memory events corrected when physical execution fails?

8. Reproducibility audit

8.1 Requirements and known gaps

Source description

Rebuilding requires synchronized observations, proprioception/instructions, action labels where available, DPFlow preprocessing, frozen latent targets, hierarchy training, and sequential memory finetuning. Hardware evaluation used two AgileX Piper 6-DoF arms and four Intel RealSense D435i cameras. e03e04e10e17

Reader analysis

Unspecified items include pretraining corpora and mixing, latent sizes, K, hierarchy depth and thresholds, boundary-ratio targets, loss weights, η, memory capacity/compression, hard-boundary gradient treatment, optimizer/batch settings, compute and software versions. Deterministic and stochastic action objectives are both described without identifying the evaluated choice. e05e06e09e10e24

Reader analysis

Reader-proposed checks: hold backbone, data and training compute fixed while comparing learned boundaries with matched-rate fixed segmentation; separately compare learned memory writing with equally budgeted periodic writing and no memory. Measure success by perturbation and memory-complexity group, alongside writes and latency. These experiments have not been run. e05e09e13e15e16e19

8.2 Proposed reproduction checks

The following checks are proposals motivated by the paper. They have not been run as part of this reading.

Check 1: Does learned boundary timing beat an equal writing budget?

Reader-proposed experiment, not performed: after resolving the backbone specification, keep the same Stage II initialization, demonstrations, memory capacity, token builder and Stage III training budget. Compare learned write gating, periodic writing matched to its mean write count, and memory disabled. Evaluate the same RMBench tasks with paired rollout seeds, reporting M(1)/M(n) success, writes per episode and inference latency. Shuffle stored tokens at evaluation as a history-use diagnostic. A consistent gain over equally budgeted periodic writing would support meaningful event selection; equal success, or no effect from shuffling, would weaken that explanation. e08e09e10e13e16e24

Check 2: Does discovered skill structure explain the Stage II robustness gain?

Reader-proposed experiment, not performed: freeze identical Stage I latents and hold planner/executor capacity, data, update count and Stage III procedure fixed. Compare learned hierarchy boundaries with fixed-length segmentation matched to the discovered mean segment length, plus the reported no-Stage-II condition. Evaluate standard LIBERO and zero-shot LIBERO-PLUS under the stated rollout protocol, recording per-perturbation success and variation across training seeds. The learned hierarchy should outperform the matched segmentation control, especially under perturbations, if its temporal structure drives the gain. Improvement only over no Stage II would leave extra pretraining and motion supervision as competing explanations. e04e05e06e10e13e14e15e24

8.3 Reading coverage

Visual audit: Inspected the title/byline, Figures 1–6, Tables 1–5, all method/equation/training pages, physical-platform details, task definitions and qualitative appendix pages. Each of the six final original PDF crops was separately viewed, with legends, headers and relevant table footnotes intact. Cross-checked the figure's write/read arrows against Eqs. (29)–(32) and Appendix A.7, the ≥ skill-boundary rule against Eqs. (14)–(15), and the strict > write rule against Eq. (32). Backbone/stage and numerical/protocol discrepancies are disclosed in the visual cautions and base report. References on pp. 9–10 and baseline descriptions on p. 19 were read as text but not rendered; no retained method or numerical claim relies on those pages. Separate supplements, code and execution traces remain outside this reading.

PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 14, 15, 16, 17, 18. Appendix coverage: reviewed.

Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.

Text reading scope & known omissions
  • Abstract; Sections 1–2: Introduction and Related Work (pp. 1–3)
  • Section 3: Method, training and inference (pp. 3–5)
  • Sections 4.1–4.4: simulation and real-world experiments (pp. 5–8)
  • Sections 5–6: Conclusion and Limitations (p. 8)
  • References (pp. 9–10)
  • Appendix A.1–A.7: signal separation, tokenization, skill discovery, unfolding, planner, memory, training and inference (pp. 11–15)
  • Appendix B.1–B.2: perturbations and task definitions; Figures 5–6 (pp. 16–18)
  • Appendix C: Baselines (pp. 18–19)

Outside the original text pass

  • Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
  • Separate supplemental material availability has not been fully verified.
  • Pages with little extracted text require visual inspection: 17
  • The supplied PDF was read in full, including all five text chunks. Text extraction does not reconstruct figure images; all six figures and five tables were therefore inspected in the PDF, including low-text page 17.
  • Separate supplemental material availability has not been fully verified. The supplementary material embedded in this PDF was reviewed.
  • Code, external links and other papers were not inspected; experiments were not reproduced.
  • Identity: the exact title and all 15 authors match the catalog after name-order normalization. The PDF is arXiv:2606.10363v1, stamped 9 June 2026; the catalog supplied a submission date of 8 June 2026. The title page additionally lists INFIFORCE, omitted from the catalog affiliation string.
  • The supplied provenance identifies the previously selected HTML as the same v1 revision. That HTML was not supplied for independent comparison; this report describes only the verified PDF, without assuming byte or presentation equivalence.

The visual audit above records the subsequent illustrated pass.

8.4 Traceable evidence

e01PDF p. 1, title, complete byline, affiliations and arXiv margin stampInspect

Exact supplied title; 15 authors from Xiaoquan Sun through Jiayu Chen; six affiliations including INFIFORCE; arXiv:2606.10363v1 dated 9 Jun 2026.

Go to primary source ↓
e02PDF pp. 1–3, Abstract, Section 1 contributions and Section 3 overview, Eq. (1)Inspect

Motivates motion/skill abstraction and task memory; factorizes controls through a skill and low-level latent chunk.

Go to primary source ↓
e03PDF p. 4, Section 3, Low-level latent actions, Eqs. (2)–(3)Inspect

Gaussian variational tokenizer uses future/current observations and flow; flow L1, masked action MSE and KL losses; tokenizer is frozen for offline extraction.

Go to primary source ↓
e04PDF pp. 11–12, Appendix A.2, Eqs. (10)–(13) and Offline latent extractionInspect

Flow is normalized by view width/height; motion and semantic features fuse across views; action alignment is label-masked; DPFlow stops after offline extraction.

Go to primary source ↓
e05PDF p. 12, Appendix A.3, Eqs. (14)–(16)Inspect

Adjacent normalized query/key dissimilarity at or above δ_s starts a segment; first token is a boundary; attention pooling produces higher-level tokens.

Go to primary source ↓
e06PDF p. 13, Appendix A.3, Eqs. (17)–(21)Inspect

Skill loss combines next-latent L1, flow reconstruction through a frozen decoder, target boundary-ratio penalty and within-segment consistency.

Go to primary source ↓
e07PDF p. 13, Appendix A.4, Eqs. (22)–(26)Inspect

Recursive start-index maps unfold final skills/boundaries to original timesteps; invalid positions in the trailing action chunk are masked.

Go to primary source ↓
e08PDF p. 4, memory module; p. 13, Appendix A.5, Eqs. (27)–(28)Inspect

Qwen3-VL-4B-Instruct planner receives RGB, language, projected proprioception and memory; continuous heads predict skill and boundary score. Planner is distinct from memory bank.

Go to primary source ↓
e09PDF p. 14, Appendix A.5, Eqs. (29)–(32); p. 4, Eqs. (5)–(6)Inspect

Attention/read gate conditions state; executor predicts latents; candidate combines adapted state, predicted skill and pooled chunk; α_t^w > η triggers append/compress under N_max.

Go to primary source ↓
e10PDF pp. 4–5, Eqs. (7)–(9); p. 14, Appendix A.6, Eqs. (33)–(40)Inspect

Three stages train tokenizer, then memory-free latent policy, then action-grounded memory policy. Skill/action-latent MSE and boundary BCE are retained; gate BCE and sparsity regularize memory.

Go to primary source ↓
e11PDF p. 15, Appendix A.6 warmup, Eqs. (41)–(42), and Appendix A.7 steps 1–7Inspect

Optional teacher-forced warmup uses target boundaries/latents. Online sequence reads current inputs/memory, predicts skill/chunk, decodes actions and writes conditionally; no future-video generation or flow estimation.

Go to primary source ↓
e12PDF p. 11, Appendix A.1, Table 5 and surrounding explanationInspect

Table distinguishes supervised from predicted latents, optional Stage I action labels, disabled Stage II memory, and absence of flow/action annotations at inference.

Go to primary source ↓
e13PDF p. 5, Section 4.1, Benchmark Selection and Evaluation MetricsInspect

LIBERO and LIBERO-PLUS use 50 rollouts/task; RMBench uses 100; all report success rate.

Go to primary source ↓
e14PDF p. 7, Table 2, HiMem-WAM, w/o Stage II and Fast-WAM rowsInspect

HiMem-WAM averages 97.7 (98.2/99.8/98.4/94.5); w/o Stage II averages 96.6; Fast-WAM 97.6. No uncertainty columns.

Go to primary source ↓
e15PDF p. 7, Table 3 caption, HiMem-WAM, w/o Stage II and HoloBrain-0 rowsInspect

Standard-LIBERO-only zero-shot protocol; averages 76.0, 72.2 and 75.3. HiMem-WAM initial-state 38.1 versus HoloBrain-0 58.2; seven perturbation columns.

Go to primary source ↓
e16PDF p. 6, Table 1, task/group/total rows and Key Finding 2Inspect

HiMem-WAM 31.6 on five M(1) tasks, 19.8 on four M(n), 26.3 overall; π0.5 10.8 and X-VLA 9.8 overall. Mem-0 is discussed as stronger but omitted from the table; no memory-off row.

Go to primary source ↓
e17PDF p. 6, Section 4.3 hardware; p. 7, Task Settings, Evaluation Settings and Training DetailsInspect

Two AgileX Piper 6-DoF arms and four D435i cameras; ten tasks across three difficulties; ST/GE evaluation; 400 demonstrations/task, five SFT epochs, 20 trials/task. Text says four GE perturbation types.

Go to primary source ↓
e18PDF p. 6, Figure 3(a)–(e); p. 7, Section 4.4, Key Finding 3Inspect

Task-specific ST/GE bars compare π0.5 with HiMem-WAM; five GE variations and platform are shown. Text reports hard-task gains of 25.0 points ST and 20.0 GE; medium gains 12.5 and 10.0.

Go to primary source ↓
e19PDF p. 7, Table 4, all rows and ST footnote; Key Finding 4Inspect

Joint-position Stage II changes: Easy 100→100, Medium 80→82.5, Hard 15→35. EE pose: 90→100, 67.5→75, 10→30. All are ST success percentages.

Go to primary source ↓
e20PDF p. 8, Section 6 LimitationsInspect

Authors identify substantial training computation, multi-stage engineering dependencies and limited validation on one dual-arm platform with ten tasks.

Go to primary source ↓
e21PDF p. 2, Figure 1 stage labels, caption and RMBench mini-chart; p. 3, Figure 2(d)Inspect

Figure 1 names Wan2.2-TI2V-5B, places skill latents under Stage I and prints π0.5 RMBench 10.4. Figure 2 depicts DPFlow feeding latent actions without an explicit deployment-stage distinction.

Go to primary source ↓
e22PDF p. 16, Appendix B.1–B.2; p. 8, Figure 4Inspect

Five GE types are position, unseen distractors, layout/height, lighting and instruction. Four easy and four medium tasks accompany two hard tasks: sequential plate placement and bread–ham–bread breakfast assembly.

Go to primary source ↓
e23PDF p. 17, Figure 5; p. 18, Figure 6Inspect

Qualitative RGB/flow rollout strips and seven LIBERO-PLUS perturbation examples are shown; no numerical latent, boundary or memory-fidelity diagnostic is provided.

Go to primary source ↓
e24PDF pp. 11–15, Appendix A.2–A.7; p. 7, Training DetailsInspect

Provides symbolic losses, boundaries, memory budget/update and optional warmup, but not numerical hyperparameters, full backbone integration, pretraining recipe, optimizer/compute/software configuration, or control timing. Real-world demonstration count and SFT epochs are given.

Go to primary source ↓

8.5 Primary sources

Scroll across the image to inspect details. Press Esc to close.