Beyond Task Success: Behavioral and Representational Diagnostics for WAM and VLA
1. Paper overview
In one sentence: Rollout diagnostics and sparse feature analysis expose architecture-dependent WAM behavior and cost, while leaving the causal value of future prediction unresolved. e02e03e04e05e14e15e18e24
| At a glance | What to know |
|---|---|
| Research problem | Source description Task success hides whether a robot steadily advances the correct object, disturbs distractors, stalls, or changes commands abruptly when replanning. The authors ask whether future prediction produces behaviorally useful representations. Their contribution is an evaluation framework applied to existing policies, rather than a new controller. e02e03 |
| Core mechanism | |
| A key reported result | LIBERO task success and target-object selectivity: Cosmos: 98.0%, 0.166, 0.017. Success (%), target progress, mean distractor path; higher success/progress and lower path are preferred.. Spatial, object, goal and long suites; 50 trials per task. Object metrics use the decoded-state subset, whose size is unspecified. pi0.5: 96.6%, 0.128, 0.687. VLA-JEPA distractor path: 0.036; FastWAM: 0.673. Similar success can conceal markedly different object disturbance. This is a checkpoint comparison, not an isolated future-prediction ablation; LingBot-VA covers only LIBERO-10. e09e10e11 |
| Reading caution | Reader analysis Evidence covers simulation and available checkpoints; decoded object traces restrict coverage. No matched architecture/training ablation establishes that imagination caused the behavioral differences. Recommending lightweight inference-time imagination remains an author proposal. e09e15 |
Core contributions
Figure 2. Where future prediction enters the evaluated policy families. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the three rows as alternative information flows. The top row sends the current observation and language into a shared world-action model, which branches to an action chunk and future observation/video. The middle row places future imagination before the action decoder; its dotted return path represents robot execution supplying another observation. The bottom row separates training from inference: a future target shapes the backbone during training, while the inference-side future output is crossed out. The legend identifies solid forward paths, dashed auxiliary/training paths and dotted feedback. The paper assigns Cosmos, LingBot-VA, and FastWAM/VLA-JEPA to these respective families. e03e09
What it supports. The location of future prediction changes what the diagnostic framework can inspect. An explicit imagination stream offers a distinct representation to analyze; an auxiliary objective may shape an action policy without retaining an inference-time rollout. The schematic explains why a shared WAM label is insufficient to predict runtime or feature organization.
Where the evidence stops. This is a family-level schematic, not a new policy architecture or proof that every depicted planning/value path runs in each checkpoint. Sections 2–3 support the three-way distinction; the diagram alone cannot establish causal benefits from any branch.
2. Motivation
2.1 The problem and the proposed response
Task success hides whether a robot steadily advances the correct object, disturbs distractors, stalls, or changes commands abruptly when replanning. The authors ask whether future prediction produces behaviorally useful representations. Their contribution is an evaluation framework applied to existing policies, rather than a new controller. e02e03
2.2 What this reading follows
A robot can finish a task while disturbing nearby objects, or appear smooth because it barely moves. This paper evaluates those differences using two complementary views of existing robot policies: executed behavior and internal sparse features. The behavioral view follows commands, target progress, distractors and runtime. The representation view asks which features track the present, recur in specific episodes, or correlate with future states and actions. The most useful reading treats these as complementary diagnostics. Their associations motivate better experiments, but unequal checkpoint coverage, weak labels and inconsistencies in the written metric definitions prevent a clean causal ranking of WAM architectures. e02e03e04e05e14e15e18e24
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Evaluation metrics |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded Evaluation metrics category is supported: this work defines behavioral and representation diagnostics for existing policies. Figure 2 classifies the evaluated systems, not a newly proposed world-action architecture. Architecture, prediction paradigm and quadrant are therefore not applicable to the diagnostic framework; no One Model judgment is warranted. e02e03e06
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Separate getting there from how the robot gets there
Begin with the executed trajectory. The diagnostic record preserves commands and query times, so a sudden plan change can be distinguished from variation inside a chunk. Next, check whether the episode succeeded and whether the policy actually moved: a stalled failure can lower delta, acceleration and jerk. Only then examine target progress and distractor paths. Final distance reduction can miss backward steps, while final distractor displacement can miss a disturbance that later reverses; the corresponding trajectory metrics recover those differences. RoboTwin adds another constraint because its commands are absolute joint targets rather than LIBERO-style delta actions. Reader interpretation: comparisons should first establish common tasks, command semantics and query schedules, then report success and object behavior alongside motion. A single smoothness ranking discards precisely the context this framework was designed to expose. e04e05e09e13e18e19
5.2 Turn a latent stream into a cautiously interpreted feature vocabulary
The representation analysis starts by centering and normalizing one model's hidden activations, then training a separate TopK SAE for each latent family. TopK followed by ReLU produces sparse activations; normalized decoder columns provide feature directions. Reconstruction plus auxiliary residual reconstruction encourages the basis to retain activation information. Each surviving feature is described by coverage, onset count, run length, magnitude, future consistency, horizon persistence and action predictiveness. Manual examples seed a probabilistic classifier rather than furnishing exhaustive semantic ground truth. A feature that appears only in a few layouts may be labeled memorized without being dead; that label does not itself demonstrate training-set copying. Reader interpretation: the resulting percentages summarize a particular learned basis and annotation procedure. They should not be treated as a census of all computations or proof that low-share models lack future information. e06e07e08e14
5.3 Climb from feature labels to behavioral evidence without skipping controls
The paper adds several checks with different evidential roles. Task-grouped probes ask whether predictive feature statistics add information about episode outcomes; removing statistics changes a predictor, not the robot. The compact FastWAM health check asks whether selected SAEs reconstruct their inputs and whether an auxiliary loss matters in those completed runs. Identical default and AUX-OFF results indicate an inactive auxiliary effect there, not that AuxK is generally unnecessary. Null controls then compare real FCS with random decoder directions and shuffled pairings. Real future streams exceed those controls, yet time reversal preserves much of the signal and a complete future-versus-past comparison was not run. Finally, five-seed heatmaps show selected stable features for a fixed rollout. Reader interpretation: these checks strengthen evidence of structured association while leaving temporal specificity, dictionary-wide robustness and causal control relevance open. e21e22e23e24
5.4 Training and inference
During training
SAEs subtract a learned pre-bias and sample mean, normalize activations, apply TopK then ReLU, and reconstruct through unit-normalized decoder directions. Training minimizes reconstruction plus AuxK residual loss weighted by 1/32. This trains diagnostic modules; no end-to-end policy improvement is established. e06e15
The feature classifier standardizes descriptors using seed-set statistics. Table 6 lists 45 audited examples per model, with uncertain cases excluded from training. Its LIBERO cross-validation accuracy is 90.5% on 222 labels; this validates agreement with weak annotations, not causal semantics. e08
During inference
The evaluated policies execute their own closed-loop action chunks. Cosmos is classified as joint; LingBot-VA imagines before decoding; FastWAM and VLA-JEPA use auxiliary future supervision without explicit inference-time rollout. The SAE and behavior probes analyze these policies and do not choose their executed actions. e03e06e09e21
5.5 Implementation flow
- Measure execution and object interaction
Canonicalize the actual executed stream before computing delta, acceleration, jerk and query-boundary jumps. Pair smoothness with failure and motion. Compute target progress and distractor displacement/path only where object states decode; RoboTwin reference objects are heuristically mapped and require benchmark-local interpretation. e04e05e19
- Describe sparse features
For each model and latent family, extract a separate sparse basis. Combine episode coverage, onset count, relative run length and activation magnitude with future consistency (FCS), conditional persistence (HS), and linear action predictiveness (AP). Align hidden and action timesteps through policy queries; impute unavailable metrics within SAE groups. e06e07
- Label and test associations
Manually audit activation examples and timing, then assign probabilistic types using a class-balanced logistic classifier. Rare activity is not automatically dead, and a future-stream feature is not automatically predictive. Episode-level probes compare all statistics against removal of predictive statistics using task-level GroupKFold. e08e21e27
6. Experiments & results
This paper evaluates how robot policies succeed, pairing executed-action and object diagnostics with sparse-autoencoder (SAE) feature analysis. Its strongest lesson is that success, smoothness, selectivity and inference cost measure different properties. WAM advantages vary by architecture and benchmark; predictive feature labels provide associative evidence, with unresolved definition and reporting inconsistencies limiting precise replication (e02, e11, e12, e14, e15, e18, e21, e24, e26).
6.1 Read the original evidence
Table 2. Object trajectories reveal differences hidden by similar task success. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the left-side progress columns: final progress measures distance reduction, while progress AUC averages normalized improvement throughout the trajectory. Reversals count movements away from the goal. Target path and directness then describe route efficiency. The two wrong-object columns answer a separate question: did distractors move, including motion that later reversed? Finally, grasp, lift, transport and place are coarse stage conditions, not substitutes for benchmark success. Read the direction arrows separately for each metric. Compare Cosmos with pi0.5 alongside Table 1 on the same page, where both have high success, before interpreting the object-level contrast. e05e09e10e11
What it supports. Cosmos reports distractor path 0.017 versus pi0.5 at 0.687, despite success rates of 98.0% and 96.6%. Auxiliary WAMs differ substantially too: VLA-JEPA reports 0.036 and FastWAM 0.673. These observations support measuring selectivity independently of task completion and independently of the broad WAM family label.
Where the evidence stops. Only episodes with decoded target states contribute, and their count is not supplied. LingBot-VA covers LIBERO-10 rather than the broader suite mix. No uncertainty is reported here, and the comparison does not isolate future prediction from other checkpoint differences.
Table 3. Behavioral advantages must be read alongside measured deployment cost. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the latency columns first: p50 is the median and p95 characterizes the slower tail of the measured distribution. Chunks per second counts policy output batches; effective Hz is a separate reported throughput quantity, not proof of equally frequent fresh observation-conditioned decisions. GPU memory adds another deployment constraint. The stated evaluation hardware is one RTX 6000 Ada. Compare rows within this setup, while retaining each checkpoint configuration. In particular, the table contains both explicit-imagination and auxiliary WAMs, so the latency ordering is an empirical property of these configurations rather than a direct measurement of the isolated cost of future prediction. e03e09e12
What it supports. Median latency is 100 ms for pi0.5, 956 ms for Cosmos and 4701 ms for LingBot-VA. FastWAM and VLA-JEPA report 1418 and 1588 ms. Thus the evaluated auxiliary configurations are not automatically faster than the joint WAM, even though their architecture removes explicit future rollout at inference.
Where the evidence stops. The neighboring prose attributes WAM overhead broadly to future imagination, but Figure 2 defines auxiliary WAMs as discarding that inference step. Preserve the measured total costs; this table cannot apportion them among imagination, model size, adapters and decoding configuration.
Table 5. Read predictive feature percentages together with their active-feature denominators. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each model has its own sparse dictionary, and each row identifies a representation stream. First inspect the active-feature count and dead percentage. Dead percentage uses all learned features, whereas the three type percentages use non-dead features with joint weak labels. Then compare current, future and action streams within a model before comparing models. These labels come from manually audited examples and a classifier over feature statistics; they are not direct measurements of how often a robot plans. A future-stream feature must supply future-oriented evidence to earn a predictive label. Small nonzero VLA entries should remain nonzero in the interpretation. e07e08e14e16e17e24e26
What it supports. LingBot-VA has a predictive share of 53.2% across 5,997 active future features, compared with Cosmos at 5.9% across 11,811. VLA-JEPA has an even larger action-row share, 59.8%, but only 92 active features. The pattern concerns recoverable sparse feature types, and percentages alone obscure the size and health of each dictionary.
Where the evidence stops. FCS definitions differ across sections, Table 7 contains out-of-range HS values, and aggregate feature counts are not fully reconciled. Appendix prose also calls VLA predictive features absent despite nonzero LIBERO entries. These issues limit precise feature-share comparisons.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO task success and target-object selectivity Spatial, object, goal and long suites; 50 trials per task. Object metrics use the decoded-state subset, whose size is unspecified. | Cosmos: 98.0%, 0.166, 0.017. Success (%), target progress, mean distractor path; higher success/progress and lower path are preferred. | pi0.5: 96.6%, 0.128, 0.687. VLA-JEPA distractor path: 0.036; FastWAM: 0.673. Similar success can conceal markedly different object disturbance. This is a checkpoint comparison, not an isolated future-prediction ablation; LingBot-VA covers only LIBERO-10. e09e10e11 |
| RoboTwin success and stalled-failure diagnostics Public-checkpoint subset on the stated 50-task/two-setup protocol; the reported episode total is inconsistent with the listed rollout factors. | LingBot-VA: 81.6 success, 6.1 low-motion failure. Success and all-episode low-motion failure (%). | FastWAM: 67.3/32.7; pi0.5: 44.3/53.0. Among pi0.5 failures specifically, 95.3% are low-motion. Low command variation can reflect stalling. RoboTwin absolute joint targets and differing query schedules prevent pooling these smoothness values with LIBERO. e09e13e18 |
| LIBERO deployment cost Reported evaluation on one RTX 6000 Ada GPU; released checkpoints and their inference configurations. | Cosmos 956; LingBot-VA 4701; FastWAM 1418; VLA-JEPA 1588. Median inference latency (ms), lower is faster. | pi0.5: 100 ms. These configurations expose a substantial deployment tradeoff, but timings do not isolate imagination cost from model size, adapters or decoding settings. e09e12 |
| LIBERO future-stream feature classification Per-model SAEs; predictive percentages among active non-dead features with joint weak labels. | LingBot-VA 53.2% of 5,997 active future features. Predictive-general feature share (%). | VLA-JEPA 16.9% of 2,537; FastWAM 7.3% of 246; Cosmos 5.9% of 11,811. Sequential future representations expose a large labeled predictive share. Different dictionaries and dead rates complicate ranking; sparse-label recoverability is not total predictive capacity. e08e14e26 |
| Incremental behavioral information in predictive SAE statistics Episode-level probes evaluated with task-level GroupKFold; Table 12 does not identify the exact pooled dataset/model composition. | Future stream: transport +0.116 AUC; target progress +0.095 R-squared. Delta = all-statistics score minus score without predictive statistics. | Future-stream acceleration: -0.204 R-squared. Predictive statistics help some object/task predictions while hurting others. These are probe-score differences, not robot success gains; absolute scores and uncertainty are not supplied. e21 |
| FastWAM SAE health and null-control validation LIBERO-10 representative seed-101 SAEs; Appendix C.5, separate from the main feature table. | Future: NMSE 0.001994, z_FCS 6.59. Normalized reconstruction error (NMSE) and null-normalized FCS. | Current: 0.000504 and 0.58. Default and AUX-OFF runs are numerically identical. The selected future stream exceeds shuffle/random-direction nulls, but time reversal preserves the signal. These checks establish neither a strict temporal arrow nor causal control relevance. e22 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 8. Condition on outcome before calling a policy smooth. Original paper, p. 24 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each success/failure pair before moving to another model. Delta, acceleration and jerk describe changes in the commanded joint-target stream. Boundary values concern transitions at policy queries; B./non-B. compares boundary and other steps. The last column uses the outcome-specific subset in that row. This differs from the all-episode low-motion failure metric in Table 4, which requires both failure and small motion. A low value in a motion column can therefore have two explanations: regular active control or little effective activity. Success-conditioned rows help expose that ambiguity, but they do not make different task and replanning distributions identical. e04e13e18e25
What it supports. For pi0.5, failed episodes have lower delta than successful episodes, 0.0336 versus 0.0576, and 95.3% of failures are low-motion. LingBot-VA has 32.9% low-motion failures on the same conditional definition. This supports reading aggregate smoothness together with outcome and activity rather than treating small command differences as sufficient evidence of competence.
Where the evidence stops. FastWAM's failure row is 99.8% low-motion, despite prose describing a weaker pattern than pi0.5. Query gaps also differ, and the exact motion threshold is unspecified. This is an outcome-stratified diagnostic, not a controlled architecture ablation or physical jerk measurement.
Figure 9; numerical values in Table 12. Predictive SAE statistics help some behavioral probes and hurt others. Original paper, p. 27 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The blue, green and orange bars correspond to current, future and action streams. Interpret the sign using the caption's explicit definition: delta equals the score with all statistics minus the score without predictive statistics. Although the axis says 'after removing', a positive bar means keeping those statistics helped. The probes use task-level GroupKFold. Success, transport and place use classification AUC; continuous targets use R-squared, so bar heights across different targets do not share one performance scale. Compare signs and streams within each target, and consult Table 12 on the same page for exact values rather than estimating them from bar height. e21
What it supports. Future-stream predictive statistics add 0.116 AUC for transport and 0.095 R-squared for target progress, but the acceleration delta is -0.204. The action stream adds 0.112 AUC for success while its wrong-object-path delta is -0.582 R-squared. The mixed signs support a selective diagnostic association, not a universal gain from predictive features.
Where the evidence stops. The ablation removes statistics from an offline probe, not features from the executing policy. Absolute probe scores, uncertainty and exact pooled dataset/model composition are not supplied. The bars cannot establish online prediction accuracy or causal effects on robot behavior.
7. Analysis & limitations
7.1 What the evidence leaves open
Evidence covers simulation and available checkpoints; decoded object traces restrict coverage. No matched architecture/training ablation establishes that imagination caused the behavioral differences. Recommending lightweight inference-time imagination remains an author proposal. e09e15
FCS differs between the main text and appendix: maximum over future offsets versus activation-weighted offset averaging. AP also shifts its action-segment start. Table 7 contains HS values above the defined probability range and Cosmos feature totals disagree across Tables 5 and 7. These remain unresolved, not corrected measurements. e16e24e26
Appendix prose describes absent VLA predictive features, but LIBERO Table 5 reports small nonzero rates. FastWAM failure-conditioned low-motion is 99.8%, exceeding pi0.5 at 95.3% despite the prose calling its pattern weaker. RoboTwin lists 50×2×10 rollouts but says 2,000 episodes. No missing factor is assumed. e09e17e18
C.5 omits a full future-versus-past grid and does not supply full health or threshold-sensitivity outputs; time reversal is not discriminated. Main-table FastWAM dead rates are not reconciled with the separate LIBERO-10 health checks. Five-seed visual stability concerns selected features on a fixed rollout, not whole-dictionary or behavioral robustness. e22e23e26
7.2 Questions for discussion
- Would object-selectivity differences persist on identical tasks and matched replanning schedules? (e09, e18)
- Would predictive labels survive an explicit future-versus-past comparison and causal feature intervention? (e21, e22, e24)
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs exact checkpoints, suite/task lists, decoded-object coverage, executed commands, query times, aligned latent streams and the stated GPU setup. Resolve the rollout count and LingBot-VA chunk decoding first; match horizons and query schedules before interpreting motion differences. e04e09e18
A complete recipe still needs action scales and thresholds, SAE k and optimizer schedule, exact extraction layers, temporal horizons, AP split details and the implemented FCS definition. The paper provides normalization and loss weighting but does not close these gaps. Proposed checks should audit metric consistency and compare future/past and shuffle controls on held-out tasks. e06e07e24e25e22
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does the smoothness comparison survive matched execution schedules?
Reader-proposed, not run: evaluate pi0.5 and FastWAM on the same RoboTwin tasks and initial states, retaining their checkpoints and a horizon of 32. Cross both policies with query gaps of 24 and 32, verify that logged chunks reconstruct the executed command stream, and predeclare one motion threshold with a sensitivity sweep. Report success, low-motion failure, command deltas, boundary jumps and decoded-object coverage; show successful and failed strata separately. If a policy ordering changes primarily with query gap or failure mixture, the original aggregate ordering is not stable evidence of architecture-specific control quality. A persistent gap in matched active rollouts would motivate a more targeted mechanism test. e04e09e13e18e25
Check 2: Does predictive labeling distinguish future structure from temporal persistence?
Reader-proposed, not run: on a fixed activation corpus, compute both written FCS definitions and verify binary HS stays within zero and one. Reconcile feature counts before comparing labels. For each future stream, compare realized future targets with equally distant past targets, time reversal, within-task episode shuffles and random decoder directions. Fit preprocessing and weak-label classifiers within training folds and evaluate on held-out tasks; retain the same SAE seeds and reconstruction checks across controls. If true-future alignment does not exceed matched-past alignment, or feature rankings reverse between the two FCS definitions, the predictive label needs a narrower interpretation. Report the full grid rather than only selected healthy streams. e06e07e08e16e21e22e24e26
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version block; Figures 1–11; Tables 1–13; the main method and evaluation pages; Appendix A metric definitions; Appendix B normalization, feature formulas, annotation and classifier pages; and Appendix C protocol, probe, null-control and seed-ablation pages. Figure 2 paths were checked against Sections 2–3, Figure 9 signs against its caption and Table 12, and appendix discrepancies against the printed formulas and tables. Every final crop was separately viewed with its labels intact. Reference pages 10–13 were read in the complete text pass but were not rendered; no retained method or numerical result relies on them. Selected phase and seed figures provide qualitative examples, not a re-audit of every labeled feature. External supplements, policy code and raw experiment traces were not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, author block and abstract (p. 1)
- Sections 1–2: Introduction and Related Work (pp. 2–3)
- Sections 3.1–3.3: evaluation protocols (pp. 3–5)
- Sections 4.1–4.4: setup, results and discussion (pp. 6–9)
- Section 5: Conclusion & Limitation (p. 9)
- References (pp. 10–13)
- Appendix A.1–A.2: behavioral metrics (pp. 14–16)
- Appendix B.1–B.3.3: SAE, metrics, labels, phase evidence and classifier (pp. 16–23)
- Appendix C.1–C.6: additional results, probes, null controls and seed ablation (pp. 23–29)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All eight supplied text chunks were read. The PDF visual pass resolves the extraction-only image omission: pages 1–9 and 14–29 were rendered and inspected. Reference pages 10–13 were read as text only.
- Identity: the title and three authors match the catalog; the title page expands the catalog name Hung Mai to Hung PQ. Mai. The inspected edition is arXiv v1, 31 May 2026 (e01). No other revision was supplied or compared.
- No code, checkpoints, raw rollout logs or external supplements were inspected; no experiments were reproduced. Incomplete source sentences and unreconciled table/formula differences were not filled in.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, author block and arXiv margin stamp
The title matches the supplied observed title. Authors are Hung PQ. Mai, Bin Zhu and Tuan Do. The artifact is arXiv:2606.01095v1 [cs.RO], dated 31 May 2026. The title block names National Economics University, Vietnam; N2TP Technology; Singapore Management University; and Phenikaa University, Vietnam.
Go to primary source ↓e02PDF pp. 1–2, Figure 1, abstract and Section 1
The proposed contribution combines rollout diagnostics with sparse-autoencoder analysis to examine behavior beyond task success. Figure 1 separates these two analysis branches.
Go to primary source ↓e03PDF pp. 3–4, Figure 2, Section 2 and Section 3.1
Joint WAMs predict future observations and actions together; sequential WAMs imagine before decoding actions; auxiliary WAMs remove explicit future prediction at inference. Figure 2 marks forward, auxiliary/training-only and closed-loop paths.
Go to primary source ↓e04PDF p. 4, Section 3.2; pp. 14–15, Appendix A.1
The canonical episode record includes executed actions, optional chunks, query times, state and runtime traces. Action diagnostics include finite differences, scaled boundary jumps and low-motion failure over all episodes. Threshold and scale symbols are given without a complete numerical configuration.
Go to primary source ↓e05PDF p. 4, Section 3.2; pp. 15–16, Appendix A.2
Object diagnostics use decoded target and goal positions for distance reduction, normalized progress AUC, reversals, path length and directness. Distractor path sums intermediate displacement; stage conditions concern grasp, lift, transport and placement.
Go to primary source ↓e06PDF p. 5, Section 3.3, SAE equations; p. 16, Appendix B.1
Separate TopK SAEs encode each model/latent family after learned pre-bias centering, per-sample mean subtraction and L2 normalization. Decoder columns are unit-normalized. Reconstruction and AuxK residual reconstruction are weighted with alpha=1/32.
Go to primary source ↓e07PDF pp. 17–19, Appendix B.2, feature metrics and temporal alignment
Descriptors comprise coverage, mean onsets, relative run length, mean episode-peak magnitude, FCS, HS and AP. HS is binary conditional persistence. AP is linear-regression R-squared against action segments, aligned to policy queries. Missing metrics are imputed within SAE groups; the implemented dead rule is exactly zero activation everywhere.
Go to primary source ↓e08PDF pp. 19–23, Appendix B.3, Table 6 and type–phase classifier
Manual seeds are assigned feature types using activation examples and phase evidence. Uncertain examples are excluded from training; class-balanced multinomial logistic regression uses standardized metrics. Table 6 lists 45 audited features per model. LIBERO classifier cross-validation accuracy is 90.5%, macro-F1 0.859 on 222 labels; RoboTwin is 93.8%, macro-F1 0.895 on 96 labels. Phase proxies may use normalized time.
Go to primary source ↓e09PDF p. 6, Section 4.1, complete setup paragraph
Seven policies are evaluated using one RTX 6000 Ada GPU. LIBERO uses spatial/object/goal/long suites and 50 trials per task, with LingBot-VA restricted to LIBERO-10. RoboTwin uses the public-checkpoint subset LingBot-VA, FastWAM and pi0.5. Its listed 50 tasks, two setups and 10 rollouts multiply to 1,000, although the paragraph reports 2,000 episodes.
Go to primary source ↓e10PDF p. 6, Table 1, Cosmos and pi0.5 rows, success and jerk columns
Cosmos reports 98.0% success and jerk L2 0.185; pi0.5 reports 96.6% and 0.220. The same table reports LingBot-VA success 98.0%, subject to its different suite coverage.
Go to primary source ↓e11PDF p. 6, Table 2, all model rows and object-metric headers
Cosmos target progress is 0.166, wrong-object motion 0.011 and wrong-object path 0.017; pi0.5 reports 0.128, 0.211 and 0.687. VLA-JEPA wrong-object motion/path are 0.030/0.036 and FastWAM 0.212/0.673. Object metrics require decoded target states; the caption gives no decoded-subset count.
Go to primary source ↓e12PDF p. 7, Table 3, p50, p95, chunks/s, effective-Hz and GPU columns; p. 6, Section 4.1
LIBERO median latencies in milliseconds are Cosmos 956, FastWAM 1418, LingBot-VA 4701, VLA-JEPA 1588 and pi0.5 100. LingBot-VA effective Hz is 3.460, pi0.5 70.963. Measurements use the stated RTX 6000 Ada setup; latency, throughput and effective Hz are separate reported quantities.
Go to primary source ↓e13PDF p. 7, Table 4, success and low-motion-failure columns and caption
RoboTwin success/low-motion-failure percentages are LingBot-VA 81.6/6.1, FastWAM 67.3/32.7 and pi0.5 44.3/53.0. Its dual-arm qpos targets differ from LIBERO delta-action commands.
Go to primary source ↓e14PDF p. 8, Table 5, current/future/action rows and denominator caption
Future-stream predictive percentages are LingBot-VA 53.2 over 5,997 active features, VLA-JEPA 16.9 over 2,537, FastWAM 7.3 over 246 and Cosmos 5.9 over 11,811. VLA-JEPA action predictive share is 59.8% over only 92 active features. Direct VLA predictive percentages are small but nonzero on LIBERO.
Go to primary source ↓e15PDF pp. 8–9, Sections 4.4 and 5
The authors explicitly limit interpretation to associative evidence, weak probabilistic SAE labels, decoded object traces, simulation benchmarks and public checkpoints. Lightweight inference-time imagination is a proposed design direction.
Go to primary source ↓e16PDF p. 18, Appendix B.2, binary HS formula; p. 22, Table 7, horizon-stability column
The defined binary conditional-persistence average is bounded by one, but Table 7 prints 23.282 for LIBERO FastWAM, 27.550 for LIBERO X-VLA and 39.527 for RoboTwin FastWAM. No rescaling is specified alongside these entries.
Go to primary source ↓e17PDF p. 8, Table 5, VLA rows; p. 26, Appendix C.2.1, VLA paragraph
Appendix prose calls pi0/pi0.5 predictive features absent, whereas Table 5 lists pi0 current/action 0.8%/0.5% and pi0.5 1.2%/0.8%. The report retains the table values and does not turn low rates into zero.
Go to primary source ↓e18PDF pp. 23–24, Appendix C.1, Table 8 and protocol interpretation
Table 8 reports low-motion percentages among failed episodes: pi0.5 95.3%, LingBot-VA 32.9%, FastWAM 99.8%. Successful/failed pi0.5 delta is 0.0576/0.0336. The prose calls FastWAM a weaker pattern, despite its higher failure-conditioned percentage. RoboTwin pi0.5 uses horizon/query gap 32/32; FastWAM uses 32/24. LingBot-VA stored chunks cannot be reconstructed into executed actions by simple flattening.
Go to primary source ↓e19PDF p. 24, Appendix C.1, Tables 9–10 and object-diagnostic paragraph
RoboTwin task/reference objects are mapped heuristically from task names. FastWAM has lower wrong-object path than LingBot-VA and pi0.5 in Table 9. Table 10 reports additional runtime comparisons; these are separate from LIBERO.
Go to primary source ↓e20PDF p. 26, Table 11, predictive column and denominator caption
RoboTwin future-stream predictive percentages are LingBot-VA 27.1% over 2,138 active features and FastWAM 3.9% over 440; pi0.5 current/action predictive entries are both 0.0%. These values belong to RoboTwin, not the LIBERO table.
Go to primary source ↓e21PDF pp. 26–27, Appendix C.4; p. 27, Figure 9, caption and Table 12
Episode-level probes use task-level GroupKFold, AUC for binary outcomes and R-squared for continuous outcomes. Delta is score with all SAE statistics minus score without predictive statistics. Future-stream deltas include transport +0.116 AUC, target progress +0.095 R-squared and acceleration -0.204 R-squared. Action deltas include success +0.112 AUC and wrong-object path -0.582 R-squared. The policy is not intervened on.
Go to primary source ↓e22PDF pp. 28–29, Appendix C.5, Table 13 and Figure 10
FastWAM LIBERO-10 seed-101 current/future NMSE is 0.000504/0.001994 and null-normalized FCS is 0.58/6.59. Default and AUX-OFF runs are identical. Real future FCS exceeds random-direction and shuffle controls, but time reversal does not substantially reduce it. The full future-versus-past grid was not run; full health and label-threshold-sensitivity results are not supplied here. Page 28 contains a repeated, abruptly unfinished sentence.
Go to primary source ↓e23PDF p. 29, Appendix C.6 and Figure 11
Five independently initialized SAEs are compared using selected stable top-five features on a fixed rollout for X-VLA and Cosmos. Normalized activation heatmaps show recurring patterns; they do not report whole-dictionary agreement or policy success uncertainty.
Go to primary source ↓e24PDF p. 5, Section 3.3 FCS/AP equations; pp. 17–19, Appendix B.2 FCS/AP definitions
Main-text FCS averages the maximum future cosine similarity across active events. Appendix FCS first activation-weights cosine similarity at each offset, then averages offsets. Main-text AP begins at t+1; Appendix B.2 begins its action segment at t. These are distinct written definitions, not silently interchangeable implementations.
Go to primary source ↓e25PDF pp. 14–20, Appendices A, B.1–B.3.2; p. 23, B.3.3; p. 28, C.5
The source supplies normalization, loss weighting, metric formulas and classifier coefficients, but no complete numeric recipe for action scale, motion/stage thresholds, TopK k, SAE optimizer schedule, horizon selection or per-model activation extraction. The compact health check covers FastWAM LIBERO-10 and does not fill the full configuration grid.
Go to primary source ↓e26PDF p. 8, Table 5, Cosmos active-feature rows; p. 22, Table 7, Cosmos LIBERO row; p. 28, Table 13
Cosmos active-feature entries 12,122, 11,811 and 4,093 in Table 5 do not sum to Table 7 aggregate 21,431. FastWAM Table 5 future dead percentage is 94.0%, whereas the separate LIBERO-10 checks in Table 13 report 0.000%; their run/subset relation is not reconciled.
Go to primary source ↓e27PDF p. 8, Figure 3; pp. 21–22, Figures 4–8 and phase-estimation text
Selected SAE timelines are aligned to manipulation phases. Figure 3 shows two Cosmos LIBERO-10 rollouts; appendix panels cover LingBot-VA, VLA-JEPA, X-VLA, pi0 and pi0.5. Phase evidence can fall back to normalized-time proxies and is auxiliary unless manually verified.
Go to primary source ↓8.5 Primary sources
Beyond Task Success: Behavioral and Representational Diagnostics for WAM and VLA ↗
PDF · 14,582 extracted words
Source fingerprint
18bc73d4d620342881c32620f393ef65985f1f17e583b8e317fef9a32d5242b6