Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Control
1. Paper overview
In one sentence: Feedback steering can improve a pretrained WAM's robustness when nuisance features occupy a transferable low-dimensional subspace, but its gains depend on task geometry and calibration. problemgeometrycontrollercosmos-cameracosmos-grippercosmos-noiseling-results
| At a glance | What to know |
|---|---|
| Research problem | Source description Video-based robot policies remain brittle to observation and initial-state changes. The paper asks whether their internal representations expose steerable robustness features that can improve executed task success without updating model weights. problemcosmos-cameracosmos-grippercosmos-noiseling-results |
| Core mechanism | Source description A three-dimensional PCA/SVM diagnostic measures how separable contrastive activation sets are, revealing dependence on model, perturbation and task. The authors propose separability as a predictor of steerability. geometrycorrelation |
| A key reported result | Cosmos-Policy camera-orientation robustness: WA-LQR 59.3% ± 4.0% Average task success rate. LIBERO-10 simulation; calibration Task 0, evaluation Tasks 0, 2, 4, 5, 9; 30 trials/task Unsteered 46.0% ± 4.1%; prompt steering 46.7% ± 4.1%; ActAdd 49.3% ± 4.1% A calculated 13.3 percentage-point gain over unsteered, across the selected group including its calibration task. ActAdd still wins individual Tasks 2 and 4. cosmos-camera |
| Reading caution | Source description The authors cannot predict transferable task groupings without setting-specific mechanistic analysis. Weak LingBot-VA results and selected LIBERO task groups limit claims of architecture-independent or universal robustness. scope-limitsling-results |
Core contributions
- Source description
A three-dimensional PCA/SVM diagnostic measures how separable contrastive activation sets are, revealing dependence on model, perturbation and task. The authors propose separability as a predictor of steerability. geometrycorrelation
- Source description
The paper adapts fixed activation addition to WAMs and develops WA-LQR: a reduced-order feedback controller that penalizes feature error and intervention magnitude while leaving pretrained weights unchanged. actadddynamicscontroller
Figure 2. The geometry of the disturbance changes with both the backbone and the transformer layer. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read panel (a) on the left as Gaussian-noise contrast and panel (b) on the right as camera contrast, following the original caption. Each model row moves from the first block to a selected best intermediate block and then the last block. Blue and orange identify positive and negative activation sets; the gray plane is the fitted SVM separator. PC1–PC3 are coordinates obtained from contrastive differences, not physical robot coordinates. The loss boxes quantify margin separation. These three-dimensional diagnostic plots should also be distinguished from the rank-64 subspaces used to construct the implemented controller. geometrycontrastive-dataprobe-setupcosmos-implementation
What it supports. Cosmos-Policy's noise conditions separate clearly in early blocks, while DiT4DiT develops separation after its initial block. Camera conditions show different geometry. This provides a reason to diagnose each perturbation and intervention site before applying a common activation-steering recipe, rather than assuming that the presence of a video backbone guarantees steerability.
Where the evidence stops. This figure concerns Task 0 and selected layers. It does not establish transfer to every task. Appendix A defines camera contrast by perturbation, whereas Appendix D.1 describes success/failure labels for mechanistic camera analysis and reports similar findings with the alternative labeling; the plotted labels alone cannot resolve that distinction.
2. Motivation
2.1 The problem and the proposed response
Video-based robot policies remain brittle to observation and initial-state changes. The paper asks whether their internal representations expose steerable robustness features that can improve executed task success without updating model weights. problemcosmos-cameracosmos-grippercosmos-noiseling-results
2.2 What this reading follows
A shifted camera can leave a robot policy unable to complete a familiar task. This paper looks inside the policy for activation directions associated with the disturbance, then asks whether changing those activations restores successful behavior. Its central progression is diagnostic: measure feature separation, build a compact local model of activation propagation, and use feedback to limit the intervention. The strongest reading is conditional. WA-LQR improves several selected LIBERO-10 groups, while a simpler fixed addition wins Cosmos-Policy's Gaussian-noise average and LingBot-VA remains difficult to steer. The illustrations connect those outcomes to the controller's assumptions and tuning requirements. problemgeometrycontrollercosmos-cameracosmos-grippercosmos-noiseling-results
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Not applicable |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded Outside quadrants / Other mechanisms classification fits an inference-time steering method spanning existing architectures. Appendix D explicitly distinguishes Cosmos-Policy's unified backbone from separate video/action modules in LingBot-VA and DiT4DiT. WA-LQR itself introduces neither a new unified predictor nor an inverse-dynamics policy. Its efficiency association is algorithmic; measured real-time performance remains unestablished. backbonecontrollercosmos-implementationmodular-implementation
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Choose a contrast that isolates the desired behavior
The first design decision is what positive and negative mean. For image noise, clean and corrupted observations provide a direct nuisance contrast. For a changed starting gripper position, an altered input can still yield a successful rollout, so Appendix A compares successful and failed perturbed rollouts. Otherwise the steering vector could push away from useful behavior. Once activations are collected, subtract negatives from positives and examine their geometry before injecting anything. The three-PC diagnostic and the controller's rank-64 SVD serve different purposes: the former tests simple separation; the latter retains a richer subspace for control. Figure 6 reports rapid contrastive-spectrum decay, but that concentration alone does not prove that every important action feature is preserved. Transfer still requires identifying tasks whose representations align. contrastive-datageometryrankcosmos-implementationscope-limits
Figure 5. Local activation predictions and shared dynamical subspaces motivate reusable feedback gains. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with panel (a). For camera-orientation perturbations, the upper curves compare directions of actual projected block outputs and their first-order approximations; the lower curves compare magnitudes. Values near one indicate agreement, with visible deviations showing where the approximation is imperfect. Colors identify five LIBERO tasks, and the horizontal index combines transformer layer l and denoising timestep t. Panel (b) compares subspaces spanned by the top 16 right singular vectors of reduced dynamics matrices from 25 inputs across five tasks. The three labeled layer/timestep heatmaps have much stronger off-diagonal overlap than the random reference; the color bar reports that overlap. linearitydynamicscontroller
What it supports. The controller has empirical support for two useful approximations: local activation propagation can be linearized, and important dynamical directions can remain similar across inputs. This helps explain why the authors can reuse reduced dynamics from a calibration input for other tasks instead of rebuilding a full activation-space controller online.
Where the evidence stops. These are activation-space diagnostics, not bounds on physical trajectory error. The tests do not prove global linearity, stability under arbitrary steering strength, or preservation of all task-relevant information. The figure's shaded regions are not identified here as confidence intervals.
5.2 Follow feedback through a single denoising pass
At a transformer block, project the realized activation into the learned contrastive subspace. Its inner product with the unit contrastive direction gives the current feature strength. WA-LQR compares that strength with the target set by lambda and the norm of the mean contrastive difference. A Riccati gain maps the resulting error to an additive block intervention, using a local linear model of activation propagation and penalties on state error and control effort. Repeat this through the layer horizon for the current denoising timestep. The scheduler then produces the next latent input and another controller horizon begins. Thus the feedback signal is internal feature error, while the existing WAM still decodes actions. The paper increases intervention cost over chunk indices to taper steering; it does not solve one combined layer-and-scheduler LQR horizon. backbonedynamicscontroller
5.3 Separate a transfer result from a universal controller claim
Read each calibration-to-task arrow before reading a bold success rate. The Cosmos camera average combines the source task with four additional tasks, and the representation groups were found empirically. That is evidence of useful transfer within the tested grouping, but it leaves open performance on tasks excluded from grouping and tuning. Then compare mechanisms within a perturbation: feedback leads camera and gripper averages, whereas ActAdd leads Cosmos noise performance. The authors hypothesize that especially clear noise separation makes fixed addition sufficient. Table 5 adds a qualification: the successful fixed strength has a sharp optimum. Finally, use LingBot as a boundary case rather than deleting its weak results. The same steering idea is implemented across differing modules and schedules, so the architecture comparison motivates further controlled experiments without isolating one architectural cause. cosmos-cameracosmos-grippercosmos-noisesensitivityling-resultsmodular-implementationscope-limits
5.4 Training and inference
During training
Training-free refers to preserving WAM weights. Offline preparation still collects contrastive activations, fits PCA/SVM and SVD representations, constructs local dynamics and precomputes Riccati gains. Appendix D.2 reports an LQR hyperparameter search before applying selected settings to other tasks of the same scene; its validation split is not specified. actaddprobe-setupcosmos-implementation
During inference
For each denoising timestep and chunk index, WA-LQR solves an independent horizon over transformer layers. Online feature error determines each additive correction. The scheduler then initializes the next denoising pass. Increasing the control penalty with chunk index weakens later interventions. This is feedback over internal activations; the pretrained policy still produces the robot actions. backbonecontroller
5.5 Implementation flow
- Define positive and negative examples
Appendix A uses clean/noised images for noise, nominal/perturbed views for camera changes, and successful/failed perturbed rollouts for gripper changes. Appendix D.1 instead describes success/failure labels for both camera and gripper mechanistic analysis, reporting similar findings with nominal/perturbed labels. These descriptions should remain distinct. contrastive-dataprobe-setup
- Diagnose activation geometry
Mean-pool token positions and action-chunk positions, compute positive-minus-negative differences, and fit PCA to those differences. Fit a linear SVM to projected activations in the top three PCs and report average hinge loss. This avoids the uninformative separability possible when activation dimension greatly exceeds sample count. geometryprobe-setup
- Construct the control subspace
Streaming randomized SVD of contrastive differences produces an orthonormal basis. Project activations into this reduced space and obtain local inter-block dynamics using Jacobian-vector or vector-Jacobian products, without materializing full activation Jacobians. Normalize the mean latent contrastive direction to define a feature-strength setpoint. actadddynamicscontroller
- Respect model-specific intervention sites
Cosmos-Policy jointly denoises actions and future proprioception, wrist images and third-person images; interventions exclude action-chunk outputs. DiT4DiT uses separate video/action modules and is steered in its action module. LingBot-VA separately tests video or action steering while holding the other module unchanged. cosmos-implementationmodular-implementation
6. Experiments & results
WA-LQR modifies pretrained world-action-model activations during inference to improve simulated manipulation under camera, gripper and image-noise perturbations. Contrastive examples identify nuisance-related directions; local activation dynamics support feedback corrections along those directions. Success depends on model and task geometry. Cosmos-Policy benefits most consistently on camera and gripper shifts, while open-loop ActAdd wins its Gaussian-noise average. LingBot-VA shows weak overall gains. These findings support selective activation steering, with calibration and transfer limits still unresolved.
The source explains the controller architecture in equations and implementation prose and supplies no standalone architecture block diagram. Original feature-separation and local-linearity figures therefore provide the mechanism visuals. Table 5 is an ActAdd sensitivity study and Figure 7 a correlation diagnostic; the supplied source has no controlled WA-LQR component-removal ablation or physical-robot experiment. These distinctions limit what the illustrated mechanism and ablation sections can establish. backbonecontrollercosmos-implementationmodular-implementationsensitivitycorrelationscope-limits
6.1 Read the original evidence
Table 1. WA-LQR leads the camera and gripper averages; ActAdd leads the noise average. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each perturbation block separately. Task i to Task j means that the projection matrices, reduced dynamics and contrastive directions are computed from Task i and then used to steer Task j; the caption specifies 30 trials per task. The group therefore includes both its calibration task and several transfer tasks. Compare the four method columns within the same row before examining the average. Camera calibration uses Task 0, gripper calibration Task 1 and noise calibration Task 6. These groups were selected using shared representations, so the displayed averages are not all-ten-task LIBERO averages or estimates for arbitrary unseen environments. cosmos-cameracosmos-grippercosmos-noisecorrelationcosmos-implementation
What it supports. For camera shifts, WA-LQR reaches 59.3% versus 46.0% unsteered; for gripper shifts, 72.7% versus 61.3%. Noise gives a different ordering: ActAdd reaches 67.3%, WA-LQR 58.7% and unsteered 26.7%. The table supports useful feedback steering while preserving a substantial case where fixed addition works better.
Where the evidence stops. The paper does not define the ± statistic, so it should not be relabeled as a confidence interval or seed standard deviation. The camera and gripper gains are calculated percentage-point differences. Neither the averages nor boldface establish that WA-LQR wins every task.
Tables 2 and 3. Steering gains differ substantially between the two models with separate video and action modules. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The upper table is DiT4DiT, with gripper and noise evaluations. The lower table is LingBot-VA, adding camera-orientation evaluations. Read the average lines within each model and perturbation; these are different task groups and should not be pooled into one cross-model score. The arrow notation records calibration-to-evaluation transfer. Appendix D identifies the intervention site as DiT4DiT's action module; LingBot's action results are shown here, with video-module alternatives in Appendix B. LingBot groups were borrowed from Cosmos because comparable task groupings did not emerge in its mechanistic analysis. Appendix Table 4 states 20 trials/task for LingBot; Table 2 does not state a DiT4DiT count. dit-resultsling-resultsmodular-implementationscope-limits
What it supports. DiT4DiT's noise average rises from 15.6% unsteered to 48.9% with WA-LQR, exceeding ActAdd's 43.3%. LingBot-VA changes from 48% to 51% for camera, 72% to 75% for gripper and 59% to 56% for noise. These outcomes support architecture-dependent applicability without establishing that architecture alone causes the difference.
Where the evidence stops. DiT4DiT's unsteered gripper Task 1 entry is printed as 69.6% with no uncertainty; preserve it. The models also differ in intervention sites, schedules and task grouping. A weak aggregate gain is not proof that every individual rollout is unsteerable.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Cosmos-Policy camera-orientation robustness LIBERO-10 simulation; calibration Task 0, evaluation Tasks 0, 2, 4, 5, 9; 30 trials/task | WA-LQR 59.3% ± 4.0% Average task success rate | Unsteered 46.0% ± 4.1%; prompt steering 46.7% ± 4.1%; ActAdd 49.3% ± 4.1% A calculated 13.3 percentage-point gain over unsteered, across the selected group including its calibration task. ActAdd still wins individual Tasks 2 and 4. cosmos-camera |
| Cosmos-Policy initial-gripper robustness LIBERO-10 simulation; calibration Task 1, evaluation Tasks 1, 2, 3, 7, 9; 30 trials/task | WA-LQR 72.7% ± 3.6% Average task success rate | Unsteered 61.3% ± 4.0%; prompt steering 60.7% ± 4.0%; ActAdd 63.3% ± 3.9% A calculated 11.4 percentage-point gain over unsteered; the group average does not establish improvement on every trial or task. cosmos-gripper |
| Cosmos-Policy Gaussian-noise robustness LIBERO-10 simulation; calibration Task 6, evaluation Tasks 0, 1, 4, 6, 7; 30 trials/task | WA-LQR 58.7% ± 4.0% Average task success rate | Unsteered 26.7% ± 3.6%; prompt steering 0.7% ± 0.7%; ActAdd 67.3% ± 3.8% WA-LQR gains 32.0 percentage points over unsteered; ActAdd gains 40.6 and exceeds WA-LQR by 8.6. These calculated differences qualify the paper's broad improvement headline. cosmos-noise |
| DiT4DiT Gaussian-noise robustness LIBERO-10 simulation; calibration Task 1, evaluation Tasks 0, 1, 2; Table 2 does not state a trial count | WA-LQR 48.9% ± 5.3% Average task success rate | Unsteered 15.6% ± 3.8%; ActAdd 43.3% ± 5.2% The calculated gain is 33.3 percentage points over unsteered; ActAdd wins Task 1. The same table reports gripper averages of 71.7% ± 4.1%, 65.7% ± 4.3% and 65.8% ± 4.3% for WA-LQR, unsteered and ActAdd. dit-results |
| LingBot-VA perturbation robustness LIBERO-10 simulation; five-task groups mapped from Cosmos-Policy; 20 trials/task stated in Table 4 | WA-LQR: camera 51.0% ± 5.0%; gripper 75.0% ± 4.3%; noise 56.0% ± 5.0% Average task success rate, action-module steering | Unsteered: 48.0% ± 5.0%, 72.0% ± 4.5%, 59.0% ± 4.9%, respectively Changes of +3, +3 and −3 percentage points show limited aggregate benefit. Table 4 reports the same averages for video steering despite different task-level outcomes. ling-results |
| ActAdd strength sensitivity Cosmos-Policy Gaussian noise; Task 6 directions transferred across five tasks; Appendix C | 67.4% at gamma=0.1; 32.6% at 0.15; 0.0% at 0.2, 0.25 and 0.5 Mean success rate versus gamma | 26.6% at gamma=0; 50.8% at 0.05 A sharp tuning dependence supports concern about oversteering. Table 5 provides no uncertainty and differs slightly from Table 1's corresponding 67.3% and 26.7%; neither table is silently corrected. sensitivitycosmos-noise |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 5. A useful contrastive direction can still fail when its fixed injection strength is poorly chosen. Original paper, p. 19 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The left column is gamma, the scalar multiplier in ActAdd's activation update. Every row uses directions acquired from Cosmos-Policy Task 6 under Gaussian-noise corruption and applies them to the same five tasks. Read down the mean column to locate the narrow high-performing region, then across individual task columns to see that sensitivity differs by task. Gamma zero supplies the no-addition reference. The table changes steering strength, not the identity of the contrastive feature. It is a sensitivity analysis of ActAdd and does not remove WA-LQR components or test the sensitivity of its setpoint and control penalties. sensitivityactaddcontrollercosmos-noise
What it supports. The reported mean rises from 26.6% at zero to 67.4% at gamma 0.1, then falls to 32.6% at 0.15 and zero at 0.2, 0.25 and 0.5. More steering is therefore not consistently better. This motivates checking whether feedback reduces damaging interventions away from the calibrated operating point.
Where the evidence stops. No uncertainty is reported. Table 5's 26.6% and 67.4% differ slightly from the corresponding 26.7% and 67.3% in Table 1; they remain table-specific values. This sweep alone cannot establish that WA-LQR is insensitive to its own hyperparameters.
Figure 7. Lower separability loss is associated with larger steering gains in the tested collection. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The vertical coordinate is hinge loss from the activation-separation diagnostic: lower values indicate a stronger linear margin between positive and negative examples. The horizontal coordinate is the success-rate difference between steered and unsteered evaluations. Points to the right of the dashed zero line represent improvement; points to the left represent degradation. Blue marks WA-LQR and green marks ActAdd. The downward fitted line and its reported coefficient summarize a pooled association across tasks and models. Individual points are not labeled by task, so the exact setting behind a dot must not be inferred from position or color alone. correlationgeometryscope-limits
What it supports. The reported coefficient is r=−0.63, consistent with the proposed use of separability as a screening diagnostic. Yet the scatter contains a broad range of gains at comparable losses. A diagnostic that helps choose where to try steering does not automatically determine the right controller, strength or transferable task group.
Where the evidence stops. This is a correlation diagnostic, not a causal ablation or held-out prediction test. The fitted line extends below zero even though Eq. (7) makes hinge loss nonnegative; read it as a descriptive trend, not a valid loss prediction throughout the plotted range.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors cannot predict transferable task groupings without setting-specific mechanistic analysis. Weak LingBot-VA results and selected LIBERO task groups limit claims of architecture-independent or universal robustness. scope-limitsling-results
Figure 7 reports correlation r=−0.63 between hinge loss and steering gain. This association does not establish that separability causes improvement; task grouping, intervention sites and tuning also differ. Figure 5 tests local activation approximations, not global nonlinear stability or physical safety. correlationlinearitycosmos-implementationmodular-implementation
The ± quantities are reproduced as printed; the paper does not define their statistical meaning. Table 2's unsteered gripper Task 1 entry is 69.6% with no uncertainty. No real-robot evaluation, controlled WA-LQR component ablation or measured end-to-end latency benchmark is supplied. cosmos-cameradit-resultsling-resultssensitivityscope-limitscontroller
7.2 Questions for discussion
- Does separability predict gains on tasks excluded from both grouping and controller tuning?
- Does feedback retain its advantage when open-loop controls receive equal calibration and intervention budgets?
8. Reproducibility audit
8.1 Requirements and known gaps
Required ingredients include the named pretrained WAMs, LIBERO-10, matched calibration rollouts, activation hooks, rank-64 SVD, Jacobian products and a Riccati solver. Cosmos uses 28 blocks and 5 denoising steps; DiT4DiT uses 16 action layers and 4 steps. LingBot uses 30-layer modules, grouped into 0–9, 10–19 and 20–29. cosmos-implementationmodular-implementation
Exact perturbation magnitudes, calibration sample counts, selected Q/R/lambda settings, tuning splits, seeds, hardware and software versions are not specified in the supplied experimental details. Proposed checks should separate calibration from evaluation and compare feedback with intervention-matched open-loop controls; these are reader proposals, not reproduced results. contrastive-datacontrollercosmos-implementationmodular-implementationsensitivity
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test feedback with matched calibration and intervention budgets
Reader-proposed experiment, not performed: use the Cosmos Task 0 camera group and keep checkpoint, perturbations, seeds, contrastive pairs and intervention sites identical. Tune ActAdd and WA-LQR on a separate calibration split with equal search budgets, then freeze all choices. Add an open-loop replay control that injects a stored WA-LQR correction schedule independent of the current activation, with matched intervention energy. On held-out rollouts measure task success, feature-tracking error and intervention norms. If feedback is the decisive mechanism, online WA-LQR should reduce feature error and damaging corrections more consistently than replay under changed observations. If matched replay performs equally well, the proposed feedback explanation needs qualification. controllercosmos-cameracosmos-implementationsensitivity
Check 2: Predict steerability before inspecting target-task outcomes
Reader-proposed experiment, not performed: reserve target tasks before selecting task groups or controller hyperparameters. Fit the PCA/SVM diagnostic on separate calibration rollouts, select any hinge-loss threshold only on source tasks, and freeze the controller construction and tuning policy. Test whether the diagnostic ranks held-out steering gains for Cosmos-Policy, DiT4DiT and LingBot-VA better than model identity and unsteered success alone. Include random-direction and shuffled-label controls with the same projection rank and intervention budget. Report per-task outcomes and uncertainty, not only a pooled correlation. A loss–gain relationship that disappears after this separation would challenge the diagnostic's prospective transfer value even if the original retrospective association remains reproducible. geometryprobe-setupcorrelationscope-limitscosmos-implementationmodular-implementation
8.3 Reading coverage
Visual audit: The title/authors/version page, all main-text method and results pages, Appendix A contrast definitions, Appendix B module results, Appendix C sensitivity table and Appendix D implementation pages were rendered and visually inspected. Figures 1–8 and Tables 1–5 were viewed in their page context; all six final crops were separately viewed. Crop explanations were cross-checked against captions, Eqs. (3)–(15) where relevant, and Appendix D intervention-site descriptions. Figure 7's negative fitted-line extension is identified as a descriptive fit, and camera-label differences between Appendices A and D.1 remain explicit. All retained method, numerical and reproduction details are supported on these declared pages. Appendix E–F plot text and captions were read in full, but their images were not individually inspected; linked code, videos and separate supplements remain outside the visual pass.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 17, 18, 19, 20, 21. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title page and abstract, p. 1: arXiv:2607.14943v1 [cs.RO], 16 July 2026; title and all five authors match the catalog; Hong and Skifstad share equal contribution. No different revision or edition was supplied.
- Sections 1–3: Introduction, Related Work, Preliminaries and Problem Statement, pp. 1–4
- Section 4: A Mechanistic Study for Interpreting WAMs, pp. 4–6
- Sections 5.1–5.2: ActAdd and WA-LQR, pp. 6–8
- Sections 6.1–6.2 and 7: activation properties, robustness evaluations, discussion and limitations, pp. 8–11
- References, pp. 12–16
- Appendices A–D.4: contrastive vectors, LingBot-VA evaluations, sensitivity and implementation details, pp. 17–21
- Appendices E–F: all supplied separability-plot text, labels and captions, pp. 22–44
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The complete supplied text was read. Visual inspection covers PDF pages 1–11 and 17–21, including every page supporting retained scientific details; appendix plot images on pages 22–44 were not individually inspected.
- Code, linked websites and videos were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title block and arXiv version stamp
The exact catalog title appears with Jihoon Hong, Julian Skifstad, Qiyue Dai, Alice Chan and Glen Chou; Georgia Institute of Technology; Hong and Skifstad have equal-contribution stars. The stamp is arXiv:2607.14943v1 [cs.RO], 16 Jul 2026.
Go to primary source ↓problemPDF p. 2, Section 1, motivation and contributions
The paper studies camera, initial-gripper and Gaussian-image perturbations and proposes mechanistic analysis plus weight-preserving inference-time steering across three WAMs.
Go to primary source ↓backbonePDF pp. 3–4, Section 3, Eqs. (3)–(5)
The scheduler chains denoising passes through transformer layers; additive control enters the block output. The existing decoder produces latent-action outputs.
Go to primary source ↓geometryPDF pp. 4–6, Figure 2 and caption; Section 4, Eqs. (6)–(7); Figures 3–4
Contrastive differences define the PCA basis; a three-PC linear SVM assesses mean hinge loss. Figure 2 contrasts noise and camera conditions at first, best and last blocks for Cosmos-Policy and DiT4DiT. Figure 3 shows shared Cosmos noise clusters; Figure 4 shows weaker LingBot camera separation.
Go to primary source ↓contrastive-dataPDF p. 17, Appendix A
Noise pairs use clean/noised images, camera pairs use unperturbed/perturbed views, and gripper pairs use successful/unsuccessful rollouts under perturbation to avoid treating successful perturbed behavior as undesirable.
Go to primary source ↓probe-setupPDF p. 20, Appendix D.1, Eq. (15)
Mechanistic camera/gripper datasets use success/failure labels, with nominal/perturbed alternatives reportedly giving similar findings. The three-PC SVM is fit by gradient descent from a zero hyperplane with regularization constant C=10.
Go to primary source ↓actaddPDF p. 6, Sections 5.1–5.2, Eqs. (8)–(10) and Dimensionality Reduction
ActAdd averages contrastive activation differences over paired examples and action-chunk positions and adds gamma times this vector. WA-LQR instead constructs a low-dimensional subspace by streaming randomized SVD.
Go to primary source ↓dynamicsPDF p. 7, Section 5.2, Eq. (11) and preceding definitions
The basis defines P=V-transpose and z=Px. Reduced dynamics use A-tilde=P-next A P-transpose and B-tilde=P-next B, computed with Jacobian products rather than explicit full Jacobians.
Go to primary source ↓controllerPDF pp. 7–8, Section 5.2, Eqs. (12)–(14) and scheduler continuation
Mean projected contrastive differences define the unit feature direction and desired strength. Independent layer-horizon LQR problems penalize state/intervention errors for each denoising timestep and chunk index. The penalty r(tau)=min(R-final,R-init exp(tau/tau-R)) suppresses later steering. Feedback gains multiply the online feature error; scheduler transitions connect passes.
Go to primary source ↓cosmos-implementationPDF p. 20, Appendices D.1–D.2
Cosmos-Policy has one unified backbone jointly denoising actions and future proprioception/images. Steering covers latent outputs except action chunks. Activation collection spans 28 blocks and 5 denoising steps; SVD rank is 64; gains use backward Riccati recursion. A hyperparameter search selects settings for other tasks of the same scene, without listing final settings or a validation split.
Go to primary source ↓modular-implementationPDF pp. 20–21, Appendices D.3–D.4
LingBot-VA predicts video then actions; either module is steered separately. Its 30 layers are pooled in three ten-layer groups, with sampled action steps 0/10/20/30/40 and video steps 0/4/9/14/19. DiT4DiT steering targets its action module across 16 layers and 4 steps, with layer groups 0–5, 6–10, 11–15. Both use rank-64 bases. Hardware, software versions and exact perturbation magnitudes are not listed.
Go to primary source ↓linearityPDF pp. 8–9, Section 6.1 and Figure 5(a–b)
Projected nonlinear and first-order activation outputs are compared by cosine similarity and magnitude ratio. Subspace overlap compares the top 16 right singular vectors of reduced dynamics matrices over 25 inputs from 5 tasks, using mean squared cosines of principal angles and a random reference.
Go to primary source ↓rankPDF p. 8, Figure 6 and Section 6.1
The camera-perturbation contrastive spectrum in Cosmos-Policy is concentrated: the figure annotates cumulative variance 0.5 at rank 3, 0.9 at rank 6 and 0.99 at rank 19 within the studied rank-64 projection.
Go to primary source ↓cosmos-cameraPDF p. 9, Table 1, caption and Camera Orientation rows
Thirty trials/task; Task 0 calibration transfers to Tasks 0/2/4/5/9. Average success is 46.0±4.1%, 46.7±4.1%, 49.3±4.1%, 59.3±4.0% for unsteered, prompt, ActAdd, WA-LQR. ActAdd exceeds WA-LQR on Tasks 2 and 4. The ± statistic is not defined in the table.
Go to primary source ↓cosmos-gripperPDF p. 9, Table 1, Initial Gripper Position rows
Task 1 calibration transfers to Tasks 1/2/3/7/9. Average success is 61.3±4.0%, 60.7±4.0%, 63.3±3.9%, 72.7±3.6% for unsteered, prompt, ActAdd, WA-LQR, respectively.
Go to primary source ↓cosmos-noisePDF p. 9, Table 1, Camera Gaussian Noise rows
Task 6 calibration transfers to Tasks 0/1/4/6/7. Average success is 26.7±3.6%, 0.7±0.7%, 67.3±3.8%, 58.7±4.0% for unsteered, prompt, ActAdd, WA-LQR, respectively.
Go to primary source ↓dit-resultsPDF p. 10, Table 2, both perturbation groups and T0→T1 gripper row
DiT4DiT gripper averages are 65.7±4.3%, 65.8±4.3%, 71.7±4.1%; noise averages are 15.6±3.8%, 43.3±5.2%, 48.9±5.3%, for unsteered, ActAdd and WA-LQR. Noise Task 1 favors ActAdd. The unsteered gripper T0→T1 value is 69.6% without ±; the caption omits trial count.
Go to primary source ↓ling-resultsPDF p. 10, Table 3 and Section 6.2; pp. 17–18, Appendix B and Table 4
Groups are inherited from Cosmos-Policy. Table 4 specifies 20 trials/task. Unsteered camera/gripper/noise averages are 48.0±5.0%, 72.0±4.5%, 59.0±4.9%; WA-LQR action and video averages are each 51.0±5.0%, 75.0±4.3%, 56.0±5.0%. Task-level outcomes differ across modules.
Go to primary source ↓correlationPDF p. 9, Figure 7, axes and legend; p. 10, Section 6.2
The figure plots hinge loss vertically and steered-minus-unsteered success-rate change horizontally. Its fitted line slopes downward and the legend reports r=−0.63. Blue/green distinguish WA-LQR/ActAdd; points are not individually task-labeled.
Go to primary source ↓sensitivityPDF p. 19, Appendix C and Table 5, Mean column
Task 6 contrastive directions steer five Cosmos tasks under Gaussian noise. Mean success for gamma 0/0.02/0.05/0.075/0.1/0.15/0.2/0.25/0.5 is 26.6/36.2/50.8/52.6/67.4/32.6/0/0/0 percent. No uncertainty or WA-LQR sensitivity sweep appears.
Go to primary source ↓scope-limitsPDF p. 10, Section 7, Limitations; p. 11, Figure 8 and caption
Authors identify setting-specific task-transfer analysis and architecture dependence as limitations. Qualitative results select LIBERO-10 rollouts where steering succeeds and unsteered execution fails; they are not an unbiased success-rate estimate or physical-robot study.
Go to primary source ↓8.5 Primary sources
Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Control ↗
PDF · 14,561 extracted words
Source fingerprint
e11eb6aacf2a81fe62488337d3e1b0ee926b325ed36b634b5713b2a772e36965