WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time
1. Paper overview
In one sentence: Human-only fast-weight adaptation can steer coupled video/action prediction, but its transfer depends on a previously learned human–robot interface and incompletely specified evaluation separation. e02e03e04e05e06e09e15e17e22
| At a glance | What to know |
|---|---|
| Research problem | Source description How can a user change robot behavior by showing egocentric human video, without collecting deployment robot actions or repeatedly fine-tuning the full policy? Human motion is not directly executable; the paper learns the cross-embodiment interface before deployment. e02 |
| Core mechanism | Source description A video-side residual memory stores deployment evidence in adaptive parameters and influences action generation through the WAM’s coupled experts. e03 |
| A key reported result | Nine-task household manipulation: 46.2 Mean partial-credit progress (%). New household setting; three embodiments; 25 trials per task/method. LDA 32.5; WAM-COTRAIN 25.3; EgoScale reimplementation 15.0; pi_0.5 14.8; WAM-ICL 7.1. The reported advantage over LDA is 13.7 percentage points. Table entries show six task wins and three losses, contradicting the prose; this is not a 46.2% binary success rate. e08e09e17 |
| Reading caution | Source description Authors identify phase-alignment sensitivity, finite memory capacity, uncharacterized task-drift limits and an RGB-only human interface. e12 |
Core contributions
- Source description
A video-side residual memory stores deployment evidence in adaptive parameters and influences action generation through the WAM’s coupled experts. e03
Figure 2. Human evidence enters through a learned video-side memory interface. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the left panel from noise at the bottom to predicted action chunks and clean video at the top. The dashed enlargement points into a video block, where the blue attention path and orange fast-weight path meet at a residual addition. Use Eqs. (2)–(3) to interpret the token labels: human features supply keys and values, while robot video queries read the adapted memory. The right panel separates a frozen WAM, marked by a snowflake, from fast weights marked by a flame. Its upward deployment arrow leads to robot execution. Meta-training and deployment are different stages: the outer training loop updates the backbone, despite its later frozen status. e03e04e05e06e07
What it supports. The architecture gives action-free human supervision a route to robot behavior through shared video/action attention. It adds a residual to video tokens rather than a direct update to the action block output. The physical action chunks remain predictions of the coupled WAM, informed by the adapted visual representation.
Where the evidence stops. The figure’s deployment label says Video Loss Update, but Eq. (7) explicitly also includes key–value reconstruction. The central reconstruction graphic is schematic: Eq. (3) compares reconstructed human values with human targets, not robot values. Follow the equations and Algorithm A.1 for the computational update.
2. Motivation
2.1 The problem and the proposed response
How can a user change robot behavior by showing egocentric human video, without collecting deployment robot actions or repeatedly fine-tuning the full policy? Human motion is not directly executable; the paper learns the cross-embodiment interface before deployment. e02
2.2 What this reading follows
A human video supplies visual evidence of an interaction without specifying executable robot commands. WAM-TTT addresses that gap by first learning, from paired demonstrations, how a human-video memory should affect robot prediction. At deployment, the video updates a small fast-weight network inside the video expert; the backbone stays fixed while its action predictions respond to the altered representation. The reported household average exceeds the underlying LDA baseline, and component ablations favor the learned memory interface. Read those gains alongside the source’s limits: household scenes already appear on the human side of meta-training, progress gives partial credit, and several appendix descriptions conflict with the main formulation. e02e03e04e05e06e09e15e17e22
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Pending verification |
| Quadrant | Pending verification |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded WAM and Dual-system labels are supported by explicitly separate video/action experts coupled through joint attention. Architecturally, prediction jointly models future video and actions; the memory readout is not a separate inverse-dynamics policy. Memory and alignment tags fit. Pending prediction/quadrant fields remain unchanged for catalog review. e03e04e05
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Teach a robot query to read a human memory
The reconstruction objective starts entirely on the human side: projected human keys enter the fast-weight MLP, and projected human values are its targets. This alone does not guarantee a useful robot controller. Meta-training supplies that missing test. After the human-side update, robot video queries read the memory, its output becomes a video residual, and the coupled WAM predicts robot observations and action chunks. The outer robot loss differentiates through that sequence, training the backbone, projections and memory initialization to make the human update useful. Appendix A gives a linear-attention analogy for the memory: a linear least-squares solution can produce a key-weighted value readout under an isotropy approximation. The deployed nonlinear MLP shares the reconstruction target, but the closed-form derivation does not directly establish its retrieval behavior. e03e04e05e07e13
5.2 Separate the inner update from the rollout
There are two learning timescales. During meta-training, the inner loop starts from a learned initialization and updates fast weights using human-only losses. The outer optimizer then changes the slower parameters using paired robot supervision; the adapted example-specific weights are discarded. At deployment, those slow parameters and the initialization are fixed. Table B.1 specifies one inner step at a test-time learning rate of 0.01, after which the resulting memory stays fixed during robot rollout. This is the source’s concrete adaptation recipe, even though high-level descriptions use broader language about online absorption. The robot continues to condition action chunks on current observations, while no robot reward or action label is needed to construct the human-side update. The number and duration of deployment video clips and the action-chunk horizon remain unspecified. e04e05e06e07e14
5.3 Ask what the household gain actually generalizes across
Reader analysis: the headline comparison is strongest as evidence for improved robot deployment under the paper’s stated data exposure. Robot demonstrations are collected in a standardized cubicle, but Appendix B says the paired human demonstrations already depict the actual household evaluation scenes. The same nine task names also appear in the meta-training dataset. Consequently, calling the New results unseen-skill or fully unseen-scene transfer would exceed what the split description establishes. Separately, progress measures weighted milestones: grasping a drink earns partial credit before transport and release. This makes 46.2% average progress meaningful without turning it into a success probability. Appendix E.2’s lab rollouts without new human video address another setting, but selected frame strips provide no aggregate rate. A stronger generalization test must separate human-side scene exposure as carefully as robot-side exposure. e08e09e15e17e19
5.4 Training and inference
During training
Meta-training updates WAM parameters, slow key/value/query/output projections and the fast-weight initialization. Thus the backbone is frozen only during deployment. Robot video and action diffusion targets provide the outer supervision. e05e07
Table B.1 specifies one inner step, learning rates 0.1 in meta-training and 0.01 at test time, reconstruction weight 0.04, and 100,000 meta-training steps. e14
During inference
Run human videos in video-generation mode, update only fast weights on the combined objective, then hold them fixed while producing robot action chunks. No deployment robot labels or action-expert gradients are required. Robot feedback enters through current observations; adaptation is not described as reward-driven online learning. e06
5.5 Implementation flow
- Keep prediction and execution distinct
LDA processes video latents and action tokens in coupled diffusion experts. TTT modifies only the video block output; subsequent joint attention can alter action predictions. There is no separately specified inverse-dynamics conversion of a generated human trajectory. e03e06
- Write human evidence into memory
Project human features into keys and values. A fast-weight MLP learns their mapping through reconstruction plus human video prediction. Robot video queries later read this mapping through learned query/output projections. e04e13
- Teach the interface before deployment
Select nearest-phase human frames for robot timesteps. Adapt all layers using a full-network human pass, then evaluate the paired robot loss and differentiate through adaptation. Reset fast weights to their learned initialization between examples. e05e07
6. Experiments & results
WAM-TTT learns a memory interface that lets action-free human videos steer an LDA world-action model. Deployment adapts only video-side fast weights, then fixes that memory for robot execution. Reported household progress improves, but scene-exposure and loss-description inconsistencies limit the generalization and reproduction claims (e03–e06, e09, e15, e22).
6.1 Read the original evidence
Figure 4. Selected robot executions illustrate the demonstration-to-control interface. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start to the left of the dashed blue separator: each row contains a human demonstration, sometimes with an inset showing the egocentric view. Continue across the robot frames in the direction of the time arrow. The rows visibly cover tabletop object handling, writing or drawing, cup arrangement, and a cooking interaction. Compare the objects and intended interaction across human and robot columns rather than expecting identical hand trajectories or camera views. These are selected physical execution sequences, distinct from the video expert’s prediction target. The figure connects the abstract memory mechanism to observable behavior, while the quantitative tables answer a different question about average progress. e18e03e09
What it supports. The paper demonstrates the interface through physical robot rollouts across visibly different activities. Human and robot appearances differ, yet the paired strips show related object interactions. This supports the existence of demonstrated executions; the frames alone do not measure how frequently the same behavior succeeds over repeated trials.
Where the evidence stops. The graphic has four rows, although the p. 7 prose describes three representative tasks. The rows have no task-name labels, so the guide uses visible activity descriptions. Sparse selected frames do not establish success rates or continuous contact accuracy.
Table 1. A higher average coexists with three task-level regressions. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each column is a task in the New household setting, and each cell averages progress over 25 trials. First compare the rightmost average column, then inspect individual tasks: Table Bussing rises to 100.0 for WAM-TTT, while Stamp Paper falls to 8.3. Progress is a weighted milestone score, so a fractional result can reflect partial completion. For a fair reading, distinguish LDA without human adaptation, the co-training and in-context WAM variants, and the broader policy baselines; EgoScale is a reimplementation. Boldface is not fully reliable here: Transfer Bottle emphasizes WAM-TTT’s 55.6 even though LDA reports 56.0. e08e09e15e17
What it supports. The reported average is 46.2% versus LDA’s 32.5%, a 13.7 percentage-point gain; WAM-ICL reports 7.1%. Directly counting the cells gives six outright task wins. WAM-TTT trails on Transfer Bottle, Stamp Paper and Pyramid Stacking, so the average gain does not establish uniform improvement.
Where the evidence stops. The prose claims seven wins and a Flip Steak tie at 10.0; the table instead reports 34.3 there. Appendix B also places meta-training human videos in the actual New scenes. Treat this as the reported robot deployment comparison, with unresolved scene separation.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Nine-task household manipulation New household setting; three embodiments; 25 trials per task/method. | 46.2 Mean partial-credit progress (%) | LDA 32.5; WAM-COTRAIN 25.3; EgoScale reimplementation 15.0; pi_0.5 14.8; WAM-ICL 7.1. The reported advantage over LDA is 13.7 percentage points. Table entries show six task wins and three losses, contradicting the prose; this is not a 46.2% binary success rate. e08e09e17 |
| Nine-task cubicle manipulation Orig. standardized cubicle; 25 trials per task/method. | 61.1 Mean partial-credit progress (%) | LDA 50.2; WAM-ICL 48.4. The separate Orig. protocol supports an in-cubicle improvement; it does not resolve household training-scene exposure. e16e15 |
| Table Bussing and Swap Place memory ablation New; 10 trials per task/configuration. | 100.0 / 88.9 Progress (%), Table Bussing / Swap Place | No meta-training 9.0 / 0.0; no reconstruction 66.7 / 72.0; no TTT 40.0 / 74.1; LoRA 30.0 / 0.0. Supports the calibrated interface and reconstruction component. These trial sets differ from the 25-trial main table; no uncertainty is reported. e10 |
| Deliver Drink under lighting and spatial shifts Section 4.4 perturbation evaluation; Table 3 gives no separate trial count. | 66.0 / 56.0 Reported progress (%), lighting / spatial | LDA 54.0 / 28.0; WAM-ICL 12.0 / 20.0. Evidence covers two quantitative perturbation rows, not every shift illustrated in Figure 5 or the appendix galleries. e11e19 |
| Robot/human data-ratio comparison Three New tasks; 25 trials per configuration/task; episode counts per task. | 74.1 at (100 robot, 100 human) Mean progress (%) | 73.7 at (200,0); 51.4 at (10,190); 59.5 at (100,0); 73.3 at (100,200). The matched-count comparison is suggestive, but statistical equivalence and equal collection cost are not established. e20 |
| Table Bussing VLM architecture ablation New; 10 trials per configuration; TTT pipeline retained. | 100.0 with pretrained VLM open during meta-training Progress (%) | DiT-only 72.0; unpretrained VLM 80.0; frozen VLM 54.0. Supports meta-training the conditioning features within this small experiment; deployment still freezes them. e21e06 |
| Human pseudo-action supervision Four New tasks; 25 trials per configuration/task. | 72.3 for the row labeled VG only Mean progress (%) | 28.9 with retargeted pseudo-actions plus forward-dynamics loss. This changes both processing and supervision, so it does not isolate tracking noise. The VG-only label conflicts with the main method’s combined loss. e22 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2. The paired memory interface matters in the small protocol ablation. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read horizontally within each task. The full method includes paired meta-training, reconstruction and deployment adaptation. WAM-LoRA substitutes a generic low-rank adaptation mechanism; the next two columns remove the learned alignment stage or the reconstruction objective; the final column removes human-video adaptation. The particularly low no-meta-training scores show that a memory branch does not automatically make human evidence useful for control. Compare no reconstruction with full WAM-TTT to assess the structured write objective. Keep this experiment separate from Table 1: its New-setting cells use 10 trials, and its full Swap Place score is 88.9 rather than the main table’s 66.7. e10e09
What it supports. Full WAM-TTT reports 100.0/88.9 on Table Bussing/Swap Place, versus 66.7/72.0 without reconstruction and 9.0/0.0 without meta-training. This supports learning the interface before deployment and including the reconstruction term. The no-TTT scores, 40.0/74.1, further suggest a benefit from adaptation in these runs.
Where the evidence stops. Ten trials per cell and no reported uncertainty limit precision. Removing meta-training changes more than a single readout operation. The table contains no clean video-prediction-loss removal, and its no-TTT row should not be substituted for LDA’s separate main-table results.
Table E.1. Human demonstrations may reduce robot-data needs, with uncertainty unresolved. Original paper, p. 26 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The first column gives robot and human episodes per task. The top row fixes a robot-only reference at 100 episodes. Within the ruled middle group, all three configurations total 200 episodes, so compare the balanced 100/100 allocation against 200 robot-only and the human-heavy 10/190 allocation. The final row adds human data without adding robot data. Each task/configuration uses 25 New-setting trials; the average spans only Transfer Bottle, Table Bussing and Deliver Drink. Notice the task tradeoff: the balanced allocation has better Transfer Bottle progress, while the robot-only 200-episode allocation has better Deliver Drink progress. e20e15
What it supports. The balanced allocation reports 74.1% mean progress, close to 73.7% for 200 robot episodes, and above 59.5% for 100 robot episodes alone. Reducing robot data to 10 episodes gives 51.4%. These displayed means motivate testing partial substitution while retaining substantial action-grounded training.
Where the evidence stops. The source calls 74.1 and 73.7 statistically indistinguishable without a reported test or uncertainty. Equal episode counts do not establish equal collection cost. Appendix B’s 2,286 paired episodes and the stated deployed 100/100-per-task subset are not reconciled.
Table E.3. Added pseudo-action supervision harms this reported configuration. Original paper, p. 28 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare rows within each task before looking at the average. The lower row adds a complete human-hand processing chain: wrist estimation, MANO hand fitting and embodiment-specific retargeting generate pseudo-actions. A forward-dynamics loss then predicts the next observation in frozen DINOv3 feature space, weighted by one. The four tasks span the paper’s three embodiments, and each configuration/task uses 25 New-setting trials. The row labels describe the source’s ablation terminology; they do not resolve whether the baseline also retained memory reconstruction. That point matters because the main method explicitly includes reconstruction in its human-side adaptation objective. e22e06
What it supports. The displayed average decreases from 72.3% to 28.9%, and all four task columns decline. Table Bussing shows the largest decrease, from 100.0 to 33.3. The evidence supports rejecting this particular pseudo-action-plus-loss configuration; it does not establish that every form of action supervision would be harmful.
Where the evidence stops. Appendix E.6 repeatedly says VG-only, conflicting with Eq. (7)’s reconstruction term. The comparison changes both retargeting and the objective, so attribution to pose noise alone is an author interpretation. The supplied source gives no independent clean-pose control.
7. Analysis & limitations
7.1 What the evidence leaves open
Authors identify phase-alignment sensitivity, finite memory capacity, uncharacterized task-drift limits and an RGB-only human interface. e12
New is a robot-scene shift, but Appendix B places meta-training human videos in those actual evaluation scenes. All nine task names also appear in meta-training. Strictly unseen-scene or held-out-skill generalization is therefore unverified. e08e15
Frozen parameters prevent direct weight overwriting, not behavior changes caused by altered residuals. Appendix A’s linear-memory derivation is not a proof that the deployed nonlinear MLP performs exact attention or preserves all skills. e03e06e13
Source inconsistencies remain: Table 1 contradicts its win/tie prose; Appendix C mislabels difference signs and co-training rank; Appendix E.6 omits reconstruction from its stated baseline. Qualitative galleries establish examples, not aggregate robustness. e09e16e19e22
7.2 Questions for discussion
- Would benefits survive holding out household scenes on both human and robot sides (e15)?
- Does better human reconstruction predict robot progress after controlling for video-prediction loss (e04, e10)?
8. Reproducibility audit
8.1 Requirements and known gaps
Requires the LDA checkpoint and inherited diffusion configuration, paired episodes, phase synchronization and embodiment-specific control. Table B.1 lists Qwen3-VL-4B-Instruct, a 16-block/1536-hidden/32-head MMDiT, TTT head size 48, fast hidden width 128, AdamW, global batch 128 and eight H800 GPUs with ZeRO-2. e05e14e15
Resolve which collected episodes enter the asserted 100/100-per-task configuration and whether reconstruction remains active in E.6. Exact deployment clip count/length, action horizon, control rate, latency, software versions, random seeds and uncertainty estimates are not specified in the relevant configuration/results. e06e14e15e20e22
Implement the published milestone rubrics, but obtain missing scoring thresholds and reconcile Swap Place’s handover instruction with its object-swap rubric before claiming comparable progress. e17
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does the memory carry skill information beyond scene appearance?
Reader-proposed check, not performed: use the same meta-trained checkpoint and matched Table Bussing starts. Compare no update, correctly matched human clips, and wrong-skill clips recorded in the same scene, with equal clip length and adaptation compute. Cross the clip conditions with video-only versus video-plus-reconstruction updates to test the main formulation and the conflicting E.6 description explicitly. Preserve the stated one-step budget and slow-parameter freeze. Score both the published grasp/bin-release progress and full completion over repeated trial sets. If wrong-skill clips help equally, or reconstruction only lowers its own loss without improving execution, that would weaken the claim that the memory stores control-relevant skill evidence. e04e06e10e14e17e22
Check 2: Hold out the scene on both sides of meta-training
Reader-proposed check, not performed: rebuild the episode manifest so a household scene is absent from both human and robot meta-training data. Keep task identities, episode budgets, training schedule and robot initial-state distribution matched to a scene-exposed control. On the held-out scene, compare frozen meta-trained weights, WAM-ICL and WAM-TTT using the same newly collected human clips for the latter two. Report per-task progress, full completion, repeated-run uncertainty and whether any training clip depicts that scene. Persistence of the TTT advantage would strengthen scene-transfer evidence; a large collapse relative to the exposed-scene control would bound the original New-setting interpretation. Reconcile the 2,286-episode collection with the 100/100 configuration before allocating splits. e08e09e14e15e17e20
8.3 Reading coverage
Visual audit: All non-reference pages of the 28-page v2 PDF were rendered and visually inspected: title/authors, Figures 1–5, method equations, Algorithm A.1, Table B.1, collection figures, full results, all progress rubrics, and Appendix E figures/tables. References on pp. 10–13 were read as text. All six final original crops were separately viewed at their retained dimensions. Figure 2 was checked against Eqs. (1)–(8) and Algorithm A.1; its abbreviated update label is disclosed. All pages supporting numerical, training, inference, evaluation and proposed-check details are included. Native-rate supplemental videos and code were not supplied or reviewed.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title and abstract (p. 1)
- 1 Introduction (p. 2)
- 2 Related Work (pp. 2–3)
- 3 Method, architecture, meta-training and deployment (pp. 4–5)
- 4 Experiments and 5 Conclusion/Limitations (pp. 5–9)
- References (pp. 10–13)
- A Meta-training algorithm and derivation (pp. 14–16)
- B Hyperparameters and datasets (pp. 15–17)
- C Full main results (pp. 17–18)
- D All nine progress rubrics (pp. 19–21)
- E.1–E.6 Additional results and ablations (pp. 21–28)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- The reviewed artifact is arXiv:2607.06988v2, dated 10 July 2026. The catalog lists 9 July 2026 as submission date; v1 was not supplied, so changes between revisions cannot be established. Title and author order agree; the catalog affiliation string omits Galbot and CASIA (e01).
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This acquisition limitation was resolved by inspecting the original PDF pages and final crops.
- Separate supplemental material availability has not been fully verified.
- Code, linked references, native-rate supplemental videos and raw trial data were not supplied or inspected; no experiments were reproduced. All seven supplied text chunks were read without truncation. References were read as text; all non-reference PDF pages were also visually inspected.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, author/affiliation block and arXiv margin
The observed title matches the catalog. All 16 authors agree in order. The artifact is arXiv:2607.06988v2, 10 July 2026. Affiliations are Peking University, Galbot, CASIA and Tsinghua University; the catalog affiliation string omits Galbot and CASIA. The supplied catalog submission date is 9 July 2026; the earlier version was not supplied for comparison.
Go to primary source ↓e02PDF pp. 1–2, Abstract and Section 1
WAM-TTT proposes human-video steering through deployment-time fast-weight memory, preceded by paired human–robot meta-training. The authors motivate reusable adaptation without human pose labels or deployment robot demonstrations.
Go to primary source ↓e03PDF p. 3, Figure 2; p. 4, Section 3.1, Eqs. (1)–(2)
LDA has video and action experts coupled by joint attention. TTT adds a projected fast-weight readout to video tokens only; the action block receives no direct TTT residual. Figure 2 shows a frozen WAM and trainable fast weights at deployment, but abbreviates the update as Video Loss Update.
Go to primary source ↓e04PDF p. 4, Section 3.2, key/value definitions and Eqs. (3)–(5)
Human features produce keys and values; robot video features produce queries. Fast weights minimize human video prediction plus lambda times per-layer human key–value reconstruction. Reconstruction is normalized by batch size, human token count and embedding dimension.
Go to primary source ↓e05PDF p. 5, Section 3.2, Eq. (6) and Human-robot data synchronization
Robot video/action diffusion targets supervise the outer loop, which differentiates through adaptation and updates WAM parameters, slow projections and the fast-weight initialization. Adapted weights reset between examples. Human frames are paired to robot time by nearest normalized phase t/T_r.
Go to primary source ↓e06PDF p. 5, Section 3.3, Eqs. (7)–(9)
At deployment the human-only objective explicitly includes both video prediction and key–value reconstruction. Only fast weights update. WAM parameters, slow projections and initialization are frozen; adapted weights remain fixed during action-chunk rollout conditioned on o_t and g.
Go to primary source ↓e07PDF p. 16, Algorithm A.1, steps 1–4
Each inner iteration runs the full human-side network, computes combined losses, and updates all layers simultaneously. The robot pass then uses adapted weights, followed by outer backpropagation through the inner update.
Go to primary source ↓e08PDF p. 5, Section 4.1; p. 6, Dataset and Metric and Baselines
Nine real-robot tasks span Unitree G1, Galbot gripper and Galbot sharpa. Orig. uses the training cubicle; New jointly changes lighting, table height and objects. Main cells average partial-credit progress over 25 trials. The paper reports 2,286 paired episodes and egocentric GoPro human video. Baselines include LDA, co-training, in-context conditioning, an EgoScale reimplementation and pi_0.5.
Go to primary source ↓e09PDF p. 7, Table 1, all rows and Avg. column; Section 4.2 quantitative-results paragraph
Reported New averages are WAM-TTT 46.2, LDA 32.5, WAM-COTRAIN 25.3, EgoScale 15.0, pi_0.5 14.8 and WAM-ICL 7.1 percent. WAM-TTT loses Transfer Bottle (55.6 versus LDA 56.0), Stamp Paper (8.3 versus 33.3), and Pyramid Stacking (10.4 versus co-training 12.0); Flip Steak is 34.3 versus co-training 34.2. Counting these displayed cells gives six outright task wins, conflicting with the prose claiming seven wins and a Flip Steak tie at 10.0. Transfer Bottle is also incorrectly emphasized in the WAM-TTT row.
Go to primary source ↓e10PDF p. 7, Section 4.3; p. 8, Table 2, both task rows and caption
New-setting protocol ablations use 10 trials per cell. For Table Bussing/Swap Place, full TTT reports 100.0/88.9, LoRA 30.0/0.0, no meta-training 9.0/0.0, no memory reconstruction 66.7/72.0, and no TTT 40.0/74.1 percent progress. The source does not give a corresponding no-video-loss-only row.
Go to primary source ↓e11PDF pp. 7–8, Section 4.4 and Figure 5; p. 9, Table 3
Deliver Drink lighting/spatial results are WAM-TTT 66.0/56.0, LDA 54.0/28.0, WAM-ICL 12.0/20.0 and pi_0.5 28.0/0.0. The table reports only those two perturbations; Figure 5 illustrates additional setup changes. A separate trial count is not stated for Table 3.
Go to primary source ↓e12PDF p. 8, Section 5, Limitations
Authors identify sensitivity to incorrect phase pairing, limited fast-weight/projection expressiveness under task drift, an empirically uncharacterized adaptation boundary, and an RGB-only egocentric human interface without pose, contact or 3-D cues.
Go to primary source ↓e13PDF pp. 14–15, Appendix A, Eqs. (A.1)–(A.4) and surrounding derivation
The deployed fast-weight memory is a nonlinear MLP. Appendix A derives a least-squares linear-memory solution and an attention-like readout under an isotropy approximation. The closed-form equalities apply to the linear special case, not directly to the deployed nonlinear network.
Go to primary source ↓e14PDF p. 16, Table B.1, all settings; Appendix A, p. 14, outer-loss notation
The configuration lists Qwen3-VL-4B-Instruct plus a DiT-L MMDiT head; 16 blocks, hidden size 1536, 32 heads, TTT head dimension 48 and fast hidden width 128. One inner step uses learning rates 0.1 during meta-training and 0.01 at test time; lambda is 0.04. Meta-training uses AdamW, 100,000 steps, global batch 128 and eight NVIDIA H800 GPUs with DeepSpeed ZeRO-2. The exact outer loss, weights and noise schedule are inherited from LDA rather than restated.
Go to primary source ↓e15PDF pp. 16–17, Appendix B, Embodiments and datasets; Figures B.1–B.2 and captions
The 2,286 paired episodes are split as 600 Unitree G1, 544 Galbot gripper and 1,142 Galbot sharpa. Robot demonstrations come from the cubicle; paired human demonstrations are recorded in the actual household scenes later used for New evaluation, with human video re-recorded at deployment. The nine task names are shared between the described meta-training dataset and evaluation; a held-out-task partition is not specified.
Go to primary source ↓e16PDF pp. 17–18, Appendix C; Table C.1, Orig./New and Avg. columns; transfer summary
Orig. averages are WAM-TTT 61.1, LDA 50.2 and WAM-ICL 48.4 percent, versus New 46.2, 32.5 and 7.1. The transfer summary labels its difference Orig. minus New but prints negative differences. Its prose calls co-training second-worst on New, whereas the Avg. column ranks 25.3 third among six methods.
Go to primary source ↓e17PDF pp. 19–21, Appendix D, progress definition and all nine rubrics
Progress adds task-specific milestone weights and is averaged across trials; it is not binary success. Table Bussing assigns half the per-item credit to grasp and half to bin release; Deliver Drink weights grasp/transport/release 0.30/0.30/0.40. Rubrics cover all nine tasks, with automatic scoring from end-effector and known object poses. Numerical pose thresholds are not provided. Swap Place has a handover instruction but an object-swap milestone list.
Go to primary source ↓e18PDF p. 6, Figure 4 and caption; p. 7, Qualitative Results
Figure 4 places human demonstrations on the left and four-frame robot execution strips on the right, with time increasing rightward. Four visible rows depict tabletop object handling, writing/drawing, cup stacking and cooking. The prose on p. 7 describes three representative tasks, whereas the graphic contains four rows.
Go to primary source ↓e19PDF pp. 21–25, Appendix E.1–E.3, Figures E.1–E.3 and captions
The appendix shows selected robot rollouts, including extra writing/drawing/wiping demonstrations outside the quantitative benchmark. E.2 shows lab-scene transfer after meta-training without new in-scene human videos; E.3 instead adapts on in-scene human videos under six perturbation axes. These galleries do not provide per-axis aggregate success rates.
Go to primary source ↓e20PDF pp. 25–26, Appendix E.4; Table E.1, all rows and Takeaways
For Transfer Bottle/Table Bussing/Deliver Drink under New, 25 trials per cell, reported average progress is 59.5 for (100,0), 51.4 for (10,190), 74.1 for (100,100), 73.7 for (200,0), and 73.3 for (100,200) robot/human episodes per task. The deployed 100/100 description is not reconciled with Appendix B episode counts. The prose asserts statistical indistinguishability and collection-cost savings without reporting an uncertainty estimate, test or cost measurement.
Go to primary source ↓e21PDF p. 26, Appendix E.5; Table E.2, configuration rows and caption
The New Table Bussing architecture ablation uses 10 trials per configuration. DiT-only scores 72.0, unpretrained VLM 80.0, frozen VLM 54.0 and pretrained VLM open to meta-training 100.0 percent. All variants retain the meta-training/test-time TTT pipeline.
Go to primary source ↓e22PDF pp. 27–28, Appendix E.6; Figure E.4; Table E.3, both rows and caption; compare p. 5, Eq. (7)
The pseudo-action variant combines MediaPipe/EgoMimic-style estimation, MANO fitting and EgoScale-style retargeting, then adds a DINOv3-feature forward-dynamics loss weighted by 1. On four New tasks with 25 trials per cell, the reported averages are 72.3 for VG only and 28.9 for VG+FD. The latter changes both supervision and the processing pipeline. Appendix E.6 repeatedly describes the baseline as video-loss-only, in conflict with the explicit reconstruction term in Eq. (7).
Go to primary source ↓8.5 Primary sources
WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time ↗
PDF · 11,986 extracted words
Source fingerprint
73058de8991363cdaae87380d875de77449e2929a5f871fdcdff26da8a7c7228