TacWAM: Anchor-Guided World Action Model with Mechanics-Aware Tactile Prediction
1. Paper overview
In one sentence: TacWAM combines mechanics-supervised tactile latents and recent contact history with a tri-modal generator whose attention lets tactile futures supervise learning while keeping them out of action inputs. e-probleme-generatore-safe-maske-maine-ablatione-force
| At a glance | What to know |
|---|---|
| Research problem | Source description Visual similarity can conceal increasing pressure, slip, or deformation. The paper asks how tactile futures can teach contact-sensitive representations while preserving the information available during deployment. Its tactile latent is explicitly an observable representation, not a complete physical state. e-probleme-saf |
| Core mechanism | Source description SAF aligns tactile appearance, dense force, and deformation flow before bilateral fusion. Reconstructing local fields and each sensor’s resultant force and torque encourages mechanics-relevant latents. e-safe-saf-implementation |
| A key reported result | Four-task real-world manipulation: 75.0% mean: chip 90.0%, cherry 70.0%, wiping 75.0%, twirling 65.0%. Task success rate; mean across four tasks. Agilex Piper with two Xense G1-WS tactile sensors; same demonstrations and epochs across methods, 20 trials per method/task, one final checkpoint. Test poses/configurations vary within unspecified predefined ranges. Mean success: VT-WAM 37.5%, RDP 32.5%, π0.5 31.3%, Fast-WAM 20.0%. TacWAM exceeds the strongest per-task baselines by 40, 10, 40, and 30 percentage points respectively. These are executed robot outcomes under task-specific constraints, including avoiding breakage/crushing/drops and sustaining wiping contact. The mean gain over VT-WAM is 37.5 percentage points. No confidence intervals or multi-seed retraining are reported. e-maine-protocole-platform |
| Reading caution | Reader analysis Only four tasks on one platform and one checkpoint per method are evaluated. RDP and VT-WAM are author reimplementations. Shared demonstrations do not make pretraining, batch size, or compute identical; generalization to new sensors, objects, or platforms is untested. e-protocole-baselines |
Core contributions
- Source description
SAF aligns tactile appearance, dense force, and deformation flow before bilateral fusion. Reconstructing local fields and each sensor’s resultant force and torque encourages mechanics-relevant latents. e-safe-saf-implementation
- Source description
A temporal encoder conditions tactile forecasting, while AGT limits action attention to current visual/tactile anchors and action tokens. Future tactile prediction remains a parallel training objective. e-generatore-historye-mask
- Author claim
The authors demonstrate higher success on four contact-rich tasks and consistent degradation across a nested sequence of history and attention changes. e-maine-ablation
Figure 1. Three prediction streams share one generator, with touch encoded as a mechanics-supervised latent. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start in panel (a): the green video, blue action, and yellow tactile paths enter modality-specific DiT experts coupled by the MoT joint attention layer. Text/state tokens provide current task and proprioceptive context. The purple history pathway summarizes recent touch and modulates the tactile expert; its contact-state head supplies an auxiliary learning signal. Panel (b) expands the tactile input: appearance passes through a Wan VAE, while force and deformation use CNN branches before spatial alignment and bilateral fusion. The implementation yields one 128-dimensional latent per tactile step. Finally, panel (c) shows that training and deployment retain restricted access to the current anchors. e-generatore-safe-saf-implementatione-historye-stage2
What it supports. The design gives tactile prediction a structured target space: local force and flow are fused with appearance, and decoder reconstruction preserves bilateral force/torque information. These components belong to one tri-modal generator. History conditions the tactile expert, and future tactile states supervise training without becoming direct action inputs.
Where the evidence stops. The drawing is an architectural summary, not an isolated test of SAF. Wrench is reconstruction supervision, not an extra spatial sensor input; the history contact-event head should not be confused with the separate force/flow/wrench decoder.
2. Motivation
2.1 The problem and the proposed response
Visual similarity can conceal increasing pressure, slip, or deformation. The paper asks how tactile futures can teach contact-sensitive representations while preserving the information available during deployment. Its tactile latent is explicitly an observable representation, not a complete physical state. e-probleme-saf
2.2 What this reading follows
A robot can appear to hold an object securely while contact force is rising toward damage or the object is beginning to slip. TacWAM uses future tactile prediction to teach a world-action model about this hidden contact evolution. Its central choices concern what to predict and who may read the prediction tokens: tactile appearance, force, and deformation define a learned target space, while an attention mask separates future sensory targets from action generation. The reported robot results favor this combination, but the staged ablations and representative force curves leave important causal and calibration questions open. Read the architecture, visibility rules, and evidence boundaries together. e-probleme-generatore-safe-maske-maine-ablatione-force
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Conflicts with the recorded classification
The tactile/multimodal WAM label is supported. However, Section 3.2 explicitly specifies one tri-modal generator with modality experts and shared masked attention, supporting a One Model architectural assessment rather than the recorded Dual-system label. This is joint sensory/action predictive co-training without inverse-dynamics extraction; tactile forecasting is not action-conditioned. The recorded Other mechanisms and Outside quadrants labels remain preserved for editorial review. Efficient inference is not established by a latency measurement. e-generatore-maske-inference
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 First define what a tactile future means
TacWAM does not predict a single raw sensor stream. Each sensor supplies appearance, a dense force field, and deformation flow with different structures. SAF aligns their spatial features before fusing the two gripper sides into a single-step latent. This makes the observed current tactile state and future prediction targets live in the same representation. Pretraining asks a decoder to recover force fields, flow, and a bilateral wrench, so the representation is supervised to retain mechanical information beyond visual appearance. The wrench supplies global force and torque supervision rather than an additional encoder input. After pretraining, both encoder and decoder are frozen. During generator training, however, gradients still travel through the decoder to penalize mechanically inconsistent predicted latents. Thus freezing the decoder stabilizes the target interpretation without removing its supervisory role. e-safe-saf-implementatione-stage1e-stage2
Table 5. Read each row as a permission list for the query stream. Original paper, p. 16 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the bottom row: an action query may read the current visual anchor V0, current tactile anchor T0, and action tokens A. The dashes under visual and tactile futures are the essential restriction. In the middle row, tactile prediction can read V0 and its own tactile sequence, but it cannot read actions or future video. The top row keeps video processing within the visual stream. A Y is permission subject to the within-stream rules, not unrestricted attention between every token. In particular, the tactile anchor reads only itself within tactile self-attention. Language and proprioception arrive through separate cross-attention. e-maske-generatore-relaxede-ablation
What it supports. The matrix makes the forecasting role precise. Because tactile futures cannot read candidate actions, TacWAM does not learn an action-conditioned tactile consequence model in this finalized design. Because actions cannot read future sensory tokens, those targets can support predictive co-training without supplying direct future information to the action branch.
Where the evidence stops. This stream-level table must be interpreted with the token-level anchor rules in Appendix B.1. The visibility constraints alone do not prove better control; the staged robot ablations provide the empirical comparison, with their own confounds.
5.2 Separate temporal context from access to future targets
At the start of a chunk, the model has recent tactile observations but no observed future. The history encoder compresses 32 tactile latents into a context vector that modulates the tactile expert through adaptive normalization. That context is not appended as attention memory for the action expert. Meanwhile, future tactile training variables are noised versions of teacher targets. If actions can read those variables, they may exploit information that differs from generated futures available at deployment. AGT blocks this direct route: actions read current visual and tactile anchors and other action tokens. Future tactile tokens also cannot read action tokens, so the learned forecast does not answer what would happen under a candidate action. These restrictions explain why the paper treats tactile forecasting as a parallel learning objective rather than a planning simulator or online corrective controller. e-historye-maske-relaxede-generator
5.3 Connect robot success to the limits of the diagnostics
Reader analysis: evaluate the control claim and the forecasting claim separately. For control, the robot predicts target joint positions, executes an action chunk, then refreshes observations. Table 2 measures physical task completion under that loop, with a 75.0% four-task average. Table 3 narrows the scope to chip grasping and wiping; its 82.5% full-model average is therefore compatible with Table 2 rather than a second overall result. For forecasting, Figure 4 displays selected decoded force traces, which can reveal lag and amplitude errors but cannot establish an aggregate accuracy gain. The staged ablations connect the two claims only partially: removing history also removes auxiliary supervision, and attention relaxations are tested after that removal. A useful reproduction must control these changes independently before attributing the task-success gain to a specific temporal or information-isolation mechanism. e-inferencee-maine-ablatione-forcee-history
5.4 Training and inference
During training
Stage 1 trains SAF and its decoder with SmoothL1 reconstruction on normalized force, wrench, and flow targets, weighted 1.0, 1.5, and 1.0. The appearance VAE stays frozen. Stage 2 freezes SAF and the decoder while training the WAM generator and history encoder; gradients still pass through the fixed decoder. e-saf-implementatione-stage1e-stage2
Stage 2 uses scheduler-weighted flow-matching MSE for video, action, and tactile latents; decoded consistency uses normalized-target MSE, and contact events use BCE-with-logits. Tactile losses exclude the clean anchor. Initialization follows Wan2.2-TI2V-5B with ActionDiT initialization for action/tactile experts. e-stage2e-history
Both stages use AdamW, learning rate 10⁻⁴, cosine decay, weight decay 10⁻², 20 epochs, bfloat16, and seed 42. Stage 2 uses eight A800 80GB GPUs, batch eight per GPU, accumulation two, and 1,000 flow timesteps per stream. Demonstrations comprise 300 episodes per task, about 500 frames each, synchronized at 30 Hz. e-stage2e-platforme-data
During inference
At chunk start, encode current observations and recent touch, then denoise with 20 flow-matching steps. The action/tactile horizon is 32; video subsampling yields nine frames including the current frame. Each 7-dimensional action is a target joint-position state. Execute one chunk, obtain new observations, and predict again. Decoded tactile futures support analysis, with no online correction role. e-inference
The 30 Hz figure describes synchronized sensing/control streams; the paper supplies no measured end-to-end generation latency establishing a 30 Hz policy update rate. e-platforme-inference
5.5 Implementation flow
- Construct a mechanics-supervised tactile state
A frozen Wan VAE supplies appearance features. Force and flow CNN features are bilinearly aligned to its grid; shared left/right encoders feed bilateral fusion, pooling, and a 128-dimensional readout. The single-frame decoder reconstructs both 35×20×3 fields per sensor and a 12-dimensional bilateral wrench. Wrench is a supervision target, not a fourth spatial input. e-safe-saf-implementation
- Summarize recent contact evolution
A residual temporal Conv1d encoder processes 32 tactile latents, using last/mean/max/standard-deviation pooling to form a 256-dimensional context. Adaptive normalization injects it only into the tactile expert. A six-label contact-event head supervises current/any contact, onset, release, and stable contact/noncontact; stability uses eight recent steps. e-history
- Enforce stream-specific visibility
One Mixture-of-Transformers generator contains modality-specific experts joined by masked mixed self-attention. Language and proprioception enter through cross-attention. Action queries read only current visual anchor V0, clean tactile anchor T0, and action tokens. Future tactile queries read V0 and tactile tokens, but no action tokens or visual futures. Thus tactile forecasting is not an action-conditioned consequence model. e-generatore-mask
6. Experiments & results
TacWAM trains one visual–tactile–action generator to predict contact-relevant tactile latents alongside visual futures and robot action chunks. A mechanics-supervised encoder, recent tactile history, and restricted attention define how this supervision reaches the learned system. Actions read current anchors, not future sensory tokens. Four real-robot tasks yield 75.0% mean success versus VT-WAM’s 37.5%; staged ablations support the complete design but leave individual mechanisms only partly isolated.
6.1 Read the original evidence
Table 2. TacWAM has the highest reported success rate in each of the four task columns. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read down each task column before using the average: the strongest comparator differs by task, so the best mean baseline is not always the best local baseline. Chip refers to fragile chip grasping and placement; cherry requires stable placement without visible crushing; wiping requires sustained board contact; twirling uses the paper's stated pen-motion criterion. Each cell summarizes 20 physical trials of one final checkpoint. Methods use the same demonstrations, epochs, and test-condition sequence, with tactile streams available only to tactile methods. RDP and VT-WAM are author reimplementations, and their input and training configurations should be considered when interpreting the ranking. e-maine-protocole-baselinese-task-discrepancy
What it supports. TacWAM reports 75.0% mean success compared with VT-WAM's 37.5%, a 37.5-percentage-point gain. Its four task rates are 90.0%, 70.0%, 75.0%, and 65.0%. The taskwise improvements over the strongest comparator are 40, 10, 40, and 30 percentage points, so the largest gains are not uniform across tasks.
Where the evidence stops. No confidence intervals or training-seed variance are supplied, and equal epochs do not mean equal compute. Figure 2's single-pen placement prompt also differs from the text's two-pen twirling description, leaving the exact pen protocol unresolved.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Four-task real-world manipulation Agilex Piper with two Xense G1-WS tactile sensors; same demonstrations and epochs across methods, 20 trials per method/task, one final checkpoint. Test poses/configurations vary within unspecified predefined ranges. | 75.0% mean: chip 90.0%, cherry 70.0%, wiping 75.0%, twirling 65.0%. Task success rate; mean across four tasks | Mean success: VT-WAM 37.5%, RDP 32.5%, π0.5 31.3%, Fast-WAM 20.0%. TacWAM exceeds the strongest per-task baselines by 40, 10, 40, and 30 percentage points respectively. These are executed robot outcomes under task-specific constraints, including avoiding breakage/crushing/drops and sustaining wiping contact. The mean gain over VT-WAM is 37.5 percentage points. No confidence intervals or multi-seed retraining are reported. e-maine-protocole-platform |
| Staged history and attention ablation on chip grasping and wiping Same Stage 2 data, objectives apart from the stated removals, horizons, and optimizer; 20 real trials per task. Only chip and wiping enter this average. | Full: 90.0% / 75.0% / 82.5%; no history: 50.0% / 60.0% / 55.0%; no history + Attn-AT: 30.0% / 45.0% / 37.5%; no history + Attn-VT: 10.0% / 5.0% / 7.5%. Success rate, chip / wiping / two-task mean | History removal costs 27.5 percentage points in the two-task average; successive visibility relaxations cost another 17.5 and 30.0 points. History removal also disables contact-event supervision. Attention relaxations are evaluated only after history removal. The authors’ shortcut explanation is plausible, but this nested design does not isolate every component or interaction. e-ablatione-relaxede-stage2 |
| Episode-level tactile force forecasting Representative wiping and two-pen twirling episodes; decode predicted latents to bilateral wrench and resultant force magnitude. | TacWAM follows several contact transitions and force peaks more closely than the history-free model; substantial underestimation remains. Qualitative force-trajectory agreement | Figure 4 overlays ground truth, TacWAM, and no-history predictions; it supplies no aggregate forecast-error metric or uncertainty. The curves are diagnostic evidence about forecasting. They do not establish calibrated mechanics or feedback correction, and their force axis has no stated physical unit. e-force |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 3. The ablation average covers chip and wiping only. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the rows in order because the changes accumulate. The full model uses history and AGT. The second row removes the history encoder, substitutes zero context, and disables the contact-event loss. The third retains that history-free setup and introduces Attn-AT, letting actions read the entire tactile prediction sequence. The fourth retains Attn-AT and adds bidirectional exchange between visual and tactile futures through Attn-VT. The two task columns are physical success rates over 20 trials each. Their average is therefore a two-task summary, distinct from the four-task average in Table 2. Other Stage 2 objectives and optimization settings are retained as described. e-ablatione-relaxede-historye-stage2
What it supports. Mean success declines from 82.5% to 55.0% when the history pathway and its auxiliary loss are removed, then to 37.5% and 7.5% under successive visibility relaxations. This supports the complete design's value on these tasks. The especially severe final row motivates testing the authors' proposed training/deployment mismatch explanation.
Where the evidence stops. This is a nested sequence, not a factorial experiment. It cannot separate history from its auxiliary loss or establish how relaxed attention behaves with history present. The shortcut explanation is an author interpretation; no independent shortcut-intervention test is reported.
Figure 4. Representative decoded force traces reveal both improved timing and remaining prediction errors. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the upper wiping episode and lower twirling episode separately; their frame axes cover different episode lengths. Blue is ground truth, red is TacWAM, and green is the history-free variant. The plotted signal is obtained by decoding tactile latents to bilateral resultant wrench and summarizing force magnitude, rather than directly plotting an action command. Compare the timing of contact rises and releases first, then whether predicted amplitudes match the blue trace. The wiping curve illustrates delayed transitions without history. In twirling, compare the two pronounced peaks and the sustained-force plateaus around them. Keep both legends and axes in view when making these comparisons. e-forcee-inference
What it supports. TacWAM follows several contact transitions and the major twirling peaks more closely than the history-free variant. However, its wiping peak is visibly too small, and parts of the twirling trace miss sustained ground-truth force. The visual evidence supports improved representative forecasts while also showing that temporal alignment and amplitude accuracy remain imperfect.
Where the evidence stops. These are representative episodes with no aggregate error, uncertainty, or explicit physical unit on the force axis. Appendix I identifies the decoded predictions as diagnostics; they are not used to correct robot actions online.
7. Analysis & limitations
7.1 What the evidence leaves open
Only four tasks on one platform and one checkpoint per method are evaluated. RDP and VT-WAM are author reimplementations. Shared demonstrations do not make pretraining, batch size, or compute identical; generalization to new sensors, objects, or platforms is untested. e-protocole-baselines
No independent SAF-input ablation, tactile-future-loss removal, or factorial history-by-mask comparison is reported. The staged results support the combined system more strongly than a unique causal explanation for each gain. e-ablatione-relaxed
The source leaves contact-force threshold values, wrench/flow clipping bounds, data partition details, numerical test-pose ranges, and the wiping cleanliness threshold unspecified. It reports neither training-seed variance nor confidence intervals/significance tests. e-saf-implementatione-historye-datae-protocol
Figure 2’s pen prompt describes rotating one pen against a box edge and placing it in a holder, whereas the task prose, Table 2, and Appendix G describe two-pen twirling. The exact evaluated motion is therefore incompletely reconciled within the supplied source. e-task-discrepancye-maine-protocol
7.2 Questions for discussion
- Does history remain beneficial when its contact-event auxiliary loss is controlled independently?
- How much of the gain survives new sensor calibration and explicitly held-out object/pose conditions?
- Would training on generated tactile futures reduce the Attn-AT mismatch without sacrificing forecast quality?
8. Reproducibility audit
8.1 Requirements and known gaps
A close reproduction requires synchronized overhead/wrist views, calibrated bilateral tactile fields, and joint-target demonstrations, plus compatible Wan2.2/ActionDiT initialization. Preserve normalization, clean anchors, future-only tactile losses, and the frozen-decoder gradient path before comparing success. e-saf-implementatione-stage2e-data
Appendix E.1 promises training/inference code upon publication under a research-use license; raw data, preprocessing/analysis scripts, trained checkpoints, and baseline reimplementation code are not released. This is a source release statement, not verified code availability. e-release
Proposed checks should independently vary history and contact-event loss, and test action-output invariance to future-token perturbations under AGT versus relaxed masks. Keep noise, anchors, data, and optimization fixed; repeat robot comparisons across training seeds. e-maske-historye-ablatione-protocol
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate tactile history from its contact-event supervision
Reader-proposed experiment, not performed: on chip grasping and wiping, keep SAF, AGT, future prediction losses, initialization, data, horizon and optimizer fixed. Compare full TacWAM, the same history encoder with contact-event loss weight set to zero, and the published zero-history/no-contact-loss variant. Use identical test-condition sequences across several training seeds; report task success with uncertainty and held-out force error and contact-transition lag. History has an independently supported benefit if the history-without-contact-loss model consistently exceeds the zero-history model. If only the full model improves, the result attributes more of the gain to auxiliary supervision than the published history ablation can distinguish. e-historye-ablatione-stage2e-protocole-force
Check 2: Test whether the action branch depends on future-token values
Reader-proposed experiment, not performed: at fixed anchors, language, proprioception, history, action noise, and flow timestep, perturb only future tactile prediction tokens and then only future visual tokens. Under finalized AGT, action predictions should remain unchanged up to numerical tolerance; systematic changes would falsify the implemented isolation claim. Repeat with matched Attn-AT and Attn-VT models, which permit additional dependencies. Then compare teacher-derived versus generated tactile futures for the relaxed models while keeping other conditions fixed. Greater action sensitivity and a larger generated-future performance loss would support the proposed shortcut explanation; invariance or no deployment gap would weaken it. Inspect masks and any hidden conditioning paths before interpreting robot trials. e-maske-generatore-relaxede-protocol
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version page, all four figures, all eight tables, and the declared supporting method, training, hardware/software, protocol and baseline pages. Page 19, Appendix D.2, directly verifies eight NVIDIA A800-SXM4-80GB GPUs for Stage 2 and the stated PyTorch 2.7.1/CUDA 12.8 software stack; its Appendix C.3 continuation also verifies shared ablation optimization settings. Page 18 supplies loss weights and optimization settings, including batch size per GPU; it does not supply the GPU count or model. Five original crops were inspected at their final stored dimensions: Figure 1, Tables 5, 2 and 3, and Figure 4. Narrow tables were rendered at 400 DPI; architecture and force plots at 200 DPI. Full text, including references and Appendices A–I, was read across all six chunks. Pages 2 and 12 were text-read but not image-inspected. Separate supplements, code and experiments remain outside this reading.
PDF pages inspected for this edition: 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title page and abstract (p. 1)
- 1 Introduction (pp. 1–2)
- 2 Related Work, 2.1–2.2 (pp. 2–3)
- 3 Method, 3.1–3.6 (pp. 3–6)
- 4 Experiments, 4.1–4.3 (pp. 6–10)
- 5 Conclusion (p. 10)
- References (pp. 10–13)
- A Implementation Details, A.1–A.6 (pp. 13–16)
- B Attention Mask Details, B.1–B.5 (pp. 16–17)
- C Training Details, C.1–C.3 (pp. 17–19)
- D Hardware and Software Environment, D.1–D.2 (p. 19)
- E Data Collection Details and E.1 Code and Artifact Release (p. 20)
- F Evaluation Protocol (p. 20)
- G Task-Specific Success Criteria (p. 20)
- H Baseline Details (p. 21)
- I Additional Qualitative Results (p. 21)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Identity/version disclosure: the inspected title and six authors match the catalog. The PDF margin identifies arXiv:2607.28391v2 [cs.RO], 24 Aug 2026; the title block separately says August 6, 2026. The catalog submittedDate is 2026-08-23. This report concerns the supplied v2 artifact; v1 and its revision differences were not supplied or compared.
- Acquisition omission preserved: text extraction does not reconstruct figure images. This reading additionally inspected all four figures and all eight tables in the retained PDF, resolving image/layout coverage for those items.
- Separate supplemental material availability has not been fully verified. The supplementary sections embedded in this 21-page PDF were read completely.
- All six supplied text chunks were read individually, including references and appendices. Code and external linked resources were not inspected; experiments were not reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title/author block, header and arXiv margin
Exact title matches the supplied primary title. Authors are Lei Jin, Yiding Ma, Xin Zhang, Chen Gao, Wei Wu, Yong Li; affiliations are Tsinghua University and Manifold AI. The margin states arXiv:2607.28391v2 [cs.RO], 24 Aug 2026, while the title block states August 6, 2026.
Go to primary source ↓e-problemPDF pp. 1–3, Abstract, §1, §3.1, Eqs. (1)–(2)
Contact force, deformation and slip motivate tactile prediction; the latent is explicitly tactile-observable rather than a full physical state. Inputs include vision, touch, proprioception and task/language.
Go to primary source ↓e-generatorPDF p. 3, Figure 1(a–c); p. 4, §3.1 Eq. (3) and §3.2
One tri-modal WAM generator uses modality-specific Mixture-of-Transformers experts with masked mixed self-attention and separate language/proprioception cross-attention. The conditioning equations do not denote separate networks; tactile prediction is parallel rather than action-conditioned.
Go to primary source ↓e-safPDF p. 4, §3.3, Eqs. (4)–(6)
SAF fuses synchronized rectified images, dense force and mesh flow bilaterally. Its single-frame decoder reconstructs local fields and each sensor’s 3D force/torque; wrench is global supervision.
Go to primary source ↓e-saf-implementationPDF pp. 13–14, Appendix A.2 and Table 4
Appearance uses frozen Wan VAE features (48 channels), force/flow branches 32 each, per-side fusion 64, bilateral pooling/readout 128. Force and flow tensors are 2×35×20×3; wrench has 12 values. Normalization uses training-set z-scores with unspecified wrench/flow clipping bounds.
Go to primary source ↓e-historyPDF p. 5, §3.5 Eq. (10); p. 15, Appendices A.3–A.5
A 32-step temporal residual Conv1d encoder forms 256-dimensional history context, with adaptive-normalization modulation of the tactile expert only. Six binary contact-event labels include stability of at least eight steps; BCE weight is 0.1. Force-threshold values are not given. No-history disables both context and contact loss.
Go to primary source ↓e-maskPDF p. 5, §3.4 and Table 1; p. 16, Table 5 and Appendix B.1; p. 17, B.4
Rows are queries, columns keys/values. Action sees V0, T0 and A only; tactile futures see V0 and tactile tokens; visual tokens do not read tactile tokens. Within tactile self-attention T0 reads itself, while later tactile tokens may read the full tactile sequence. Future tactile tokens cannot read actions.
Go to primary source ↓e-stage1PDF p. 6, Eq. (11); p. 17, Appendix C.1, Eq. (16), Table 6
SAF pretraining uses SmoothL1 force/wrench/flow losses weighted 1.0/1.5/1.0. The appearance VAE stays frozen; SAF and decoder are frozen after this stage.
Go to primary source ↓e-stage2PDF p. 18, Appendices C.2–C.3, Eq. (17), Tables 7–8
Stage 2 trains the generator/history encoder with fixed SAF and differentiable fixed decoder. Loss weights are 1/1/1/0.5/0.1; tactile losses cover future indices only. Table 8 gives AdamW, 10⁻⁴ learning rate, cosine decay, 10⁻² decay, 20 epochs, bfloat16, seed 42; Stage 2 batch eight/GPU, accumulation two, 1,000 timesteps/stream, shift 5.0 and 20 denoising steps. Initialization uses Wan2.2-TI2V-5B and compatible ActionDiT.
Go to primary source ↓e-inferencePDF p. 6, §3.6 deployment paragraph; pp. 15–16, Appendix A.6
Action tokens are next-transition target joint positions, dimension seven. Horizon is 32, visual sequence nine frames after subsampling, inference 20 denoising steps. One chunk executes before observations refresh. Decoded tactile predictions are analysis-only, not online corrections; no measured generation latency is supplied.
Go to primary source ↓e-platformPDF p. 6, §4.1 Platform; p. 19, Appendix D.1 and D.2 software-stack and hardware-detail bullets
Agilex Piper, two Xense G1-WS sensors, overhead/wrist cameras and 30 Hz synchronized streams. Appendix D.2 explicitly gives GPU model NVIDIA A800-SXM4-80GB (80 GB) and a count of eight GPUs for TacWAM Stage 2 training. Its software-stack bullets state PyTorch 2.7.1 with CUDA 12.8 wheels, and the hardware-detail bullets state CUDA toolkit 12.8.
Go to primary source ↓e-dataPDF p. 6, §4.1 Data; p. 20, Appendix E
Each of four tasks has 300 episodes averaging about 500 frames. All sensor/action streams are synchronized. No explicit train/validation partition or numerical test-pose range is supplied.
Go to primary source ↓e-mainPDF p. 8, Table 2, all rows and Chip/Cherry/Wiping/Twirling/Avg. columns; §4.2
Success percentages: π0.5 10/60/35/20, mean 31.3; Fast-WAM 15/40/20/5, mean 20; RDP 50/45/0/35, mean 32.5; VT-WAM 45/45/30/30, mean 37.5; TacWAM 90/70/75/65, mean 75. Each task uses 20 real trials.
Go to primary source ↓e-ablationPDF p. 9, Table 3 and §4.3; pp. 18–19, Appendix C.3
Chip/wiping/mean percentages: full 90/75/82.5; no history 50/60/55; no history+Attn-AT 30/45/37.5; no history+Attn-VT 10/5/7.5. These are nested changes on two tasks; no-history zeros history and disables its auxiliary loss. There is no standalone SAF or tactile-future-loss ablation in the reported sequence. Appendix C.3 on p. 19 states that all ablation variants use the full model’s Stage 2 optimization hyperparameters.
Go to primary source ↓e-relaxedPDF pp. 9–10, Figure 3 and §4.3; pp. 16–17, Appendices B.2–B.5
Attn-AT lets actions read the full tactile prediction sequence after history removal. Attn-VT retains that access and adds bidirectional visual/tactile future exchange. Authors attribute degradation to target-derived training shortcuts and generated-future mismatch; this explanation is not independently isolated.
Go to primary source ↓e-forcePDF p. 10, §4.3 Future tactile state prediction; p. 11, Figure 4, both panels and caption; p. 21, Appendix I
Ground truth, TacWAM, and no-history force-magnitude rollouts are shown for representative wiping and twirling episodes. TacWAM better tracks several transitions/peaks but underestimates others. Axes give frame and resultant strength without a physical unit; no aggregate prediction-error metric is reported. These predictions do not correct actions online.
Go to primary source ↓e-protocolPDF p. 8, Evaluation protocol; p. 20, Appendices F–G
Same demonstrations, epochs and test-condition sequence; 20 trials per task/method; one checkpoint. No confidence intervals, significance tests or multi-seed training. Success excludes chip breakage/drop, cherry drop/visible crushing, lost wiping contact, and falling pens; numerical cleanliness and pose-range specifications are absent.
Go to primary source ↓e-baselinesPDF p. 8, baseline descriptions; p. 21, Appendix H
π0.5 is fine-tuned; Fast-WAM retains visual predictive training. RDP and VT-WAM are author reimplementations. Baseline settings differ, including π0.5 horizon 33 versus TacWAM 32, and batch sizes; equal epochs do not establish equal compute.
Go to primary source ↓e-releasePDF p. 20, Appendices E and E.1
Raw data are not public because of calibration, synchronization and management constraints. Code is promised upon publication under a research-use license; raw data, preprocessing/analysis scripts, trained checkpoints and baseline reimplementation code are not released.
Go to primary source ↓e-task-discrepancyPDF p. 7, Figure 2 second task prompt and §4.1 Two-pen twirling bullet; p. 20, Appendix G
The illustrated prompt describes one pen rotated upright against a box edge and placed into a holder. The text describes two-pen twirling without dropping. The supplied paper does not reconcile these descriptions.
Go to primary source ↓8.5 Primary sources
TacWAM: Anchor-Guided World Action Model with Mechanics-Aware Tactile Prediction ↗
PDF · 8,516 extracted words
Source fingerprint
c20dbfc77f39c943793d09d958cb571bac2209c341e1d9c128cd43de3136da47