VT-WAM: Visual-Tactile World Action Model for Contact-Rich Manipulation
1. Paper overview
In one sentence: VT-WAM predicts tactile evolution alongside actions and guides contact-phase attention, retaining tactile computation while dropping future visual prediction during control. problemarchitecturemaskguidancemain-resultspredictionablationrecovery
| At a glance | What to know |
|---|---|
| Research problem | Source description Contact deformation, slip and pressure can change while a wrist-camera image barely changes. The authors argue that dense visual evidence dominates joint learning because useful touch is temporally sparse. Their question is whether predicting tactile evolution, and encouraging its use specifically during contact, improves executed manipulation. problem |
| Core mechanism | Source description Couple future visual, tactile and action prediction through three modality-specific experts connected within one MoT backbone. architectureobjective |
| A key reported result | Six real-world contact-rich manipulation tasks: VT-WAM: 71.67% overall; surface 81.67%, insertion 61.67%. Board/vase/cucumber/plug/card/tube: 90/85/70/60/70/55%. Reported success rate: average task score as a percentage. Separate task training on the same 100 demonstrations per task; 20 independent physical trials per method/task on the same robot. Surface tasks allow scores 0/0.5/1; insertion is binary. Fast-WAM: 45.00% overall, 56.67% surface, 33.33% insertion; task scores 70/55/45/20/55/25%. OmniVTLA: 35.83% overall. The overall advantage over Fast-WAM is 26.67 percentage points; versus OmniVTLA it is 35.84 points using displayed rounded means. The metric is not a uniform binary completion probability. Same demonstrations do not equalize pretrained models or tactile availability. implementationprotocolmain-results |
| Reading caution | Reader analysis The authors explicitly leave multi-task training and scaling laws unexplored. Task-specialized results do not establish transfer to new tasks or objects. limitationsprotocol |
Core contributions
- Source description
Couple future visual, tactile and action prediction through three modality-specific experts connected within one MoT backbone. architectureobjective
- Author claim
Demonstrate improvements across six real-world tasks, supported by tactile prediction metrics and two-task mechanism ablations. main-resultspredictionablation
Figure 2. Asymmetric attention keeps the visual anchor and tactile future available to action queries. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at panel (a): visual and tactile VAEs produce latents, action projection produces action tokens, and state/language enter through cross-attention. Orange layers connect the three experts. In panel (b), rows are queries and columns are keys; colored cells permit attention. Action rows see the first visual frame and all tactile positions. Tactile rows cannot see action keys, and visual rows cannot see either other modality. The inference mask removes future visual columns and rows while retaining tactile prediction. In panel (c), the red stop-gradient marker applies to the keys in an auxiliary attention calculation, and the lock marks training-only guidance. architecturemaskguidanceobjectiveinference
What it supports. The model’s asymmetry is an information-flow choice: actions can use evolving predicted touch without requiring denoised future video at deployment. The architecture therefore supports joint tactile/action generation inside an integrated multi-expert backbone. The stop-gradient operation confines the auxiliary key-gradient path; it does not imply that the whole visual or tactile expert is frozen.
Where the evidence stops. Panel (c) says tactile weight should exceed visual weight, while Section III-C’s hinge also gives zero loss at equality. The graphic shows only the visual anchor; the auxiliary equation writes visual keys without an explicit mask. The exact key subset and contact threshold remain implementation questions.
2. Motivation
2.1 The problem and the proposed response
Contact deformation, slip and pressure can change while a wrist-camera image barely changes. The authors argue that dense visual evidence dominates joint learning because useful touch is temporally sparse. Their question is whether predicting tactile evolution, and encouraging its use specifically during contact, improves executed manipulation. problem
2.2 What this reading follows
When a wiping tool loses contact with a vase, a wrist-camera image may change little even though the task has stopped progressing. VT-WAM makes tactile evolution part of action generation: separate visual, tactile and action experts exchange information through an asymmetric attention mask. A training-only guidance loss encourages action queries to use tactile evidence during contact. The paper reports stronger physical task scores and better tactile prediction than its baselines, but these are distinct evaluations. Read the architecture first, then the task table, prediction table and ablations. The final diagnostic connects the proposed attention mechanism to an illustrative contact disturbance. problemarchitecturemaskguidancemain-resultspredictionablationrecovery
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Joint prediction |
| Quadrant | Q3 · Dual-system × Joint prediction |
3.1 Evidence-based assessment
Conflicts with the recorded classification
WAM, tactile sensing and Joint prediction are supported. The recorded Dual-system/Q3 architecture conflicts with the described integrated backbone: Figure 2 connects three experts through shared attention layers and predicts modalities through their heads, without separate planning and execution systems. This favors a One Model reading on architectural evidence, beyond merely sharing a loss. The efficiency subcategory is design-supported but lacks timing measurements. The catalog snapshot is preserved for editorial review. architecturemaskobjectiveinference
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Keep the visual anchor while predicting contact evolution
The key distinction is between what the model learns to predict and what it must compute for control. During training, visual, tactile and action experts each predict a flow velocity field. Their queries, keys and values meet in MoT attention, but the mask restricts cross-modal access. In particular, the tactile expert cannot depend on future visual tokens or action keys, while action queries can read the tactile sequence and current visual anchor. At deployment, future visual tokens can therefore be removed while tactile and action latents are denoised together. The tactile future is a model prediction available during denoising, not a future sensor measurement handed to the controller. Joint inference restores visual prediction for the paper’s prediction analysis. Keeping these two modes separate explains why Figure 5 illustrates capability without depicting the actual control computation. architecturemaskobjectiveinferenceprediction
5.2 Teach action queries when touch deserves attention
An allowed attention connection does not ensure that a model uses it. AVTAG addresses this by constructing a second attention distribution from action queries to visual and tactile keys. The keys are detached in this auxiliary calculation, so the loss guides action queries without directly updating those keys through that path. For each action token, the paper sums attention by modality and normalizes the two masses into relative visual and tactile weights. A hinge penalizes visual weight exceeding tactile weight, but only for tokens marked as contact phases by pronounced deformation. Equal weights already satisfy the formal loss despite the graphic’s strict greater-than shorthand. This is training supervision, not an online threshold rule that overrides robot actions. Reproducing it requires resolving the unreported numerical contact threshold and exact auxiliary key selection. guidancemaskobjective
5.3 Separate task improvement from a mechanism explanation
Reader analysis: the strongest control result is Table I, but its average combines partial-credit surface scores with binary insertion success. Table II asks a different question: how closely predicted tactile deformation matches reference fields under joint inference. Neither table alone identifies why control improves. Table III moves closer by contrasting first-frame and full-sequence tactile readout, then adding AVTAG to the same asymmetric architecture. The gains support the temporal-tactile hypothesis, while the final tube improvement is small relative to the trial count. Figure 6 supplies an intuitive contact-disturbance story, showing changed attention and reported recovery, but the plots remain selected examples. A convincing reproduction should therefore measure task score and recovery across repeated controlled disturbances, and test whether disrupting temporal tactile information changes performance beyond merely changing attention weights. protocolmain-resultspredictionablationrecovery
5.4 Training and inference
During training
Train a separate model per task using 100 kinesthetic demonstrations, with all streams synchronized to 30 Hz. The visual backbone is pretrained Wan2.2-5B; tactile and action experts are 1B-scale DiTs. The three flow-loss weights equal 1, and the AVTAG coefficient is 0.05. implementationprotocol
Optimization uses AdamW, learning rate 1e-4, weight decay 1e-2, bf16, gradient clipping 1.0, cosine decay and 5% warmup on NVIDIA A100 80GB GPUs. Training stages, frozen-parameter policy, batch size, update count and GPU count are not specified. implementation
During inference
Prediction analysis jointly denoises visual, tactile and action tokens. Physical control instead removes future visual tokens and denoises tactile and action latents against the current visual anchor, using 10 denoising steps on a remote A100 server. This is a computational design benefit without a reported millisecond latency measurement. inferenceimplementation
The action head supplies control chunks; the paper does not specify executable action coordinates, chunk length, execution fraction, feedback/replanning schedule or end-to-end control rate. The 30 Hz sensing/resampling rate should not be treated as measured policy throughput. architectureimplementationinference
5.5 Implementation flow
- Encode three streams
A Wan2.2 video VAE encodes wrist images; a pretrained tactile VAE encodes six deformation channels from two sensor surfaces. Action chunks receive linear projection. Language and state condition each expert through cross-attention. architecture
- Route information asymmetrically
Each expert forms its own queries, keys and values. Visual queries cannot read tactile or action keys. Tactile queries read the first visual frame but no action keys. Action queries read that visual anchor and the full tactile latent sequence. Thus predicted touch can inform actions without a downstream inverse-dynamics module. architecturemask
- Guide contact-phase queries
AVTAG constructs auxiliary attention using action queries and detached visual/tactile keys. It compares total attention mass by modality and penalizes visual dominance only for action tokens identified by pronounced tactile deformation. Stop-gradient blocks this auxiliary path into keys; it does not establish that the encoders or backbone are globally frozen. guidance
6. Experiments & results
VT-WAM learns visual futures, tactile deformation and robot actions in an integrated multi-expert flow model. During control it keeps a current visual anchor and predicts tactile evolution alongside actions. A contact-gated training loss encourages action queries to use touch. Its six-task average is 71.67% versus Fast-WAM’s 45.00%, but surface-task scores include partial credit. Evidence supports task-specialized contact manipulation, with missing latency and uncertainty measurements limiting broader claims.
6.1 Read the original evidence
Table I. VT-WAM leads on all six tasks under the paper’s mixed scoring protocol. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the six task columns before the far-right average. Board wiping, vase wiping and cucumber peeling are surface tasks; plug insertion, card swiping and tube insertion form the constrained-insertion group. The shaded row gives VT-WAM, and Fast-WAM immediately above it provides the strongest overall baseline. Section IV-B on page 6 defines an essential distinction absent from the table header: surface scores permit half credit for more than half the target region, whereas insertion scores are binary. All methods use the same task demonstrations and platform, with 20 independent trials per task, but their modalities and pretrained backbones differ. protocolmain-resultsimplementation
What it supports. VT-WAM reports 71.67% overall against Fast-WAM’s 45.00%, a 26.67-percentage-point gain. Its surface and insertion averages are 81.67% and 61.67%, compared with 56.67% and 33.33%. The six task columns all favor VT-WAM, so the overall improvement is not confined to one interaction regime.
Where the evidence stops. The aggregate mixes partial-credit surface scores with binary insertion success. It should not be read as the probability of completely finishing a random task. No confidence intervals or seed variability are reported; cross-model comparisons do not isolate one architectural change.
Table II. VT-WAM improves both reported tactile prediction metrics. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the arrows in the column headers: smaller l2 is better, while larger cosine similarity is better. The evaluation description starts in Section IV-C.2 on page 6 and continues on page 7. It computes distance across the complete three-dimensional deformation field, but directional agreement only where deformation is non-zero. These columns therefore test different aspects of predicted touch. The shaded VT-WAM row is compared with exUMI and UVA using the same task demonstrations. Figure 5 on page 6 supplies qualitative visual/tactile examples from joint inference; the table summarizes tactile prediction, not robot task completion. predictioninference
What it supports. VT-WAM’s l2 error is 0.077, compared with 0.083 for UVA and 0.091 for exUMI. Its cosine similarity is 0.749, versus 0.667 and 0.618. Both reported criteria favor its tactile predictions, providing complementary evidence to the physical task results rather than another measure of the same outcome.
Where the evidence stops. The paper does not specify a held-out split, prediction horizon, physical units or uncertainty for these values. Joint prediction analysis includes visual generation, whereas deployed control uses visual-cache inference. The table does not measure tactile prediction quality under every deployment condition.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Six real-world contact-rich manipulation tasks Separate task training on the same 100 demonstrations per task; 20 independent physical trials per method/task on the same robot. Surface tasks allow scores 0/0.5/1; insertion is binary. | VT-WAM: 71.67% overall; surface 81.67%, insertion 61.67%. Board/vase/cucumber/plug/card/tube: 90/85/70/60/70/55%. Reported success rate: average task score as a percentage | Fast-WAM: 45.00% overall, 56.67% surface, 33.33% insertion; task scores 70/55/45/20/55/25%. OmniVTLA: 35.83% overall. The overall advantage over Fast-WAM is 26.67 percentage points; versus OmniVTLA it is 35.84 points using displayed rounded means. The metric is not a uniform binary completion probability. Same demonstrations do not equalize pretrained models or tactile availability. implementationprotocolmain-results |
| Tactile deformation prediction Joint inference; all compared methods evaluated on the same task demonstrations. Held-out split and prediction horizon are unspecified. | VT-WAM: 0.077 l2 and 0.749 cosine. Full-field l2 distance (lower); cosine similarity over non-zero deformation regions (higher) | exUMI: 0.091 and 0.618; UVA: 0.083 and 0.667. Both metrics favor VT-WAM under this reported protocol. Physical error units and uncertainty are absent; these prediction metrics do not independently establish control success. prediction |
| Tactile readout and AVTAG ablation on wipe vase and insert tube Two task representatives; Table III compares Fast-WAM-based variants under the paper’s task-scoring protocol. | M0 baseline 55/25; M1 symmetric tactile sequence 65/40; M2 asymmetric first tactile frame 40/30; M3 asymmetric full sequence 70/50; M4 with AVTAG 85/55. Reported wipe-vase / insert-tube success score (%) | M2 to M3 adds 30/20 percentage points; M3 to M4 adds 15/5 points. Temporal tactile readout has the larger observed effect. M3–M4 isolates AVTAG according to the authors; 5 points on binary tube insertion equals one additional success in 20 trials, so its stability remains untested. protocolablation |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table III, including Notes. Temporal tactile access and AVTAG have separable ablation comparisons. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the model descriptions and retain the notes: M0 is the visual-only Fast-WAM baseline; M1 adds tactile-sequence prediction with symmetric attention. M2 uses asymmetric attention but lets action queries access only the first tactile frame. M3 exposes the full tactile sequence under asymmetric attention. M4 adds AVTAG to M3. Compare M2 with M3 to examine temporal tactile readout, then M3 with M4 to examine training guidance. The latter pair has the same asymmetric architecture according to Section IV-D. Wipe vase uses partial-credit surface scoring, while insert tube uses binary success under the evaluation protocol on page 6. protocolablationmaskinference
What it supports. Full tactile-sequence access raises vase/tube scores from 40/30 to 70/50. Adding AVTAG then raises them to 85/55. The temporal readout comparison has the larger observed gain. M1 also improves over M0, but symmetric fusion carries a different inference requirement, so that comparison does not isolate the cache mechanism.
Where the evidence stops. Only two tasks are ablated, and no seed variance or uncertainty intervals are given. Under 20 binary tube trials, the final five-point gain represents one additional success. The symmetric/asymmetric comparison also changes inference computation, which the table does not quantify.
Figure 6. A contact disturbance illustrates different attention patterns with and without training guidance. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the upper trial without AVTAG, then the lower trial with it. Red is relative tactile attention, blue is relative visual attention, and the dashed curve uses the right-hand force axis. Numbered points link action-execution steps to image snapshots. The support moves down and contact is disturbed; the upper panel labels subsequent contact weakening, while the lower panel labels maintained contact. Section IV-D describes the guided model re-establishing contact. Only the wrist images are available as visual input to the policy: side images and contact-force traces are explanatory visualizations. Compare qualitative event sequences, since the two panels use different step spans and force scales. guidancerecoveryablation
What it supports. Without guidance, visual attention stays above tactile attention through the illustrated disturbance. With AVTAG, tactile attention rises above visual attention, and the source reports recovery and task completion. This is consistent with the intended contact-sensitive use of touch, complementing the M3–M4 score comparison in Table III.
Where the evidence stops. These are illustrative traces, not an aggregate recovery experiment or a causal intervention on attention. Different axis scales prohibit direct timing or force-magnitude comparisons by eye. The plotted force is not a documented extra policy input or inference-time guidance signal.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors explicitly leave multi-task training and scaling laws unexplored. Task-specialized results do not establish transfer to new tasks or objects. limitationsprotocol
Tables I–III give point estimates without confidence intervals or seed variability. Figure 6 illustrates disturbed vase wiping, but attention traces are diagnostic associations, not proof that attention weights alone cause recovery. main-resultspredictionablationrecovery
Removing future visual prediction is architecturally motivated, but no measured latency, throughput, or matched-compute accuracy comparison establishes real-time performance. inferenceimplementationablation
7.2 Questions for discussion
- Would temporally shuffled predicted touch reduce control performance when its marginal deformation statistics remain intact?
- Does contact-gated guidance remain beneficial with calibrated tactile noise or changed contact thresholds?
- How much latency is saved by removing visual futures at matched action quality?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the xArm7/Robotiq 2F-85 platform, wrist RGB and paired Xense sensing, task demonstrations, and compatible visual/tactile VAEs and DiT initialization. RGB is 128×128; each sensor provides a 35×20 3D deformation field at 30 Hz. architectureimplementation
Resolve the numerical contact threshold, token-to-contact alignment, auxiliary attention aggregation, action representation/horizons, flow sampling recipe and optimization duration before exact replication. The supplied paper does not provide these details or a declared prediction train/test split. architectureguidanceobjectiveimplementationprediction
Proposed checks: repeat M3–M4 with matched disturbance schedules and multiple seeds; separately compare full tactile readout with first-frame-only and temporally shuffled readout while holding architecture and compute fixed. Measure recovery and task score, not attention alone. maskguidanceablationrecovery
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Repeat AVTAG’s contact-recovery comparison with matched disturbances
Reader-proposed experiment: train M3 and M4 from matched initializations using the same demonstrations, data order, optimizer schedule and denoising budget across five seeds. Resolve and record the contact-gate threshold before training. On vase wiping and tube insertion, use at least the paper’s 20 trials per task per seed, with matched initial states; for vase wiping, randomize a prespecified support-lowering schedule and include undisturbed controls. Report the original task score, recovery frequency and recovery delay with uncertainty. The guidance hypothesis predicts more reliable recovery without a large undisturbed penalty. A tactile-attention increase without a reproducible behavioral gain would weaken that explanation. guidanceimplementationprotocolablationrecovery
Check 2: Test whether temporal tactile content drives the readout gain
Reader-proposed experiment: reproduce matched M2 and M3 training with identical expert sizes, data, optimization and denoising steps. Evaluate both on vase wiping and tube insertion. Then evaluate M3 with its first tactile anchor intact but predicted future tactile content permuted among fixed time positions; retain token count and compute, and compare against its unmodified readout. Log task scores and tactile prediction errors separately. If temporal contact evolution matters, full coherent readout should outperform the first-frame variant and temporal corruption should degrade control. A corruption result alone is insufficient because it introduces distribution shift; the matched trained M2–M3 comparison supplies the complementary control. maskimplementationprotocolpredictionablation
8.3 Reading coverage
Visual audit: All eight original PDF pages were rendered and actually viewed. Page 1 verifies title, ten authors, affiliations and arXiv v1 dated 2 July 2026, and contains Figure 1. Page 2 supplies architecture notation. Pages 3–4 verify Figure 2, cross-modal masks, stop-gradient, hinge equations, flow objectives and Figure 3 hardware. Page 5 verifies Table I, Figure 4, visual-cache inference and all retained hardware/training settings. Page 6 verifies task scoring, baseline protocol and Figure 5 prediction examples. Page 7 verifies Table II, Table III with notes and Figure 6 diagnostic axes/caption. Page 8 supplies the stated limitation and references; no appendix is present. Every final crop was viewed at its native generated resolution, preserving original labels and necessary table notes. Cross-modal mask directions agree with the equations; the strict inequality shorthand in Figure 2(c) is distinguished from the formal zero-loss equality condition. Separate supplements, code and external sites were not supplied or inspected; no experiment was reproduced.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- PDF p. 1: title, authors, affiliations, v1 stamp, abstract and Section I
- PDF p. 2: Sections I–II and III-A
- PDF pp. 3–4: Sections III-A–D.1, attention masks, AVTAG and objectives
- PDF pp. 5–6: Sections III-D.2 and IV-A–C, implementation, protocol and results
- PDF p. 7: Sections IV-C–D, prediction metrics, ablations and recovery diagnostic
- PDF p. 8: Section V and all references; all four supplied text chunks read in full
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction-only figure omission was addressed by inspecting all eight PDF pages, all six figures, all three tables, and every final crop.
- No separate supplement was supplied. Only the supplied arXiv v1 was reviewed; no other edition was supplied for comparison. Title and author identity agree with the catalog.
- Code, project website and external references were not inspected; experiments were not reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title/author/affiliation block and arXiv margin stamp
The observed title matches the supplied title. The ten authors, in order, are Shuai Tian, Yupeng Zheng, Yuhang Zheng, Songen Gu, Yujie Zang, Yuxing Qin, Weize Li, Haoran Li, Wenchao Ding, and Dongbin Zhao. The artifact is arXiv:2607.02503v1 [cs.RO], 2 July 2026. All five supplied affiliations match the title page; no title or author revision discrepancy is observed.
Go to primary source ↓problemPDF pp. 1–2, Abstract, Section I and Figure 1(a)
The authors motivate tactile dynamics through sparse contact information and visual dominance; Figure 1(a) plots effective contact ratios for the six tasks, without defining its error bars in the caption.
Go to primary source ↓architecturePDF p. 2, Section III-A; p. 3, Figure 2(a) and Section III-A continuation
Wan2.2 video VAE tokens, pretrained tactile VAE tokens from two 3D deformation surfaces, and projected action chunks enter modality-specific experts connected by MoT attention in one backbone. State and language use cross-attention. Modality-specific heads predict flow velocity fields.
Go to primary source ↓maskPDF pp. 3–4, Section III-B cross-modal mask equations; p. 3, Figure 2(b)
Rows are queries and columns are keys. Visual queries cannot read tactile/action keys; tactile queries read the first visual frame and no action keys; action queries read the first visual frame and full tactile sequence. Figure 2 removes future visual tokens for inference and keeps first-frame anchors isolated from future same-modality tokens.
Go to primary source ↓guidancePDF p. 4, Section III-C, auxiliary attention, normalization and hinge-loss equations; p. 3, Figure 2(c)
AVTAG uses action queries and stop-gradient visual/tactile keys to form auxiliary attention. It sums and normalizes attention by modality, then penalizes max(0,p_v-p_t) on action tokens with pronounced tactile deformation. The formal loss has zero penalty at p_t >= p_v; the graphic uses the stricter shorthand Encourage p_t > p_v. No numerical contact threshold is specified.
Go to primary source ↓objectivePDF p. 4, Section III-D.1, flow matching and full training objective equations
Visual, tactile and action squared velocity errors form a weighted joint flow loss; the full training loss adds weighted contact-gated AVTAG. Predicted fields and target fields are defined, but the detailed flow interpolation/sampling procedure is not specified here.
Go to primary source ↓inferencePDF p. 5, Section III-D.2, Efficient Visual-Cache Inference
Joint inference denoises visual, tactile and action tokens for prediction analysis. Real-world control retains the current visual anchor, removes future visual prediction, and denoises tactile and action latents. No measured latency comparison is reported.
Go to primary source ↓implementationPDF p. 4, Figure 3; p. 5, Sections IV-A.1–2
The platform is a 7-DoF xArm7 with Robotiq 2F-85, a RealSense D435 labeled in Figure 3, and paired Xense sensors. Data comprise 100 kinesthetic demonstrations per task, synchronized at 30 Hz; RGB is 128 by 128 and each tactile field 35 by 20 by 3. Wan2.2-5B and 1B-scale tactile/action DiTs use AdamW, learning rate 1e-4, weight decay 1e-2, bf16, clipping 1.0, cosine decay and 5% warmup. Loss weights are 1,1,1 and AVTAG 0.05. Training uses A100 80GB GPUs; inference uses a remote A100 and 10 denoising steps.
Go to primary source ↓protocolPDF p. 5, Figure 4 and Section IV-A; p. 6, Sections IV-A.3 and IV-B.1–2
Methods train separately per task on the same demonstrations and share platform, tasks and metrics. Each method/task has 20 independent trials. Surface tasks score 0, 0.5 for more than half the region, or 1 for complete coverage; insertion tasks score binary target attainment. The three surface tasks and three insertion tasks are explicitly listed.
Go to primary source ↓main-resultsPDF p. 5, Table I, all method rows and task/average columns; p. 6, Section IV-C.1
VT-WAM scores 90/85/70/60/70/55 percent on board/vase/cucumber/plug/card/tube, with surface/insertion/overall averages 81.67/61.67/71.67. Fast-WAM gives 70/55/45/20/55/25 and averages 56.67/33.33/45.00. OmniVTLA averages 33.33/38.33/35.83. No uncertainty intervals accompany Table I.
Go to primary source ↓predictionPDF p. 6, Figure 5 and Section IV-C.2; p. 7, Table II and Section IV-C.2 continuation
Joint inference produces qualitative visual/tactile sequences. Tactile l2 is evaluated over the full 3D field and cosine similarity over non-zero deformation regions, on the same task demonstrations for all methods. Table II reports exUMI 0.091/0.618, UVA 0.083/0.667 and VT-WAM 0.077/0.749. The evaluation split, prediction horizon, physical error units and uncertainty are not specified.
Go to primary source ↓ablationPDF p. 7, Table III including Notes, and Section IV-D, Ablations 1–2
On wipe vase/insert tube, M0 Fast-WAM gives 55/25; M1 symmetric tactile-sequence prediction 65/40; M2 asymmetric first-tactile-frame readout 40/30; M3 asymmetric full tactile-sequence readout 70/50; M4 adds training-only AVTAG and gives 85/55 percent. The text says M3 and M4 differ only in AVTAG.
Go to primary source ↓recoveryPDF p. 7, Figure 6(a–b), caption and Section IV-D, Ablation 2
A vase-wiping disturbance lowers the support. Red tactile and blue visual attention are compared without/with AVTAG; dashed contact force and side views are visualization only. The top panel labels contact weakening, and the bottom panel labels maintained contact; the text describes contact recovery. Panels have different action-step spans and force-axis ranges.
Go to primary source ↓limitationsPDF p. 8, Section V, Conclusion
The authors explicitly restrict the study to individual-task specialization and leave multi-task training and scaling laws to future work. The supplied eight-page PDF ends with references and has no appendix.
Go to primary source ↓8.5 Primary sources
VT-WAM: Visual-Tactile World Action Model for Contact-Rich Manipulation ↗
PDF · 6,107 extracted words
Source fingerprint
237eb6230bd43016d80f437963050804aa0bcaee2a4c65c1df29ec53278d62d5