FAWAM: Force-Aware World Action Models for Closed-Loop Contact-Rich Manipulation
1. Paper overview
In one sentence: FAWAM turns predicted wrist wrenches into references for fast, gated action correction, improving contact-rich task completion while requiring additional intervention data and careful gate selection. e02e03e05e06e08e09e10e19
| At a glance | What to know |
|---|---|
| Research problem | Source description Wiping, peeling and pivoting require sustained contact that images alone may not reveal. A force-conditioned chunk policy can still miss disturbances occurring between policy queries. FAWAM asks whether anticipated contact forces can become execution-time references for correcting those deviations. e02e03e06 |
| Core mechanism | |
| A key reported result | Four-task real-world contact-rich manipulation: FAWAM: 15/20, 19/20, 17/20, 17/20; average 85.00%. Successful trials and average success rate. Franka FR3; Erase Board, Peel Cucumber, Pivot Box and Wipe Vase under varied contact geometry. Each base/baseline uses 90 demonstrations per task; FAWAM adds 20 intervention episodes for correction. Twenty evaluation trials per task, at most 60 seconds each. GE-Act 48.75%; π0.5 47.50%; ForceVLA and TA-VLA each 63.75%; FAWAM without residual correction 73.75%. Differences from the strongest vision-only and force-aware baselines are 36.25 and 21.25 percentage points. The abstract’s percent wording should not be read as relative improvement. Extra correction data means the full comparison is not data-matched. e01e08e09 |
| Reading caution | Author claim The authors identify missing large-scale synchronized video–force data, limited correction episodes and indirect contact observability from wrist wrenches. Force does not condition video generation; dense tactile signals and local contact frames remain future work. e14 |
Core contributions
Figure 2. Separate prediction and correction pathways connect anticipated contact to robot feedback. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the bottom left: video and text tokens enter the Video DiT, while force tokens enter a separate encoder. Their arrows meet at the Force-Action DiT, whose joint output is an action–force sequence. The enlarged block shows force-adaptive normalization and a residual connection, consistent with the modulation in Equation (2). On the right, the upper diagram describes correction-data collection: the snowflake marks the fixed base model and the flame marks residual training. The lower diagram is deployment. Predicted actions and forces enter the corrector from the left; measured force returns from the robot on the right. Residual and gate outputs modify the command before execution. e03e04e05e06e07e14
What it supports. The mechanism uses future wrench prediction twice: as supervision for the action decoder and as a reference during execution. The depicted 1 Hz and 10 Hz pathways explain how local contact corrections can arrive within a base action chunk, without rerunning the entire visual prediction pathway for every update.
Where the evidence stops. The overview omits explicit tracking-error and proprioception inputs, which Section 3.3.1 supplies. It also omits the gate inequality: corrections activate only for g_t^res > 0.8. The drawing is not a latency benchmark or evidence that force conditions video generation.
2. Motivation
2.1 The problem and the proposed response
Wiping, peeling and pivoting require sustained contact that images alone may not reveal. A force-conditioned chunk policy can still miss disturbances occurring between policy queries. FAWAM asks whether anticipated contact forces can become execution-time references for correcting those deviations. e02e03e06
2.2 What this reading follows
A robot wiping a board may follow the right visual path while pressing too weakly or getting stuck. FAWAM addresses this mismatch by giving force three jobs: describing recent contact, predicting the contact expected from a planned motion, and correcting the motion when feedback departs from that expectation. Its slow model generates action and wrench chunks; a faster residual network updates execution selectively. The real-robot results are encouraging, but the strongest comparison includes extra human correction data. Read the architecture together with the component ablation and gate sweep to distinguish useful predictive supervision from demonstrated execution-time benefits. e02e03e05e06e08e09e10e19
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Joint prediction |
| Quadrant | Q3 · Dual-system × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded Dual-system × Joint prediction quadrant is supported: a separate video world model feeds the action decoder, and a faster residual network closes the loop. Joint prediction directly couples actions and future wrenches. The joint video-action subcategory needs qualification: the paper does not establish a single shared video/action denoiser, and its video branch is frozen during action training. e03e05e06e14
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Give the action decoder an expectation of contact
The first learning problem is to connect intended motion with what the wrist should feel. FAWAM begins with GE-Act’s separation between a video model and an action decoder. Recent wrench measurements are compressed into a contact feature, then converted into offsets for the action decoder’s normalization parameters. Zero initialization makes these offsets initially neutral. The video model is adapted first and frozen while the force-aware decoder learns. A shared output projection and joint flow-matching objective supervise both future actions and future wrenches, so force is a prediction target as well as an observation. This does not put force into the video model. The important architectural distinction is between the visual representation that conditions action generation and the action–wrench representation that learns anticipated contact. e03e04e05e14
5.2 Turn predicted contact into a conditional command update
During execution, the predicted wrench trajectory becomes a reference rather than merely a training target. The residual corrector compares recent measured and predicted wrenches, and also receives previews of upcoming actions and expected wrenches. Its outputs have different roles: the residual head proposes a correction, while the gate decides whether to apply it. Equation (7) adds the residual only when the gate exceeds the threshold. Human intervention supplies the action-difference targets, and autonomous segments teach zero corrections and inactive gates. Keeping both kinds of samples in balanced training batches is intended to preserve nominal behavior while learning recovery. The ten-hertz correction loop operates inside the one-hertz base-model cycle; the follower’s impedance controller then executes the resulting commands. A learned force expectation thus influences actual feedback control. e03e06e07e17
5.3 Separate system improvement from proof of the reference mechanism
The results answer several questions at different levels. Table 1 shows that the complete trained system completes more of these real tasks. Table 2 shows that observation, joint prediction and residual correction make different contributions, while Figure 15 shows sensitivity to intervention selectivity. Reader analysis: this combination is consistent with the proposed mechanism, but does not isolate every causal link. In particular, the full-versus-residual-only comparison changes the base policy as well as the availability of wrench references; the full system also receives additional correction data relative to the main baselines. The perturbation images illustrate plausible recovery, yet they do not measure prediction error or gate timing. A discriminating follow-up should keep the base policy and corrective supervision fixed while changing only the information carried by the predicted reference. e08e09e10e11e13e19
5.4 Training and inference
During training
First fine-tune GE-Base’s video model using latent flow matching. Freeze it for the second stage, which jointly trains the force encoder and action–wrench decoder. Interpolated noisy targets train velocity prediction, with separate action and wrench squared-error losses. e05
Both stages use eight NVIDIA A100 GPUs, batch 64 and 30,000 steps each; learning rates are 3×10⁻⁵ and 1.5×10⁻⁴. The wrench-loss weight is one. Appendix A.1 specifies bfloat16, DeepSpeed ZeRO-2, AdamW, 1,000 warmup steps and gradient clipping at norm 1.0. e16
FACTR interventions label corrected-command minus base-command residuals. Non-intervention targets and gate labels are zero; intervention gate labels are one. The frozen force encoder feeds a dual-head MLP with two shared width-256 layers, GELU and dropout 0.1. Train for 500 epochs using balanced batches of 128, AdamW at 3×10⁻⁴, and equal residual/gate loss weights. e07e17
During inference
The base model runs at 1 Hz; the residual corrector runs at 10 Hz. Recent measured wrenches are compared with the predicted reference, and corrections activate only when the gate exceeds 0.8. The follower uses impedance control. Predicted trajectories alone are not executed outcomes. e03e06e07
5.5 Implementation flow
- Build a contact-sensitive decoder
GE-Act supplies the video-to-action backbone. A force MLP encodes wrench history; block-specific MLPs add offsets to the action DiT’s AdaLN-Zero shift, scale and gate parameters. Zero-initialized final layers initially preserve the original decoder. e03e04
- Predict motion and its contact reference
A joint linear projection produces concatenated action and wrench trajectories from a shared decoder, rather than an isolated auxiliary force head. The adapted video model provides visual features; force is introduced in the action branch. e05e14
- Correct within an action chunk
A lightweight residual MLP receives contact features, wrench-error history, future action/wrench previews and proprioception. Its action head proposes local changes; its gate determines whether those changes reach the executed command. e06
6. Experiments & results
FAWAM predicts action chunks and wrist wrenches from a video-derived representation, then uses measured–predicted wrench discrepancies to gate faster residual corrections during real robot execution. Its strongest evidence is improved completion of four contact-rich tasks; the full system also requires additional human-intervention training data.
6.1 Read the original evidence
Table 1. FAWAM leads the reported four-task comparison, with a larger training-data budget for correction. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each task column as successful trials out of twenty, then use Avg. SR to compare the equally sized task evaluations. The first two rows are vision-only baselines; ForceVLA and TA-VLA are the force-aware baselines. The final two rows separate FAWAM’s base policy from the full residual-corrected system. Section 4.1 places these counts on a Franka FR3 under changes in contact geometry, with a sixty-second trial cap. All base policies use the same ninety demonstrations per task, while the residual corrector receives twenty additional intervention episodes. This training distinction matters when interpreting the bottom row’s advantage. e01e08e09
What it supports. The full model reaches 85.00%, versus 48.75% for GE-Act and 63.75% for either force-aware baseline. These are differences of 36.25 and 21.25 percentage points. Without residual correction, FAWAM still reaches 73.75%, so the reported improvement is not confined to the corrective branch.
Where the evidence stops. The full model benefits from extra intervention supervision. No uncertainty estimates accompany the counts, and the paper does not precisely formalize each task’s success criterion or held-out geometry split. The table establishes this evaluation’s outcomes, not general robustness across robots.
Figure 4. Shorter reported execution times accompany the full model’s higher success rate. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the vertical axis in seconds and compare bars within each task group. The printed numbers give the averages directly, and the salmon Full bar is the shortest in all four groups. Keep the original legend visible: it names Base, Obs, Obs+Obj, Res and Full. The label Obs+Obj differs from Table 2’s Pre terminology, so the Base-versus-Full comparison is the least ambiguous numerical reading. Consult Table 2 alongside the chart: execution duration is meaningful only with task completion. Appendix A.3 explicitly warns that an early or brittle termination can make an unsuccessful rollout appear fast. e10e12
What it supports. The full model’s reported times are 29.3 seconds for board erasing, 36.3 for peeling, 23.2 for box pivoting and 31.8 for vase wiping. The respective Base values are 47.9, 37.9, 38.7 and 42.6 seconds. These are rollout-duration measurements, not model-inference latency measurements.
Where the evidence stops. Failure handling and the population used for timing averages are not specified precisely. Appendix A.3 says some peeling ablations are faster, yet every plotted peeling bar exceeds Full’s 36.3 seconds. Preserve that source inconsistency rather than turning the prose into a contradictory numerical claim.
Figure 5. Selected disturbed rollouts illustrate how local correction can restore progress after contact changes. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the left and right halves as separate rollouts without and with residual correction. Within each task block, follow the image sequence from the first row into the second; blue arrows locate the introduced change. The upper task tilts the erasing surface, and the lower task displaces the box. Red and green outlines highlight the authors’ contrasting final outcomes. Section 4.4 interprets the failures as jamming or toppling and the corrected runs as recovery. Figures 7–10 in the appendix provide further selected sequences, while Figures 11–14 compare vision-only, force-observation and full-model behavior. e06e13e18
What it supports. The images provide concrete examples of the intended feedback behavior: the robot changes subsequent execution after the contact situation changes. Table 3 separately reports board recovery improving from 0/5 to 3/5 with correction, consistent with the illustrated case. The image sequence itself does not supply a recovery probability.
Where the evidence stops. These selected frames contain no synchronized wrench-error or gate trace establishing the cause of each adjustment. Table 3’s third task is labeled “Slice cucumber” although this figure and the text describe Pivot Box; its 4/5 result cannot be assigned to box pivoting without clarification.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Four-task real-world contact-rich manipulation Franka FR3; Erase Board, Peel Cucumber, Pivot Box and Wipe Vase under varied contact geometry. Each base/baseline uses 90 demonstrations per task; FAWAM adds 20 intervention episodes for correction. Twenty evaluation trials per task, at most 60 seconds each. | FAWAM: 15/20, 19/20, 17/20, 17/20; average 85.00%. Successful trials and average success rate | GE-Act 48.75%; π0.5 47.50%; ForceVLA and TA-VLA each 63.75%; FAWAM without residual correction 73.75%. Differences from the strongest vision-only and force-aware baselines are 36.25 and 21.25 percentage points. The abstract’s percent wording should not be read as relative improvement. Extra correction data means the full comparison is not data-matched. e01e08e09 |
| Force-component ablation The four-task real-robot protocol; Table 2 toggles force observation (Obs), joint prediction (Pre) and residual correction (Res). | Base 48.75%; Obs 56.25%; Obs+Pre 73.75%; Res-only 67.50%; full 85.00%. Average success rate | Adding Pre to Obs yields 17.50 percentage points; full exceeds Obs+Pre by 11.25 points. Res-only removes both future wrench preview and tracking error while retaining sensed contact features. It also changes the base policy, so full versus Res-only does not isolate predicted-reference guidance. e08e10e11 |
| Execution-time comparison Figure 4, same four tasks; timing population and failure handling are not precisely specified. | Full: 29.3, 36.3, 23.2, 31.8 in task order. Reported average execution time, seconds | Base: 47.9, 37.9, 38.7, 42.6 seconds. The full-model bars are shortest in each group. Appendix A.3 cautions that short rollouts can reflect early termination; its statement that some peeling variants are faster conflicts with these plotted values. e12 |
| Execution-time perturbation recovery Five real-world trials per perturbation; manual board tilt, cucumber-height change, box displacement and vase-tail-height change are described. | Erase board 3/5 versus 0/5; Peel cucumber 5/5 versus 2/5; Wipe vase 4/5 versus 1/5. Successful trials with versus without correction | The remaining Table 3 column reads “Slice cucumber” and reports 4/5 versus 0/5, while the text and Figure 5 describe Pivot Box. Retain the unresolved column label without assigning its counts to box pivoting. The unambiguous columns favor correction, but five trials per setting provide limited precision. e13 |
| Residual activation threshold sensitivity Appendix A.6 tests thresholds 0, 0.2, 0.5, 0.8 and 0.99 on Peel Cucumber and Wipe Vase. | At 0.8: 95% peeling and 85% wiping; at 0.99: 90% and 80%. Success rate | At zero: 80% on both tasks. Wiping also reaches 85% at 0.5. Selective activation helps these reported settings. The appendix does not state sweep-specific trial counts or show uncertainty, and this is not a universal optimal-threshold result. e19 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2. The component ablation supports complementary benefits but leaves the wrench-reference mechanism partly entangled. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the three left columns before reading the scores. Obs means force-conditioned observation in the base model; Pre means joint action–force prediction; Res means residual correction. The empty row is the vision-only base. Moving from Obs to Obs+Pre holds the observation component present while adding the predictive objective. The Res-only row is a different comparison: Appendix A.2 says its corrector still receives sensed contact features, base-action preview and proprioception, but cannot use future wrench predictions or tracking error. Therefore, the missing Obs checkmark does not mean that this residual network has no force input. e08e10e11e17
What it supports. Observation alone raises average success from 48.75% to 56.25%. Adding joint prediction reaches 73.75%; the full configuration reaches 85.00%. Residual-only reaches 67.50%. Together these rows support useful predictive and corrective components, while the full-versus-base comparison also includes intervention supervision.
Where the evidence stops. Full versus Res-only changes both the base model and the corrector’s reference inputs. The table lacks a matched-base comparison that removes only predicted guidance. The residual-only variant also uses the same frozen force encoder described in the appendix.
Figure 15. The residual gate trades frequent correction against missed opportunities to recover. Original paper, p. 19 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The horizontal axis lists tested values of the activation threshold tau; the vertical axis is task success rate, not the fraction of actions corrected. Blue circles represent Peel Cucumber, and orange squares represent Wipe Vase. Lowering tau makes the gate easier to activate because Section 3.3.1 uses the strict inequality g_t^res > tau. Compare the discrete measured points rather than treating the connecting lines as a calibrated response curve. Both tasks start at eighty percent for zero threshold; increasing selectivity helps over part of the tested range, but the highest threshold loses some of that gain. e06e19
What it supports. At tau = 0.8, peeling reaches 95% and wiping reaches 85%. Wiping also reaches 85% at 0.5. At 0.99 the scores fall to 90% and 80%, respectively. These observations support the authors’ choice of a selective gate while showing that greater conservatism is not always better.
Where the evidence stops. Appendix A.6 does not state sweep-specific trial counts, show uncertainty, or report actual activation frequency. The plot therefore supports a threshold sensitivity result on two tasks, not proof of sparse activation or a universally optimal value of 0.8.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors identify missing large-scale synchronized video–force data, limited correction episodes and indirect contact observability from wrist wrenches. Force does not condition video generation; dense tactile signals and local contact frames remain future work. e14
The evidence concerns four tasks on one robot, with no uncertainty estimates or direct quantitative wrench-prediction metric. Additional intervention supervision and incomplete ablation combinations prevent attributing all gains to prediction quality alone. e08e09e10e13e19
7.2 Questions for discussion
- Would accurate wrench references still help when the base policy and correction-training data are held fixed?
- Does the gate suppress harmful interventions on unseen contact geometries, or mainly fit the observed intervention distribution?
8. Reproducibility audit
8.1 Requirements and known gaps
Required sensing includes an ATI Axia80-M8 wrist sensor and three RealSense D435i cameras. RGB/proprioception are recorded at 30 Hz and wrench data at 120 Hz. Resize RGB from 640×480 to 256×192; filter wrenches with a second-order 10 Hz Butterworth low-pass filter and encode the latest 0.5 seconds, or 60 frames. e08e15
The supplied method leaves numerical preview/error/correction horizons, denoising-step counts, action/wrench normalization, exact success criteria and train/evaluation geometry separation insufficiently specified. These are prerequisites for a faithful replication despite the detailed optimizer recipe. e05e06e08e15e16e17
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Isolate the information in the predicted wrench reference
Reader-proposed experiment, not performed: freeze one trained force-envisioned base policy and its force encoder. Train matched-capacity residual correctors on the same intervention/non-intervention samples, using either all published inputs or sensed contact features, action preview and proprioception without predicted-wrench preview or tracking error. Keep the training budget, gate threshold and base actions fixed. Test both on identical held-out geometry and perturbation schedules, recording success, wrench-tracking error and gate activation fraction. Add a temporally shuffled-reference diagnostic to the full-input model. If correct references outperform both the sensed-only and shuffled conditions, that supports reference-specific information; comparable outcomes would weaken the claim that anticipated contact, beyond generic force feedback, drives recovery. e06e07e10e11e17
Check 2: Test gate selection on held-out nominal and disturbed execution
Reader-proposed experiment, not performed: keep the trained base and residual weights fixed and sweep the five published thresholds on Peel Cucumber and Wipe Vase. Separate threshold selection from final evaluation by choosing on validation conditions and reporting held-out conditions. Include both nominal runs and controlled height or inclination changes, with the same trial budget and sixty-second cap for each threshold. Record success, completion time with an explicit failure rule, peak measured wrench and actual gate activation fraction. The proposed mechanism predicts that permissive gates can harm nominal runs while overly strict gates miss useful recovery. If activation patterns or held-out success do not follow that tradeoff, Figure 15’s interpretation would need revision. e06e08e12e13e19
8.3 Reading coverage
Visual audit: Visually inspected the v2 title/author page; all original Figures 1–15 and Tables 1–3; method and training equations on pages 3–5; evaluation protocol on page 6; limitations on pages 8–9; hardware, preprocessing and optimizer details on page 13; and residual-only and timing analysis on page 14. Appendix qualitative pages 15–18 and threshold analysis on page 19 were included. All six final crops were individually inspected; the narrow ablation table and threshold plot were rendered at higher native PDF resolution. Diagram arrows, frozen/trained markers and the additive correction pathway were checked against Sections 3.2–3.3; the strict gate inequality comes from page 5. Reference-only pages 10–12 were read as text. Separate supplements, linked code and experiments remain outside this reading.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 13, 14, 15, 16, 17, 18, 19. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1 Introduction
- 2 Related Work
- 3 Method (3.1–3.3.2)
- 4 Experimental Results (4.1–4.4)
- 5 Conclusion
- 6 Limitation
- References
- A.1 Hardware and Training Details
- A.2 Residual-Only Ablation Details
- A.3 Task-Wise Ablation-Time Analysis
- A.4 Additional Perturbation Examples
- A.5 Additional Visualization Supplement
- A.6 Additional Analysis on Residual Activation Threshold
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- The inspected title page identifies arXiv:2606.08555v2, dated 12 June 2026. Title and all five authors match the catalog. The catalog submission date is 11 June 2026; the earlier version was not supplied, so revision-content differences cannot be determined. Identity evidence: e01.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This acquisition limitation was addressed by inspecting the original PDF figures, tables and supporting pages for this edition.
- Separate supplemental material availability has not been fully verified.
- All four supplied text chunks were read. PDF pages 1–9 and 13–19 were visually inspected; reference-only pages 10–12 were read as text. No external cited works, project site, code or data were inspected, and no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/author block, arXiv margin and Abstract
The exact title and authors Haotian He, Zeyu Yan, Qipeng Liu, Ning Guo and Wenzhao Lian appear with two university affiliations; the margin identifies arXiv:2606.08555v2, 12 Jun 2026. The abstract expresses gains as 36.25% and 21.25%.
Go to primary source ↓e02PDF pp. 1–2, Section 1 and Figure 1
Contact ambiguity motivates using force for perception, consequence prediction and execution-time correction; Figure 1 contrasts force-only conditioning with a predicted reference and feedback correction.
Go to primary source ↓e03PDF p. 3, Sections 3.1–3.2; p. 4, Figure 2(a–b)
The base force-envisioned action model runs at 1 Hz and residual correction at 10 Hz. The GE-Act video-to-action pathway is augmented with force modulation; Figure 2 separates the video DiT, force-action DiT and residual network.
Go to primary source ↓e04PDF p. 3, Section 3.2, Eq. (1); p. 4, Eq. (2) and following paragraph
A force MLP encodes wrench histories. Block-specific offsets modify AdaLN-Zero shift, scale and gate values; zero-initialized final layers initially recover the original action decoder.
Go to primary source ↓e05PDF p. 4, Section 3.2, training paragraphs and Eqs. (3)–(4)
Video latent flow matching precedes a frozen-video action stage. A joint projection and velocity prediction supervise actions and wrenches together. The source switches from F_t in prose to W_t in its concatenation/equations.
Go to primary source ↓e06PDF p. 5, Section 3.3.1, Eqs. (5)–(7) and gate definition
Residual inputs include measured-minus-predicted wrench history, frozen-encoder contact feature, future action/wrench chunks and proprioception. The MLP predicts residuals and a scalar gate; activation is strictly g_t^res > 0.8.
Go to primary source ↓e07PDF p. 5, Section 3.3.2 and Eq. (8)
FACTR correction collection uses an impedance-controlled follower and a leader tracking policy commands. Human-minus-base commands supply residual labels; balanced intervention/non-intervention batches train residual regression and binary cross-entropy.
Go to primary source ↓e08PDF p. 6, Figure 3 and Section 4.1
The FR3 setup uses an ATI Axia80-M8 and three D435i cameras. Four tasks vary board/vase inclination, cucumber table height or box position. The text specifies 90 demonstrations per task, 20 extra intervention episodes, sensor sampling rates and 20 trials per task with a 60-second cap.
Go to primary source ↓e09PDF p. 7, Table 1, all policy rows and task/Avg. SR columns
FAWAM records 15/20, 19/20, 17/20 and 17/20, averaging 85%. GE-Act is 48.75%, π0.5 47.50%, ForceVLA/TA-VLA 63.75%, and FAWAM without residual correction 73.75%.
Go to primary source ↓e10PDF p. 7, Table 2 and Section 4.3
Obs, Pre and Res mean force-conditioned observation, joint prediction and residual correction. Average success is 48.75%, 56.25%, 73.75%, 67.50% and 85.00% for the five displayed combinations.
Go to primary source ↓e11PDF pp. 13–14, Appendix A.2 and Eqs. (9)–(10)
Residual-only lacks predicted wrench preview and measured-minus-predicted error. It uses a frozen-encoder contact feature, base-action preview and proprioception; gate threshold, labels and intervention protocol match the full residual corrector.
Go to primary source ↓e12PDF p. 7, Figure 4 and Section 4.3; p. 14, Appendix A.3
Full execution times are 29.3/36.3/23.2/31.8 s versus Base 47.9/37.9/38.7/42.6 s. The plot legend uses Obs+Obj, whereas Table 2 uses Pre. A.3 warns about early termination and says some peeling variants are faster, despite the plotted full-model minimum.
Go to primary source ↓e13PDF pp. 7–8, Section 4.4; p. 8, Table 3 and Figure 5
Five perturbation trials per setting are reported. Without/with correction: board 0/5→3/5, peeling 2/5→5/5, the column labeled Slice cucumber 0/5→4/5, vase 1/5→4/5. The third label conflicts with the box-pivoting task described and pictured.
Go to primary source ↓e14PDF pp. 8–9, Section 6 Limitation
The authors state that force is not integrated into video generation, correction data may miss failure modes and large deviations, and end-effector wrenches supply indirect contact information. Local contact frames and dense tactile sensing are proposed future directions.
Go to primary source ↓e15PDF p. 13, Appendix A.1 Hardware and Figure 6
RGB streams are 640×480 and resized to 256×192. Wrenches are sampled at 120 Hz and filtered by a second-order 10 Hz Butterworth filter. The force encoder uses 0.5 seconds/60 frames.
Go to primary source ↓e16PDF p. 13, Appendix A.1 Training, first paragraph
Eight A100 GPUs train each base stage for 30,000 steps at batch 64, with bfloat16, ZeRO-2, AdamW, gradient clipping 1.0 and 1,000 warmup steps. Video/action learning rates are 3×10⁻⁵/1.5×10⁻⁴; lambda_F is 1; the force MLP has three layers and width 256.
Go to primary source ↓e17PDF p. 13, Appendix A.1 Training, residual-corrector paragraph
The frozen force encoder feeds a residual MLP with two shared width-256 GELU layers, dropout 0.1 and two heads. Training uses balanced batches of 128 for 500 epochs, AdamW at 3×10⁻⁴ and gate-loss weight 1.
Go to primary source ↓e18PDF p. 14, Appendix A.4; pp. 15–16, Figures 7–10; pp. 16–18, Appendix A.5 and Figures 11–14
Additional selected rollouts compare correction on/off and vision-only/force-observation/full variants across board erasing, cucumber peeling, box pivoting and vase wiping. These are qualitative sequences without trial-level aggregate statistics.
Go to primary source ↓e19PDF p. 19, Appendix A.6 and Figure 15
At thresholds 0/0.2/0.5/0.8/0.99, peeling success is 80/80/85/95/90% and wiping is 80/75/85/85/80%. The section studies selective activation but does not provide sweep-specific trial counts or uncertainty bars.
Go to primary source ↓8.5 Primary sources
FAWAM: Force-Aware World Action Models for Closed-Loop Contact-Rich Manipulation ↗
PDF · 6,914 extracted words
Source fingerprint
26345090da1c9a637bcacb94faba4f928143171da91d9ce7499257d5c97c3d26