Awesome-WAMs
GitHub
PAPER REPORTENAll readings ↗

Compact Visuotactile World Models for Lifting: Prediction, Reward Alignment, and Force Constraints

English reading report: Method, equations, original figures, experiments and reproducibility.

Authors: Qinzhen Ma; Sida Peng

Affiliations: Rice University; Zhejiang University

Source: 2609.09597 ↗ · Catalog record

Reading: 17 / 558 · 6 original figures & tables · ~20 min ·

1. Paper overview

In one sentence: Aligned touch improves force forecasts, but reward alignment improves lifting without overcoming the separate limits of dynamics, calibration and force regulation. e-designe-persistencee-public-coveragee-reward-revisione-followup

At a glanceWhat to know
Research problem
Source description

Local force accuracy can conceal persistent contacts, rare peaks and reward mismatch. The paper follows prediction through calibration to executed control, while keeping public optical sensing separate from simulator force aggregates. Its target is explicit force-budgeted lifting, without identifying friction or damage thresholds. e-designe-limitations

Core mechanism
Author claim

Aligned touch improves the compact model over a parameter-matched vision-only control, but persistence prevents attributing this gain to useful transition prediction alone. e-modele-persistence

A key reported resultMatched height-reward lifting on fresh ID environments: Height RL: 93.3% strict and 33.3% joint success.

Strict lift and full-episode 8 N force-budgeted success.. Second cohort, ten shared ID environments; three fixed policies per learned method.

Matched native RL: 20.0%/10.0%; force feedback: 100.0%/70.0%. Strict gain: 73.3 percentage points, paired 95% CI [66.7, 83.3]. Task attainment improves, but mean peak rises 0.432 N, CI [0.175, 0.711]. The interval conditions on fixed policies. e-reward-revisione-followupe-paired

Reading caution
Source description

Only forty independent control initial conditions underlie 680 executions. One geometry, simulated force aggregates and the missing vision-only height-reward ablation limit attribution; no real-robot or sensor-transfer result is established. e-computee-limitations

Core contributions

  • Author claim

    Aligned touch improves the compact model over a parameter-matched vision-only control, but persistence prevents attributing this gain to useful transition prediction alone. e-modele-persistence

  • Author claim

    Trajectory calibration reveals reliability failures hidden by frame coverage and exposes large force allowances that can make control excessively conservative. e-public-coveragee-sim-coverage

  • Author claim

    A matched reward change separates improved task attainment from force regulation, using actions actually executed on fresh shared simulator environments. e-reward-revisione-followup

Figure 1. The simulator branch trains a policy in a frozen predictor; the public tactile branch studies sensing independently. Original paper, p. 2 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Follow the upper arrows from simulator recordings through the 652,157-parameter world model, then through behavior-initialized actor–critic learning, to executed control. The downward arrow from the predictor reaches episode calibration, whose diagonal arrow supplies a force margin to imagined learning. This matches the soft penalty in Eq. (4); it is not a separate hard admission gate. The public tactile-recording box has no connecting arrow into policy learning. Its GelSight images and measured forces support the independent sensing study. Finally, use the matched-comparisons box as a checklist: modality, persistence, cloning, reward choice, feedback and margin penalties answer different questions. e-designe-modele-actore-calibration

What it supports. The diagram establishes a staged system: a learned predictor supports a separate actor’s training, and MuJoCo execution assesses the resulting actions. This architecture makes it possible to hold the world model fixed while changing the task reward. Public tactile regression does not supply evidence that the simulator policy works with optical tactile sensors.

Where the evidence stops. This is an experimental-design overview, not a layer-level architecture diagram. Sections III-A and III-C supply the encoder and actor details. Its force-margin arrow denotes a penalty during training, without a closed-loop safety guarantee.

2. Motivation

2.1 The problem and the proposed response

Source description

Local force accuracy can conceal persistent contacts, rare peaks and reward mismatch. The paper follows prediction through calibration to executed control, while keeping public optical sensing separate from simulator force aggregates. Its target is explicit force-budgeted lifting, without identifying friction or damage thresholds. e-designe-limitations

2.2 What this reading follows

A robot can predict contact forces accurately and still make poor lifting decisions. This paper tests that gap with two deliberately separate experiments: optical tactile force regression on public recordings, and a compact model that trains lifting policies in simulator imagination. Read the evidence in stages. First ask what touch adds beyond vision and persistence. Then check whether calibration protects whole trajectories rather than isolated frames. Finally compare executed lifting with the stricter requirement of staying inside a force budget. The most striking reward-revision result improves task attainment, while a simple feedback controller remains better at completing the task within the budget. e-designe-persistencee-public-coveragee-reward-revisione-followup

3. Research context

We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.

Catalog dimensionRecorded classification
Major categoryNot assigned
ArchitectureNot assigned
Prediction paradigmNot assigned
QuadrantNot assigned

This table preserves the labels recorded at reading time. The current major category is WAMs. View the current classification.

3.1 Evidence-based assessment

Insufficient evidence to decide

Reader analysis

The snapshot is unassigned, so no existing quadrant can be confirmed. Architecture evidence shows an action-conditioned latent world model plus a separately trained actor and critic, rather than one network jointly predicting futures and actions. It is not inverse dynamics: actions condition transitions, and policy optimization uses imagined returns. Auxiliary RGB reconstruction is training supervision, not execution. e-modele-actore-training

This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.

4. Problem formulation

4.1 Inputs and outputs

InputsOutputs
  • Three 64×64 external RGB frames, three simulated tactile force observations and three proprioceptive observations.
  • Actions condition latent transitions; separate public-data regression takes GelSight images.
  • Endpoint forces, per-finger interval-peak normal forces, relative height, support/contact, task reward and auxiliary 16×16 RGB.
  • A separate actor outputs vertical motion and continuous gripper rate.

4.2 Equations and their role

zt=eθ(ot−2:t),z^t+k+1=dθ(z^t+k,at+k)z_t=e_\theta(o_{t-2:t}),\qquad \hat z_{t+k+1}=d_\theta(\hat z_{t+k},a_{t+k})
Eq. (1): the encoder maps observation history o to latent z; transition d uses action a to predict future latent states. Hats denote predictions. e-model
gμ(F)=Fx2+Fy2−μFn,Si=max⁡tsitg_\mu(F)=\sqrt{F_x^2+F_y^2}-\mu F_n,\qquad S_i=\max_t s_{it}
Eqs. (2)–(3) and text: F contains tangential components and compression-positive normal force. The joint residual s maximizes underprediction of g at coefficients 0.3 and 0.8 and of normal force; S is its trajectory maximum. Coefficients are analysis conditions. e-calibration
r~k=r^k−12∑j=12([max⁡(0,p^kj)+q−B]+B)2−0.0052∥uk∥22\tilde r_k=\hat r_k-\frac12\sum_{j=1}^{2}\left(\frac{[\max(0,\hat p_{kj})+q-B]_+}{B}\right)^2-\frac{0.005}{2}\lVert u_k\rVert_2^2
Eq. (4): predicted task reward is reduced by excess per-finger peak p above budget B=8 N and action magnitude u. The positive part penalizes only excess. Margin q is zero or a fixed calibration allowance; returns use discount 0.95. e-actor
L=LF+Lp+Lh+0.25(LS+LC)+0.5Lr+0.5LI+20Lz\mathcal L=\mathcal L_F+\mathcal L_p+\mathcal L_h+0.25(\mathcal L_S+\mathcal L_C)+0.5\mathcal L_r+0.5\mathcal L_I+20\mathcal L_z
Eq. (5): terms supervise endpoint force F, peak p, height h, support S, contact C, reward r, RGB I and latent consistency z. The last is an auxiliary training target. e-training

5. Method in detail

5.1 Trace information from force sensing to an executed action

Source description

Start at an observed history, not at a future image. The simulator model encodes external RGB, per-finger force aggregates and proprioception into a compact latent. Its transition takes an action and predicts the next latent; heads decode force, height and other training targets. Future observations and privileged labels supervise learning without becoming actor inputs. The actor is then initialized by behavior cloning and optimized through five-step rollouts of this frozen model. Its objective includes predicted task reward, peak-force excess, action magnitude and deviation from a frozen behavior prior. A target critic bootstraps the imagined return. At evaluation, the learned actor supplies vertical and gripper commands in MuJoCo after a shared scripted approach. The public GelSight regressor is a separate experiment, not the actor’s tactile encoder. e-modele-actore-control-protocole-public-datae-design

5.2 Protect a whole trajectory, then ask whether the bound is useful

Source description

In the public-data study, the contact function combines tangential force magnitude with compression-positive normal force under a specified coefficient. The joint residual checks both coefficient endpoints and normal-force underprediction; maximizing it over time makes each trajectory one calibration unit. The corrected rank is the ceiling of (n+1)(1−alpha), with negative quantiles clipped to zero and insufficient calibration units yielding an infinite bound. Simulator calibration uses different residuals: episode maxima for interval-peak underprediction and absolute height error, with separate component bounds. With only 24 calibration episodes, each 95% component uses the largest residual. Two resulting peak allowances already exceed 8 N before adding predicted force. This explains why a statistically motivated allowance can be impractical as a hard gate and overly conservative even as a soft training penalty. e-calibratione-sim-coveragee-actor

5.3 Interpret the reward intervention without conflating its outcomes

Reader analysis

The first control round produces the motivating mismatch: visuotactile RL increases native return while strict lifting falls relative to behavior cloning. The follow-up changes the imagined reward to clipped predicted height, keeping the world model and initial actor fixed and re-evaluating both rewards on a fresh shared cohort. This is strong evidence for a reward-related task-attainment improvement within that comparison. My interpretation is that the force-budget columns provide the essential second test: height-reward RL still trails feedback, its mean episode peak increases, and conservative margins continue to suppress completion. The source also lacks a vision-only height-reward arm, so this intervention cannot establish that touch causes the control improvement. A fresh confirmatory comparison on feasible shifted tasks would be needed before treating the finding as general reliable control. e-initiale-reward-revisione-followupe-pairede-feasibilitye-limitations

5.4 Training and inference

During training

Source description

The simulator dataset has 160 episodes, split 80/16/24/20/20 for training, validation, calibration, ID and OOD testing. Each contains 150 actions. Both modalities train for 25 epochs over three seeds; vision-only zeros normalized tactile inputs. Five-transition supervision combines normalized Huber losses, contact/support classification, auxiliary RGB and stop-gradient EMA latent consistency. e-sim-datae-modele-training

Source description

The frozen-model actor starts from behavior cloning: 1,000 BC updates precede 1,500 imagined updates using 3,520 encoded starts, batch 256. Five-step discounted returns use a target critic, detached critic targets and a behavior-prior deviation penalty. Actor/critic learning rates are 2e-4/3e-4. No simulator reward or object-state queries occur during imagined optimization. e-actor

Source description

The separate force CNN uses training-only normalization, at most 30 AdamW epochs and validation-selected checkpoints over three seeds; trajectory grouping prevents neighboring frames crossing partitions. e-public-data

During inference

Source description

Learned actors execute closed-loop vertical/gripper commands during actions 44–134 after a common scripted approach; lowering starts at 135. The approach uses initial object position, while learned policies lack object state, mass and friction. No online data collection or world-model retraining occurs during policy learning. e-control-protocole-actor

Source description

The follow-up changes only imagined task reward to clipped predicted height divided by 0.10 m, matching frozen models and BC initialization. It rewards progress but does not explicitly encode the required ten-observation hold. e-reward-revision

5.5 Implementation flow

  1. Encode and predict

    Visual, tactile and proprioceptive encoders produce 256-, 64- and 32-dimensional features, projected into a 128-dimensional SimNorm latent. Two 384-unit transition layers predict action-conditioned futures. The 652,157-parameter model starts randomly. Privileged object state, contact labels and interval peaks supervise heads but never enter the encoder or actor. e-model

  2. Calibrate the relevant unit

    Public-data calibration maximizes joint contact-function underprediction over each trajectory. Simulator calibration instead maximizes peak-force underprediction and absolute height error over each episode’s evaluated windows and five horizons, allocating half the error probability to each quantity. Bounds apply to recorded-action forecasts under exchangeability, not arbitrary optimized policies. e-calibration

  3. Separate sensing from control

    The independent 361,091-parameter force CNN uses 102,628 paired GelSight/ATI frames from 1,513 trajectories. Sphere train/validation/calibration/test partitions contain 453/75/114/116 entire trajectories; batch 6 and flat/sharp shapes are held out. These records lack world-model action/reward sequences. e-public-data

6. Experiments & results

This study asks whether better contact prediction produces better lifting decisions. A compact visuotactile model improves forecasts over vision alone, but persistence challenges the value of its dynamics. An exploratory reward revision improves executed simulator lifting, while simple force feedback remains stronger under a force budget. A separate GelSight experiment exposes the gap between frame and trajectory reliability.

6.1 Read the original evidence

Table I. Within-shape force accuracy reverses under exploratory shape shift. Original paper, p. 3 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Start with the test-distribution column, because the rows describe different generalization questions. The first sphere row holds out entire trajectories from batches used for the main split; the next holds out batch 6. Flat and sharp records test unseen shapes. The second column separates trajectory counts from frame counts, preserving the grouping used to avoid neighboring-frame leakage. Compare CNN MAE with the training-mean baseline within each row, using newtons throughout. These are three-component force errors from GelSight Mini images paired with ATI Nano17 measurements, not errors of the simulator world model or robot lifting success rates. e-public-datae-public-resultse-design

What it supports. On held-out sphere trajectories, the CNN reduces MAE from 0.11870 to 0.04234 N; it also performs well on batch 6. Flat and sharp errors rise to 0.32685 and 0.41304 N, both worse than their constant baselines. The table therefore supports local sensing accuracy alongside a clear generalization failure in this protocol.

Where the evidence stops. Batch identifiers do not establish cross-day or cross-sensor transfer. Flat/sharp tests are exploratory and retain only paired entries after unmatched trailing indices are excluded. The public records contain no action/reward sequences for training this world model.

Table II. Choosing trajectories as calibration units exposes the gap hidden by frame coverage. Original paper, p. 4 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read across each row before comparing methods. Frame coverage counts individual observations, whereas trajectory coverage requires the joint contact-function bounds to hold throughout a complete contact trajectory. All calibrated comparisons target nominal 90% coverage. Direct frame calibration nearly reaches that level for frames, but its trajectory column collapses. Moving to trajectory maxima changes the unit of protection. The rightmost column then shows the price paid in allowance. As specified in the original table caption, allowances average over frames, three tested coefficient values and training seeds; the propagated-box allowance is not the coordinate radius of its underlying box. e-calibratione-public-coverage

What it supports. Direct trajectory calibration raises complete-trajectory coverage from 15.80% under frame calibration to 87.36%, using a 0.3338 N mean allowance. Propagated coordinate boxes cover 99.14% of trajectories but require 0.6878 N. Thus the narrower direct construction is less conservative, while its observed coverage remains below the nominal 90% target.

Where the evidence stops. The direct trajectory coverage interval is [81.90%, 92.24%]. The guarantee assumes a fixed predictor and exchangeable trajectories; it does not extend automatically to new shapes, optimized policies or error conditional on envelope acceptance.

Table V. Reward alignment improves lifting; the stricter force-budget comparison still favors feedback. Original paper, p. 6 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. First compare the native and height rows for ordinary V+T RL. They share world-model checkpoints, behavior-cloning initialization and the same fresh test environments; only the imagined task reward changes. Next move from strict lift to lift within 8 N: the latter intersects the ten-sample height criterion with no force violation anywhere in the episode, including approach and lowering. The two rightmost ID columns show violations and mean episode peaks separately. Finally inspect margin variants and force feedback. Learned rows average three fixed policies on ten ID and ten OOD environments; the source caption reports zero OOD strict and joint success. e-control-protocole-reward-revisione-followupe-limitationse-measurement

What it supports. Height-reward RL reaches 93.3% ID strict success versus 20.0% for native reward, but only 33.3% succeeds within budget, compared with feedback’s 70.0%. Adding the margin reduces violations to 6.7% yet leaves only 13.3% joint success. The paper’s central tradeoff is visible in these separate columns: task attainment and force regulation improve differently.

Where the evidence stops. This follow-up was motivated by the first round and remains exploratory. The ten shared ID environments are the independent units, not thirty policy executions. No vision-only height-reward arm is provided, so the improvement cannot be attributed to touch.

6.2 Results and evaluation conditions

Task & protocolReported resultComparison & interpretation
Public tactile force regression

Held-out sphere trajectories, 116 trajectories/9,003 paired frames; three training seeds.

0.04234 ± 0.00306 N (training-seed SD); trajectory-bootstrap 95% CI [0.03823, 0.04675] N.

Mean absolute force-component error, N; lower is better.

Training-mean baseline: 0.11870 N. Flat/sharp CNN errors: 0.32685/0.41304 N versus baseline 0.13783/0.12432 N.

Accurate within-shape sensing does not transfer to the exploratory held-out shapes. e-public-results

Public contact-function trajectory coverage

Held-out sphere test; nominal 90% joint coverage.

Direct trajectory calibration: 87.36% coverage, 0.3338 N allowance; coverage 95% CI [81.90%, 92.24%].

Complete-trajectory coverage and mean contact-bound allowance.

Frame calibration: 15.80% trajectory coverage despite 89.62% frame coverage. Propagated box: 99.14% trajectory coverage, 0.6878 N allowance.

Smaller direct bounds trade observed coverage against conservatism; this is offline sensing reliability. e-public-coverage

Recorded-action simulator force forecasting

Twenty ID test episodes, 600 windows, horizons 1–5; three seeds.

V+T: 0.228 ± 0.029 / 0.523 ± 0.051 N.

Endpoint/interval-peak MAE, N; mean ± sample SD.

Vision: 1.058 ± 0.062 / 2.724 ± 0.287 N. Persistence: 0.095 / 0.498 N.

Touch improves modality comparison, but persistence wins both ID force targets. Learned dynamics improve OOD peaks over persistence: 0.849 versus 0.966 N. e-sim-datae-forecaste-persistence

Initial executed lifting

First cohort: ten ID plus ten OOD environments; learned rows average three fixed policies.

V+T RL: return 61.365; strict success 6.7%.

Native return and strict lift success.

V+T BC: return 55.943; strict success 15.0%.

Higher native reward accompanies worse evaluated lifting, motivating the exploratory follow-up. e-initial

Matched height-reward lifting on fresh ID environments

Second cohort, ten shared ID environments; three fixed policies per learned method.

Height RL: 93.3% strict and 33.3% joint success.

Strict lift and full-episode 8 N force-budgeted success.

Matched native RL: 20.0%/10.0%; force feedback: 100.0%/70.0%. Strict gain: 73.3 percentage points, paired 95% CI [66.7, 83.3].

Task attainment improves, but mean peak rises 0.432 N, CI [0.175, 0.711]. The interval conditions on fixed policies. e-reward-revisione-followupe-paired

Calibrated-margin control in the reward follow-up

Same second-cohort ID environments and fixed-policy averaging.

Margin-height RL: 6.7% violations, 13.3% joint success.

Force-violation and joint-success rates.

Unmargined height RL: 60.0%/33.3%; margin-native: 6.7%/20.0%.

Few violations coexist with poor completion; the height revision does not repair conservative margins. e-followup

6.3 Ablations and diagnostic examples

Read component removals and qualitative examples within their stated evaluation conditions.

Table III. Touch improves every displayed forecast target, but this modality comparison does not settle whether dynamics are useful. Original paper, p. 4 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Compare vision and V+T within each split, rather than reading the OOD rows as a uniformly harder version of ID. Every entry averages forecast horizons one through five along recorded actions; the plus/minus terms are sample standard deviations across three training seeds. Endpoint force averages six signed/contact components, while interval peak averages two per-finger normal-force maxima over physics substeps. The height column uses millimeters. The models have identical parameter counts: the vision-only variant masks normalized tactile inputs to zero. This makes the table a controlled modality comparison, but it does not compare learned transitions against persistence. e-modele-sim-datae-forecaste-persistence

What it supports. For ID forecasts, adding touch reduces endpoint MAE from 1.058 to 0.228 N and peak MAE from 2.724 to 0.523 N. Section V-B supplies the crucial counterexample: persistence obtains 0.095 and 0.498 N, respectively. Touch is useful input, yet these force metrics do not demonstrate an advantage from predicting state transitions.

Where the evidence stops. Persistence holds measured endpoint touch constant, but holds the model’s current estimate for peak and height targets. OOD has a different contact mix; lower all-phase error cannot establish improved contact generalization. These are forecasts, not control outcomes.

Figure 2. The paired comparison isolates reward choice while keeping force violations distinct from successful lifting. Original paper, p. 6 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read each horizontal position as height reward minus native reward, measured in percentage points. The left panel omits the calibration margin and the right includes it. The dashed vertical line marks no change. Blue rows represent successful task outcomes, so movement to the right is favorable; the red force-violation row explicitly labels positive movement as worse. Error bars are 95% intervals from resampling the ten shared ID environments while holding the three trained policies fixed. The top row tests ten sampled heights; the middle also requires the full-episode force budget. The bottom plots violation-rate differences, not changes in peak force measured in newtons. e-pairede-followupe-reward-revisione-measuremente-feasibility

What it supports. Without a margin, strict lifting improves by 73.3 percentage points, with interval [66.7, 83.3]. The joint-success gain is smaller and less precise: 23.3 points, interval [0.0, 50.0]. The margin panel does not show a corresponding lifting gain. Reward alignment therefore repairs task attainment in ordinary RL without demonstrating improved force regulation.

Where the evidence stops. These intervals measure environment variation conditional on fixed policies, not uncertainty across repeated policy training. The reward comparison is matched but exploratory, and the OOD group’s feasibility confound prevents extending the ID conclusion to general shifted-task competence.

7. Analysis & limitations

7.1 What the evidence leaves open

Source description

Only forty independent control initial conditions underlie 680 executions. One geometry, simulated force aggregates and the missing vision-only height-reward ablation limit attribution; no real-robot or sensor-transfer result is established. e-computee-limitations

Source description

OOD changes mass and friction jointly. Its ideal friction-only pinch cannot support even the lightest object within budget; zero success therefore does not isolate a learning-generalization failure. e-feasibility

Source description

Simulator peak allowances are 13.204, 6.945 and 9.438 N, with ID joint coverage 80/85/90% and OOD 65/50/45%. Recording-based bounds also degrade under learned-policy distributions; retrospective actual-action diagnostics are not online safety tests. e-sim-coveragee-policy-coverage

Reader analysis

The height criterion checks sampled observations, whereas force peaks inspect every physics substep. These success definitions should remain separate from native simulator success and imagined return. e-control-protocole-measurement

7.2 Questions for discussion

  1. Does action conditioning beat persistence when contacts change rapidly?
  2. Would height-reward gains remain with vision-only inputs and fresh feasible shifts?

8. Reproducibility audit

8.1 Requirements and known gaps

Source description

Use robosuite 1.5.1/MuJoCo 3.3.7, episode-disjoint splits and exact substep peak aggregation. Appendix A specifies world-model AdamW at 3e-4, batch 64, clipping 20 and EMA rate 0.01. Public-data reproduction needs the pinned dataset revision and trajectory grouping. e-sim-datae-traininge-public-data

Reader analysis

The source reports ancillary code and split/result manifests; their availability needs verification. Reported six-model training time is 178.17 seconds on an RTX 5090 Laptop GPU, excluding data generation, controller execution and analysis, so it is not end-to-end reproduction cost. e-artifactse-compute

8.2 Proposed reproduction checks

The following checks are proposals motivated by the paper. They have not been run as part of this reading.

Check 1: Does action conditioning beat persistence when contacts change?

Reader-proposed check, not executed: train the full visuotactile predictor and a parameter-matched action-masked transition model with identical episode splits, initialization seeds, losses and update counts. Compare both with the paper’s endpoint and current-estimate persistence baselines on the same recorded-action test windows. Report endpoint, interval-peak and height errors separately for all windows and for a contact-changing subset defined before testing. Bootstrap whole episodes. A reproducible full-model advantage over both action masking and persistence on changing contacts would support useful dynamics; a gain only over vision masking, with no persistence advantage, would weaken that interpretation. e-modele-traininge-sim-datae-forecaste-persistence

Check 2: Confirm reward gains with modality and physical feasibility controlled

Reader-proposed check, not executed: predefine a vision/visuotactile by native/height-reward comparison, matching each modality’s frozen world model and BC initialization across rewards. Include the original force-feedback controller and a fixed-margin arm. Evaluate all policies on fresh shared ID environments and on mass-only and friction-only shifts chosen to permit lifting under both the contact budget and actuator limits. Retain whole-episode force peaks, failed grasps and the ten-sample height metric. Report paired environment intervals separately from training-seed variation. Persistent height gains across modalities would identify a reward effect; force-budgeted gains over feedback would be needed to support improved constrained control. e-reward-revisione-followupe-feasibilitye-sim-datae-control-protocole-measuremente-limitations

8.3 Reading coverage

Visual audit: All eight original PDF pages were rendered and visually inspected. Page 1 verifies title, both authors, affiliations and v1 date. Page 2 supplies Figure 1, encoder details and calibration equations; page 3 supplies the actor objective and Table I/public protocol. Page 4 supplies Tables II–III, persistence controls, simulator protocol and calibration limits. Page 5 supplies Table IV, force-budget feasibility, retrospective diagnostics and matched reward-revision details. Page 6 supplies Table V, Figure 2, paired intervals, compute and conclusions. Page 7 supplies limitations, Eq. (5), training settings, substep measurements, statistical units and ancillary-material statements; page 8 contains the remaining references. Both figures and all five tables were inspected. The six final original crops were individually viewed, retaining their complete graphic/table content; long captions are excluded and relevant qualifications are restated with evidence. Figure 1 arrows agree with the staged method and soft margin in Eq. (4); Figure 2 signs and units agree with Table V and its caption. Ancillary code, expanded metric files and separate supplements remain outside this supplied reading.

PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8. Appendix coverage: reviewed.

Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.

Text reading scope & known omissions
  • Abstract and title metadata (p. 1)
  • I. Introduction (p. 1)
  • II. Related Work (p. 1)
  • III. Method, A–C (pp. 2–3)
  • IV. Public Tactile-Force Study, A–C (pp. 3–4)
  • V. Simulator World-Model and Interaction Study, A–H (pp. 4–6)
  • VI. Limitations and Conclusion (pp. 6–7)
  • Appendix: Implementation and Reproducibility Details, A–B (p. 7)
  • Data, Code, and Tool Use (p. 7)
  • References (pp. 7–8)

Outside the original text pass

  • Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
  • Separate supplemental material availability has not been fully verified.
  • All four supplied text chunks and all eight PDF pages were read; the extraction limitation was addressed by visual inspection of both figures, all five tables, equations and appendix layout.
  • Ancillary code, split manifests and metric files mentioned in the paper were not supplied or inspected; experiments were not reproduced.
  • Version scope: the inspected artifact is arXiv:2609.09597v1, dated 9 September 2026. Its title and both authors match the catalog. No other revision or edition was supplied or compared.

The visual audit above records the subsequent illustrated pass.

8.4 Traceable evidence

e-identityPDF p. 1, title, byline, affiliation footnotes and arXiv version stampInspect

The title matches the supplied catalog. Qinzhen Ma is affiliated with Rice University and Sida Peng with Zhejiang University. The stamp identifies arXiv:2609.09597v1 [cs.RO], 9 September 2026.

Go to primary source ↓
e-designPDF pp. 1–2, Introduction, Related Work and Figure 1 with captionInspect

The public GelSight sensing study and simulator force-aggregate control study are separate. Figure 1 routes simulator recordings through a compact world model and frozen-model actor–critic learning to executed control, with an episode-calibration force margin. This adapts TD-MPC2 building blocks rather than its complete algorithm.

Go to primary source ↓
e-modelPDF p. 2, Section III-A, Eq. (1)Inspect

Three RGB/touch/proprioceptive observations feed 256/64/32-dimensional encoders and a 128-dimensional SimNorm latent; the transition has two 384-unit hidden layers. The randomly initialized model has 652,157 parameters. Heads include endpoint force, interval peaks, height, support, contact, reward and auxiliary RGB. Privileged labels are targets only; vision-only masks normalized touch to zero.

Go to primary source ↓
e-calibrationPDF pp. 2–3, Section III-B, Eqs. (2)–(3) and continuationInspect

Public contact-function residuals maximize over two coefficient endpoints and normal-force underprediction, then over time for trajectory calibration. Corrected split-conformal ranks assume a fixed predictor and exchangeable trajectories. Simulator scores separately maximize interval-peak underprediction and absolute height error over episode windows and five horizons, splitting error probability between the quantities.

Go to primary source ↓
e-actorPDF p. 3, Section III-C, Eq. (4)Inspect

A behavior-initialized two-output actor learns five-step imagined returns in the frozen world model. Eq. (4) adds normalized peak-excess and action penalties; ordinary RL uses zero margin. Discount is 0.95, behavior-prior deviation weight is two, and critic targets are detached. Training uses 3,520 starts, 1,000 BC and 1,500 imagined updates, batch 256, and actor/critic learning rates 2e-4/3e-4, without simulator queries during updates.

Go to primary source ↓
e-public-dataPDF p. 3, Section IV-A; p. 7, Appendix AInspect

TacBench GelSight Mini images paired with ATI Nano17 forces provide 102,628 frames from 1,513 trajectories, at revision 136db55485b6501605b1d2648ce0fe44740e302e. Sphere batches 1–5 use trajectory-disjoint train/validation/calibration/test counts 453/75/114/116. Batch 6 and flat/sharp shapes are held out. Only paired records are retained; five unmatched trailing image indices occur in flat/sharp data. The 361,091-parameter CNN uses validation selection, three seeds and training-only normalization; derived slip labels are excluded from independent slip evaluation.

Go to primary source ↓
e-public-resultsPDF p. 3, Table I and Section IV-B; p. 7, Appendix BInspect

Sphere-test CNN MAE is 0.04234 N versus 0.11870 N for the training-mean baseline; batch 6 is 0.03915 versus 0.11322 N. Flat and sharp CNN MAEs are 0.32685 and 0.41304 N, worse than baselines 0.13783 and 0.12432 N. Sphere seed SD is 0.00306 N and its fixed-regressor trajectory-bootstrap 95% interval is [0.03823, 0.04675] N. MAE averages three force components and paired frames.

Go to primary source ↓
e-public-coveragePDF p. 4, Table II and caption; p. 3, Section IV-CInspect

At nominal 90%, direct frame calibration gives 89.62% frame and 15.80% trajectory coverage with mean allowance 0.1096 N. Direct trajectory calibration gives 99.24%/87.36% with 0.3338 N, versus propagated coordinate-box calibration 99.87%/99.14% with 0.6878 N. Allowances average frames, three tested coefficient values and seeds; the box allowance is not its coordinate radius. Direct trajectory coverage has bootstrap interval [81.90%, 92.24%]. At coefficient 0.8 and a 2 N ceiling, direct calibration accepts 18.84% of frames with 0.192% conditional proxy violations; box propagation accepts none.

Go to primary source ↓
e-sim-dataPDF p. 4, Section V-AInspect

Modified robosuite 1.5.1 Panda Lift runs in MuJoCo 3.3.7. Its 160 episodes have 150 actions at 20 Hz and 25 physics substeps per action, split 80/16/24/20/20 into train/validation/calibration/ID/OOD. The cube is fixed; mass and friction shift jointly. Per-finger actuator limits are 3, 6, 12 and 20 N, distinct from the contact budget. Stride-five evaluation gives 600 windows per test split following recorded actions.

Go to primary source ↓
e-forecastPDF p. 4, Table III, caption and Section V-BInspect

Mean ± sample SD over three training seeds, averaged over horizons 1–5: ID V+T endpoint/peak/height MAEs are 0.228±0.029 N, 0.523±0.051 N and 2.822±0.336 mm versus vision 1.058±0.062 N, 2.724±0.287 N and 3.946±0.561 mm. OOD V+T gives 0.373±0.024 N, 0.849±0.100 N and 2.321±0.405 mm. Endpoint force averages six components; interval peaks average two normal maxima.

Go to primary source ↓
e-persistencePDF p. 4, Section V-B, contact-conditioned errors, persistence and cross-episode tactile controlInspect

Endpoint persistence gives 0.095/0.203 N ID/OOD MAE, better than learned dynamics. Persistence holds current model estimates for other targets: ID peaks are 0.498 versus 0.523 N and height 5.043 versus 2.822 mm; OOD peaks are 0.966 versus learned 0.849 N. Cross-episode tactile histories raise ID endpoint/peak errors to 1.736/4.599 N. Contact counts differ, so lower all-phase OOD vision error does not establish better contact generalization.

Go to primary source ↓
e-sim-coveragePDF p. 4, Section V-CInspect

With 24 calibration episodes, rank ceil(25×0.95)=24 selects the maximum residual per component. Peak allowances across three seeds are 13.204, 6.945 and 9.438 N; joint episode coverage is 80/85/90% ID and 65/50/45% OOD. Two allowances exceed the entire 8 N budget before any nonnegative predicted peak is added. Soft margins are penalties, not safety certificates.

Go to primary source ↓
e-control-protocolPDF pp. 4–5, Section V-D; p. 7, Appendix BInspect

Matched controllers share a scripted approach using initial object position, a declared privileged setup component. Actions 44–134 change only vertical motion and gripper rate; lowering starts at 135. Strict lift requires ten consecutive 20 Hz samples at relative height at least 0.10 m before lowering. Joint success additionally forbids any per-finger peak above 8 N throughout all 150 actions; failures remain in the denominator.

Go to primary source ↓
e-feasibilityPDF p. 4, Section V-A; p. 5, Section V-DInspect

OOD mass is 0.65 or 0.90 kg and friction 0.12 or 0.22. Under an ideal horizontal two-finger friction-only pinch, two 8 N normal forces with friction at most 0.22 support at most 3.52 N, below the lightest OOD weight 6.38 N. This simplified infeasibility calculation does not rule out every geometric or dynamic MuJoCo strategy.

Go to primary source ↓
e-initialPDF p. 5, Table IV and Section V-EInspect

Initial control uses twenty fresh environments, ten ID and ten OOD; learned rows average three fixed policies. Across both domains V+T BC/RL native returns are 55.943/61.365 while strict success is 15.0/6.7%. Margin RL gives 8.3% force violations and 5.0% joint success, versus feedback 30.0% violations and 25.0% joint success. All OOD strict/joint successes are zero; learned-policy repetitions do not create sixty independent environments.

Go to primary source ↓
e-policy-coveragePDF p. 5, Section V-F; p. 6, Section V-G continuationInspect

Retrospective forecasts condition on subsequently executed future actions with original calibration unchanged. First-cohort ID joint coverage is [0,60,40]% for V+T RL versus [80,100,80]% for BC. Second-cohort coverage is [10,80,80]% for height RL, [0,60,50]% for native RL and [100,100,100]% for BC. These are different induced distributions, not online planning or safety evaluations.

Go to primary source ↓
e-reward-revisionPDF p. 5, Section V-GInspect

The exploratory follow-up replaces only the imagined task reward with clipped predicted height divided by 0.10 m. Frozen world-model checkpoints, BC actor initial parameters, margins, penalties, action ranges and training settings are matched. Native and new policies are all executed on twenty fresh shared environment seeds, ten ID and ten OOD, avoiding cross-cohort reward comparisons.

Go to primary source ↓
e-followupPDF p. 6, Table V and Section V-G continuationInspect

Second-cohort ID V+T native/height RL strict success is 20.0/93.3%, joint success 10.0/33.3%, violations 56.7/60.0%, and mean peak 8.608/9.041 N. Feedback obtains 100.0% strict and 70.0% joint success with 30.0% violations. Margin native/height RL gives 23.3/20.0% strict and 20.0/13.3% joint success, both at 6.7% violations. All OOD strict and joint successes remain zero.

Go to primary source ↓
e-pairedPDF p. 6, Figure 2, caption and Section V-G continuation; p. 7, Appendix BInspect

Figure 2 plots height-minus-native percentage-point differences with fixed-policy environment-bootstrap 95% intervals. Unmargined RL improves strict success by 73.3 points [66.7,83.3] and joint success by 23.3 [0.0,50.0]. Mean peak rises 0.432 N [0.175,0.711]. Height-RL joint success trails feedback by 36.7 points, difference interval [-63.3,-10.0]. Paired resampling uses whole shared environments, not repeated policy training.

Go to primary source ↓
e-computePDF p. 6, Section V-HInspect

Both rounds comprise 680 simulator executions on forty independent initial conditions. Reported RTX 5090 Laptop GPU training totals are 178.17 seconds for six world models, 215.13 seconds for the first nine actor-training runs and 129.45 seconds for six follow-up runs; data generation, controller execution and analysis are excluded.

Go to primary source ↓
e-limitationsPDF pp. 6–7, Section VIInspect

The source identifies one rigid geometry, simulated force aggregates, small independent cohorts, poor shape transfer and conservative calibration. The follow-up lacks a vision-only height-reward ablation. It establishes neither public-sensor transfer to the simulator policy nor real-robot success, identified friction, deformable-object damage thresholds or optimized-policy safety.

Go to primary source ↓
e-trainingPDF p. 7, Appendix A, Eq. (5); p. 2, Section III-AInspect

World-model losses weight force/peak/height by one, support/contact by 0.25, reward/RGB by 0.5, and latent consistency by 20. Supervision includes initial and five future states; latent consistency uses future states only with a stop-gradient EMA encoder. AdamW uses learning rate 3e-4, weight decay 1e-4, batch 64, gradient clipping 20 and EMA rate 0.01; seeds 0–2 run 25 epochs with validation-loss selection. The actor has two 128-unit hidden layers and 33,282 parameters.

Go to primary source ↓
e-measurementPDF p. 7, Appendix BInspect

Ten height samples at 20 Hz span 0.45 seconds between first and last, without testing continuous intermediate height. Force violations inspect every 0.002-second physics substep. Environment-bootstrap intervals use 2,000 paired, domain-stratified resamples and fixed trained policies, distinct from training-seed standard deviations.

Go to primary source ↓
e-artifactsPDF p. 7, Data, Code, and Tool UseInspect

The source states that code, split manifests and per-seed/per-episode results accompany the submission as ancillary material. It identifies the public dataset license as CC BY-NC 4.0 and TD-MPC2 revision e9f5932 with the MIT license retained; no pretrained world-model weights are used.

Go to primary source ↓

8.5 Primary sources

Scroll across the image to inspect details. Press Esc to close.