FolDeX: A Physical-World Benchmark for Long-Horizon Robotic Manipulation of Deformable Objects

1. Paper overview

In one sentence: FolDeX turns complete physical garment folding into a controlled test of robot-data reuse, with promising recovery-augmentation results but incomplete scoring and transfer specifications. e-interfacee-trackse-baselinese-recoverye-missing-appendixe-transfer

Core contributions

• Source description

  A real-robot dataset and benchmark with 2,000+ hours, 20+ tasks, and 10+ embodiments, centered on garment folding. Its common schema records synchronized vision, proprioception, timestamped actions, language instructions, and episode metadata. e-data

• Author claim

  Four data-reuse tracks separate recovery augmentation, cross-task transfer, scene changes, and embodiment changes. FoldChallenge is described as an operational external-policy evaluation platform with held-out physical objects, controlled initializations, and auditable rollouts. e-platforme-tracks

• Reader analysis

  Demonstration-only π0 reference comparisons, a recovery-augmented comparison, and FoldScore connect execution success to final garment quality and efficiency. These are benchmark reference results, not a new predictive world-model architecture. e-metrice-baselinese-recovery

↗ Enlarge figure

How to read it. Read downward through three different roles. The upper cartoons motivate the sim-to-real and rigid-to-deformable gaps; they are not measured comparisons. The middle row shows what changes in each research track: recovery examples, task content, scene appearance or layout, and robot embodiment. Along the bottom, solid arrows run from benchmark data to the evaluation pipeline, rollout repository, and leaderboard. A dashed arrow returns from the repository toward the data side, illustrating reuse of deployment experience. The caption and track descriptions support this reading. These are data and evaluation relationships, not neural-network layers or a future-video-to-action computation graph. e-platforme-trackse-baselinese-visual-disclosure

What it supports. FolDeX's proposed contribution is control over what data are reused and how the resulting policies are tested. The repository and leaderboard complete an evaluation cycle around physical execution. The authors describe held-out objects and controlled initialization as safeguards for comparing submitted policies under consistent conditions.

Where the evidence stops. This is a conceptual overview, not proof of platform availability or a measured simulation gap. Its leaderboard uses rounded values; use Table 2 for precise results. The authors disclose AI-assisted visual drafting, and the PDF retains contradictory public-access/anonymity language.

2. Motivation

2.1 The problem and the proposed response

2.2 What this reading follows

Folding a garment is a chain of dependent physical decisions: a poor retrieval can make flattening harder, and a misplaced fold can spoil the final stack. FolDeX evaluates that complete chain and organizes real-robot experience around recovery, task, scene, and embodiment changes. Its π0 comparisons show why both the training data and the metric matter: recovery augmentation improves average success, while task-specific and multi-task policies trade places when ranked by success versus FoldScore. Read the figures as a benchmark design and the tables as preliminary physical evidence. The supplied version leaves the detailed quality rubric, some execution settings, and quantitative transfer comparisons unresolved. e-interfacee-trackse-baselinese-recoverye-missing-appendixe-transfer

3. Research context

We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.

This table preserves the labels recorded at reading time. The current major category is Benchmarks & simulators. View the current classification.

3.1 Evidence-based assessment

Classification assessment not applicable

This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.

4. Problem formulation

4.1 Inputs and outputs

4.2 Equations and their role

5. Method in detail

5.1 Follow the episode before interpreting the model

Reader analysis

Begin with the state the robot actually receives: the garment is in a basket, not already arranged for folding. Retrieval creates a new table configuration, which determines what flattening can accomplish; flattening then constrains the folds and the final stack. The policy repeatedly receives camera observations and proprioception and predicts future end-effector poses. This source-defined interface makes physical feedback central, but it does not introduce a predictive world model. The tutorial implication is that a local manipulation success cannot substitute for completion of the whole chain. Figure 2 explains why the trial definition matters: the photographs sample stages and task diversity, whereas the success rule on page 6 requires an entire episode without human intervention or unrecoverable failure. e-interfacee-protocol

↗ Enlarge figure

How to read it. Use the source caption to interpret the four rows. The top row samples stages of a long folding episode; the second shows garment diversity; the third adds other deformable and rigid-object tasks; the bottom varies robot embodiment and scene. The adjacent task definition supplies the full sequence: retrieve from a basket, flatten, fold, and place or stack. The montage should therefore be read together with that definition, rather than as four independent benchmark successes. At inference, the policy uses two wrist-camera views, an external workspace view, an instruction, and proprioception to predict future end-effector poses. e-interfacee-datae-protocole-baselinese-transfer

What it supports. The benchmark tests whether progress survives a sequence of contacts and changing garment configurations. A successful isolated fold is insufficient for the complete task. The surrounding task and embodiment diversity motivates transfer research, while the main numerical tables focus on four garment categories.

Where the evidence stops. Still images show examples of states and setups, not uninterrupted autonomous rollouts. They do not establish success rates or demonstrate cross-embodiment transfer. The paper describes the camera/action interface but omits detailed control frequency and trajectory-horizon settings.

5.2 Separate delayed execution from recovery experience

Reader analysis

Two changes address different problems. Training-time RTC simulates inference delay and conditions the policy on action prefixes already committed for execution. Its reference comparison changes the training configuration while keeping the multi-task, demonstration-only setting. Recovery augmentation instead changes the experience available for learning: operators collect corrections from deployed-policy states, and the track combines those trajectories with independent demonstrations for a separately initialized policy. Table 3 applies that idea to a single RTC π0 trained across four garments. A plausible reader interpretation is that recovery examples cover states missing from clean demonstrations, but the paper does not measure that coverage or isolate it from additional data volume. The experiments support a useful combination, while a budget-matched control is still needed to identify why it helps. e-rtce-trackse-baselinese-recovery

↗ Enlarge figure

How to read it. Both horizontal axes count trajectories, not hours or success percentages. The left panel groups records by embodiment, with orange deformable and blue rigid portions stacked where both appear. The right panel groups trajectories by task using the same colors. Aloha dominates the embodiment panel, while Fold shirt is the largest task category. Keep the panels separate: they are marginal summaries, not a task-by-robot matrix. The common data schema described on page 4 includes object-instance, scene, embodiment, and completion metadata, which would be needed to construct controlled subsets from these aggregate distributions. e-datae-baselinese-recoverye-transfer

What it supports. The collection spans several task families and robots, but its diversity is not balanced coverage. That distinction matters for interpreting data reuse: a gain from mixing datasets could reflect which categories dominate the mixture. This is a reader inference motivating controlled sampling, not a measured explanation of the reported transfer failures.

Where the evidence stops. The bars lack exact value labels and do not specify the actual training mixtures for Tables 2–3. Do not convert approximate bar lengths into exact split sizes or infer per-embodiment performance from these dataset counts.

5.4 Training and inference

During training

During inference

5.5 Implementation flow

1. Define a dependent physical episode

   Start with a garment in a basket. Retrieve it onto the table in a random configuration, flip and flatten it, perform multiple folds, then place or stack it. Earlier physical errors propagate into later stages; stage completion alone is insufficient. e-interface

2. Organize heterogeneous experience

   Record task, object category and instance, scene, embodiment, operator, data source, completion status, and failure or termination reason. Platform diversity includes Aloha, YAM, Piper, Astribot, Franka, UR5, a mobile dual-arm platform, and AgiBot G2; a shared record format does not itself align their action spaces. e-datae-transfer

3. Construct the recovery track

   Deploy a shared policy, collect human intervention and recovery trajectories, and combine them with independently collected demonstrations to train a separately initialized policy. The evaluation tests subsequent autonomous behavior, not success achieved through intervention during the scored episode. e-trackse-protocol

4. Hold evaluation conditions fixed

   Use the same tasks, garment instances, initialization procedure, maximum episode duration, and robot execution protocol. Unless otherwise specified, evaluate each garment category in 30 independent physical trials with randomized configurations. Success requires the whole sequence without human intervention or unrecoverable failure. e-metrice-protocol

5. Assess outcome quality

   Neatness has six levels, 0–5, based on final shape, compactness, flatness, structural stability, and edge alignment. At least two evaluators independently assess qualitative scores; the same evaluators assess all methods and a score is recorded only on agreement. e-metrice-protocol

6. Experiments & results

FolDeX evaluates complete physical garment-folding episodes and asks whether expensive robot experience can be reused across recovery states, tasks, scenes, and embodiments. Its reference policies use π0. Recovery-augmented training raises reported average success from 80.75% to 95.00%, but transfer studies remain preliminary and the missing quality-scoring appendix prevents independent reconstruction of FoldScore from the supplied paper alone.

6.1 Read the original evidence

↗ Enlarge figure

How to read it. This table evaluates one multi-task π0 policy trained with RTC on demonstrations plus recovery trajectories from all four garments. Read each row across success, average time in minutes:seconds, and FoldScore. The metric definition uses time over successful trials, so time should be interpreted alongside success rather than as a failure-inclusive duration. To assess augmentation, compare this table with the RTC Multi column of Table 2, not with task-specific training or the supplementary pre-flattened Shirt experiment. The recovery track collects human corrections during earlier deployment, then evaluates the resulting policy under the complete-task rule without human intervention. e-recoverye-baselinese-trackse-protocole-metrice-missing-appendixe-supplementary

What it supports. Average success rises from 80.75% to 95.00%, and FoldScore from 75.59 to 82.53. Pants and Towel reach 100.00% success, while Shirt and Skirt remain at 90.00%. The reported aggregate improvement is strong, but it is not a uniform gain: Shirt success falls from Table 2's printed 93.00%.

Where the evidence stops. Adding recovery trajectories changes both data content and data quantity. Without an equal-budget extra-demonstration control, the result cannot isolate recovery-specific benefit. Finite trial counts, missing uncertainty, and unavailable scoring details further limit claims of robust superiority.

6.2 Results and evaluation conditions

6.3 Ablations and diagnostic examples

Read component removals and qualitative examples within their stated evaluation conditions.

↗ Enlarge figure

How to read it. Each garment has a success-rate row and a FoldScore row. All columns use π0 and expert demonstrations; RTC means real-time chunking during training. Compare RTC Multi against Standard Multi to examine the RTC configuration within joint garment training. Compare RTC Multi against RTC Single to examine shared versus category-specific policies. The bottom two rows summarize the four categories, but the task rows expose tradeoffs hidden by those averages. For example, RTC Multi improves Pants success relative to Standard Multi while its Pants FoldScore is lower. FoldScore additionally incorporates final-state quality and successful-trial completion time, as defined on page 5. e-rtce-baselinese-protocole-metrice-missing-appendix

What it supports. RTC Multi reaches 80.75% average success versus 72.50% for Standard Multi, a calculated 8.25-percentage-point increase. RTC Single achieves the highest average success, 82.50%, but RTC Multi has the highest FoldScore, 75.59. Thus neither training choice nor metric produces a universally dominant policy across garments.

Where the evidence stops. This is a configuration comparison, not a fully controlled architecture ablation. No uncertainty is reported. Shirt's printed 93.00% needs raw-count clarification under the stated default of 30 trials; the absent quality rubric also limits independent FoldScore auditing.

7. Analysis & limitations

7.1 What the evidence leaves open

7.2 Questions for discussion

1. Would recovery augmentation retain its advantage over an equal budget of additional demonstrations?
2. How stable are FoldScore rankings across garment instances, raters, and episode timeouts?
3. Can embodiment transfer preserve complete-episode performance once action conventions are explicitly aligned?

8. Reproducibility audit

8.1 Requirements and known gaps

8.2 Proposed reproduction checks

The following checks are proposals motivated by the paper. They have not been run as part of this reading.

8.3 Reading coverage

8.4 Traceable evidence

8.5 Primary sources