ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations

English reading report: Method, equations, original figures, experiments and reproducibility.

Authors: Tongzhou Mu; Zhan Ling; Fanbo Xiang; Derek Yang; Xuanlin Li; Stone Tao; Zhiao Huang; Zhiwei Jia; Hao Su

Affiliations: University of California San Diego

Source: NeurIPS 2021 Datasets and Benchmarks Track · ref-18fe7d2bfb0e93f4c45e ↗ · Catalog record

Reading: 504 / 558 · 5 original figures & tables · ~18 min ·

1. Paper overview

In one sentence: ManiSkill makes object-level manipulation generalization measurable with curated simulation and successful demonstrations, while its baselines reveal how little fixed-object competence guarantees transfer. motivation formulation interface demonstrations single-object generalization-protocol drawer-result

At a glance	What to know
Research problem	Source description A robot that opens one cabinet has not necessarily learned to open another with different geometry or articulation. ManiSkill targets this within-category transfer through low-level physical interaction. Partial egocentric observations, occlusion and changing contact dynamics make object identity a meaningful evaluation split, beyond repeating new starts on familiar assets. motivation formulation tasks
Core mechanism	Source description Four tasks cover revolute, prismatic, wheel-ground planar and unconstrained object motion. The paper reports 162 articulated objects across cabinet, chair and bucket categories, processed for physical manipulation. motivation tasks asset-verification
A key reported result	OpenCabinetDoor — unseen-object generalization: BC PointNet + Transformer: test 0.11±0.02; training 0.30±0.06. Mean success rate ± standard deviation across runs. Point clouds; 300 demonstrations per training environment; 150,000 gradient steps; ten test environments × 50 trajectories; five runs. PointNet BC test: 0.04±0.03. BCQ and TD3+BC test: both 0.04±0.02. The architecture improves reported test success, but the absolute success remains low and below its training performance. generalization-protocol door-result
Reading caution	Source description The authors acknowledge limited assets and incomplete household-skill coverage. No sim-to-real experiment is reported, so simulated success cannot establish physical-robot transfer. limitations

Core contributions

Source description
Four tasks cover revolute, prismatic, wheel-ground planar and unconstrained object motion. The paper reports 162 articulated objects across cabinet, chair and bucket categories, processed for physical manipulation. motivation tasks asset-verification
Source description
A collection pipeline supplies approximately 36,000 successful trajectories and 1.5 million renderable frames. Separate training tracks distinguish demonstration-only learning, online fine-tuning without external annotations, and unrestricted methods. demonstrations tracks

Two rows of differently shaped cabinets, swivel chairs and buckets illustrate geometric and articulation diversity. — **Figure 2.** Object variation changes the manipulation problem before any policy is trained. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.

2. Motivation

2.1 The problem and the proposed response

Source description

A robot that opens one cabinet has not necessarily learned to open another with different geometry or articulation. ManiSkill targets this within-category transfer through low-level physical interaction. Partial egocentric observations, occlusion and changing contact dynamics make object identity a meaningful evaluation split, beyond repeating new starts on familiar assets. motivation formulation tasks

2.2 What this reading follows

Opening a familiar drawer and opening an unfamiliar cabinet test different abilities. ManiSkill turns that distinction into a benchmark: articulated objects vary in shape and topology, cameras move with the robot, and policies must produce low-level control commands. Its demonstration pipeline makes these difficult tasks accessible to imitation and offline reinforcement learning, without requiring every researcher to train the collectors. The central result is a mismatch between promising learning on one object and weak performance across objects. Read the visuals as a sequence from asset diversity and observation design to the evaluation split, then to the evidence that limits claims about generalizable skills. motivation formulation interface demonstrations single-object generalization-protocol drawer-result

3. Research context

We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.

Catalog dimension	Recorded classification
Major category	Benchmarks & simulators
Architecture	Not applicable
Prediction paradigm	Not applicable
Quadrant	Not applicable

3.1 Evidence-based assessment

Supports the recorded classification

Reader analysis

The recorded benchmark/simulator category and not-applicable architecture, prediction-paradigm and quadrant fields fit this source. It supplies physical simulation, demonstrations and generalization protocols. The baseline networks predict actions; the paper does not propose a joint learned world/action model. MPC-assisted reward design does not change that architectural assessment. formulation demonstrations tracks baselines

This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.

4. Problem formulation

4.1 Inputs and outputs

Inputs	Outputs
Egocentric point clouds or RGB-D observations, robot state and provided task-relevant segmentation masks. Training-object demonstrations stored as internal environment states; permitted interaction depends on the track.	Normalized robot-controller targets that cause simulated physical interaction. Task-specific binary success and mean success rates over held-out object environments.

4.2 Equations and their role

\mathcal{T}=\{T_{o,l}:o\in\mathcal{O},\;l\in\mathcal{L}_o\},\qquad \mathcal{E}_o=\{T_{o,l}:l\in\mathcal{L}_o\}

Section 2.1 defines task or skill \(\mathcal{T}\) over objects \(o\) in category set \(\mathcal{O}\), with environment parameters \(l\) in object-specific set \(\mathcal{L}_o\). Each \(T_{o,l}\) is a finite-horizon POMDP instance. Environment \(\mathcal{E}_o\) fixes the object while varying its parameters. Holding out objects therefore differs from changing parameters or initial conditions of one object. formulation

5. Method in detail

5.1 Treat an object as an environment family

Source description

Start with the distinction between a task, an object and an initial condition. Section 2.1 defines a task as a collection of POMDP instances across objects and their environment parameters. Fixing one object still leaves a family of environments through parameters such as friction. Object-level generalization requires withholding the object itself, rather than only varying its starting configuration. Table 1 then gives this definition an operational split: the cabinet, chair and bucket tasks each reserve objects for testing. Their manipulation goals remain category-specific, such as opening the designated articulated part or moving a bucket onto a platform. Thus a single-object demonstration experiment and the multi-object benchmark answer different questions, even when both report an average success rate for the same named task. formulation splits tasks single-object generalization-protocol

An RGB view and matching depth view of a chair appear beside the fused point cloud, with robot arms and missing surfaces visible. — **Figure 4.** The policy sees partial egocentric geometry, even after panoramic fusion. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.

5.2 Separate demonstration construction from the evaluated controller

Source description

The collection pipeline solves a problem that the final benchmark intentionally leaves difficult. A dense reward template is designed for each task and checked using CEM-based MPC. Separate SAC agents then specialize to individual objects and supply successful trajectories from varied starts. A benchmark learner must combine experience across training objects into one policy for that task. In the reported visual baselines, point features include position, color, masks and robot state. The richer encoder processes mask groups separately before Transformer aggregation and attention pooling; BC then learns to match demonstrated actions. At execution, controller targets drive the simulated robot and its physical contacts. The stored demonstration states support rendering observations, but the generalization evaluation uses visual inputs rather than exposing privileged object states to the policy. demonstrations baselines interface formulation

Dataset table listing four tasks, total/train/test object counts, dual-arm requirements, qualitative motion-planning difficulty and task DoF, with explanatory notes. — **Table 1.** The test split holds out objects, not merely new starts on known geometry. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.

5.3 Read the generalization gap alongside the training deficit

Reader analysis

The tempting reading of Table 3 is that these policies simply overfit. Reader analysis suggests a more careful decomposition. The richer BC model opens doors on training objects with a mean success rate of 0.30, then falls to 0.11 on test objects. Both values are low, so the test deficit sits on top of incomplete learning across the training distribution. The drawer row makes the distinction sharper: its multi-object training score is 0.37, despite the separate fixed-object experiment reaching 0.85 with 300 demonstrations. Those protocols differ and cannot be subtracted as a controlled transfer penalty. Instead, they motivate reporting fixed-object learning, multi-object training success and held-out-object success separately. The chair row further cautions that a richer architecture need not improve the final test metric. door-result drawer-result chair-result generalization-protocol single-object

5.4 Training and inference

During training

Source description

Demonstration collection first checks a shared dense reward template for each task using CEM-based MPC. It then trains a separate SAC agent per object to collect 300 successful trajectories with varied starts. These specialized collectors do not demonstrate a single policy generalizing across objects. demonstrations

Source description

The evaluated policies learn from demonstrations using BC, BCQ or TD3+BC. BC minimizes L2 distance between predicted and demonstrated actions. Multi-object models receive 150,000 gradient steps. For PointNet plus Transformer on one RTX 2080Ti, reported training times are approximately 5, 35 and 9 hours respectively. baselines generalization-protocol

During inference

Source description

The visual policy produces controller targets from observations and robot state; PID controllers execute joint commands in simulation. Single-arm robots have 13 joints and dual-arm robots 22; operational-space control is also supported. MPC is described as a reward-verification tool, not inference-time planning for these evaluated learned policies. interface demonstrations baselines

Source description

Test evaluation uses unseen objects from the same category, with visual rather than privileged object-state inputs. Table 3 evaluates ten held-out environments with 50 trajectories each and reports mean and standard deviation across five runs. Tracks must be compared separately. formulation tracks generalization-protocol

5.5 Implementation flow

Define object-level transfer
Each object induces an environment family with variable parameters such as joint friction. Train/test counts are 42/10 door cabinets, 25/10 drawer cabinets, 26/10 chairs and 29/10 buckets. Cabinet counts differ from the numbers of individual doors or drawers shown in parentheses in Table 1. formulation splits
Create solvable contact tasks
Open one designated door or drawer; push a swivel chair without toppling; or lift a bucket containing a ball onto a platform. Asset screening, manual convex decomposition and collision repairs address annotation errors and exploitable or obstructive geometry. tasks asset-verification
Observe and encode
Three 400×160 robot-mounted cameras spaced 120 degrees apart supply panoramic observations. Baseline point features combine position, RGB, segmentation masks and robot state, followed by downsampling. The richer encoder processes mask groups with separate PointNets, then a Transformer, attention pooling and an MLP; the simpler encoder uses one PointNet and an MLP. interface baselines

6. Experiments & results

ManiSkill benchmarks simulated manipulation of unseen objects within familiar categories. Its contribution is a curated set of articulated environments, successful demonstrations and controlled evaluation tracks. Point-cloud policies can learn one fixed object reasonably well, but the reported baselines struggle across object variation. This is a benchmark and demonstration pipeline, with no learned future-world predictor proposed.

Source and visual limitations

Source description

The supplied main-paper PDF contains Figures 1–4 and Tables 1–3, but Section 3 places the detailed network diagram in supplementary Figure 7, which is not supplied. Figure 4 therefore illustrates the observation mechanism; it is not presented as a network diagram. Table 2 supplies the diagnostic comparison, but varies demonstration count and gradient steps together and does not isolate a single architectural component. No physical-robot experiment is reported. supplement-dependencies interface baselines single-object limitations

6.1 Read the original evidence

Training and test success rates with standard deviations for four manipulation tasks and four algorithm/architecture combinations. — **Table 3.** Low test success is accompanied by incomplete learning even on training objects. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.

6.2 Results and evaluation conditions

Task & protocol	Reported result	Comparison & interpretation
OpenCabinetDrawer — fixed-object imitation Point-cloud policies on one fixed object; 300/1,000 demonstrations, 20,000/40,000 gradient steps respectively; 100 evaluation trajectories.	BC PointNet + Transformer: 0.85 / 0.90. Average success rate (fraction; no uncertainty reported)	PointNet BC: 0.68 / 0.76; PointNet + Transformer BCQ: 0.45 / 0.55; TD3+BC: 0.31 / 0.57. The richer BC architecture wins these comparisons. Data and optimization budgets change together, and this is not unseen-object evaluation. single-object
OpenCabinetDoor — unseen-object generalization Point clouds; 300 demonstrations per training environment; 150,000 gradient steps; ten test environments × 50 trajectories; five runs.	BC PointNet + Transformer: test 0.11±0.02; training 0.30±0.06. Mean success rate ± standard deviation across runs	PointNet BC test: 0.04±0.03. BCQ and TD3+BC test: both 0.04±0.02. The architecture improves reported test success, but the absolute success remains low and below its training performance. generalization-protocol door-result
OpenCabinetDrawer — unseen-object generalization Point clouds; 300 demonstrations per training environment; 150,000 gradient steps; ten test environments × 50 trajectories; five runs.	BC PointNet + Transformer: test 0.12±0.02; training 0.37±0.06. Mean success rate ± standard deviation across runs	PointNet BC test: 0.11±0.03; BCQ: 0.11±0.03; TD3+BC: 0.10±0.02. The training gain provides little test improvement here. Table 2’s much higher fixed-object result uses a different training problem. generalization-protocol drawer-result single-object
PushChair — unseen-object generalization Point clouds; 300 demonstrations per training environment; 150,000 gradient steps; ten test environments × 50 trajectories; five runs.	BC PointNet + Transformer: test 0.08±0.01; training 0.18±0.02. Mean success rate ± standard deviation across runs	PointNet BC test: 0.09±0.02; BCQ and TD3+BC: both 0.08±0.01. The richer architecture has no reported test advantage on this task; a blanket claim of better generalization would be misleading. generalization-protocol chair-result
MoveBucket — unseen-object generalization Point clouds; 300 demonstrations per training environment; 150,000 gradient steps; ten test environments × 50 trajectories; five runs.	BC PointNet + Transformer: test 0.08±0.01; training 0.15±0.01. Mean success rate ± standard deviation across runs	PointNet BC test: 0.02±0.01; BCQ: 0.06±0.01; TD3+BC: 0.03±0.01. The relative improvement over PointNet BC leaves most evaluated bucket trials unsuccessful. generalization-protocol bucket-result

6.3 Ablations and diagnostic examples

Read component removals and qualitative examples within their stated evaluation conditions.

Five demonstration-count columns pair increasing gradient-step budgets with success rates for PointNet BC and PointNet-plus-Transformer BC, BCQ and TD3+BC. — **Table 2.** A fixed drawer can be learned, but this diagnostic changes data and training budget together. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.

7. Analysis & limitations

7.1 What the evidence leaves open

Source description

The authors acknowledge limited assets and incomplete household-skill coverage. No sim-to-real experiment is reported, so simulated success cannot establish physical-robot transfer. limitations

Author claim

Successful-only demonstrations and control difficulty are the authors’ conjectured explanation for offline RL trailing BC, not an isolated causal result. limitations

Reader analysis

Table 2 couples demonstration count to gradient steps, while the architecture comparison changes segmentation-specific processing and Transformer aggregation together. Neither isolates one mechanism; Table 3 also shows low training success, so failure is not solely a test-distribution issue. single-object baselines generalization-protocol drawer-result

7.2 Questions for discussion

How much of the fixed-object data trend remains when gradient steps are held constant?
Does segmentation-based grouping, Transformer aggregation or model capacity explain the architectural gains?
How should low training success be separated from true object-generalization failure?

8. Reproducibility audit

8.1 Requirements and known gaps

Source description

Reproduction needs the processed assets, object split, stored demonstrations, observation/mask construction and controller configuration. Preserve track restrictions and success definitions. The main text defers reward details, success thresholds, downsampling, network specifications and algorithm settings to unsupplied supplementary Sections B–D. splits interface tracks supplement-dependencies

Reader analysis

For collection-cost planning, the reported MPC check can find a successful trajectory within 15 minutes using 20 CPUs, conditional on success and for one start. This is not the cost of collecting the complete dataset; SAC training and rendering require separate accounting. demonstrations

Reader analysis

A useful first replication would separate optimization budget from demonstration count on the fixed drawer, then test whether the PointNet-plus-Transformer advantage survives controlled segmentation and capacity comparisons on held-out objects. These are proposed checks, not completed experiments. single-object baselines generalization-protocol

Source description

The paper describes an open-source stack, but this PDF does not identify a pinned software release or explicit license terms. Reuse and exact-version reproduction would require checking those source details separately. release-description

8.2 Proposed reproduction checks

The following checks are proposals motivated by the paper. They have not been run as part of this reading.

Check 1: Disentangle demonstrations from optimization steps

Reader-proposed check, not performed: use the same fixed OpenCabinetDrawer object and PointNet-plus-Transformer BC architecture as Table 2. Cross its demonstration counts with a common 40,000-step training budget, while also repeating the paper’s count-dependent budgets. Keep observation processing, optimizer and evaluation starts fixed; repeat seeds and evaluate 100 trajectories per condition. If success still improves with more demonstrations at fixed steps, the data effect survives this control. If the trend largely disappears, optimization budget explains an important part of the original pattern. Exact implementation settings require the absent supplement. single-object baselines supplement-dependencies

Check 2: Locate the source of the encoder advantage

Reader-proposed check, not performed: compare the single-PointNet BC baseline, grouped PointNets with a capacity-matched pooling head, and grouped PointNets plus Transformer. Give each the same available point features and masks, demonstrations, update budget and object split, with no added interaction. Evaluate training and held-out door and chair objects under Table 3’s protocol across five runs. If grouped pooling matches the Transformer, grouping or capacity may explain the gain. If only training success rises, the change improves fitting without demonstrating better object transfer. Report paired run-level differences; recover missing architecture settings before attempting exact replication. baselines generalization-protocol door-result chair-result tracks supplement-dependencies

8.3 Reading coverage

Visual audit: Visually inspected the title, author affiliation and equal-contribution footnote on p. 1; the design motivation on pp. 2–3; Figures 1–4; Table 1 and all explanatory notes on p. 4; the POMDP notation and task definitions on p. 5; sensor/controller details and Figure 4 on p. 6; reward collection, MPC compute, tracks and asset processing on p. 7; baseline information flow and Table 2 on p. 8; Table 3, its protocol, GPU/timing details and limitations on p. 9; and the end of References on p. 15. All five final original crops were viewed. Table 1 retains its notes because they define parenthesized counts and task DoF. Figure 4’s panels were checked against its caption; Figure 3’s world-camera illustration is explicitly distinguished from benchmark observations. No network arrows could be checked because supplementary Figure 7 is absent. References on pp. 10–14 were read as text; external cited works, code, data and unsupplied supplements were outside this pass.

PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 15. Appendix coverage: not present.

Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.

Text reading scope & known omissions

Abstract (p. 1)
1 Introduction and Related Works (pp. 2–4)
2 ManiSkill Benchmark (p. 4)
2.1 Basic Terminologies and Setup (p. 5)
2.2 Tasks with Diverse Motions and Skills (pp. 5–6)
2.3 Robots, Actions, Visual Observations, and Rewards (p. 6)
2.4 RL-Based Demo Collection and MPC-Assisted Reward Template Design (pp. 6–7)
2.5 Multi-Track Training-Evaluation Protocol (p. 7)
2.6 Asset Selection, Re-Modeling, Postprocessing, and Verification (pp. 7–8)
3 Baseline Architectures, Algorithms, and Experiments (p. 8)
3.1 Single Environment Results (pp. 8–9)
3.2 Object-Level Generalization Results (p. 9)
4 Conclusion and Limitations (p. 9)
Acknowledgement and References (pp. 10–15)

Outside the original text pass

Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
Separate supplemental material availability has not been fully verified.
All five supplied text chunks were read individually, covering all 15 PDF pages. Original Figures 1–4 and Tables 1–3 were visually inspected; page images inspected are listed in the illustrated edition.
Supplementary Sections B–D and architecture Figure 7 are cited in the paper but absent from the supplied PDF. They were not read; no separate supplement was supplied.
Identity and edition: the title and all nine authors match the catalog. The title-page footer identifies the NeurIPS 2021 Datasets and Benchmarks edition. No numbered revision or revision date is printed; no version difference is established, and no later edition was substituted.
Code, datasets and linked resources were not accessed or inspected. No installation, demonstration rendering from stored states, or experiment was performed.

The visual audit above records the subsequent illustrated pass.

8.4 Traceable evidence

identityPDF p. 1, title, author block, equal-contribution footnote and venue footerInspect

The title matches the catalog. The nine authors are Tongzhou Mu, Zhan Ling, Fanbo Xiang, Derek Yang, Xuanlin Li, Stone Tao, Zhiao Huang, Zhiwei Jia and Hao Su, affiliated with University of California San Diego. The first five contributed equally. The footer identifies NeurIPS 2021 Track on Datasets and Benchmarks.

Go to primary source ↓

motivationPDF pp. 2–3, Section 1, object diversity discussion and benchmark summary; p. 3, Figure 2 and captionInspect

ManiSkill targets manipulation of unseen objects within a familiar category, using full-physics simulation and diverse articulated geometry. The paper reports 162 objects across three categories. Figure 2 illustrates cabinets, chairs and buckets, with manually processed geometry and physical parameters.

Go to primary source ↓

splitsPDF p. 4, Table 1, all task rows and explanatory notesInspect

Train/test object counts are 42/10 for OpenCabinetDoor, 25/10 for OpenCabinetDrawer, 26/10 for PushChair and 29/10 for MoveBucket. Parentheses count individual doors or drawers, not cabinets. DoF denotes the manipulated task system; six free-body DoF are included for chairs and buckets. Motion-planning difficulty is qualitative.

Go to primary source ↓

formulationPDF p. 5, Section 2.1, task/environment definitions and train/test partitionInspect

A task is a family of finite-horizon POMDPs over objects and their environment parameters. An environment holds the object fixed while parameters vary. Object identities are partitioned into training and test sets. Visual modes replace privileged object states with observations while retaining robot states.

Go to primary source ↓

tasksPDF p. 5, Section 2.2, four named task descriptionsInspect

Doors rotate about a revolute joint; drawers translate along a prismatic joint. A dual-arm robot pushes a swivel chair to a target without toppling it, or lifts a bucket containing a ball onto a platform. Chair underactuation and bucket balancing introduce distinct challenges.

Go to primary source ↓

interfacePDF p. 6, Section 2.3 and Figure 4 with captionInspect

Robots combine a mobile platform, Sciurus body and one or two Franka Panda arms, with 13 or 22 joints. Actions are normalized controller targets; PID joint control and operational-space control are supported. Three 400-by-160 cameras spaced 120 degrees apart supply panoramic RGB-D or fused point clouds, with task-relevant masks. Figure 4 contrasts one camera’s RGB/depth with all-camera point-cloud fusion.

Go to primary source ↓

demonstrationsPDF p. 6, Section 2.4, dataset paragraph; p. 7, Section 2.4, two-stage collection pipelineInspect

Approximately 36,000 successful trajectories and 1.5 million renderable frames are provided, with 300 trajectories per training object per task. Stored internal states permit rendering. Shared task-level dense reward templates are tested with CEM-based MPC; separate SAC agents per object collect randomized successful demonstrations. Successful MPC checks take within 15 minutes using 20 CPUs for one initial state.

Go to primary source ↓

tracksPDF p. 7, Section 2.5, three tracks and evaluation metricInspect

No Interactions permits only supplied demonstrations for training. No External Annotations allows online fine-tuning in training environments without new annotations. No Restrictions also permits new data/environments and handcrafted planning. Each track is evaluated separately using mean success on predetermined test instances; detailed success conditions are deferred to supplementary Section B.4.

Go to primary source ↓

asset-verificationPDF pp. 7–8, Section 2.6Inspect

Assets are screened for annotation errors, manually decomposed when convex decomposition causes artifacts, and repaired when collision geometry prevents successful manipulation. The stated verification process seeks solvable environments, rather than accepting every original PartNet-Mobility asset unchanged.

Go to primary source ↓

baselinesPDF p. 8, Section 3, point-cloud features, two architectures and learning algorithmsInspect

Position, RGB, segmentation masks and robot state form point features; point clouds are downsampled. One baseline uses PointNet followed by an MLP. The other applies separate PointNets to mask groups, then a Transformer, attention pooling and a final MLP. BC minimizes L2 action distance; BCQ and TD3+BC are offline-RL baselines.

Go to primary source ↓

single-objectPDF p. 8, Table 2, all columns and caption; p. 9, Section 3.1 discussionInspect

For one fixed OpenCabinetDrawer object, BC with PointNet plus Transformer scores 0.16/0.35/0.51/0.85/0.90 as demonstrations increase 10/30/100/300/1000 and gradient steps increase 2000/4000/10000/20000/40000. At 300 demonstrations, PointNet BC, BCQ and TD3+BC score 0.68/0.45/0.31. At 1000 they score 0.76/0.55/0.57. Each reported average uses 100 evaluation trajectories, without uncertainty estimates in this table.

Go to primary source ↓

generalization-protocolPDF p. 9, Table 3 caption and Section 3.2 training paragraphInspect

Table 3 gives mean and standard deviation across five runs with point-cloud inputs and 300 demonstrations per training environment. Test rates use ten held-out environments and 50 trajectories each. Models train for 150,000 gradient steps. PointNet plus Transformer training takes about 5 hours for BC, 35 for BCQ and 9 for TD3+BC on one NVIDIA RTX 2080Ti GPU.

Go to primary source ↓

door-resultPDF p. 9, Table 3, OpenCabinetDoor row, all algorithm/architecture columnsInspect

BC PointNet plus Transformer obtains training/test success 0.30±0.06/0.11±0.02, versus PointNet BC 0.18±0.02/0.04±0.03. BCQ and TD3+BC, both using PointNet plus Transformer, each achieve test success 0.04±0.02.

Go to primary source ↓

drawer-resultPDF p. 9, Table 3, OpenCabinetDrawer row, all algorithm/architecture columnsInspect

BC PointNet plus Transformer obtains training/test success 0.37±0.06/0.12±0.02, versus PointNet BC 0.24±0.03/0.11±0.03. BCQ test success is 0.11±0.03; TD3+BC test success is 0.10±0.02.

Go to primary source ↓

chair-resultPDF p. 9, Table 3, PushChair row, all algorithm/architecture columnsInspect

BC PointNet plus Transformer obtains training/test success 0.18±0.02/0.08±0.01, versus PointNet BC 0.11±0.02/0.09±0.02. BCQ and TD3+BC each obtain test success 0.08±0.01.

Go to primary source ↓

bucket-resultPDF p. 9, Table 3, MoveBucket row, all algorithm/architecture columnsInspect

BC PointNet plus Transformer obtains training/test success 0.15±0.01/0.08±0.01, versus PointNet BC 0.03±0.01/0.02±0.01. BCQ and TD3+BC achieve test success 0.06±0.01 and 0.03±0.01 respectively.

Go to primary source ↓

limitationsPDF p. 9, Section 4, final paragraph; Section 3.1, offline-RL discussionInspect

The authors identify limited asset count, incomplete household-skill coverage and no sim-to-real experiments. Their explanation for BC outperforming offline RL invokes successful-only demonstrations and control difficulty, explicitly as conjecture.

Go to primary source ↓

supplement-dependenciesPDF p. 5, Section 2.2; p. 6, Section 2.3; p. 7, Sections 2.4–2.5; p. 8, Section 3; p. 15, end of ReferencesInspect

The main text directs task details, masks and success definitions to supplementary Section B; dense rewards and collection details to C; downsampling, architecture and algorithm settings to D. The detailed architecture diagram is explicitly identified as supplementary Figure 7. The supplied PDF ends with References, not these supplementary sections.

Go to primary source ↓

illustration-viewpointPDF p. 3, Figure 3 and captionInspect

The four point-cloud examples are displayed using cameras mounted in the world frame for visibility. The caption explicitly says that actual task cameras are mounted on the robot head and offer an egocentric view.

Go to primary source ↓

release-descriptionPDF p. 1, Abstract and repository link; p. 4, paragraph preceding the contribution listInspect

The authors describe the simulator, environment, SDK and baselines as open-sourced and the stack as free and entirely open-source. The supplied main text does not provide a pinned software release or explicit license terms.

Go to primary source ↓

8.5 Primary sources

ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations ↗

PDF · 8,523 extracted words

Source fingerprint

cd7f0b127de0e036cd7786dab8797af4c15c46164f620af1a0f80450277bc535