RLBench: The Robot Learning Benchmark & Learning Environment

1. Paper overview

In one sentence: RLBench makes manipulation tasks, observations and generated demonstrations reusable, while leaving learner design and the proof of generalization to subsequent experiments. e01e02e04e05e07e09e11e13e16

Core contributions

• Source description

  A common robot scene supports 100 hand-designed tasks, spanning reaching and object manipulation through longer composed activities; Figure 1 illustrates 24 examples. Diversity is a design goal rather than a measured claim of complete household-task coverage. e02

• Reader analysis

  Motion-planned demonstrations and extensible task-building tools make repeated data generation possible. The claimed infinite supply means episodes can be generated on demand; it does not establish infinite independent skills or infallible planning. e07e08

• Source description

  Few-shot challenge v1.0 separates held-out tasks from familiar-task configurations, encouraging evaluation beyond changing object colours within a single manipulation template. e05e11

↗ Enlarge figure

How to read it. Read the montage as a collection of different objectives, not successive frames of one robot trajectory. The caption identifies a block pyramid at the upper left, peg insertion at the upper right, checkers setup at the lower left and watering a plant at the lower right. The recurring table and robot distinguish the shared scene from the task objects placed inside it. This is the first step toward understanding the paper's evaluation proposal: new placements of blocks are episodes of a familiar task, whereas these panels illustrate a broader range of manipulation objectives. e02e04e05e16

What it supports. The benchmark combines a common physical setup with varied objects and interaction goals. The 24 illustrated examples make the authors' diversity objective concrete; the count of 100 tasks comes from the accompanying source text, not from extrapolating this montage. Task breadth is the resource contribution being illustrated.

Where the evidence stops. These snapshots do not establish that a learned policy solves any task. They also do not show every variation or the full action sequence needed for the depicted objectives.

2. Motivation

2.1 The problem and the proposed response

2.2 What this reading follows

A manipulation benchmark must define more than a list of objects. It must say what changes between trials, what the controller can observe, how expert data is obtained and what counts as success. RLBench addresses these questions with a shared simulated robot scene and 100 hand-designed tasks. Its task–variation–episode hierarchy is central: learning a new placement of familiar blocks differs from adapting to an unseen manipulation task. This reading follows the supplied September 2019 arXiv v1, whose quantitative graphic describes demonstration lengths rather than policy scores. The illustrations explain the environment and protocol, and keep that empirical boundary visible. e01e02e04e05e07e09e11e13e16

3. Research context

We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.

3.1 Evidence-based assessment

Supports the recorded classification

This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.

4. Problem formulation

4.1 Inputs and outputs

4.2 Equations and their role

5. Method in detail

5.1 Decide what is actually held out

Source description

Begin with Figure 4 before interpreting the few-shot challenge. RLBench groups related objectives into a task, gives each variation several descriptions, and samples configurations as episodes. Learning to stack a different number of red blocks therefore need not mean learning an entirely different task. Section V instead proposes holding out 10% of the 100 tasks, then supplying K demonstrations of an unseen task and testing new episodes. Users report 1-, 5- and 20-shot success. Section VI defines a separate multi-task setting that may train on all tasks before testing fresh examples. The difference is the training exposure, not merely the test image. The exact held-out identities are deferred to the webpage, and the protocol's later switch from K to N for demo count remains an unresolved notation inconsistency. e05e11e12e14

↗ Enlarge figure

How to read it. Follow the red callout arrows from the labels to the scene components. The upper callout locates the over-the-shoulder stereo pair, while the next arrow identifies the monocular camera near the wrist. The lower arrows locate the seven-degree-of-freedom Panda and the tabletop task workspace. These are component annotations, not a neural-network information-flow graph. Cross-checking the caption and Section IV-A confirms that the scene is shared across tasks and also provides joint and end-effector information. The visible arrangement explains why the environment can expose both external views and an observation that moves with the manipulator. e04e03e16e17

What it supports. RLBench standardizes the embodiment and sensing platform while letting users choose the learning or classical control system. The wrist and shoulder cameras provide different viewpoints of the same workspace; neither is a predicted future frame. The figure supports the benchmark classification rather than a particular world-model architecture.

Where the evidence stops. The scene illustration does not specify camera calibration or noise, and the paper disclaims full photorealism and generally realistic physics. A familiar robot model alone does not verify physical transfer.

5.2 Follow a demonstration from task authoring to execution

Source description

A task author supplies scene objects and demonstration waypoints in a V-REP model, with Python logic for variation and success. The expert algorithm π* uses OMPL motion planning to generate observation–action trajectories. This explains the promise of on-demand demonstrations without requiring a human to record every rollout. A validator attempts demonstration collection, although its allowed failure rate is not quantified. A learner can ingest these data, but the paper does not prescribe its objective or architecture. At execution, the example agent acts on an observation and the environment returns the next observation, sparse reward and termination flag. Figure 6 anchors the final evaluation: the saucepan lid must be held and sensor-detected. The explicit condition checks and simulator action execution are separate from any prediction a later learner might make. e05e06e07e08e09e16

↗ Enlarge figure

How to read it. Move down the rows to compare RGB appearance, depth and segmentation masks. Move across the columns to compare the two shoulder views, grouped beneath the stereo heading, with the monocular wrist view at the right. The same workspace appears differently as viewpoint and modality change. The depth row uses a colour rendering, whereas the mask row separates regions into distinct colours. Section IV-A identifies all three as observations available at each frame, alongside proprioception. Read this figure as an inventory of possible inputs; it does not state that every downstream method must consume every channel. e04e06e16

What it supports. The environment can support visual control and geometric approaches through a common task collection. Reader analysis: an RGB-only agent and an agent given segmentation or depth face different information conditions, so their success scores would need an explicit sensor specification before they could be compared fairly.

Where the evidence stops. There is no depth colour bar from which to recover metric distances. These are supplied simulated observations, not an evaluated perception model's predictions, and the figure measures no benefit from adding modalities.

5.3 Interpret the only quantitative diagnostic at its own scale

Reader analysis

Figure 7 offers a useful workload description, but its bars should not become an algorithm leaderboard. The lower panel averages five demonstrations per sampled task from the first variation. Reader interpretation: a long generated episode can motivate testing long-horizon control, yet its length also reflects the planner and selected action interface. A short episode could still demand difficult perception or precision; the paper does not measure that relationship. The caption's 100–1000 timestep range also conflicts with the visibly shorter bars at the right, so exact bounds need underlying measurements. Reproduction should therefore recover the generator settings and record variability before using these lengths to stratify an evaluation. No success-rate table, component ablation or physical deployment study in this version supplies the missing causal or transfer evidence. e06e07e13e16e17

↗ Enlarge figure

How to read it. Start with the vertical stack_blocks label: every group still belongs to that task. The first groups ask for one, two or three red blocks; the final illustrated group changes the target to a maroon block. Within each group, the episode panels show different configurations. The short text lists under the images are alternative descriptions of that variation, not extra tasks. Section IV-B formalizes an episode as an observation–action trajectory sampled from a variation. The important relationship is containment: episodes belong to variations, and variations belong to tasks. The ellipsis and final index labels are schematic rather than an exhaustive enumeration. e05e11e14

What it supports. The hierarchy prevents a colour or configuration change from automatically being counted as a new task. The few-shot challenge holds out tasks, whereas familiar-task evaluation can hold out only episodes. These protocols test different capabilities even when their final metric is task success.

Where the evidence stops. This example explains the grouping convention but does not enumerate the actual meta-test task list. Task boundaries remain partly a design choice, which the authors explicitly motivate with the apple-versus-banana example.

↗ Enlarge figure

How to read it. Read this as the paper's printed task example, not as a validated installation recipe. The initialization obtains the lid and success sensor, then registers the lid as graspable. The condition set combines GraspedCondition with DetectedCondition before passing it to the backend. The caption resolves the intended logic: the lid must both be held and be detected by the sensor for success. The lower methods return a textual description and a variation count of one. The waypoints that produce demonstrations are stored separately in the scene model, so their absence from this Python excerpt does not imply demonstration generation is learned. e07e08e09e10

What it supports. Success has a concrete operational meaning: moving the lid into the detected region is insufficient if it is not held. Reader analysis: benchmark reproduction must preserve the condition logic along with the visual scene, because matching final appearances would not necessarily reproduce the source's completion decision.

Where the evidence stops. Section IV-E assigns descriptions to init_variation(int i), but this figure returns one from init_episode(self, index). The faithful crop preserves that API discrepancy; the supplied paper does not resolve which interface to implement.

5.4 Training and inference

During training

During inference

5.5 Implementation flow

1. Load a shared scene

   V-REP supplies simulation and PyRep the robotics interface. A seven-degree-of-freedom Panda is fixed to a table under three directional lights. Tasks enter the workspace with the gripper initially holding nothing, so tool use includes acquiring the tool. e04

2. Separate objective from configuration

   A task groups variations; each variation has descriptions and can generate episodes. In stack_blocks, changing target colour or requested block count changes the variation, while resampling placements produces another episode. This hierarchy defines what kind of generalization is tested. e05

3. Generate the expert data

   Task authors specify waypoints in a .ttm scene model and success/variation logic in Python. The expert algorithm π* uses OMPL to obtain demonstrations. Validation attempts demonstrations and tolerates only a small, unspecified number of planning failures. e07e08

4. Execute and evaluate actions

   Environment exposes task environments. Users choose absolute or delta joint velocities, positions or torques, or end-effector velocities or poses. The simulator executes actions and returns feedback; task completion yields +1. No learned future-prediction module is prescribed. e06e16

5. Make success explicit

   For removing a saucepan lid, the illustrated condition set requires both grasping the lid and detecting it with the designated proximity sensor. The task logic operationalizes success, so judging only a visually plausible final image would not reproduce this criterion. e09

6. Experiments & results

RLBench standardizes simulated manipulation around 100 hand-designed tasks, multimodal observations and waypoint-generated demonstrations. Its central evaluation distinction is between new configurations of familiar tasks and genuinely held-out tasks. This v1 paper establishes an environment and few-shot protocol, with descriptive task statistics but no learned-policy performance comparison (e02, e05, e11, e13, e16).

6.1 Read the original evidence

↗ Enlarge figure

How to read it. The horizontal labels identify tasks and the vertical axis gives episode length in timesteps. Each green bar averages five demonstrations from the first variation of a sampled task; the caption states that 75 tasks were sampled. Read the ordered bars as a description of the demonstration generator's trajectories. The long left tail includes set_the_table and block_pyramid, while the right end includes reach_target. No second colour, baseline series or error bars is present. The upper vocabulary-frequency panel of Figure 7 was also inspected but is outside this crop; it is a separate descriptive statistic. e13e06e07e16e17

What it supports. The paper supplies evidence of varied generated trajectory horizons, not comparative policy performance. Its caption states a 100–1000 timestep range, yet several bars at the right fall below 100. The broad variation is clear, but the exact lower bound cannot be settled by silently accepting the caption.

Where the evidence stops. The plot covers only the first variation of sampled tasks and shows no uncertainty. Demonstration duration depends on planning and control settings, so it cannot by itself establish task difficulty or learner success.

6.2 Results and evaluation conditions

6.3 Ablations and diagnostic examples

No ablation or diagnostic visual is included in this edition. The source-specific visual limitations are recorded above.

7. Analysis & limitations

7.1 What the evidence leaves open

7.2 Questions for discussion

1. How much apparent few-shot progress would disappear under task-disjoint evaluation instead of new episodes of familiar task templates? (e05, e11, e14)
2. Does planner-generated trajectory length predict learner difficulty once action mode and observation access are controlled? (e06, e13)

8. Reproducibility audit

8.1 Requirements and known gaps

8.2 Proposed reproduction checks

The following checks are proposals motivated by the paper. They have not been run as part of this reading.

8.3 Reading coverage

8.4 Traceable evidence

8.5 Primary sources