Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
1. Paper overview
In one sentence: Meta-World makes generalization measurable by combining shared manipulation structure with distinct task families, exposing the gap between learning known skills and adapting to new ones. e-designe-interfacee-protocolse-verificatione-split-conflictse-rewards
| At a glance | What to know |
|---|---|
| Research problem | Source description Changing a goal location is a narrower generalization problem than learning a different interaction, such as opening a window after reaching for pucks. The benchmark seeks task distributions broad enough to require new behaviors but structured enough for experience to transfer. Its premise is that neither highly disjoint games nor tiny parametric variations adequately test this balance. e-designe-formalism |
| Core mechanism | Source description A suite of 50 MuJoCo manipulation tasks recombines reaching, pushing and grasping around a common Sawyer arm. Within-family object/goal variation complements differences in objects and joints, while shaped rewards aim to make individual tasks learnable. e-designe-interfacee-tasks |
| A key reported result | MT10 multi-task manipulation: Approximately 85% for multi-headed SAC, as summarized in Section 5. Average task success rate. Ten known task families; fixed object/goal positions; one-hot task identity; average over training families. Approximately 40% for single-head SAC; below 30% for PPO, TRPO and task embeddings. Task-specific heads substantially improve this known-task comparison. The rounded difference is about 45 percentage points; it is not evidence of held-out-task adaptation. e-protocolse-mt-resultse-sac |
| Reading caution | Reader analysis State observations and dense shaping simplify perception and exploration. Images, sparse rewards, long-horizon composition, infrequent resets and online task arrival are proposed extensions, not demonstrated capabilities. The experiments concern simulated manipulation. e-interfacee-future |
Core contributions
- Source description
A suite of 50 MuJoCo manipulation tasks recombines reaching, pushing and grasping around a common Sawyer arm. Within-family object/goal variation complements differences in objects and joints, while shaped rewards aim to make individual tasks learnable. e-designe-interfacee-tasks
- Source description
Five evaluation modes separate within-family adaptation, simultaneous learning of known families, and adaptation to held-out families. Independent single-task policies provide a solvability check before interpreting multi-task failures. e-protocolse-verification
Figure 3. The experimental split changes what successful learning means. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across a row before comparing rows. ML1 retains the pick-and-place family while changing the goal; Section 4.3 says the goal is hidden and must be discovered through experience. MT10 repeats the same families on both sides: the right-hand column is evaluation on known tasks, not a held-out-family test. ML10 instead places different interactions in its test column. This is the benchmark’s method diagram rather than a neural-network architecture. Check the figure against the protocol prose: the MT10 illustration includes window closing, whereas the prose on page 6 lists open box. That inventory disagreement remains unresolved. e-formalisme-protocolse-split-conflicts
What it supports. The diagram separates three questions: adapting to a new configuration, learning several known skills, and acquiring a held-out skill family. A high MT10 score answers the second question. It cannot by itself establish the family-level generalization sought by ML10 and ML45.
Where the evidence stops. ML10 labels also conflict with Figure 5: this diagram includes dial turning in training and sweeping off the table in testing; the results chart instead labels open door and sweep into goal. The crop preserves the original labels.
2. Motivation
2.1 The problem and the proposed response
Changing a goal location is a narrower generalization problem than learning a different interaction, such as opening a window after reaching for pucks. The benchmark seeks task distributions broad enough to require new behaviors but structured enough for experience to transfer. Its premise is that neither highly disjoint games nor tiny parametric variations adequately test this balance. e-designe-formalism
2.2 What this reading follows
A robot that reaches a new location has not necessarily learned how to acquire a new kind of skill. Meta-World turns that distinction into a benchmark: related tasks share a Sawyer arm and workspace, but differ in objects, joints and required interactions. The evaluation separates learning known tasks from adapting to held-out families, then checks whether failures arise even before generalization is required. Read the protocol diagram first, the success definition next, and the performance charts last. The central evidence concerns policies acting in simulation. This proceedings PDF also contains conflicting task labels and reward notation that a reproduction must resolve. e-designe-interfacee-protocolse-verificatione-split-conflictse-rewards
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Benchmarks & simulators |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded benchmark/simulator category is supported by the task suite, simulator interface and evaluation protocols. Architecture, prediction paradigm and quadrant are appropriately not applicable to the resource itself. Shared policies or multiple heads in the evaluated baselines do not turn Meta-World into a One Model world-action architecture; it supplies tasks and measurements rather than a joint future/action predictor. e-designe-interfacee-algorithms
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 First decide which kind of novelty the split tests
Meta-World deliberately combines two scales of variation. Moving a puck changes a parameter inside a task family; replacing that interaction with window opening changes the kind of behavior required. The benchmark keeps a common robot and workspace so that the second change still offers some reusable structure. Now connect that design to information available to the policy. MT10 and MT50 provide task identity and keep positions fixed, so they ask whether the learner can jointly acquire known skills. ML1 hides the goal and tests unseen configurations within a family. ML10 and ML45 withhold task IDs and introduce held-out families. The number in the benchmark name therefore does not by itself define generalization: one must inspect both the family split and the information exposed during evaluation. e-designe-formalisme-protocols
Table 3. Success is defined geometrically, with a different tolerance for each task. Original paper, p. 17 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each right-hand entry is an indicator: it records whether the distance between the task-relevant position o and goal g is smaller than the printed tolerance. Read the units statement at the bottom before interpreting the decimals. Reach uses 0.05 meters, push uses 0.07, press button uses 0.02, and open door uses 0.08. Thus the example 5 cm threshold in the main text is not a universal rule. These entries define the success criterion used to evaluate policies; they are not the reward functions optimized during training, which appear separately in Appendix B and Table 2. e-thresholdse-rewardse-interface
What it supports. The reported success averages combine task-specific geometric tests. That provides an interpretable alternative to comparing raw shaped returns, but the meaning of success still depends on the selected tolerance. Reading this table is therefore necessary before comparing algorithms or implementing a benchmark evaluation.
Where the evidence stops. These are metric definitions, not measured performance. The table alone does not specify episode-level success aggregation or fully explain the extraction of every task-specific position; matching its thresholds is necessary but not a complete evaluator specification.
5.2 Follow the interaction loop through training and adaptation
The benchmark’s physical dynamics come from MuJoCo. A policy consumes the permitted position observations and emits actions in the stated end-effector position space; simulator interaction then supplies observations and reward. Shaped reward components encourage reaching and the subsequent manipulation stages, while the separate distance test measures success. Multi-task methods fit policies across the supplied task identities. Meta-RL methods instead learn how to use a short interaction history: RL² carries recurrent state across episodes, MAML changes the policy through gradients, and PEARL infers an experience-conditioned task embedding for its actor and critic. Appendix D allocates ten trajectories to adaptation. None of this implies that the benchmark proposes a learned dynamics model: the evaluated policies execute in the simulator, and their training cost remains separate from that short adaptation budget. e-interfacee-rewardse-thresholdse-algorithmse-curves
5.3 Use the diagnostics to locate a failure before explaining it
A useful reader interpretation is to treat the experiments as a sequence of checks. Figure 6 first establishes that independent SAC policies can obtain success across the inventory. Figure 5 then asks whether sharing a policy or adaptation procedure preserves that competence, and only its held-out rows address novel-family adaptation. If a method already fits its training families poorly, weak test performance cannot be attributed entirely to a failure of transfer. The head comparison suggests that policy parameterization matters, but it does not distinguish task-specific routing from every increase in effective capacity. Finally, Figure 8 adds training cost to the picture without clearly identifying the meta panels’ train/test split. These checks support a bottleneck diagnosis, not a unique causal explanation or a claim that wider task distributions always improve adaptation. e-verificatione-mt-resultse-meta-resultse-sace-curvese-reporting-gaps
5.4 Training and inference
During training
Multi-task baselines are PPO, TRPO, SAC, SAC with one policy head per task, and an on-policy task-embedding method. Meta-training compares recurrent RL², gradient-based MAML, and off-policy PEARL. The paper describes PPO as the training algorithm for both RL² and MAML; these are benchmarked algorithms, not new Meta-World components. e-algorithms
The two multi-task SAC variants list three 400-unit layers, batch size 128 times task count, policy/Q learning rates 0.0003, discount 0.99 and horizon 150. Both learn task-specific temperatures. Matching these listed settings does not establish equal parameter counts after adding task heads. e-sace-reporting-gaps
During inference
During multi-task evaluation, the policy acts with its supplied task identity. During meta-test adaptation, RL² retains recurrent state across episodes, MAML updates the policy through gradient steps, and PEARL conditions actor and critic on an experience-derived probabilistic task embedding. Feedback comes from simulator interaction. e-formalisme-algorithmse-interface
Appendix D states that adaptation requires ten trajectories. That small adaptation budget is separate from the potentially large meta-training budget; success after adaptation does not establish inexpensive training or physical-robot transfer. e-curvese-future
5.5 Implementation flow
- Build related task MDPs
The simulator supplies state, receives the policy action, and produces subsequent observations and rewards. Section 4.2 specifies nine position coordinates, using either one object plus goal or two objects. This benchmark does not propose a learned future-state predictor or an imagined-rollout planner. e-formalisme-interface
- Choose the generalization question
ML1 trains on 50 position configurations and tests ten held-out configurations within reaching, pushing or pick-and-place, hiding goal positions. MT10/MT50 supply task IDs and fix positions, measuring known-task competence. ML10/ML45 randomize positions and withhold IDs, adapting from ten/45 training families to five held-out families. e-protocols
- Separate optimization reward from evaluation
Policies optimize shaped rewards assembled from common behavioral components. Evaluation uses task-specific distance thresholds instead of raw return, so a high shaping reward is not itself counted as successful manipulation. e-interfacee-rewardse-thresholds
6. Experiments & results
Meta-World tests whether reinforcement learning can share manipulation skills across 50 simulated task families and adapt to held-out families. Shared robot and control interfaces make transfer plausible, while task diversity exposes failures hidden by goal-only benchmarks. Multi-headed SAC helps on ten known tasks, but broad multi-task learning and few-shot adaptation remain difficult. These are simulated policy-execution results, with important protocol and reporting ambiguities.
This is a benchmark paper rather than a proposed model architecture, so Figure 3 supplies the method view. Quantitative outcomes are published as charts, with no numerical results table; the original tables here instead document success thresholds and experimental settings. The source has no dedicated component-ablation section: the head comparison, independent-policy verification and learning curves serve as qualified diagnostics. Figure 8 also omits a PEARL curve from its ML45 panel. No missing experiment or curve has been reconstructed. e-designe-algorithmse-verificatione-curvese-sace-thresholdse-reporting-gaps
6.1 Read the original evidence
Figure 5. Known-task learning and held-out-task adaptation remain different bottlenecks. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the bold average rows, then inspect individual tasks that contribute to those averages. The left panels evaluate MT10 and MT50 on their training families. The right panels separate training and test families with dashed horizontal lines; use the test-average row when discussing adaptation. Consult each panel’s legend because algorithm colors are not consistent across the entire figure. The horizontal axes use fractions from zero to one, unlike the percentage axes in Figure 8. The spread across individual tasks also matters: strong performance on a subset can coexist with a modest aggregate and many near-zero task scores. e-protocolse-mt-resultse-meta-resultse-reporting-gaps
What it supports. Section 5 summarizes multi-headed SAC at about 85% on MT10, falling to about 40% on MT50. For held-out families, MAML leads the reported ML10 test comparison at about 40%, while PEARL reaches about 30% on ML45. There is no single method that resolves all four evaluation modes.
Where the evidence stops. These are rounded prose summaries supported by the bar patterns, not exact tabulated means. No seed counts or uncertainty definitions are supplied for these comparisons. ML10 and ML45 also change task composition, so their difference does not isolate training-set size.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| MT10 multi-task manipulation Ten known task families; fixed object/goal positions; one-hot task identity; average over training families. | Approximately 85% for multi-headed SAC, as summarized in Section 5. Average task success rate | Approximately 40% for single-head SAC; below 30% for PPO, TRPO and task embeddings. Task-specific heads substantially improve this known-task comparison. The rounded difference is about 45 percentage points; it is not evidence of held-out-task adaptation. e-protocolse-mt-resultse-sac |
| MT50 multi-task manipulation All 50 known task families; fixed positions and task identity supplied. | Approximately 40% for multi-headed SAC in the Section 5 prose. Average task success rate | The other four multi-task methods are summarized as below 30%. Scaling the task set remains difficult. These are rounded prose values; the Figure 8 endpoint is lower and is not explicitly aligned to Figure 5 or the prose evaluation checkpoint. e-protocolse-mt-resultse-curvese-reporting-gaps |
| ML10 held-out-family adaptation Ten training families and five held-out families, randomized positions, no input task IDs; ten adaptation trajectories. | Approximately 40% for MAML and 10% for RL². Average meta-test task success rate | PEARL is described as unable to generalize on this split; its test-average bar is at zero. Some adaptation occurs, but fitting the training families is also incomplete. The listed task-family discrepancies prevent treating the paper alone as an unambiguous executable split specification. e-protocolse-meta-resultse-curvese-split-conflicts |
| ML45 held-out-family adaptation 45 training families and five held-out families; randomized positions, hidden task IDs and ten adaptation trajectories. | Approximately 30% for PEARL. Average meta-test task success rate | Approximately 20% for MAML and RL². The authors associate PEARL’s improvement with broader experience. ML10 and ML45 use different task sets, so this is not a controlled estimate of the causal effect of adding training families. e-designe-protocolse-meta-resultse-curves |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Appendix E.3 and E.4 hyperparameter tables. The head comparison shares listed training settings, but does not establish equal model capacity. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare corresponding rows in the upper and lower tables. The explicit architectural change is from a feedforward network to one head per task; both list three fully connected layers with 400 units. Batch size is 128 times the number of tasks, while the interaction-to-update row lists task-count environment steps per training step. Both use Adam, policy and Q-function learning rates of 3e-4, discount 0.99 and horizon 150. The last row says the temperature is learned and disentangled by task in both variants. These details help interpret the MT10 comparison, but the table contains settings rather than success measurements. e-sace-algorithmse-mt-resultse-reporting-gaps
What it supports. Together with Figure 5 and Section 5, the tables support a useful architectural comparison: multi-headed SAC is summarized at about 85% MT10 success versus about 40% for single-head SAC. The shared listed settings narrow the explanations, while leaving total capacity and implementation details to be checked.
Where the evidence stops. One head per task changes the parameterization and may change parameter count. The paper does not provide a parameter-count-matched control, so the comparison does not isolate task-specific routing from every capacity-related explanation.
Figure 6. Independent policies check that the task suite is learnable before testing shared learning. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each task row compares separately trained policies, not a single policy that must perform all listed tasks. Light blue denotes SAC and salmon denotes PPO; the top row aggregates performance. Scan both the nearly saturated rows and the shorter bars toward the bottom. Those shorter SAC bars matter because Appendix C says all tasks are learnable to some degree, rather than guaranteeing perfect performance on every task. The authors explicitly describe this experiment as benchmark verification outside the official evaluation modes. Its purpose is to test whether poor multi-task results could simply reflect tasks that the selected learning algorithms cannot solve at all. e-verificatione-sace-mt-results
What it supports. The plot supports the claim that SAC obtains some success on every individual task, while PPO handles most of them. This makes the weakness of shared learning more informative: a task suite can be individually tractable yet difficult for a policy or adaptation procedure trained across many families.
Where the evidence stops. Independent policies have separate training and parameters. This experiment is neither an equal-compute comparison with multi-task learning nor a proof that every task reaches perfect success. It is a solvability diagnostic, not a controlled component ablation.
Figure 8. The cost of acquiring transferable experience is separate from the cost of adaptation. Original paper, p. 13 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the horizontal label literally: the unit is 1K samples, so an axis position of 1000 represents one million samples. The vertical values are percentages, not the zero-to-one fractions in Figure 5. Compare algorithms within a panel before comparing panels, because the plotted sample ranges differ. The light-blue MT10 trajectory makes the multi-headed SAC advantage visible over training, while MT50 remains substantially harder. Appendix D distinguishes these long training processes from the ten trajectories used for adaptation. Also separate this figure from Figure 7 above it in the PDF: Figure 7 plots average return, whereas Figure 8 plots success rate. e-curvese-mt-resultse-reporting-gapse-meta-settings
What it supports. The MT10 advantage is visible as a learning trajectory rather than only a final bar, and scaling to MT50 leaves substantial headroom. The authors argue that off-policy learning can use data more efficiently. The figure helps examine training cost, but does not provide a uniform wall-clock or hardware comparison.
Where the evidence stops. The meta panels do not explicitly identify train versus test, and ML45 omits PEARL. Endpoints are not reconciled with Figure 5. The caption says off-policy on both sides of its comparison; Appendix D’s paragraph instead contrasts off-policy with on-policy learning.
7. Analysis & limitations
7.1 What the evidence leaves open
State observations and dense shaping simplify perception and exploration. Images, sparse rewards, long-horizon composition, infrequent resets and online task arrival are proposed extensions, not demonstrated capabilities. The experiments concern simulated manipulation. e-interfacee-future
Split descriptions conflict: MT10 prose lists open box, while Figures 3 and 5 show close window. ML10’s diagram uses dial turning in training and sweeping off the table in testing, whereas Figure 5 labels open door and sweep into goal. The ML45 heading says 50 training tasks; the paragraph and Figure 1 specify 45. e-split-conflicts
The printed reward exponentials have positive squared distances, with a positive 0.01 denominator in Table 2, and switch from object-to-goal distance in Appendix B to gripper-to-goal distance in the table. Read literally, the positive exponent increases with distance. The intended implementation cannot be established by silently changing a sign or symbol. e-rewards
Exact aggregate scores, seed counts and uncertainty definitions are absent. Figure 8 does not clearly label meta-training versus meta-test curves and omits PEARL from ML45. Its caption mistakenly compares off-policy methods with off-policy methods; the surrounding paragraph makes the intended on-policy comparison. e-reporting-gapse-curves
7.2 Questions for discussion
- How much of the benefit of task-specific heads survives equal total parameter and environment-sample budgets?
- Would broader meta-training improve adaptation when held-out families and total experience are kept fixed?
- How sensitive are reported success rankings to the task-specific distance thresholds?
8. Reproducibility audit
8.1 Requirements and known gaps
A faithful reproduction needs the proceedings-era task definitions, position randomization, observation masking, rewards and success thresholds, plus the listed algorithm settings. Appendix E supplies several learning rates and rollout budgets, but does not pin software versions, hardware or seeds; the precise split conflicts also require resolution. e-interfacee-protocolse-sace-meta-settingse-thresholdse-split-conflicts
Reader-proposed checks: compare SAC heads under matched sampling and parameter budgets; separately test the printed reward formulas for distance monotonicity and agreement with success thresholds before launching training. These checks target the paper’s strongest architectural comparison and a concrete implementation ambiguity. e-sace-mt-resultse-rewardse-thresholds
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does the MT10 head advantage survive matched capacity and sampling?
Reader-proposed experiment, not performed: first resolve the conflicting MT10 inventory and freeze one explicitly documented split. Train the single-head and per-task-head SAC variants with the Appendix E.3/E.4 settings, equal task sampling, identical environment-step budgets and multiple independent seeds. Add a wider single-head control matched to the multi-head model’s total parameter count. Evaluate all variants at the same checkpoints with the same Table 3 success tests, reporting per-task results as well as the mean. Persistence of the head advantage against both controls would support an explanation involving task-specific parameterization; disappearance against the capacity-matched control would weaken that interpretation. Track learning curves so that a transient optimization advantage is not mistaken for a stable final improvement. e-sace-mt-resultse-protocolse-split-conflictse-thresholdse-reporting-gaps
Check 2: Do the printed rewards point toward the stated success region?
Reader-proposed consistency check, not performed: before training, transcribe the Appendix B equations and Table 2 separately, preserving their printed signs and the difference between object-to-goal and gripper-to-goal distances. Hold the reach/lift gates active and compare rewards as the relevant distance decreases toward a Table 3 success threshold. Also vary gripper and object positions independently to expose the symbol mismatch. The positive exponential printed with denominator 0.01 predicts larger reward at larger distance, which should be documented rather than silently corrected. Any later implementation-based resolution must identify its version and explain which printed expression it departs from. This check can falsify consistency of the published formulas without pretending to establish what uninspected code actually implements. e-rewardse-interfacee-thresholds
8.3 Reading coverage
Visual audit: The title/author page, all method and experiment pages, task descriptions, reward equations, success definitions and Appendix E settings were visually inspected. Figures 1–8, Tables 1–3 and the E.1–E.10 parameter tables were read in the PDF. All six final crops were inspected, including revised margins on the learning curves and SAC tables. Reference pages 9–10 were read in the complete text but were not visually inspected. No separate supplement, code, project video or later release was examined. The protocol discrepancies and reward-sign/symbol issues were cross-checked against their corresponding prose, labels and equations. Every page supporting the retained scientific claims and proposed checks is included above.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 14, 15, 16, 17. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and title block
- 1 Introduction
- 2 Related Work
- 3 The Multi-Task and Meta-RL Problem Statements
- 4 Meta-World
- 4.1 The Space of Manipulation Tasks: Parametric and Non-Parametric Variability
- 4.2 Actions, Observations, and Rewards
- 4.3 Evaluation Protocol
- 5 Experimental Results and Analysis
- 6 Conclusion and Directions for Future Work
- Acknowledgments
- References
- A Task Descriptions
- B Task Rewards and Success Metrics
- C Benchmark Verification with Single-Task Learning
- D Learning curves
- E Hyperparameter Details, E.1–E.10
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This acquisition limitation was addressed by inspecting the original PDF pages and final crops.
- Separate supplemental material availability has not been fully verified.
- The complete supplied 17-page paper was read through all six text chunks. References on pages 9–10 were read as text; all other pages were also inspected visually. No separate supplement was supplied.
- Code, project videos and external documentation were not inspected; no experiments were reproduced.
- Identity: the observed title and all seven authors match the catalog. The PDF footer identifies CoRL 2019; catalog publicationYear is 2020 and its BibTeX year is 2019. No revision identifier or revision history is supplied. This report covers only the supplied proceedings PDF and makes no claims about later Meta-World releases.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author/affiliation block, equal-contribution note and conference footer
The title and seven authors match the catalog. Tianhe Yu, Deirdre Quillen and Zhanpeng He share equal contribution. Five institutions are listed. The footer identifies CoRL 2019 in Osaka, Japan; no revision number is printed.
Go to primary source ↓e-designPDF pp. 2–4, Figure 1 and caption; Sections 2, 3 and 4.1; Figure 2 and caption
The benchmark contains 50 simulated manipulation tasks sharing a Sawyer arm and workspace. It combines variation in object/goal positions with qualitatively different interactions. Figure 1 depicts 45 training and five held-out task families.
Go to primary source ↓e-formalismPDF p. 3, Section 3, multi-task and meta-RL problem statements
Multi-task RL optimizes a task-conditioned policy over training tasks, without a separate task test set. Meta-RL uses training tasks to acquire an adaptation procedure for unseen tasks drawn from the assumed task distribution.
Go to primary source ↓e-interfacePDF p. 5, Section 4 and Section 4.2, Actions, Observations, and Rewards
Tasks use MuJoCo with Multiworld/Gym interfaces. The stated action space is 3D end-effector positions. The nine-dimensional state contains end-effector/object/goal positions or end-effector/two-object positions. Rewards combine shared reaching, grasping and placing components, or simpler subsets.
Go to primary source ↓e-protocolsPDF p. 5, Figure 3 and caption; p. 6, Section 4.3, ML1, MT10/MT50 and ML10/ML45 paragraphs
ML1 uses 50 training and ten held-out position configurations within each evaluated family, with goal positions hidden. MT10/MT50 fix positions and supply one-hot task IDs. ML10/ML45 use ten/45 training families and five held-out families, randomize positions and omit task IDs.
Go to primary source ↓e-split-conflictsPDF p. 5, Figure 3, MT10 and ML10 rows; p. 6, Section 4.3; p. 8, Figure 5, task labels
The MT10 prose lists open box, but Figures 3 and 5 show close window. Figure 3 shows ML10 dial turning in training and sweeping off the table in testing; Figure 5 instead labels open door in training and sweep into goal in testing. The ML45 heading says 50 meta-training tasks while its paragraph and Figure 1 say 45.
Go to primary source ↓e-algorithmsPDF p. 6, Section 5, algorithm descriptions
Multi-task comparisons include PPO, TRPO, SAC, multi-headed SAC and task embeddings. Meta-RL compares PPO-trained recurrent RL², PPO-trained gradient-based MAML, and off-policy PEARL with a probabilistic experience embedding supplied to actor and critic.
Go to primary source ↓e-mt-resultsPDF p. 7, Section 5, multi-task results paragraph; p. 8, Figure 5, upper-left MT10 and lower-left MT50 average rows
The prose reports approximately 85% MT10 success for multi-headed SAC versus 40% for single-head SAC and below 30% for the on-policy alternatives. For MT50 it reports approximately 40% for multi-headed SAC and below 30% for the others. These are rounded prose summaries, not exact tabulated estimates.
Go to primary source ↓e-meta-resultsPDF p. 7, Section 5, ML10/ML45 paragraph; p. 8, Figure 5, right-hand test-average rows and train/test separators
On ML10, the prose reports about 40% MAML and 10% RL² test success, with PEARL unable to generalize. On ML45, PEARL reaches about 30%, versus about 20% for MAML and RL². The figure distinguishes training-family results from held-out-family results.
Go to primary source ↓e-futurePDF pp. 7–8, Section 6, Future directions for algorithm design and Future extensions of the benchmark
The authors identify difficulty fitting diverse training tasks and propose image observations, sparse rewards, compositional long horizons, fewer resets and online task arrival as extensions. They warn that images can expose task identity and encourage memorization.
Go to primary source ↓e-tasksPDF p. 11, Appendix A, Table 1, task descriptions
Table 1 describes the task inventory and randomized positions, including doors with revolving joints, drawers, stick manipulation, nut assembly, box closing and bin transfer.
Go to primary source ↓e-rewardsPDF p. 5, Section 4.2, symbol definitions; p. 12, Appendix B, displayed reward equations; p. 16, Table 2
Rewards add reaching, gated grasping/lifting and gated placing or pushing terms. The printed exponential uses a positive squared distance; Table 2 prints denominator 0.01. Appendix B uses object-to-goal distance in the exponential, whereas Table 2 uses gripper-to-goal distance. The source provides no reconciliation.
Go to primary source ↓e-verificationPDF p. 12, Figure 6 and caption; Appendix C
Independent SAC policies achieve some success on all 50 tasks, while PPO succeeds on most. The appendix explicitly describes this as task solvability verification, not an official benchmark evaluation mode.
Go to primary source ↓e-curvesPDF p. 13, Appendix D; Figures 7 and 8, axes, legends and captions
Appendix D states that adaptation uses ten trajectories and distinguishes this from costly meta-training. Figure 7 plots average return for ML1. Figure 8 plots percentage success against numbers of 1K samples for MT10, ML10, MT50 and ML45; the ML45 panel has no PEARL curve, and meta panels do not explicitly label train versus test. Its caption repeats off-policy on both sides of the efficiency comparison, whereas the paragraph contrasts off-policy with on-policy.
Go to primary source ↓e-sacPDF p. 14, Appendix E.1, E.3 and E.4, SAC hyperparameter tables
The multi-task SAC variants list three 400-unit fully connected layers, batch size 128 times task count, 1000 uniform exploration steps, task-count environment steps per training update, Adam, policy/Q learning rates 3e-4, discount 0.99, horizon 150, reward scale 1 and learned task-specific temperatures. E.4 specifies one head per task.
Go to primary source ↓e-meta-settingsPDF p. 15, Appendix E.5–E.10, hyperparameter tables
PEARL lists 12000 environment steps per 10000 training steps and horizon 150. RL² lists ten rollouts per task, horizon 150 and meta-batch size 40. MAML lists ten rollouts per task, meta-batch size 20 and inner learning rate 0.05. Hardware, software release versions and random-seed counts are not specified in these tables.
Go to primary source ↓e-thresholdsPDF p. 6, Section 4.3, Success metrics; p. 17, Table 3, reach, push, press button, open door rows and units caption
Success is a binary distance-threshold event. Table 3 gives thresholds in meters: reach 0.05, push 0.07, press button 0.02 and open door 0.08. Thresholds vary across tasks rather than using one universal 5 cm cutoff.
Go to primary source ↓e-reporting-gapsPDF p. 7, Figure 4 and Section 5; p. 8, Figure 5; p. 13, Figures 7–8; pp. 14–15, Appendix E
Performance is shown in charts rather than a numerical results table. Exact aggregate values, random-seed counts and definitions of plotted uncertainty are not supplied. Learning-curve endpoints are not explicitly aligned with the per-task bars or rounded prose summaries; no parameter-count-matched head comparison is specified.
Go to primary source ↓8.5 Primary sources
Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning ↗
PDF · 9,695 extracted words
Source fingerprint
96646a2e41cb7aa6be2ce2b0dde3c60248dcbcdd333bf0949977b1c8343d242e