LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
1. Paper overview
In one sentence: LIBERO makes lifelong robot learning measurable through controlled task changes, exposing the tension between acquiring new skills and retaining old ones. e-probleme-generatione-suitese-datae-policye-algorithm-results
| At a glance | What to know |
|---|---|
| Research problem | Source description Robot forgetting can concern what an object is, where it is, or how to manipulate it. LIBERO makes these factors experimentally accessible through a task-conditioned policy learning a sequence with restricted access to prior tasks and demonstrations. It studies declarative and procedural transfer together, beyond supervised recognition alone. e-probleme-suites |
| Core mechanism | Source description A Robosuite-based procedural task generator turns human-activity language templates into executable initial configurations and goal predicates, yielding 130 standardized tasks with 50 expert demonstrations per task. e-generatione-suitese-data |
| A key reported result | Lifelong learning on LIBERO-Long: ER: 0.48 ± 0.02 / 0.32 ± 0.04 / 0.32 ± 0.01. FWT ↑ / NBT ↓ / AUC ↑. Ten long-horizon tasks; ResNet-T; Table 2; three seeds, mean ± standard error. SeqL: 0.54 ± 0.01 / 0.63 ± 0.01 / 0.15 ± 0.00. PackNet: 0.22 ± 0.01 / 0.08 ± 0.01 / 0.25 ± 0.00. ER balances learning and retention. PackNet forgets less but learns these tasks more slowly; SeqL leads FWT while retaining much less. e-suitese-metricse-algorithm-results |
| Reading caution | Reader analysis The controlled suites are useful interventions, but success alone cannot diagnose what internal knowledge was forgotten. The study establishes simulation results under known task conditioning, without demonstrating physical deployment or generalization to arbitrary unseen instructions. e-probleme-suitese-policye-languagee-limitations |
Core contributions
- Source description
A Robosuite-based procedural task generator turns human-activity language templates into executable initial configurations and goal predicates, yielding 130 standardized tasks with 50 expert demonstrations per task. e-generatione-suitese-data
- Author claim
The initial study crosses three policy architectures with lifelong-learning baselines and probes task ordering, language identifiers and supervised pretraining. The authors find that preventing forgetting can compromise new-task learning. e-policye-algorithmse-algorithm-resultse-languagee-orderinge-pretraining
Figure 1. LIBERO organizes distribution shifts and lifelong-learning questions; the two left branch labels conflict with the text. Original paper, p. 2 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the central procedural generator, then follow the four colored branches. The cube and pyramid symbols distinguish declarative and procedural knowledge in the authors' framing. The right branches show Goal, with fixed objects and layout but different goals, and LIBERO-100, with mixed variations. Read the left branches together with Section 4.2: the graphic connects Object to changed layouts and Spatial to changed objects, whereas the text and caption assign spatial relationships to Spatial and object identity to Object. The bottom strip lists the five experimental axes: distribution shifts, algorithms, architectures, task ordering and pretraining. e-figure1e-suitese-problem
What it supports. The benchmark's contribution is controlled comparison across different kinds of task change. Section 4.2 specifies ten tasks in each of Spatial, Object and Goal, plus 100 tasks with entangled variation. Those written definitions support the interpretation of the later results.
Where the evidence stops. The Spatial/Object inconsistency is in the supplied original figure. This crop preserves it exactly. Use Section 4.2 and the figure caption for suite definitions; the diagram alone cannot resolve the mislabeled branches.
2. Motivation
2.1 The problem and the proposed response
Robot forgetting can concern what an object is, where it is, or how to manipulate it. LIBERO makes these factors experimentally accessible through a task-conditioned policy learning a sequence with restricted access to prior tasks and demonstrations. It studies declarative and procedural transfer together, beyond supervised recognition alone. e-probleme-suites
2.2 What this reading follows
A robot that learns a new manipulation task can lose an older skill for several reasons: it may confuse the objects, forget their spatial relations, or lose the behavior needed to finish the task. LIBERO turns those possibilities into a simulation benchmark with controlled task suites and human demonstrations. Its initial experiments show why a single score or architecture ranking is insufficient: sequential finetuning learns new tasks quickly, while methods that protect past knowledge can learn more slowly. This reading follows task construction, policy execution and the evaluation protocol, then examines the architecture, language and pretraining evidence within the limits of the supplied proceedings paper. e-probleme-generatione-suitese-datae-policye-algorithm-results
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Benchmarks & simulators |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded benchmark/simulator category is supported: LIBERO defines simulated manipulation tasks, demonstration data and lifelong evaluation. Its baseline policies predict actions directly, without a learned future-state/action prediction system. One Model, prediction-paradigm and quadrant assignments are therefore not applicable to this benchmark contribution. e-generatione-suitese-policye-data
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Turn a language task into an executable test
Begin with the distinction between an instruction and a success test. LIBERO draws behavioral templates from Ego4D annotations, chooses compatible objects and scenes, and expresses initialization and goals in PDDL. In the drawer example, the initial configuration places objects in named regions; the goal combines an open drawer with the bowl inside it. The policy receives observations and task language, while the simulator determines whether the predicates hold. Spatial, Object and Goal then constrain which task factors change across a sequence. As a reader's interpretation, this design improves diagnostic control without making failures self-explanatory: failing the goal still does not reveal whether perception, memory or action selection was responsible. Figure 1's swapped left branch labels make the written definitions in Section 4.2 essential. e-generatione-suitese-figure1e-policy
Figure 2. Language templates become simulator tasks through explicit initialization and conjunctive success conditions. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the downward arrows numbered 1, 2 and 3: language annotations yield behavioral templates, a template becomes an instruction, and that instruction is associated with a scene and PDDL description. The colored branches unpack that description. Block A declares objects, fixtures and regions; block B initializes object locations; block C specifies the goal. For the drawer example, the goal contains an AND joining an open-drawer predicate with a bowl-in-drawer predicate. Dashed lines connect configuration symbols to the rendered scene. These connections describe task construction; they do not show the learned policy's neural computation. e-generatione-policye-source-gaps
What it supports. LIBERO can vary what a robot starts with and what counts as success while retaining a shared manipulation environment. The simulator checks explicit goal predicates, making task execution measurable. The policy still has to learn actions from demonstrations; the PDDL description does not supply those actions.
Where the evidence stops. A conjunction specifies conditions that must hold together, not a complete temporal plan. Figure 2 illustrates benchmark generation. The policy architecture diagram referenced as Figure 6 is absent from the supplied PDF.
5.2 Follow the observations through the policy to action
The mathematical state representation is observation history because a single sensory observation may be non-Markovian. The implemented policies encode images, task language and temporal information differently. ResNet-RNN combines visual features with an LSTM and incorporates BERT language through FiLM and LSTM inputs. ResNet-T sends visual and language tokens into a temporal transformer decoder. ViT-T also uses a vision transformer, injecting language at both visual and temporal stages. Each temporal output becomes the input to a Gaussian-mixture action head. Behavioral cloning trains the action distribution from demonstrations; execution samples a continuous end-effector action and obtains new observations. The learned quantity is therefore an action policy, not a predicted future scene. Table 1 evaluates these architectural alternatives, but the absent Figure 6 prevents inspection of the original detailed network diagram. e-bce-policye-datae-architecture-resultse-source-gaps
5.3 Separate learning a task from keeping it
Imagine tracking one task while the robot moves through a sequence. During that task's training, LIBERO averages success at eleven evaluated checkpoints to obtain its FWT contribution. It then follows success on that same task after later tasks are learned. NBT measures the subsequent loss relative to the selected learned-task score, while AUC combines initial learning with later retained performance. The protocol keeps the earliest best checkpoint and flattens the remaining current-task curve, so reproducing only the final epoch changes the measurement. This explains how SeqL can lead FWT while ER wins Long AUC, and why PackNet can forget little yet learn Long poorly. Reader interpretation: report all three scores and the underlying curves before attributing a gain to knowledge transfer. The undefined final-task NBT convention must also be resolved explicitly. e-metricse-algorithm-results
5.4 Training and inference
During training
Behavioral cloning fits demonstrated actions rather than optimizing sparse reward directly. ER retains experience, EWC regularizes parameter updates, and PackNet allocates subnetworks. SeqL and joint multitask learning provide reference protocols. The paper supplies 50 Spacemouse-teleoperated trajectories per task; old datasets are not fully available to lifelong learners. e-bce-algorithmse-data
The pretraining experiment applies BC to LIBERO-90 for 50 epochs, saving every five epochs and selecting the best-performing checkpoint per architecture before downstream lifelong learning on Long. Encoder freezing, optimizer settings and detailed selection/evaluation splits are not specified in the supplied body. e-pretraininge-source-gaps
During inference
The policy processes the current observation history and task embedding, then samples a continuous end-effector action from its GMM output. Subsequent observations provide feedback. The simulator checks goal predicates and terminates when all hold. This is direct policy execution; the described models do not predict future videos or plan through a learned world model. e-policye-bce-generation
5.5 Implementation flow
- Define a task through initialization and success
Ego4D annotations supply templates such as opening or placing. Available objects and compatible scenes instantiate instructions. PDDL specifies objects, placements and initial status, followed by conjunctive unary or binary goal predicates. The illustrated drawer task requires both opening the drawer and placing the bowl inside it. e-generation
- Control the distribution shift
Spatial, Object and Goal each have ten tasks: distinguish identical bowls by location, change the pick-place object, or change behaviors while keeping objects and spatial relations fixed. LIBERO-100 mixes factors; its 90 short tasks support pretraining and ten long tasks form LIBERO-Long. Figure 1's two left branch labels conflict with these textual definitions. e-suitese-figure1
- Encode observations and task identity
ResNet-RNN combines ResNet visual features with an LSTM; BERT language enters via FiLM and LSTM inputs. ResNet-T replaces temporal recurrence with a transformer decoder and includes a language token. ViT-T uses a vision transformer and temporal decoder, with language tokens in both. Temporal latents feed a GMM action head. e-policy
- Evaluate learning and retention separately
At epochs 0, 5, …, 50, evaluate execution success. Retain the earliest best current-task checkpoint and hold subsequent current-task curve values at that maximum. FWT summarizes learning on new tasks; lower NBT means less subsequent loss; higher AUC balances learning and retained performance. FWT here is not a gain relative to an independently trained baseline. e-metrics
6. Experiments & results
LIBERO tests whether a robot can learn successive manipulation tasks while retaining earlier skills. Its controlled task suites distinguish spatial, object and goal changes; its experiments reveal a tradeoff between learning new tasks quickly and preserving previous performance. The reported outcomes are simulated policy execution, with strong dependence on architecture, learning algorithm and evaluation protocol.
The supplied proceedings PDF includes task-generation diagrams, quantitative tables and ablations, all represented here. Its policy architecture diagram is only referenced as Figure 6; algorithm details, task specifics, experimental setup and additional diagnostics are referred to Appendices B.1, C, D and E, which are absent from the supplied artifact. Figure 2 therefore illustrates the benchmark mechanism, while policy architecture is explained only to the level supported by Sections 4.3–4.4. Separate supplement availability remains unverified. e-generatione-policye-source-gaps
6.1 Read the original evidence
Table 1. The preferred visual backbone depends on both the task suite and the lifelong-learning algorithm. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Choose a task-suite block, then compare architecture rows inside either the ER or PackNet half. FWT and AUC favor higher values; NBT favors lower values. The paper reports means plus or minus standard errors across three seeds. Bold identifies the best value, while purple additionally indicates the authors' two-tailed test at p = 0.05, described as using equal sample sizes and unequal variances. On Long, compare ER's ResNet-T row with ViT-T, then repeat in the PackNet half. Finally inspect Object under ER to see why the Long ranking cannot be generalized across distribution shifts. e-architecture-resultse-metricse-policy
What it supports. On Long, ER favors ResNet-T by AUC: 0.32 ± 0.01 versus ViT-T's 0.25 ± 0.02. With PackNet, ViT-T reaches 0.34 ± 0.01 versus ResNet-T's 0.25 ± 0.00. Object under ER instead favors ViT-T, at 0.57 ± 0.01 versus 0.44 ± 0.06.
Where the evidence stops. These are architecture comparisons under specific learning algorithms, not a parameter-matched isolation of attention or recurrence. Low NBT can accompany poor acquisition. Three-seed standard errors are neither standard deviations nor confidence intervals.
Table 2. Fast acquisition and strong retention lead to different algorithm rankings. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Keep the policy fixed at ResNet-T and read across each algorithm's FWT, NBT and AUC entries. Long is the upper-left block; Goal is the lower-right block. Compare SeqL, ER and PackNet within Long before moving to Goal. Means and standard errors come from three seeds; purple marks reported significance under the caption's two-tailed test at p = 0.05. The MTL row has only one reported reference value per suite, positioned under AUC, and should not be assigned missing FWT or NBT values. MTL is a joint-training reference rather than the same sequential-access experiment. e-algorithm-resultse-algorithmse-metricse-suites
What it supports. Long illustrates the tradeoff: SeqL has the highest FWT, 0.54 ± 0.01, but AUC of only 0.15 ± 0.00. ER raises AUC to 0.32 ± 0.01. PackNet reduces NBT to 0.08 ± 0.01 but reaches AUC 0.25 ± 0.00. On Goal, PackNet's AUC rises to 0.75 ± 0.01.
Where the evidence stops. The authors' explanation that PackNet lacks sufficient per-task capacity on Long is a conjecture, not an isolated capacity ablation. The printed metrics also require a terminal-task NBT convention that the supplied text does not specify.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Lifelong learning on LIBERO-Long Ten long-horizon tasks; ResNet-T; Table 2; three seeds, mean ± standard error. | ER: 0.48 ± 0.02 / 0.32 ± 0.04 / 0.32 ± 0.01. FWT ↑ / NBT ↓ / AUC ↑ | SeqL: 0.54 ± 0.01 / 0.63 ± 0.01 / 0.15 ± 0.00. PackNet: 0.22 ± 0.01 / 0.08 ± 0.01 / 0.25 ± 0.00. ER balances learning and retention. PackNet forgets less but learns these tasks more slowly; SeqL leads FWT while retaining much less. e-suitese-metricse-algorithm-results |
| Lifelong learning on LIBERO-Goal Ten goal-varying tasks; ResNet-T; Table 2; three seeds, mean ± standard error. | PackNet: 0.75 ± 0.01. AUC ↑ | ER: 0.49 ± 0.02; SeqL: 0.22 ± 0.00; EWC: 0.06 ± 0.00. The ranking differs from Long. A single suite cannot establish a universally best lifelong algorithm. e-suitese-metricse-algorithm-results |
| Policy architecture comparison Table 1; Long and Object evaluated separately; three seeds, mean ± standard error. | Long with ER: ResNet-T 0.32 ± 0.01; ViT-T 0.25 ± 0.02; ResNet-RNN 0.08 ± 0.01. AUC ↑ | Long with PackNet: ViT-T 0.34 ± 0.01 versus ResNet-T 0.25 ± 0.00. Object with ER: ViT-T 0.57 ± 0.01 versus ResNet-T 0.44 ± 0.06. Architecture preference depends on both algorithm and distribution shift; these comparisons do not isolate temporal modeling from all capacity differences. e-policye-metricse-architecture-results |
| Language-embedding ablation on LIBERO-Long ER with ResNet-T; Table 3; three seeds, mean ± standard error. | BERT: 0.32 ± 0.01; Task-ID: 0.33 ± 0.01. AUC ↑ | CLIP: 0.35 ± 0.01; GPT-2: 0.30 ± 0.01. No statistically significant difference is reported. Task-ID is BERT encoding an identifier string; the experiment does not establish equivalence or language grounding on unseen instructions. e-languagee-metrics |
| Sensitivity to task ordering ResNet-T; five orderings in Figure 4; error bars are standard deviations. The figure and adjacent text do not identify the suite. | ER ordering means span 0.24–0.32. AUC, as labeled on the x-axis | PackNet ordering means span 0.25–0.33. The authors report significant ordering dependence for PackNet. The plotted y-axis says Success Rate; preserve the explicit AUC label and avoid inventing the omitted suite. e-ordering |
| Supervised pretraining before LIBERO-Long LIBERO-90 BC pretraining; downstream Long; Figure 5. | ER with ResNet-T: 0.22 with pretraining. Plotted Success Rate; aggregation and error-bar semantics unspecified | Same combination without task pretraining: 0.32; multitask reference: 0.48. This pretraining recipe reduces the plotted downstream score. Unpretrained numbers coincide with table AUC entries, but the figure's caption does not explicitly define that aggregation. e-pretraining |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 3. Task identifiers perform similarly to descriptive embeddings in this fixed benchmark setting. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. All rows share LIBERO-Long, ER and ResNet-T, so the intended intervention is the task embedding. Read the dimension column before comparing metrics: CLIP uses 512 dimensions, while the other rows use 768. Task-ID means that a string such as Task 5 is encoded by pretrained BERT; it is not the removal of task conditioning. The three score columns retain the paper's higher/lower directions. The caption reports means and standard errors over three seeds and says no statistically significant difference was found. Bold entries mark numerical leaders without establishing a significant advantage. e-languagee-metrics
What it supports. AUC is 0.32 ± 0.01 for BERT descriptions and 0.33 ± 0.01 for BERT task identifiers, with CLIP at 0.35 ± 0.01 and GPT-2 at 0.30 ± 0.01. The reported comparison therefore provides no detected advantage for semantically detailed descriptions in this particular protocol.
Where the evidence stops. Nonsignificance does not establish equivalence or prove that semantic information is unused. The experiment does not test unseen-instruction grounding, and CLIP's different embedding dimension changes more than language-model identity alone.
Figure 5. Behavioral-cloning pretraining on LIBERO-90 reduces the displayed downstream scores for all five tested combinations. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each group as a fixed algorithm and architecture. The legend assigns red to pretraining, brown to no pretraining and cyan to multitask learning. The described pretraining uses LIBERO-90 for 50 BC epochs, selecting a checkpoint from saves every five epochs, followed by lifelong learning on LIBERO-Long. Compare red with brown within a group before consulting the cyan reference. In the ER/ResNet-T group, the labels are 0.22, 0.32 and 0.48. The vertical axis explicitly says Success Rate; neither the figure caption nor its labels specify how that score is aggregated over learning. e-pretraininge-suites
What it supports. Every displayed red bar is below its paired brown bar. ER/ResNet-T drops from 0.32 without task pretraining to 0.22 with it; ER/ViT-T drops from 0.25 to 0.10. The evidence supports a failure of this supervised pretraining recipe to help these downstream lifelong learners.
Where the evidence stops. Preserve the Success Rate axis label: matching unpretrained table AUC values does not fully specify the plot's aggregation. Error-bar semantics are also unstated here. These results do not test all pretraining objectives or adaptation strategies.
7. Analysis & limitations
7.1 What the evidence leaves open
The controlled suites are useful interventions, but success alone cannot diagnose what internal knowledge was forgotten. The study establishes simulation results under known task conditioning, without demonstrating physical deployment or generalization to arbitrary unseen instructions. e-probleme-suitese-policye-languagee-limitations
Low NBT is insufficient when initial learning is weak. PackNet's capacity explanation is an author conjecture; language nonsignificance is not equivalence; harmful BC pretraining does not establish that pretraining generally hurts. e-metricse-algorithmse-languagee-pretraining
Source ambiguities remain: Figure 1 reverses the Spatial/Object branch mapping; Eq. (1) uses an undefined L after defining horizon H; Eq. (3) omits the terminal NBT convention. The authors also identify human-data privacy as a longer-term concern. e-figure1e-bce-metricse-limitations
7.2 Questions for discussion
- Would matched per-task capacity explain PackNet's different rankings on Goal and Long?
- Would conclusions change with fixed final checkpoints rather than success-selected checkpoints?
- Can task descriptions improve generalization when wording and task identity are deliberately decoupled?
8. Reproducibility audit
8.1 Requirements and known gaps
A faithful replication needs task definitions, demonstrations, observation/action preprocessing, task orders, encoder configuration, replay memory, EWC strength, PackNet allocation, optimizer and evaluation-rollout settings. Appendix D and related supplements are referenced, but their details, hardware and software versions are absent here. The checklist's affirmative compute/training answers do not recover those specifications; no usable license terms are supplied. e-generatione-algorithmse-policye-datae-source-gaps
Proposed checks should separate acquisition from retention: compare SeqL, ER and PackNet with matched task orders and report raw checkpoint success as well as the paper's metrics. A second check should compare BERT descriptions with BERT task identifiers under controlled paraphrases. These are proposals, motivated by the reported tradeoffs and language result. e-algorithm-resultse-metricse-orderinge-language
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate acquisition limits from retention gains
Reader-proposed experiment, not performed: compare SeqL, ER and PackNet using ResNet-T on the same LIBERO-Long sequences, demonstrations, initializations and evaluation episodes. Obtain and fix the missing implementation settings before attempting numerical replication; record replay budgets and per-task active capacity. Save raw success at every five epochs through epoch 50 and after each subsequent task. Report the paper's best-checkpoint metrics alongside a fixed-final-checkpoint sensitivity analysis, with an explicit convention for the final-task NBT term. Repeat across matched task orderings. The reported tradeoff predicts higher SeqL acquisition but poorer retention, and low PackNet NBT alongside weaker acquisition. If that pattern disappears with controlled capacity or checkpoint selection, those factors qualify the algorithmic explanation. e-algorithm-resultse-algorithmse-metricse-orderinge-source-gaps
Check 2: Test whether language conveys more than task identity
Reader-proposed experiment, not performed: hold ER, ResNet-T, LIBERO-Long data, task order and the 768-dimensional BERT encoder configuration fixed. Train one condition with original descriptions and another with stable Task-ID strings. Evaluate the description condition on original wording, independently verified meaning-preserving paraphrases and deliberately mismatched goal descriptions, using identical simulator states; retain original wording as a control. Report both aggregate metrics and per-task action success with uncertainty across seeds. A semantic contribution predicts robustness to paraphrases but sensitivity to changed goals. Comparable performance from stable identifiers would reproduce the narrower task-identification result. Either outcome must be separated from training-task success, and encoder update rules must be documented because the supplied body leaves them unspecified. e-languagee-policye-generatione-metricse-source-gaps
8.3 Reading coverage
Visual audit: Viewed the title/author/affiliation page, all main-paper method and experimental pages, and the checklist. Visually inspected Figures 1–5 and Tables 1–3; inspected every final crop after extraction. Checked Figure 2's numbered flow and conjunctive goal against Section 4.1, metric directions against Eq. (3), and table headers/uncertainty against captions. Figure 1's Spatial/Object branch conflict is disclosed. Figure 4's AUC label and unspecified suite, and Figure 5's Success Rate label and unspecified aggregation/error bars, are preserved. These pages cover every retained method, numerical, training, evaluation and reproduction-source claim. Reference-only pages 11–15 were read as text. Referenced appendix material, including Figure 6 and saliency maps, is absent from this supplied PDF and was not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 16. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, authors, affiliations and Abstract (p. 1)
- 1 Introduction (pp. 1–3)
- 2 Background: MDPs and lifelong imitation learning (p. 3)
- 3 Research Topics in LLDM (pp. 3–4)
- 4 LIBERO: generation, suites, algorithms and architectures (pp. 4–6)
- 5 Experiments: metrics and all main-paper studies (pp. 6–10)
- 6 Related Work and 7 Conclusion and Limitations (p. 10)
- References (pp. 10–15)
- Checklist (p. 16)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All five supplied text chunks were read individually, covering the entire 16-page artifact. Full-text-reviewed refers to its complete main paper, references and checklist; appendices are not included in this artifact.
- Identity/version scope: the inspected title and seven authors match the catalog. The supplied artifact is the NeurIPS 2023 Datasets and Benchmarks proceedings paper. The catalog also lists an arXiv PDF; no arXiv revision identifier or cross-edition comparison is established by this supplied PDF.
- Acquisition limitation preserved: separate supplemental material availability has not been fully verified. Referenced Appendices B.1, C, D and E, Figure 6, additional tables and attention maps were not supplied or read.
- Text extraction does not reconstruct figure images. This limitation was addressed for the supplied main-paper visuals by viewing rendered pages 1–10 and 16, all five figures, all three tables and all six final crops. Reference-only pages 11–15 were read as text.
- Code, downloadable datasets, project links and licenses were not inspected; no experiments were reproduced. URLs are provenance only.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author/affiliation block, equal-contribution footnote and venue footer
The title and seven authors match the catalog. Affiliations are The University of Texas at Austin, Sony AI and Tsinghua University. Bo Liu, Yifeng Zhu and Chongkai Gao have equal-contribution marks. The footer identifies NeurIPS 2023, Track on Datasets and Benchmarks; no arXiv revision number is printed.
Go to primary source ↓e-problemPDF pp. 1–3, Abstract, Section 1 and Sections 2.1–2.2
Lifelong robot learning combines declarative and procedural knowledge. Tasks vary their initial-state distributions and goal predicates while sharing state/action spaces, dynamics and horizon. A task-conditioned policy learns sequentially with restricted access to past tasks and demonstration data.
Go to primary source ↓e-generationPDF p. 4, Figure 2 and Section 4.1; p. 5, Initial State Distribution and Goal Specifications
Ego4D language annotations provide behavioral templates. Instructions and available simulator objects determine scenes and PDDL initial configurations and conjunctive goals. The example requires an open top drawer and the bowl inside it. Robosuite underlies the pipeline; simulation terminates when all goal predicates hold.
Go to primary source ↓e-suitesPDF p. 5, Section 4.2, LIBERO-X and LIBERO-100 paragraphs
Spatial, Object and Goal contain ten tasks each, varying spatial relations, object identity and goals respectively. Spatial selects between identical bowls; Object changes the pick-place target; Goal keeps objects and relations fixed. LIBERO-100 splits into 90 short-horizon pretraining tasks and ten long-horizon downstream tasks.
Go to primary source ↓e-figure1PDF p. 2, Figure 1, top-left branches and caption; p. 5, Section 4.2
Figure 1 connects LIBERO-Object to 'Different layouts, same objects' and LIBERO-Spatial to 'Different objects, same layout'. These branch labels conflict with Section 4.2 and the caption's mapping of Spatial to spatial relationships and Object to objects. The bottom row lists five research topics.
Go to primary source ↓e-bcPDF p. 3, Section 2.2 and Eqs. (1)–(2)
Non-Markovian sensory observations include perception, joint and gripper information; observation history represents state. Behavioral cloning uses a supervised loss, such as negative log likelihood, for a Gaussian-mixture policy. Earlier demonstration datasets are not fully available. Eq. (1) uses an upper limit L although the preceding horizon is H; L is not defined here.
Go to primary source ↓e-algorithmsPDF p. 5, Section 4.3; p. 8, Table 2 Findings
ER is memory-based, EWC regularization-based and PackNet a dynamic-architecture method. SeqL and MTL are reference baselines. The authors explain PackNet's smaller per-task capacity through subnetworks, and conjecture this limits LIBERO-Long learning. Detailed algorithms are referred to Appendix B.1.
Go to primary source ↓e-policyPDF pp. 5–6, Section 4.4
ResNet-RNN uses ResNet, FiLM language conditioning and an LSTM; ResNet-T uses ResNet and a temporal transformer decoder with a language token; ViT-T uses a vision transformer and temporal decoder with language tokens in both. BERT supplies default instruction embeddings. Each temporal output feeds a GMM head from which continuous end-effector actions are sampled. Figure 6 is referenced.
Go to primary source ↓e-dataPDF p. 6, final paragraph of Section 4.4
All evaluated algorithms and architectures use behavioral cloning. Each generated task has 50 expert demonstration trajectories collected through teleoperation using a 3Dconnexion Spacemouse.
Go to primary source ↓e-metricsPDF p. 6, Section 5.1, Eq. (3) and footnote 5; p. 7, Figure 3
Success is evaluated at epochs 0, 5, …, 50. The earliest best current-task checkpoint is retained, with current-task scores thereafter held at that best value. FWT averages within-task checkpoint success; NBT averages subsequent losses from the learned-task score; AUC combines initial learning and later retained performance. Metrics are averaged over tasks. Eq. (3) leaves the NBT terminal-task zero denominator convention unspecified.
Go to primary source ↓e-architecture-resultsPDF p. 7, Table 1, LIBERO-Long and LIBERO-Object AUC columns, and caption; p. 8, opening Findings paragraph
On Long, ER AUC is 0.08 ± 0.01 for ResNet-RNN, 0.32 ± 0.01 for ResNet-T and 0.25 ± 0.02 for ViT-T; PackNet gives 0.03 ± 0.00, 0.25 ± 0.00 and 0.34 ± 0.01 respectively. On Object, ER gives 0.44 ± 0.06 for ResNet-T versus 0.57 ± 0.01 for ViT-T. Values are means ± standard errors across three seeds. Bold indicates best; purple additionally marks the reported two-tailed test at p = 0.05.
Go to primary source ↓e-algorithm-resultsPDF p. 8, Table 2, LIBERO-Long and LIBERO-Goal rows/columns, caption and Findings
With ResNet-T on Long, SeqL has FWT/NBT/AUC 0.54 ± 0.01 / 0.63 ± 0.01 / 0.15 ± 0.00; ER 0.48 ± 0.02 / 0.32 ± 0.04 / 0.32 ± 0.01; PackNet 0.22 ± 0.01 / 0.08 ± 0.01 / 0.25 ± 0.00. On Goal, PackNet AUC is 0.75 ± 0.01, ER 0.49 ± 0.02, SeqL 0.22 ± 0.00 and EWC 0.06 ± 0.00. Means and standard errors use three seeds. SeqL leads FWT on all four suites. MTL is a separate reference.
Go to primary source ↓e-languagePDF p. 8, Study on Language Embeddings; p. 9, Table 3 and caption
On Long with ER and ResNet-T, BERT/CLIP/GPT-2/Task-ID dimensions are 768/512/768/768 and AUC values are 0.32 ± 0.01 / 0.35 ± 0.01 / 0.30 ± 0.01 / 0.33 ± 0.01. Task-ID is BERT applied to strings such as 'Task 5'. Values are means ± standard errors over three seeds, with no statistically significant difference reported.
Go to primary source ↓e-orderingPDF p. 9, Figure 4, caption and Study on task ordering/Findings
Five ordering scores span 0.24–0.32 for ER and 0.25–0.33 for PackNet with ResNet-T. The x-axis identifies AUC although the y-axis says Success Rate. Error bars are standard deviations for a fixed ordering. The authors report significant ordering differences for PackNet. The adjacent text and caption do not name the task suite.
Go to primary source ↓e-pretrainingPDF p. 9, Figure 5, legend, labeled bars, caption and pretraining paragraph; p. 10, continuation
BC pretraining uses LIBERO-90 for 50 epochs, selects the best checkpoint among saves every five epochs, and evaluates downstream on Long. Figure 5 labels ER ResNet-T as 0.22 with pretraining, 0.32 without and 0.48 multitask; ER ViT-T is 0.10/0.25/0.46. Each of five red pretraining bars is below its brown no-pretraining partner. The axis says Success Rate; the caption does not specify aggregation or error-bar semantics.
Go to primary source ↓e-source-gapsPDF p. 5, references to Appendices B.1 and C; p. 6, references to Figure 6 and Appendix D; p. 7, reference to Appendix E.1; p. 10, reference to Appendix E.4; p. 16, Checklist 3(a)–(d) and 4(b)
The main paper delegates algorithm, task, experiment and additional-result details to appendices, and points to an architecture figure and attention visualizations outside the supplied body. The checklist asserts provision of training details and compute information but does not enumerate them; it marks asset-license discussion N/A.
Go to primary source ↓e-limitationsPDF p. 10, Section 7, Conclusion and Limitations
The contribution is a simulation benchmark with 130 standardized tasks and initial algorithm/architecture experiments. The authors identify architecture design, forward transfer and useful pretraining as open directions, and human-data privacy as a longer-term concern.
Go to primary source ↓8.5 Primary sources
LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning ↗
PDF · 8,100 extracted words
Source fingerprint
a99676bd4887c071966e424f93a2bf6971e02ad87b4fffb25c60a75912c2092c