SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning
1. Paper overview
In one sentence: SERL turns demonstration-assisted RL into an effective real-robot training stack, while its integrated results leave the contribution of individual engineering choices unresolved. e01e02e03e06e07e10e12e20
| At a glance | What to know |
|---|---|
| Research problem | Source description Real robot learning must jointly address scarce interaction, visual reward specification, mechanical resets and contact forces. SERL treats implementation and integration as adoption bottlenecks, offering a selected working stack rather than proposing a new RL algorithm. e02 |
| Core mechanism | |
| A key reported result | PCB component insertion: RL 100/100; 20 minutes Successful trials; training wall-clock time. Real Franka Panda; two wrist cameras; randomized initial pose in the Table 2 10 cm × 10 cm region; state-based reward. RL: 20 demos plus online data; BC: 100 demos; 100 evaluation trials each. BC 10/100. A large observed advantage under the same stated evaluation protocol; training includes online interaction unavailable to BC. e09e10e11e12 |
| Reading caution | Source description The authors limit the framework’s scope: it is not a comprehensive algorithm library, and classifier rewards or forward/backward resets may fail in other settings. e15 |
Core contributions
Figure 2. Learning and physical action execution proceed through separate, periodically synchronized components. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the actor node in the upper left. Its policy samples an action from the current observation and calls the environment. Follow the left arrow downward: these actions reach a robot environment containing the controller and sensors. The right arrow returns transition tuples to the learner, whose replay buffer supports repeated updates. The blue band states that training runs in parallel and the policy synchronizes periodically. This matches Section 4.4 and the caption: the learner updates parameters while the actor maintains interaction. The small code fragments illustrate interfaces; they are not a complete deployment specification. e03e06e07
What it supports. Separating action selection from parameter updates makes it possible to reuse each physical transition for multiple gradient steps without placing all computation directly in the control loop. This is the software counterpart of RLPD’s high update-to-data design. The robot environment supplies actual experience rather than model-generated rollouts.
Where the evidence stops. The diagram provides data flow, not measured scheduling performance. It does not specify synchronization intervals, communication delays or the numerical UTD setting, and the paper does not isolate asynchronous execution in an ablation.
2. Motivation
2.1 The problem and the proposed response
Real robot learning must jointly address scarce interaction, visual reward specification, mechanical resets and contact forces. SERL treats implementation and integration as adoption bottlenecks, offering a selected working stack rather than proposing a new RL algorithm. e02
2.2 What this reading follows
A robot that learns to insert a connector needs more than an optimizer: it needs useful observations, a reward, a way to start again, and a controller that can make contact without excessive reference error. SERL assembles these ingredients around RLPD and demonstrates physical manipulation with modest online training times. Read this paper as a systems contribution whose central evidence is executed task performance. The diagrams explain how learning and control cooperate; the tables and plots show the actual evaluation conditions. This edition reviews the supplied arXiv v4 artifact, with its documented timing discrepancies and missing component ablations preserved. e01e02e03e06e07e10e12e20
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Foundational work |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded foundational-action-policy and technical-resource categories fit this integration paper. Architecture, prediction paradigm and quadrant are not applicable to the world/action-model taxonomy: SERL learns model-free values and policies, with optional separate reset agents, rather than a joint future/action predictor or inverse-dynamics world model. e02e03e05e06e09
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Follow the learning loop, then ask where reward comes from
Begin with one real transition: the actor observes the scene, chooses an end-effector command and receives a new observation after the robot executes it. RLPD mixes such online transitions with demonstrations, using equal contributions from the two buffers to fit its critic and improve its actor. Multiple updates reuse scarce physical experience. That loop still needs a reward. SERL can infer success from a classifier, and its optional VICE route adds policy-visited states as negatives to counter reward exploitation. The evaluated tasks use state-based or binary-classifier rewards as listed in Table 2; VICE support does not mean it was used in every experiment. Reader interpretation: the system’s sample efficiency depends on the interaction between learning, reward quality and execution, so the aggregate success plot cannot identify which ingredient mattered most. e03e04e06e10e20
5.2 Why reference limiting belongs inside the fast controller
A large target error can make an impedance controller demand a large interaction force. Simply shrinking every RL action would also slow the approach to the object, because the policy updates less frequently than the robot controller. SERL instead limits reference error at the real-time layer. The paper illustrates a 10 Hz policy above a 1 kHz controller, so one policy block contains many opportunities to advance a bounded reference. Its free-space condition relates the number of low-level steps, the error bound and the maximum action displacement. Figure 3 then supplies the behavioral check: the measured position stops at contact while the clipped reference remains nearby, yet free-space motion still approaches the commanded target. This explains the design logic; the position trace does not replace direct force measurements or establish a universal safety guarantee. e07
5.3 Keep spatial generalization separate from physical resets
The relative-frame design and the backward agent solve different problems. For spatial representation, SERL randomizes the initial end-effector pose, describes proprioception relative to that initial frame, and expresses actions relative to the current end-effector frame. The appendix explains how the robot interface receives a base-frame twist, although its printed adjoint notation contains inconsistencies that a reproduction must resolve. This representation is intended to reduce dependence on an absolute target location. Physical resets require another intervention: after relocation moves the object into the destination bin, a separately learned backward policy moves it back. The two agents have separate critics and rewards. Reader interpretation: location robustness does not imply autonomous reset capability, and an automated reset does not establish generalization to new objects or target arrangements. e05e08e16e18
5.4 Training and inference
During training
Each task starts with 20 Space Mouse demonstrations and continues collecting real robot experience. BC receives 100 expert demonstrations, roughly matching the converged RL replay-buffer data volume. Training uses one Nvidia RTX 4090. The paper does not specify whether the pretrained vision backbone is frozen. e09e10e11
PCB and cable episodes randomize the initial end-effector pose within fixed regions. Relocation starts above bin centers and learns both directions. These task-specific choices matter when interpreting sample efficiency. e10
During inference
Fresh images and proprioception drive closed-loop 6D end-effector commands. The illustrated hierarchy uses a 10 Hz policy and 1 kHz impedance controller. Reset execution comes from a separately trained agent, not inverse dynamics or an imagined future rollout. e05e07e09
5.5 Implementation flow
- Encode observations
An ImageNet-pretrained ResNet-10 vision backbone connects to a two-layer MLP. Images and robot proprioception inform the policy; no future-image or dynamics prediction model is described. e09e03
- Learn values and actions
RLPD updates a Q-function and an entropy-regularized actor. High update-to-data ratios reuse experience, half of each batch comes from prior data and half from online replay, and critic layer normalization supports repeated updates. e03
- Specify success and resets
PCB insertion uses a state-based reward assuming a rigidly held object. Cable routing and relocation use binary classifiers. Optional VICE refreshes classifier negatives with policy-visited states. Relocation uses independent forward and backward agents, each with its own policy, critic and reward. e04e05e10
- Separate learning from control
The actor sends actions to the robot environment; transition tuples feed the learner and policy parameters synchronize periodically. The low-level controller physically tracks the actor commands while learning proceeds asynchronously. e06e07
- Limit contact and remove absolute coordinates
The controller clips reference error at the fast control layer. Proprioception is expressed relative to the episode-initial end-effector frame, while policy actions use the current end-effector frame; the appendix describes conversion to the robot base frame. e07e08e16
6. Experiments & results
SERL integrates demonstration-assisted off-policy RL, reward specification, learned resets and compliant robot control into a practical software stack. It learns physically executed manipulation policies from images and proprioception; the reported system reaches 100/100 successes on three tasks. The evidence supports an effective integrated implementation, without isolating which component causes each gain.
The supplied paper contains no controlled component ablation. Figure 3 is included as a diagnostic of reference limiting, while Figures 6–7 compare the complete RL stack against behavioral cloning. These visuals cannot isolate the contribution of RLPD, clipping, relative frames or asynchronous learning. The source also lacks a quantitative perturbation-robustness breakdown. e08e20
6.1 Read the original evidence
Figure 5. The reset agent performs a second manipulation task that reverses the forward task. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the upper-left sequence as connector insertion and the upper-right sequence as routing a cable into a clip. Each sequence ends in a green-bordered high-reward state. The lower-left sequence moves the object from the right bin into the left bin; the lower-right sequence returns it. These directions agree with the task description and Section 4.3’s forward/backward design. The panels make the meaning of reset-free concrete: a learned agent physically rearranges the scene for another attempt. They also show why visual reward specification matters for a free object whose location cannot be inferred from end-effector position alone. e04e05e10e18
What it supports. The system demonstrates contact-rich insertion, deformable-object manipulation and free-object relocation. The lower pair illustrates the extra policy required to automate resets. This is a physical recovery of the task’s starting condition, not a reversal of stored observations or a backward prediction through a learned dynamics model.
Where the evidence stops. Selected successful sequences do not establish success probabilities or robustness under perturbations. Green borders identify high-reward states in the paper; they do not validate the classifier on unseen failures or show all attempted resets.
Table 2. Training time belongs to a particular task, sensor layout, reward and reset protocol. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across a row before comparing training times. All three rows list 20 demonstrations, but PCB and cable use two wrist cameras, whereas relocation uses one wrist and one side camera. The random-reset column refers to initial end-effector sampling: the caption specifies uniform random starts for PCB and cable and starts above bin centers for relocation. The size column gives 10 cm × 10 cm, 20 cm × 20 cm and 20 cm × 30 cm, respectively. Ground-truth reward is used for PCB; the other rows use binary classifiers. The relocation time covers the forward/backward pair, as clarified by the Results text. e05e09e10e11
What it supports. The displayed times are 20 minutes for PCB, 31 minutes for cable and 105 minutes for the relocation pair. These are task-specific reported training times, not an across-task average or a complete end-to-end accounting of demonstration collection and hardware preparation. The camera and reset choices should accompany any reproduction comparison.
Where the evidence stops. The prose says PCB and cable converge in under 30 minutes, conflicting with cable’s 31-minute table entry. The generic setup mentions two wrist cameras, but this table specifies relocation’s wrist/side arrangement. The report retains the task-specific entries.
Figure 6. RL succeeds in all reported trials, with a substantial advantage over the supplied behavioral-cloning baseline. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The vertical axis counts successes out of 100 trials, so each labeled count also equals its percentage under this protocol. Match blue BC and red RL within each task: PCB is 10 versus 100, cable 19 versus 100, and relocation 58 versus 100. Section 5 states that both methods use the same task evaluation conditions and the same initial-pose randomization scheme. Their training data differ: RL starts from 20 demonstrations and gathers online experience, whereas BC fits 100 expert demonstrations. The authors say that BC’s demonstration data volume roughly matches the converged RL replay buffer; this is not an identical-data comparison. e10e11e12e20
What it supports. The strongest supported finding is that the complete RL system reaches 100/100 in all three plotted task categories under the authors’ evaluation. It substantially exceeds BC even though it starts with fewer demonstrations. This supports the usefulness of continued interactive learning in the implemented system, without attributing gains to a single component.
Where the evidence stops. An observed 100/100 is not a proof of a true success probability of one. No training-seed uncertainty or component ablation is displayed, and the relocation bar does not distinguish the two directional policies.
Figure 7. Reported RL executions complete faster than BC in each task, with a small caption-versus-number discrepancy for PCB. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the horizontal axis, measured in seconds, rather than comparing bar heights. Shorter is faster. Unlike Figure 6, this plot lists relocation first and PCB last. Red denotes RL and blue denotes BC: relocation is 7.53 versus 18.94 seconds, cable 4.08 versus 13.58, and PCB 5.22 versus 10.14. The caption describes these as average times taken to succeed. Pair this figure with the success plot because BC fails many more evaluation trials. A completion-time mean and a success count answer different questions; the PDF does not explain precisely how unsuccessful attempts enter the timing statistic. e11e12e13
What it supports. All displayed RL means are lower than the corresponding BC means. Reader-calculated BC/RL time ratios are approximately 2.52 for relocation, 3.33 for cable and 1.94 for PCB. Thus the plot supports faster reported completion, but the PCB numbers do not literally satisfy the caption’s at-least-2× statement.
Where the evidence stops. The caption overstates the PCB ratio slightly. The PDF also omits timing uncertainty and failed-trial aggregation details, so these means cannot by themselves establish expected throughput across repeated attempts or a reliability-adjusted speed advantage.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| PCB component insertion Real Franka Panda; two wrist cameras; randomized initial pose in the Table 2 10 cm × 10 cm region; state-based reward. RL: 20 demos plus online data; BC: 100 demos; 100 evaluation trials each. | RL 100/100; 20 minutes Successful trials; training wall-clock time | BC 10/100. A large observed advantage under the same stated evaluation protocol; training includes online interaction unavailable to BC. e09e10e11e12 |
| Cable routing Real robot, two wrist cameras; randomized initial pose in a 20 cm × 20 cm region; classifier reward; same 20-demo RL versus 100-demo BC protocol and 100 evaluation trials. | RL 100/100; 31 minutes in Table 2 Successful trials; training wall-clock time | BC 19/100. Supports deformable-object manipulation in this setup. The prose says under 30 minutes for both cable and PCB; the table reports 31 for cable. This discrepancy remains unresolved. e10e11e12 |
| Object relocation Real robot; one wrist and one side camera; classifier rewards; 20 cm × 30 cm bins; starts above bin centers. Forward and backward agents learn relocation and reset. | RL 100/100 for the plotted task; 105 minutes total for forward/backward training Successful trials; combined training time | BC 58/100 with 100 demonstrations. The result concerns physical relocation with learned resets. Figure 6 does not separately report forward/backward success counts, and 105 minutes is not a single-policy training time. e05e10e11e12 |
| Manipulation cycle time Figure 7 reports average time to succeed for the same three task categories and RL/BC comparison. | RL: PCB 5.22; cable 4.08; relocation 7.53 Average cycle time in seconds; lower is better | BC: PCB 10.14; cable 13.58; relocation 18.94. All plotted means favor RL. PCB is about 1.94× faster by the displayed values, slightly below the caption’s claim of at least 2× for every task. Failure-time handling and uncertainty are unspecified. e13e11 |
| University of Washington peg insertion Author-reported deployment at another institution using 3D-printed Functional Manipulation Benchmark parts and 20 initial demonstrations. | 100/100; 19 minutes training; under 3 hours preparation Successful trials; convergence and preparation time | No matched BC result is reported for this deployment. A portability demonstration reported in the paper, not a reproduction conducted for this reading. e14 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 3. Reference limiting changes the contact trajectory while preserving fast free-space movement. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the legend before comparing panels: orange is the 10 Hz RL target, red the 1 kHz interpolated target, blue the clipped target and green the measured end-effector position. Both horizontal axes are time in milliseconds; vertical axes are position in meters. In the left panel, the command continues toward a hard surface while the measured position stops short, and the clipped reference stays closer to the measured position than the unbounded target. In the right panel, the clipped and measured traces approach the target during free-space motion. Section 4.5 and the caption agree on this contact/free-space distinction. e07e20
What it supports. The trace supports the intended controller behavior: reference error is limited during contact without reducing every high-level action to a tiny displacement. Figure 4 and Section 4.5 explain why multiple low-level updates within one policy step can allow useful free-space travel while enforcing a local reference constraint.
Where the evidence stops. These are position traces, not measured force curves or a success-rate ablation with clipping removed. They support a mechanism diagnosis but do not quantify damage risk, force uncertainty or the controller’s isolated contribution to learning.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors limit the framework’s scope: it is not a comprehensive algorithm library, and classifier rewards or forward/backward resets may fail in other settings. e15
The main evaluation reports no training-seed uncertainty or controlled removal of RLPD, clipping, relative frames or asynchronous learning. Figure 3 is a controller diagnostic, not an RL component ablation. Perturbation robustness is described without a dedicated quantitative stress-test breakdown. e08e11e12e13e20
Table 1 compares different insertion tasks, hardware/setup choices, reward shaping and demonstration budgets. It supplies literature context, not a controlled cross-system ranking. e17
7.2 Questions for discussion
- How much of the contact-task gain depends on fast reference limiting rather than the learning algorithm?
- Would relative observations and actions still help under controlled target displacement when image statistics and starting poses are matched?
8. Reproducibility audit
8.1 Requirements and known gaps
Reconstruct the Panda setup, camera placement, demonstrated tasks, randomization and reward labels before comparing training time. The PDF supplies a GPU model and backbone but omits a full hyperparameter configuration, including the numerical UTD ratio, learning rates, classifier dataset sizes and controller gains/clipping thresholds. e03e04e07e09e10e11
Resolve the appendix’s frame notation before implementing action conversion: its prose names an initial-frame transform while the displayed adjoint uses current-time blocks, and the final rotation block changes the episode superscript from i to t. The intended current-frame-to-base mapping is described, but the printed inconsistencies are not repaired here. e16
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Isolate where reference limiting acts
Reader-proposed experiment, not performed: on the PCB setup, compare SERL’s real-time reference limiting with clipping only the high-level policy displacement, holding controller gains, observations, reward, starting-pose distribution, demonstrations and learner settings fixed. Use conservative preset limits for both conditions. First replay matched free-space and contact commands; measure displacement per policy block, tracking error and interaction force with independently calibrated instrumentation. Then compare learning curves and 100-trial success/cycle-time evaluations across repeated training runs, reporting all resets and interventions. The proposed mechanism predicts that fast-layer limiting preserves useful free-space speed while bounding contact reference error. If matched limits produce the same travel and learning behavior, the claimed placement advantage would not be supported. e07e09e10e11e12e20
Check 2: Separate the effects of relative observations and relative actions
Reader-proposed experiment, not performed: use a two-by-two comparison of absolute versus episode-relative proprioception and base-frame versus current-end-effector-frame actions on PCB insertion. Re-express the same demonstrations for each condition and keep images, rewards, controller settings, training budget and starting-pose sampling matched. Resolve the appendix’s transform notation first and verify round-trip frame conversions on known poses. Train with a fixed target; evaluate both the training arrangement and predefined target displacements, including matched mid-episode perturbations. Record success, cycle time and recovery attempts across repeated runs. An advantage that appears specifically under displacement would support the spatial-generalization rationale; equal generalization after these controls would weaken it. e08e09e10e11e16
8.3 Reading coverage
Visual audit: Visually inspected the title, ten authors, affiliations and v4 date on p. 1; framing and problem on pp. 2–3; losses and reward definition on p. 4; architecture and reset design on p. 5; controller equations, hierarchy and diagnostic on p. 6; tasks, prior-work table and network setup on p. 7; Table 2, success/cycle-time plots and GPU/protocol details on p. 8; cross-institution result, limitations and appendix start on p. 9; and frame equations on p. 10. Figures 1–8 and Tables 1–2 were visually read. All six final crops were separately viewed. Figure 2 arrow directions and Figure 3 legend were checked against captions and method text. The Table 2 timing discrepancy, Figure 7 caption discrepancy and appendix notation inconsistencies are disclosed. Pages 11–14 contain references and were read as text; no supporting scientific claims rely on visually uninspected pages. External code, videos and supplements were not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, author block and abstract (p. 1)
- 1. Introduction (pp. 1–2)
- 2. Related Work (pp. 2–3)
- 3. Preliminaries and Problem Statement (p. 3)
- 4. Sample Efficient Robotic Reinforcement Learning in the Real-World, including 4.1–4.6 (pp. 3–7)
- 5. Experiments, including cross-institution reproducibility (pp. 7–9)
- 6. Discussion and Acknowledgments (p. 9)
- 7. Appendix / 7.1. Details on Relative Observation and Action Frame (pp. 9–10)
- References (pp. 10–14)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The supplied artifact is arXiv:2401.16013v4, dated 20 March 2025, with a January 2024 title-page header. Its title and all ten authors match the catalog. The catalog cites ICRA 2024; that proceedings edition and earlier revisions were not supplied, so version-to-version content differences cannot be established.
- The extraction omitted figure images; this was addressed by inspecting original PDF pages 1–10 and every final crop. Reference-only pages 11–14 were read as text.
- Separate supplemental material availability has not been fully verified; no supplements were supplied.
- Code, project documentation and videos were not inspected. No installation or experiments were performed.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, author/affiliation block and arXiv margin
Exact title and ten authors agree with the supplied identity. Jianlan Luo and Zheyuan Hu share equal contribution. Four affiliations are printed. Margin: arXiv:2401.16013v4, 20 Mar 2025; header: January 2024.
Go to primary source ↓e02PDF pp. 2–3, Sections 1–3
SERL integrates existing RL, rewards, resets, adapters and control as a community resource; the authors explicitly do not propose a new algorithm or benchmark.
Go to primary source ↓e03PDF p. 4, Section 4.1 and unnumbered critic/actor losses
RLPD uses high UTD, equal prior/online batch sampling, critic layer normalization, target-network Q regression and entropy-regularized actor optimization with adaptive alpha.
Go to primary source ↓e04PDF pp. 4–5, Section 4.2
Reward choices include state-based rules, binary success classifiers with log event probability, and VICE updates adding policy-visited states as negatives. PCB reward assumes a rigidly held object.
Go to primary source ↓e05PDF p. 5, Section 4.3
Reset-free training uses two independent RL agents with separate policies, Q-functions and rewards; forward execution and backward reset are both learned.
Go to primary source ↓e06PDF p. 5, Section 4.4 and Figure 2
Gym-like adapters interface with robots. Figure 2 sends actor actions to the environment and environment transitions to the learner; it labels parallel training and periodic policy synchronization.
Go to primary source ↓e07PDF pp. 5–6, Section 4.5, unnumbered impedance equations and Figures 3–4
The hierarchy illustrates 10 Hz RL and 1 kHz tracking. Reference error is bounded at the real-time layer; the free-space condition is M times absolute Delta at least maximum absolute action. Figure 3 plots target, interpolated target, clipped target and measured position for contact and free-space movement.
Go to primary source ↓e08PDF pp. 6–7, Section 4.6
Initial end-effector poses are randomized; proprioception is expressed in the initial end-effector frame and actions in the current frame. The text describes adaptation to moving or perturbed objects without reporting a separate quantitative perturbation table.
Go to primary source ↓e09PDF p. 7, Section 5, experimental setup
Franka Panda; an ImageNet-pretrained ResNet-10 connected to a two-layer MLP; camera images plus pose, twist, force and torque; 6D end-effector delta-pose actions. Backbone freezing is not specified.
Go to primary source ↓e10PDF p. 8, Table 2, all task rows, caption and Results paragraph
PCB/cable/relocation rows report 20 demonstrations and 20/31/105 minutes. Camera layouts, rewards, random reset flags and dimensions are task-specific. Relocation time covers both agents. Cable timing conflicts with the prose’s under-30-minute statement.
Go to primary source ↓e11PDF p. 8, Section 5, demonstration collection and Results paragraphs
RL begins with 20 Space Mouse demonstrations; BC uses 100 expert demonstrations, roughly matching RL buffer volume. Both use the stated initial-pose protocols. Training uses one Nvidia RTX 4090. Evaluation uses 100 trials per task; training time includes computation, resets and intended stops.
Go to primary source ↓e12PDF p. 8, Figure 6, all labeled bars and caption
RL has 100 successes in each 100-trial task. BC has 10 for PCB, 19 for cable and 58 for relocation. No error bars or separate relocation-direction counts are shown.
Go to primary source ↓e13PDF p. 8, Figure 7, labeled bars and caption
RL/BC cycle times in seconds are PCB 5.22/10.14, cable 4.08/13.58 and relocation 7.53/18.94. The caption claims at least 2× faster in all tasks, though the PCB ratio from labels is below 2. Averaging and failed-attempt handling are not detailed.
Go to primary source ↓e14PDF p. 9, Reproducibility paragraph and Figure 8
University of Washington peg insertion uses FMB 3D-printed parts, 20 demonstrations, under 3 hours setup, 19 minutes convergence and 100/100 success.
Go to primary source ↓e15PDF p. 9, Section 6, limitations paragraph
The framework is not comprehensive, may exclude non-manipulation settings, and does not solve all reward-specification or reset-free learning problems.
Go to primary source ↓e16PDF pp. 9–10, Section 7.1, relative transform and adjoint equations
The appendix transforms pose into the episode-initial frame and describes mapping current end-effector twist to base coordinates. Its adjoint-definition prose references the initial transform while matrix entries reference the current time; the bottom-right rotation block also has an inconsistent episode superscript.
Go to primary source ↓e17PDF p. 7, Table 1; pp. 8–9, Comparison to prior systems
Prior insertion reports differ in tasks, demonstration budgets, rewards and setup. The authors acknowledge comparability limits. The prose points to Table 2 for this comparison, but the actual prior-work comparison is Table 1.
Go to primary source ↓e18PDF p. 7, Figure 5 and Section 5 task descriptions
Four photo sequences show PCB insertion, cable routing and relocation in both directions. Green borders denote high-reward completion states; forward relocation moves the object right-to-left and backward reverses it.
Go to primary source ↓e20PDF p. 6, Figure 3; pp. 7–9, complete Section 5 and Figures 5–8
The empirical material comprises controller traces, task demonstrations, RL/BC success and cycle-time comparisons, literature context and cross-institution deployment. No controlled component ablation is reported.
Go to primary source ↓8.5 Primary sources
SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning ↗
PDF · 9,088 extracted words
Source fingerprint
4a45bc106add1571a0f5e02f295522ce8ae75186943f66c9339a5954a81bac16