World Action Models Enable Continual Imitation Learning with Recurrent Generative Replays
1. Paper overview
In one sentence: A world-action policy can rehearse old skills through recurrent synthetic trajectories, but retention depends on whether its imagined observations and actions remain coherent. ev-identityev-recurrenceev-updateev-liberoev-vlaev-diagnostics
| At a glance | What to know |
|---|---|
| Research problem | Source description A pretrained robot policy must acquire new tasks without retaining earlier demonstrations. Only earlier task instructions remain available. Ordinary fine-tuning provides no old-task supervision, while experience replay violates this information constraint. REGEN asks whether the policy's own joint action-and-observation generator can provide usable replacement supervision. ev-framing |
| Core mechanism | Source description REGEN turns an existing WAM into its own recurrent replay generator, avoiding a separately trained dynamics model and stored old demonstrations. ev-framingev-jointev-recurrence |
| A key reported result | LIBERO-Object continual imitation learning: REGEN: 95.3 / 26.1 / 65.5 FWT ↑ / NBT ↓ / AUC ↑, reported scale. Six base tasks, four sequential additions, 50 demonstrations/task; 50 randomized trials per observed task after each stage. Seq-FT: 92.7 / 82.6 / 24.9; ER: 95.7 / 4.8 / 93.4; RAR: 96.9 / 3.0 / 95.2. Generated replay improves retention substantially. ER uses stored demonstrations and RAR simulator interaction, so their information/access budgets differ. RAR actually exceeds ER here despite the prose broadly calling ER strongest. ev-sim-protocolev-metricsev-libero |
| Reading caution | Reader analysis Recursive visual artifacts accumulate within rollouts and over adaptation stages. The authors associate declining PSNR with forgetting and hypothesize weakened action-observation alignment; the diagnostic comparison does not isolate a single causal mechanism. ev-diagnosticsev-stage-blur |
Core contributions
- Source description
REGEN turns an existing WAM into its own recurrent replay generator, avoiding a separately trained dynamics model and stored old demonstrations. ev-framingev-jointev-recurrence
- Author claim
The authors demonstrate improved retention in LIBERO and physical manipulation, then identify visual degradation and action-observation inconsistency as bottlenecks. Their priority claim of being the first such framework is an author claim, not independently established here. ev-framingev-liberoev-robotev-diagnostics
Figure 2. Available scene observations seed rehearsal of a different, previously learned instruction. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the bottom-left demonstration strip: the available task puts cheese in a bowl, while the instruction supplied to the policy asks it to put the bowl on the stove. In the expanded sequence on the right, blue marks initialization from a real observation and green marks subsequent generation. Follow the curved gray arrows from predicted observations into later inputs. Action outputs leave the policy separately and enter the replay record; this diagram does not show those actions being executed to obtain the next image. The caption simplifies the sequence to H=1. For the experimental H=16, Eq. (2) supplies the timing: a recurrent input was predicted H timesteps earlier. ev-recurrenceev-updateev-training
What it supports. The mechanism makes old-task supervision available without retrieving old demonstrations. It changes the instruction while borrowing an available scene, then reuses its own predicted observations. Eq. (3) pairs each actual input observation with the first action of its predicted chunk before the dataset is used for adaptation.
Where the evidence stops. The caption calls the left side unrolled, although the explicit unrolling is on the right. Its schematic pair labels also should not replace Eq. (3)'s input/action indexing. Use Eqs. (2)–(3) and Algorithm 1 for timing and pairing.
2. Motivation
2.1 The problem and the proposed response
A pretrained robot policy must acquire new tasks without retaining earlier demonstrations. Only earlier task instructions remain available. Ordinary fine-tuning provides no old-task supervision, while experience replay violates this information constraint. REGEN asks whether the policy's own joint action-and-observation generator can provide usable replacement supervision. ev-framing
2.2 What this reading follows
Suppose a robot has learned to put a bowl on a stove, then receives demonstrations for putting cheese in a bowl. REGEN retains the old instruction and asks the existing policy to generate the missing rehearsal data from available observations. Its future images become inputs to further predictions, creating synthetic observation-action sequences for continual fine-tuning. This reading follows that feedback loop, compares its retention with real and simulator-collected replay, and examines why a convincing imagined success can still produce unsuccessful actions. The supplied v1 contains useful evidence and several unresolved reporting differences, which remain visible throughout the bundle. ev-identityev-recurrenceev-updateev-liberoev-vlaev-diagnostics
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | One Model |
| Prediction paradigm | Joint prediction |
| Quadrant | Q1 · One Model × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded One Model × Joint prediction classification is supported by the shared latent diffusion architecture for action, observation and reward generation, not merely by joint training. REGEN adds continual rehearsal around that WAM; it does not introduce a separate inverse-dynamics action extractor or an inference-time planning search. Generated observations actively condition replay generation. ev-jointev-recurrenceev-update
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Build rehearsal by changing the instruction and recycling the predicted scene
The crucial operation occurs before adapting the policy. Take the preceding checkpoint, select an old instruction, and initialize it with observations from a current-task demonstration. For the first H timesteps those observations are real; afterward Eq. (2) supplies observations generated H timesteps earlier. Algorithm 1 advances time one step at a time, so H=16 creates a delayed prediction pipeline rather than a sequence of independent one-frame guesses. At each step, save the input observation and the first predicted action. The synthetic trajectory therefore contains supervision for the old instruction even though its original demonstration is unavailable. Only after constructing the replay set does the algorithm update the policy with mixed data. This distinction matters: recurrent imagination is an adaptation-data mechanism, while robot evaluation tests execution of the resulting policy. ev-recurrenceev-updateev-trainingev-robot
5.2 Separate acquisition, retention and access to past environments
Reader analysis: the strongest comparison for the stated constraint is REGEN against sequential fine-tuning within the same suite. Both acquire new tasks, but generated rehearsal supplies old-task examples and changes retention markedly. The paper's FWT definition averages acquisition scores; it does not subtract a no-transfer baseline, so high FWT alone cannot quantify positive transfer from earlier tasks. NBT and AUC address what remains available after further adaptation. ER and RAR then help interpret the residual gap, but their extra access must remain explicit: ER has old demonstrations, and RAR can revisit the simulator. Spatial introduces another exception by seeding from previous-task object configurations. The lesson is to compare outcomes together with the information each replay generator receives, rather than selecting the best number across suites. ev-metricsev-liberoev-sim-protocolev-spatial
5.3 Trace how an imperfect prediction becomes imperfect supervision
Reader analysis: REGEN's generator also determines what the learner rehearses. A blurry or misplaced object in a predicted observation becomes part of the next conditioning input, and an inconsistent observation-action pair enters the training set. Figure 5 and Appendix Figures 8–9 support these failure modes, but they do not establish that image quality alone causes forgetting. The termination experiment is useful because it changes how long recurrent errors can accumulate: the goal-reward rule yields higher PSNR than either fixed horizon. However, shorter trajectories also alter which frames and actions enter replay. A careful interpretation therefore separates visual fidelity, consistency with executed actions, and retention after fine-tuning. The ablation establishes a difference in reported image quality; verifying the complete causal chain requires the controlled checks proposed below. ev-updateev-diagnosticsev-stage-blurev-action-mismatchev-stopping
5.4 Training and inference
During training
The base stage uses 10K iterations; each subsequent stage uses 2K from the preceding checkpoint, with ten pseudo-trajectories per previous task. Appendix A.1 describes joint latent denoising under a flow-matching objective; Eq. (4) gives the replay update abstractly as behavioral cloning. ev-trainingev-jointev-update
Table 6 specifies Adam at peak learning rate 10^-4, 1,000 warm-up steps, 30,000 cosine-decay steps, batch size 40 per GPU, four GPUs and 12 accumulation steps. Inputs are 224×224, actions seven-dimensional and proprioception nine-dimensional; augmentations are random crop, color jitter and Gaussian blur. ev-training
During inference
Replay generation uses model predictions without simulator interaction or robot execution. Table 6 specifies five denoising steps for actions and one for observations/value. Robot evaluation executes the adapted policy; the reward head's imagined completion signal is not physical success verification. The supplied paper does not fully specify deployment action scheduling. ev-recurrenceev-trainingev-robotev-diagnostics
5.5 Implementation flow
- Encode a joint prediction problem
Cosmos-Policy starts from Cosmos-Predict2-2B. Wan2.1 tokenizes images and pretrained T5-XXL encodes instructions. Normalized actions and proprioception become latent frames; one latent diffusion model handles actions, future observations and reward. ev-joint
- Seed the old instruction with available observations
For each previous instruction, use real current-task observations for the first H timesteps. The instruction selects the old behavior while the visual context supplies the scene. This assumes the scene contains objects needed by the old task. ev-recurrenceev-spatial
- Feed predictions back at the correct horizon
For t≥H, use the observation predicted at t−H. The algorithm advances t by one and stores only the first action of each predicted chunk. Figure 2 simplifies this pipeline to H=1; experiments use H=16. ev-recurrenceev-updateev-training
- Stop and assemble replay
Stop at the maximum horizon or when three consecutive progress predictions exceed 0.99 and at least one reaches 1.0. Aggregate synthetic pairs across old tasks, then train on their union with current demonstrations. ev-update
6. Experiments & results
REGEN reuses a world-action policy to generate rehearsal trajectories for old instructions while learning a new task. Current-task observations seed recurrent prediction; synthetic observation-action pairs then supplement new demonstrations. Retention improves substantially over sequential fine-tuning, but real replay generally remains stronger, and plausible imagined outcomes can disagree with executed actions.
6.1 Read the original evidence
Table 1. Generated replay recovers substantial retention while leaving a gap to privileged replay. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each suite as a separate three-column comparison. FWT is the paper's average acquisition score, NBT measures relative forgetting, and AUC aggregates performance over later evaluations; their arrows indicate preferred directions. Begin with Seq-FT and REGEN to see what generated rehearsal changes. Then compare ER, whose gray row uses stored real demonstrations, and RAR, which obtains replay through simulator interaction. Those rows have different access assumptions from REGEN. The protocol starts with six tasks and introduces four more, evaluating every observed task with 50 randomized trials after each stage. The dagger beside Spatial REGEN denotes its special initialization from previous-task object configurations. ev-metricsev-sim-protocolev-liberoev-spatialev-vla
What it supports. On Object, REGEN reports NBT 26.1 and AUC 65.5 versus Seq-FT's 82.6 and 24.9, while FWT remains high. ER reaches AUC 93.4 and RAR 95.2, so this table supports substantial recovery rather than equivalence to real replay. RAR also exceeds ER on Object despite the main text's broad description of ER as strongest.
Where the evidence stops. Table 2 reports Goal NBT 38.7 instead of this table's 44.9. Spatial uses the prior-configuration exception, and the tables provide no training-seed uncertainty. These qualifications prevent treating every column as a clean test of the same instruction-only assumption.
Table 3. Physical manipulation improves under the paper's partial-scoring evaluation. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across a row to keep acquisition, forgetting and aggregate performance together. The xArm7 starts with carrot-in-bowl, adapts to carrot-on-plate, then to eggplant-in-bowl. Each task supplies 50 teleoperated demonstrations at 15 Hz. The reported metrics summarize the two continual stages, using ten randomized trials per task with changes in object placements and initial gripper configurations. Appendix B.3 is essential to interpreting this small table: it assigns 50 points for touching the target and another 50 for reaching the goal. Accordingly, the displayed values summarize scored behavior and should not be read as the fraction of trials completing the entire task. ev-robotev-metrics
What it supports. REGEN raises reported FWT from 50 to 80 and AUC from 13.8 to 53.8, while NBT falls from 96.3 to 60.5. This provides physical-execution evidence beyond generated videos. The source attributes the acquisition improvement to replay regularization in the small base-task regime; that causal explanation is not separately isolated.
Where the evidence stops. The three tasks share a pick-and-place structure, and each task has only ten evaluation trials. Partial credit and the absence of reported seed uncertainty limit claims about binary completion rates, broader manipulation generalization or the precision of the improvement.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO-Object continual imitation learning Six base tasks, four sequential additions, 50 demonstrations/task; 50 randomized trials per observed task after each stage. | REGEN: 95.3 / 26.1 / 65.5 FWT ↑ / NBT ↓ / AUC ↑, reported scale | Seq-FT: 92.7 / 82.6 / 24.9; ER: 95.7 / 4.8 / 93.4; RAR: 96.9 / 3.0 / 95.2. Generated replay improves retention substantially. ER uses stored demonstrations and RAR simulator interaction, so their information/access budgets differ. RAR actually exceeds ER here despite the prose broadly calling ER strongest. ev-sim-protocolev-metricsev-libero |
| LIBERO-Goal continual imitation learning Same six-plus-four protocol and randomized evaluation as Object. | Table 1 REGEN: 90.6 / 44.9 / 40.8 FWT ↑ / NBT ↓ / AUC ↑ | Seq-FT: 90.6 / 100 / 10.3; ER: 94.0 / 7.2 / 92.4. Table 2 instead reports REGEN NBT 38.7, with FWT/AUC unchanged. Acquisition is preserved while retention improves, but the remaining ER gap is large. The conflicting NBT entries remain unresolved and must not be averaged or silently substituted. ev-sim-protocolev-liberoev-vla |
| LIBERO-Spatial continual imitation learning with prior configurations Same task count and evaluation protocol, but REGEN† seeds generation using previous-task object configurations. | REGEN†: 87.2 / 17.6 / 76.9 FWT ↑ / NBT ↓ / AUC ↑ | Seq-FT: 87.4 / 99.8 / 10.8; ER: 86.4 / -0.28 / 87.8. Strong retention under a modified initialization assumption; this is not a demonstration that current-task observations alone suffice for every suite. ev-sim-protocolev-liberoev-spatial |
| xArm7 sequential pick-and-place learning Carrot-in-bowl → carrot-on-plate → eggplant-in-bowl; 50 demonstrations/task at 15 Hz; ten randomized trials/task and two adaptation stages. | REGEN: 80 / 60.5 / 53.8 FWT ↑ / NBT ↓ / AUC ↑ from the reported partial-scoring protocol | Seq-FT: 50 / 96.3 / 13.8. Physical experiments support improved scored performance. Appendix B.3 awards 50 points for touching the object and 50 for reaching the goal; these aggregates must not be presented as binary task-completion percentages. ev-robotev-metrics |
| Replay quantity and termination ablations Replay count on LIBERO-Object; stopping criterion on LIBERO-Goal. | Five → ten replays: 96.5/33/62.5 → 96.1/31/62.3. Goal-reward stopping PSNR: 20.3. FWT / NBT / AUC for replay count; PSNR ↑ for stopping | Fixed horizons 200 and 150: PSNR 18.4 and 19.5. More replay modestly reduces NBT without improving AUC. The ten-replay row differs from Table 1 without explanation. Higher PSNR under early stopping does not establish improved executed success. ev-replay-countev-stoppingev-libero |
| Imagined versus action-grounded replay success LIBERO-Goal pseudo-trajectories across four continual stages; predicted actions executed in the simulator. | Imagined / grounded: S1 83/42, S2 65/20, S3 62/30, S4 44/38. Successful trajectories (%) | Each stage compares visual predictions with execution of predicted actions. The generation can depict success that its actions do not realize. Figure 5 diagnoses inconsistency, not a guarantee that improving image fidelity alone would close the retention gap. ev-diagnosticsev-action-mismatch |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Tables 4–5. Replay quantity and early stopping address different parts of replay quality. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Treat the left and right tables as separate experiments. On the left, moving from five to ten replays changes the amount of generated rehearsal for each old Object task; compare all three metrics before deciding whether the added replay helps. On the right, the Goal experiment changes how a generated trajectory ends and reports image PSNR, not policy success. The goal-reward rule requires a three-prediction completion window. The fixed-horizon rows use H=200 and H=150 in the original table, although the method elsewhere uses H for the action chunk and T_max for the rollout bound. That notation reuse does not change the experimental chunk length to 200. ev-replay-countev-stoppingev-updateev-trainingev-libero
What it supports. Ten replays reduce NBT from 33 to 31, but FWT falls from 96.5 to 96.1 and AUC from 62.5 to 62.3. Goal-based stopping yields PSNR 20.3 versus 18.4 and 19.5 for the fixed horizons. These results support modest retention benefit from extra replay and better reported image fidelity from stopping early.
Where the evidence stops. The ten-replay Object row differs from Table 1's default result, with no supplied explanation. PSNR reference alignment and averaging are not fully specified, and comparing different rollout lengths does not alone show better action supervision or downstream success.
Figure 5. Visual degradation and action inconsistency expose two limits of generated rehearsal. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the shared stage labels S1 through S4, but keep the vertical axes separate. The left panel tracks generated-frame PSNR; the caption identifies its error bars as standard deviations. The middle panel compares NBT for REGEN and simulator-collected RAR, showing the retention advantage of the latter. The right panel is the most direct consistency diagnostic: blue bars count trajectories judged successful from imagined observations, while orange bars count successful execution of predicted actions in the simulator. Its printed bar values are percentages. Appendix Figure 8 provides a qualitative counterpart to the left panel, showing progressively blurrier bowl-placement sequences across the same number of stages. ev-diagnosticsev-sim-protocolev-stage-blur
What it supports. At S1, 83% of imagined trajectories appear successful but only 42% succeed when the predicted actions are grounded in the simulator. At S4, those values are 44% and 38%. The smaller final gap accompanies a decline in imagined success; it should not be mistaken for evidence of steadily improving control.
Where the evidence stops. The panels show association, not a controlled isolation of visual fidelity as the cause of forgetting. RAR also changes where observations come from. The sample counts and detailed judging procedure for imagined-success percentages are not fully specified.
Figure 9. An imagined successful outcome can be inconsistent with the actions predicted alongside it. Original paper, p. 21 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each task from left to right in time, then compare the two rows within that task. In panel (a), the instruction asks the robot to push the plate in front of the stove. Panel (b) asks for the black bowl at the table center to be placed on the plate. The upper green-bordered row in each pair contains future observations imagined by the WAM; the lower red-bordered row comes from executing its predicted actions in the simulator. Watch the target object's final placement, not just whether the arm moves plausibly. The task labels are retained beneath each pair so the desired outcome remains explicit. ev-action-mismatchev-diagnosticsev-update
What it supports. These selected examples make the state-action mismatch concrete: a sequence can depict the desired object displacement while the corresponding actions fail to produce it. For REGEN, this matters because the generated observations and actions become training pairs for later rehearsal. Their visual plausibility alone cannot certify that they provide coherent behavioral supervision.
Where the evidence stops. These are illustrative cases, not a measured failure frequency; Figure 5 supplies the aggregate diagnostic. The frames do not isolate whether the inconsistency originates in action prediction, recurrent state degradation, or another implementation detail, and they cannot verify contact dynamics by themselves.
7. Analysis & limitations
7.1 What the evidence leaves open
Recursive visual artifacts accumulate within rollouts and over adaptation stages. The authors associate declining PSNR with forgetting and hypothesize weakened action-observation alignment; the diagnostic comparison does not isolate a single causal mechanism. ev-diagnosticsev-stage-blur
FWT here is average acquisition performance, not improvement over an untrained baseline. Main result tables lack uncertainty across training seeds. Figure 4's lower representation drift for REGEN is supportive but cannot independently establish task success. ev-metricsev-liberoev-robotev-drift
7.2 Questions for discussion
- Would state-action-consistent replay improve retention more than a PSNR-matched visual improvement? (ev-diagnostics, ev-action-mismatch)
- How far can instruction-only retention extend when required old-task objects are absent from current scenes? (ev-spatial)
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the specified Cosmos/Wan/T5 components, camera/proprioception preprocessing, fixed task orders, generated replay and stagewise checkpoints. Preserve the appendix's partial scoring and Spatial exception. Table 6 gives GPU count but no GPU model, runtime or software versions. ev-jointev-trainingev-sim-protocolev-robotev-spatial
Unspecified details include mixed-data sampling weights, which pretrained modules are frozen, exact PSNR reference/alignment, and how the learning-rate schedule restarts across stages. Also resolve NBT table conflicts, base-stage aggregation in the metric formulas, and zero acquisition-score handling before comparing implementations. ev-updateev-jointev-trainingev-stoppingev-metricsev-vla
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test stopping quality while controlling replay length and training exposure
Reader-proposed, not run: freeze one LIBERO-Goal checkpoint and generate replay from identical current-task seed observations and old instructions using the published goal-reward rule, fixed 150 steps and fixed 200 steps. Add a fixed-rule subset matched to the goal-rule trajectory-length distribution, and equalize the number of training pairs and optimizer updates across adaptation runs. Specify the PSNR reference and frame alignment before scoring. Execute the generated actions in the simulator and measure completion separately from visual PSNR, then compare old-task NBT after adaptation across repeated seeds. If goal stopping improves only unadjusted PSNR but not consistency or NBT after length matching, that would weaken the interpretation that it improves rehearsal supervision beyond removing difficult late frames. ev-updateev-stoppingev-diagnosticsev-trainingev-sim-protocol
Check 2: Test whether action-consistent observations improve replay beyond visual sharpness
Reader-proposed, not run: at one adaptation stage, generate a fixed set of old-task action sequences from matched LIBERO-Goal seeds. Build one replay set with their imagined observations and another with observations recorded while executing exactly those actions in the simulator, retaining the same instructions and input/first-action alignment. Add a third set whose simulator observations are blurred to approximately match the imagined set's PSNR, and match trajectory lengths, sample counts and fine-tuning updates. This is a diagnostic requiring privileged simulator access, not an implementation of the no-interaction deployment setting. Compare grounded success and downstream NBT across seeds. If simulator-grounded replay retains its advantage after approximate fidelity matching, that would support state-action coherence as a factor beyond sharpness; disappearance of the advantage would weaken that explanation. ev-updateev-diagnosticsev-action-mismatchev-sim-protocolev-training
8.3 Reading coverage
Visual audit: The title/author/version page, all method and numerical-evidence pages, Algorithm 1, task orders, evaluation rubric, hyperparameters, all nine figures and all six tables were visually inspected. The six final crops were individually inspected for legible labels, complete bounds and faithful content. Figure 2's left/right caption wording, horizon notation reuse in Table 5, Goal NBT disagreement between Tables 1–2, and the differing ten-replay Object row are disclosed. Reference-only pages 10–13 were read as supplied text but not rendered; external cited works and separate supplements were not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 14, 15, 16, 17, 18, 19, 20, 21. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, authors and arXiv v1 stamp (p. 1)
- Abstract and Sections 1–2: Introduction and Related Work (pp. 1–3)
- Section 3: World Action Models; Sections 4.1–4.2: formulation, replay and objectives (pp. 3–5)
- Sections 5.1–5.4: implementation, metrics, simulation, robot experiments and analyses (pp. 5–8)
- Sections 6–7: limitations and conclusion (pp. 8–9)
- Acknowledgments and References (pp. 10–14)
- Appendix A.1–A.2, Algorithm 1, B.1–B.3 and Table 6 (pp. 14–17)
- Appendix C.1–C.4 and Figures 6–9 (pp. 17–21)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction's image omission was addressed by inspecting the supplied PDF figures and tables. All five supplied text chunks were read individually; no supplied body or appendix section was omitted.
- Only arXiv:2606.27374v1, dated 25 June 2026, was supplied and reviewed. Title and author identities match the catalog; catalog surname-first formatting is the only author-list difference. No later revision or separate edition was inspected.
- Code, external project pages and separately linked references were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
ev-identityPDF p. 1, title block and arXiv margin stamp
The title and five authors match the catalog. The PDF identifies arXiv:2606.27374v1 [cs.RO], 25 June 2026, and the Department of Computer Science, University of North Carolina at Charlotte, United States.
Go to primary source ↓ev-framingPDF pp. 1–2, Abstract, Section 1 and Figure 1; pp. 3–4, Section 4.1
REGEN addresses continual imitation learning when previous-task instructions remain available but previous action-observation demonstrations do not. Generated replay is mixed with current-task demonstrations.
Go to primary source ↓ev-jointPDF p. 3, Section 3, Eq. (1); p. 14, Appendix A.1
Cosmos-Policy uses a latent video diffusion model initialized from Cosmos-Predict2-2B. Wan2.1 encodes images and pretrained T5-XXL encodes language; actions and proprioception are normalized to [-1,+1] and converted into latent frames. The model jointly denoises action, future-observation and reward latents under the described flow-matching objective.
Go to primary source ↓ev-recurrencePDF p. 4, Section 4.2, Figure 2 and Eq. (2); p. 15, Algorithm 1, lines 4–15
For t<H the input is a current-task real observation; afterward it is the observation predicted H timesteps earlier. Algorithm 1 increments t by one. Figure 2 illustrates H=1; its caption calls the left side unrolled although the explicit unrolled sequence is on the right.
Go to primary source ↓ev-updatePDF p. 5, Section 4.2, Termination, Eqs. (3)–(4); p. 15, Algorithm 1, lines 12–25
Generation stops at the maximum horizon or after three consecutive reward predictions above 0.99 with at least one reaching 1.0. Each replay entry pairs the input observation with the first predicted action. The policy is then updated using current real data plus generated replay.
Go to primary source ↓ev-trainingPDF p. 5, Section 5.1; p. 17, Table 6
The base model trains for 10K iterations and each continual stage for 2K, generating 10 replays per previous task. Table 6 specifies 224×224 images, action dimension 7, proprioception dimension 9, H=16, Adam, learning rate 10^-4, 1,000 warm-up steps, 30,000 cosine-decay steps, four GPUs, batch size 40 per GPU, accumulation 12, crop/color-jitter/Gaussian-blur augmentation, and five action versus one observation/value denoising steps.
Go to primary source ↓ev-metricsPDF p. 5, Section 5.1, FWT, NBT and AUC definitions
FWT averages diagonal task-acquisition scores r_n,n; NBT averages later relative drops (r_n,n-r_p,n)/r_n,n; AUC averages each task's scores from its acquisition onward. Higher FWT/AUC and lower NBT are preferred.
Go to primary source ↓ev-sim-protocolPDF p. 6, Section 5.2, Setting and Baseline Methods; pp. 15–16, Appendix B.1–B.3
Each LIBERO suite has ten tasks and 50 demonstrations per task; six tasks form the base stage and four arrive sequentially. The appendix gives fixed task orders shared across methods. Evaluation uses 50 randomized trials per observed task after each stage. ER retains real demonstrations; RAR collects simulator rollouts.
Go to primary source ↓ev-liberoPDF p. 6, Table 1, REGEN, Seq-FT, ER and RAR rows across all three suites
Object FWT/NBT/AUC: REGEN 95.3/26.1/65.5, Seq-FT 92.7/82.6/24.9, ER 95.7/4.8/93.4, RAR 96.9/3.0/95.2. Goal: REGEN 90.6/44.9/40.8, Seq-FT 90.6/100/10.3, ER 94.0/7.2/92.4. Spatial: REGEN† 87.2/17.6/76.9, Seq-FT 87.4/99.8/10.8, ER 86.4/-0.28/87.8.
Go to primary source ↓ev-spatialPDF p. 7, Section 5.2 continuation, paragraph explaining REGEN†
Spatial replay is initialized with object configurations sampled from previous tasks because required objects must be present. The authors state that trajectories involving absent objects cannot be reliably synthesized.
Go to primary source ↓ev-vlaPDF p. 7, Table 2 and VLA vs. WAM Continual Learning
Table 2 reports π0.5 Seq-FT at 96.8/88/35.5 and Cosmos-Policy REGEN at 90.6/38.7/40.8 on Goal. REGEN's NBT 38.7 differs from 44.9 in Table 1, with unchanged FWT/AUC; no reconciliation is supplied.
Go to primary source ↓ev-robotPDF p. 7, Section 5.3, Table 3 and Figure 3; p. 16, Appendix B.3, Real-world
The xArm7 learns carrot-in-bowl, carrot-on-plate, then eggplant-in-bowl with 50 demonstrations per task at 15 Hz and ten randomized evaluation trials per task. Table 3 gives Seq-FT FWT/NBT/AUC 50/96.3/13.8 and REGEN 80/60.5/53.8. Appendix B.3 awards 50 points for touching the target and 50 for reaching the goal.
Go to primary source ↓ev-driftPDF p. 8, Figure 4 and Section 5.4, Action representation drift and Visualization of predicted actions
After the first continual stage, mean action-latent centroid L2 distances across six base tasks are 0.30 for Seq-FT, 0.04 for ER and 0.12 for REGEN. The accompanying XY trajectory example shows REGEN closer to the demonstration than Seq-FT.
Go to primary source ↓ev-replay-countPDF p. 8, Section 5.4, Number of replays; p. 9, Table 4
For Object, five replays yield FWT/NBT/AUC 96.5/33/62.5 and ten yield 96.1/31/62.3. The ten-replay row differs from the main Object result; the supplied text does not explain the setting or run difference.
Go to primary source ↓ev-stoppingPDF p. 8, Section 5.4, Termination criterion; p. 9, Table 5
Goal-reward stopping yields reported PSNR 20.3 versus 18.4 for fixed horizon 200 and 19.5 for fixed horizon 150 on LIBERO-Goal. Table 5 labels these horizons H, although the method uses H for chunk length and T_max for maximum rollout length.
Go to primary source ↓ev-diagnosticsPDF p. 9, Figure 5, all panels, and Section 6
Generated-frame PSNR declines across stages; REGEN NBT remains above RAR. Imagined versus simulator-grounded success percentages are 83/42, 65/20, 62/30 and 44/38 for stages 1–4. The authors hypothesize that degraded observations weaken state-action alignment.
Go to primary source ↓ev-qualitativePDF p. 18, Figure 6 and caption; p. 19, Figure 7 and caption
Figure 6 pairs green expert frames with blue generated frames for four tasks. Figure 7 compares current-task, Seq-FT and REGEN rollouts on selected previous tasks in Goal, Object, Spatial and real-world settings.
Go to primary source ↓ev-stage-blurPDF p. 20, Figure 8 and caption; p. 17, Appendix C.3
Generated sequences for putting the bowl on top of the cabinet become progressively blurrier across the four continual stages.
Go to primary source ↓ev-action-mismatchPDF p. 21, Figure 9 and caption; p. 17, Appendix C.4
For pushing a plate and placing a black bowl, the upper green-bordered imagined sequences appear successful while the lower red-bordered simulator executions fail to realize the imagined states.
Go to primary source ↓8.5 Primary sources
World Action Models Enable Continual Imitation Learning with Recurrent Generative Replays ↗
PDF · 8,904 extracted words
Source fingerprint
b003b876ee0587bc5c068ebb42540ff8391d1fcb9d1ee647a3b4a6d6e32da6b1