NoiseGate: Learning Per-Latent Timestep Schedules as Information Gating in World Action Models
1. Paper overview
In one sentence: NoiseGate improves joint video–action control by learning when to denoise each imagined frame, at the cost of simulator rollouts for reward-based scheduler training. e03e05e06e08e10e14e15e17
| At a glance | What to know |
|---|---|
| Research problem | Author claim A shared video timestep exposes all imagined frames at the same nominal noise level, although their usefulness for action generation may differ. The authors propose keeping uncertain futures partially masked while exposing useful ones earlier. The question is whether task reward can learn this allocation inside a joint denoiser, rather than prescribing one ordering across future frames. e02e03 |
| Core mechanism | |
| A key reported result | RoboTwin random-scene: 50-task average: 94.28% reported Mean task success rate (%). Scaled Fast-WAM-style batch-1024 configuration; 100 evaluation episodes per task, with randomized poses, colors, and backgrounds. Stage-1 WAM 92.58% (+1.70 percentage points); Fast-WAM 91.78%, LingBot-VA 91.50%, π0.5 76.76%, Motus 87.02%. The same-backbone Stage-1 comparison best isolates scheduler training. Table 5 adapts prior-method columns from an earlier detail table, limiting fully controlled external comparisons. No confidence intervals are reported. e07e08 |
| Reading caution | Source description Simulator rollouts dominate scheduler-training cost; sparse successes slow convergence. A general scheduler scaling across large multi-task suites remains future work. Driving, navigation, and video-generation applications are proposed, not tested here. e15 |
Core contributions
Figure 2. Train the joint model to tolerate heterogeneous noise, then learn only its video scheduler. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the lower left: action and video encoders feed separate expert streams that meet at Joint Self-Attention. The different video-token shades indicate heterogeneous noise; the action stream retains a global timestep. Now follow the upper-right loop. Snowflakes mark the frozen video and action backbones, while the orange reward path returns a GRPO update to the GPN only. The lower-right inset expands the scheduler: latent encoding, observation-conditioned fusion, temporal processing, and a squashed-Gaussian output. Its positive Δt values are decrements, subtracted from video times according to Equations (5)–(6). The fixed observation and independent action clock constrain what this learned module can change. e03e04e05e06e16e17e18
What it supports. NoiseGate changes the video context consumed during action denoising without directly retraining the action expert in Stage 2. The diagram separates the preparation needed to handle heterogeneous noise from reward learning of the schedule itself. This supports joint prediction through coupled experts, rather than a video-generation-then-inverse-dynamics pipeline.
Where the evidence stops. The diagram’s decrement relation follows the main-text scale convention. Appendix C and Algorithm 1 use inconsistent scale/log-density notation; equivalent update magnitudes do not by themselves resolve the policy-probability implementation. Language conditioning is specified in the equations but not expanded here.
2. Motivation
2.1 The problem and the proposed response
A shared video timestep exposes all imagined frames at the same nominal noise level, although their usefulness for action generation may differ. The authors propose keeping uncertain futures partially masked while exposing useful ones earlier. The question is whether task reward can learn this allocation inside a joint denoiser, rather than prescribing one ordering across future frames. e02e03
2.2 What this reading follows
A visually clean imagined future can still be wrong at the moment a robot needs to grasp an object. NoiseGate treats future-video noise as a control variable inside a joint video–action model: some frames become available early, while others remain partially masked. Its Gating Policy Network learns those schedules from task success while the action clock stays fixed. Read the architecture first, then compare the full benchmark with the smaller controlled study. Finally, use the schedule and attention diagnostics to assess the authors’ information-gating explanation. The evidence supports improved simulated task success, with unresolved reporting and policy-density inconsistencies that matter for reproduction. e03e05e06e08e10e14e15e17
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Joint prediction |
| Quadrant | Q3 · Dual-system × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
Joint prediction is supported by co-denoising video and actions. Dual-system is defensible at the expert-module level: distinct video and action DiTs retain modality-specific parameters while sharing attention. This is tightly coupled generation, not a sequential planner/controller pipeline. The GPN controls video scheduling during inference; reward post-training updates that scheduler rather than the action expert. e03e05e06
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Treat imagined video as a changing input to the action expert
The useful starting point is the joint sequence. Future video and action tokens interact repeatedly through self-attention, so an imagined frame can influence action refinement before generation finishes. A shared scalar video timestep reveals every future latent at the same nominal noise level. NoiseGate first trains the backbone to tolerate heterogeneous levels, then gives a separate policy control over their evolution. The reader’s interpretation is that scheduling changes the evidence available to the action expert without explicitly multiplying attention weights by a learned mask. That distinction matters: the GPN directly controls noise times, while reduced attention is an empirical association measured later. Clean observation conditioning remains available, the action clock is fixed, and the GPN does not output motor commands itself. e02e03e04e05e12
Figure 4. Frame priorities change across the two illustrated denoising chunks. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the horizontal axis from the beginning to the end of one denoising chunk. The black conditioning-latent line stays at zero, matching the pinned clean observation. Pink and yellow trace the two predicted latents: a faster downward trajectory means the corresponding video frame is denoised earlier. In Task A the two curves stay close, with the yellow curve ending below the pink. In Task B they separate more strongly, and the pink curve reaches zero first. The action clock is not plotted; according to the method it advances on its own fixed schedule throughout this adaptation. e05e13e18
What it supports. The examples demonstrate that the GPN does not always use one shared video timestep or one fixed ordering of future frames. Task B retains residual noise in the second latent while the first becomes clean. This is compatible with selective exposure of imagined context, rather than requiring every future frame to reach zero noise.
Where the evidence stops. Task A and Task B are not identified by benchmark task names. These are single-chunk illustrations, not distributions or matched success comparisons. The curves support schedule diversity, but cannot independently establish why a particular schedule improves actions.
5.2 Follow one timestep update and identify the notation boundary
Take the current latent stack and video times as the scheduler’s state. Its encoder summarizes each frame, fuses the clean observation, and combines temporal and timestep features before producing a Gaussian actor distribution. Under the main-text convention, a squashed sample gives a relative scale between zero and two. Multiplying by the nominal decrement and subtracting with a zero floor can make one frame advance faster than another. This is monotone progress within each frame, without an imposed ordering across frames. Algorithm 1 then advances the action clock and calls the frozen backbone. The reproduction problem is the actor probability: Appendix C mixes the scaled and unscaled sigmoid conventions. Equivalent decrement formulas are possible, but its printed inverse and density cannot be accepted as mutually consistent without clarification. e05e16e17e18
5.3 Separate performance evidence from the explanation of that performance
The scaled Stage-1 versus NoiseGate comparison asks whether reward-trained scheduling improves a fixed backbone. Its average is positive, while the microwave regression shows that the benefit is not uniform. The smaller ablation asks which components matter: independent-noise training, heterogeneous inference, and learned scheduling are introduced in sequence. Keep its configuration and task set separate from the main table, and preserve the discrepancy between its printed averages and row arithmetic. Attention curves and residual-noise plots then suggest a mechanism, but do not randomly intervene on that mechanism. As reader analysis, a stronger claim of adaptive information selection would require the current-state scheduler to beat a replayed schedule with the same denoising budget. Matching these controls would distinguish useful feedback from a good fixed task-level timing pattern. e07e08e09e10e11e12e13e20
5.4 Training and inference
During training
Stage 1 demonstration-finetunes the backbone on official RoboTwin data, independently sampling video-frame times and a separate action time. The paper describes diffusion-noise or flow-velocity targets without fixing a complete implemented loss or timestep-sampling distribution. e04e07
Stage 2 freezes both DiT experts and trains only the GPN. GRPO uses binary episodic success, group-normalized advantages, per-latent new/old policy ratios, clipping, and a decaying entropy bonus. There is no learned value network or reference-policy KL term. e06
During inference
Each step encodes the current video state, samples relative scales, updates future-video times, advances the fixed global action clock, and runs the frozen MoT. Observation time remains zero. Appendix D specifies ten denoising steps. e05e18e19
Generated actions are executed in RoboTwin; imagined frames differ from subsequent observations. Figure 5 contrasts a failed baseline grasp with NoiseGate success, which the authors attribute to uncertainty retained in a critical future frame. The case alone does not establish causality. e14
5.5 Implementation flow
- Couple video and actions through attention
A chunk-bidirectional Wan 2.2-TI2V-5B video DiT and Action-Expert DiT process a joint sequence. They share self-attention while retaining modality-specific feed-forward layers. The current observation stays clean. Future-video tokens participate in action refinement during generation. e03
- Encode the current denoising state
The GPN uses two strided 3-D convolutions, combined attention/average/max pooling, and 256-dimensional latent tokens. Observation-conditioned gated fusion precedes a three-layer, eight-head temporal Transformer. Time embeddings and pooled features feed a squashed-Gaussian actor. e16
- Choose relative denoising speeds
The actor scales each predicted frame’s nominal decrement within (0,2), then subtracts it with a zero floor. No ordering across frames is imposed. Each frame’s time nevertheless cannot increase across denoising steps. e05
6. Experiments & results
NoiseGate learns how quickly each imagined future video latent becomes clean while a joint video–action model generates robot actions. A reward-trained scheduler changes video timesteps while leaving the action clock fixed. The reported 50-task RoboTwin average rises from 92.58% to 94.28%; controlled ablations support scheduling benefits, but their printed baseline averages and the policy-density notation contain unresolved inconsistencies.
6.1 Read the original evidence
Table 5. Read the same-backbone improvement alongside the individual task regressions. Original paper, p. 16 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The two rightmost columns provide the most direct comparison: demonstration-trained Stage-1 WAM and that backbone with the learned NoiseGate scheduler. Entries are percentages from the random-scene protocol, not fractional rates as in the ablation crop. Begin with the average row, then scan Hanging Mug and Open Microwave to see opposite task-level changes. The earlier columns supply context from Fast-WAM variants and other policies. Their provenance matters: the caption says these prior-method values were adapted from a corresponding RoboTwin detail table, whereas Stage-1 is the authors’ evaluation. This full table also restores rows hidden by the ellipsis in main-text Table 1. e07e08e09
What it supports. The reported mean increases from 92.58% to 94.28%, or 1.70 percentage points. Hanging Mug rises from 60% to 69%, but Open Microwave falls from 77% to 69%. Thus the average improvement is compatible with meaningful local regressions; it does not mean the scheduler improves every task.
Where the evidence stops. Each task has 100 evaluation episodes, with no confidence intervals or repeated-seed dispersion supplied. Adapted external baselines are not evidence of identical training conditions. These are simulated execution success rates, not physical deployment or video-quality scores.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| RoboTwin random-scene: 50-task average Scaled Fast-WAM-style batch-1024 configuration; 100 evaluation episodes per task, with randomized poses, colors, and backgrounds. | 94.28% reported Mean task success rate (%) | Stage-1 WAM 92.58% (+1.70 percentage points); Fast-WAM 91.78%, LingBot-VA 91.50%, π0.5 76.76%, Motus 87.02%. The same-backbone Stage-1 comparison best isolates scheduler training. Table 5 adapts prior-method columns from an earlier detail table, limiting fully controlled external comparisons. No confidence intervals are reported. e07e08 |
| Hanging Mug and Open Microwave Scaled random-scene evaluation, 100 episodes per task. | Hanging Mug 69%; Open Microwave 69% Task success rate (%) | Stage-1 WAM: 60% and 77%, respectively; changes are +9 and −8 percentage points. The positive average hides a substantial task regression. Section 5.2 calls regressions small, but the microwave row warrants separate attention. e07e09 |
| Controlled schedule ablation: 15 tasks Motus-derived batch-128 configuration with VLM removed; 100 episodes per task. | NoiseGate 67.5% Reported overall success rate (%) | Printed baselines: Shared-t 57.5%, Stage-1 WAM 61.3%, hand-crafted monotone schedule 63.4%. Printed gains are +10.0, +6.2, and +4.1 percentage points. Shared-t removal changes training and inference. Reader arithmetic over Table 4 gives baseline means 57.07%, 60.93%, and 62.60%, inconsistent with its average row; NoiseGate gives 67.53%, consistent after rounding. e07e10e11 |
| Attention and residual-noise diagnostics Trained NoiseGate probes; controlled configuration; figures show two future frames. | Attention generally decreases with noise in early/late layers; middle layers are comparatively flat. Final noise differs across frames and tasks. Action-to-video attention and final video noise | Layer groups 0–9, 10–19, and 20–29 are shown separately. These are associations, not direct measurements of causal information utility. Figure 7’s success labels differ from the tables, and frame ordering is not universal. e07e12e20 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 4. Separate independent-noise training from the additional benefit of learned scheduling. Original paper, p. 15 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read this table left to right as changes in training substrate and inference schedule. Shared-t lacks both independent per-latent noise training and the GPN. Stage-1 WAM introduces independent-noise training but uses shared-scalar inference. Hand-crafted keeps that substrate and uses a fixed monotone frame schedule. NoiseGate learns its schedule with GRPO. All rows belong to the smaller Motus-derived batch-128 study, with the VLM removed; they are not the batch-1024 main results. Values are success fractions, so 0.94 means 94%. The average row is reproduced faithfully, including its inconsistency with the first three columns’ displayed task values. e07e10e11
What it supports. The printed summary ranks NoiseGate at 67.5%, ahead of hand-crafted scheduling at 63.4%, Stage-1 at 61.3%, and Shared-t at 57.5%. This supports a benefit beyond merely permitting different noise levels. The reported 10.0-point comparison to Shared-t combines training and inference changes; the 4.1-point hand-crafted comparison is more focused on schedule choice.
Where the evidence stops. Reader arithmetic over the 15 rows yields baseline means of 57.07%, 60.93%, and 62.60%, rather than printed 57.5%, 61.3%, and 63.4%. NoiseGate’s 67.53% rounds consistently. The ranking survives, but exact aggregate gain magnitudes require clarification; no alternative weighting is specified.
Figure 6. Noise-associated attention attenuation varies across transformer depth. Original paper, p. 17 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across columns to compare the two predicted frames, and down rows to compare early layers 0–9, middle layers 10–19, and late layers 20–29. Every horizontal axis is video noise, not physical time; every vertical axis measures attention from action tokens to video features. Points give binned means, with shaded ±1-standard-deviation bands. Early-layer attention drops as noise increases. Middle-layer curves are comparatively flat, with small local increases. Late-layer attention trends downward overall but is not perfectly monotone in every panel. This depth-resolved view qualifies the stronger monotone-gating wording accompanying the aggregate Figure 3 on page 8. e07e12
What it supports. The diagnostic is consistent with video noise affecting how strongly action tokens attend to future-frame features, especially in early and late layers. It also shows why an aggregate curve is insufficient: middle layers behave differently. The measured effect concerns attention allocation, while reliable task-relevant information remains the authors’ interpretation.
Where the evidence stops. Shaded standard deviations are not confidence intervals or success-rate uncertainty. Local increases contradict a strictly monotone reading of the captions. Observational attention probes do not isolate noise from latent content or demonstrate that attenuated features caused the success gains.
Figure 7. Learned schedules can finish with different residual noise, but the frame ordering is task-dependent. Original paper, p. 17 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each row represents a task. Compare the pink and yellow dot positions to see which future frame retains more final-step noise; horizontal bars show ±1 standard deviation across the evaluated chunks. Farther right means less completely denoised, not greater prediction accuracy. The SR annotations are the figure’s diagnostic success labels and should not be substituted for the benchmark table values. In several rows, frame 2 retains more noise. However, Place A2B Right and Place Bread Basket put the frame-1 mean farther right. The latter distinction is important when testing claims that the more distant future should always stay more masked. e05e20
What it supports. The scheduler does not force all future latents to the same clean endpoint. The plot supports heterogeneous residual masking, but not a universal rule that frame 2 retains more uncertainty. In particular, Place A2B Right reverses that ordering even though Appendix F.2 cites it while discussing larger farther-future residual noise.
Where the evidence stops. The source leaves diagnostic sampling and table alignment unresolved: Place A2B Right is labeled SR 90% here, versus 78% in Table 4 and 98% in Table 5. Residual noise is a scheduler output, not a validated uncertainty-calibration measure.
7. Analysis & limitations
7.1 What the evidence leaves open
Simulator rollouts dominate scheduler-training cost; sparse successes slow convergence. A general scheduler scaling across large multi-task suites remains future work. Driving, navigation, and video-generation applications are proposed, not tested here. e15
Figure 3’s caption describes monotone attenuation, but its mean curve has local increases; Figure 6 shows depth dependence. Attention weight is an incomplete proxy for feature reliability or causal influence. e12
Appendix C alternates between scales in (0,1) and (0,2). It pairs r=2σ(u) with u=logit(r) in Eq. (11), whereas Algorithm 1 uses r=σ(u) and doubles the nominal decrement. The updates can agree, but the printed scaled-action density is inconsistent. e17
Physical-robot deployment, held-out-task transfer, repeated-seed uncertainty, and latency improvement are not established. Diagnostic success labels and ablation aggregate arithmetic remain unresolved. e07e08e11e15e20
7.2 Questions for discussion
- Does the GPN require the current chunk, or would a learned task-specific fixed schedule suffice?
- How much of the attention trend survives a controlled intervention on one frame’s noise?
8. Reproducibility audit
8.1 Requirements and known gaps
Required ingredients include RoboTwin demonstrations and randomized rollouts, video/action experts, independent-noise Stage-1 training, and the GPN. Table 3 specifies group size 8, four inner epochs, clip 0.2, AdamW at 10^{-4}, gradient clip 1.0, entropy coefficient 0.01 with 0.999 decay, 50 epochs, eight episodes per epoch, eight GPU rollout workers, and seed 42. e03e04e07e19
Missing details include GPU models, wall-clock costs, full Stage-1 settings, precise handcrafted schedule, action representation/horizon, and the 15-task selection criterion. Resolve the scale/log-density convention before implementing policy ratios; prose does not establish actual code behavior. e04e07e10e16e17e19
Proposed checks compare learned schedules with within-task replayed and cross-task swapped schedules, and verify equivalent actor-scale conventions yield identical decrements and policy ratios. No such experiments were performed here. e05e10e13e17e18
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does the current chunk need an adaptive schedule?
Reader-proposed experiment: freeze one independently-noise-trained backbone and compare the learned GPN against a within-task replay of schedules collected from different episodes, a cross-task schedule swap, and shared-scalar inference. Keep the ten denoising calls, action clock, initial noise, and scene seeds matched; evaluate 100 episodes per task across additional seeds. Record task success and frame-timestep trajectories. If replay matches the GPN, current-chunk adaptation is not needed to explain the gain. If cross-task swaps preserve it too, the task-specific gating interpretation weakens. Report per-task changes and arithmetic means, explicitly checking the aggregation inconsistency found in Table 4. e05e07e10e11e13e18e19
Check 2: Can the two actor-scale conventions reproduce the same GRPO update?
Reader-proposed implementation check: use identical Gaussian samples and scheduler states to implement both Algorithm 1’s unit-range scale with a doubled nominal decrement and the main text’s double-range scale with an undoubled decrement. Use the correct change-of-variable density for each convention, preserving pre-clipping actions when video time reaches zero. Compare decrements, updated times, per-latent likelihood ratios, and gradients of one clipped GRPO update on a fixed reward batch. They should agree after accounting for constant density offsets. A difference falsifies convention equivalence in that implementation. Also resolve whether stored summed log-probabilities or individual frame log-probabilities drive the actual ratio; do not silently repair the source formula and call it verified. e05e06e17e18
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version block and Figure 1 on page 1; motivation and notation on pages 2–3; architecture, equations, training, and evaluation on pages 4–6; Tables 1–2 on page 7; Figures 3–4 on page 8; the execution/prediction case on page 9; limitations and GPN architecture on page 13; Algorithm 1 and all Table 3 settings on page 14; Table 4 and Appendix F prose on page 15; the complete Table 5 on page 16; and Figures 6–7 on page 17. All six final original crops were inspected for labels, bounds, and legibility. Reference pages 10–12 were read as text; no scientific claim relies on visually uninspected reference-page content. Separate supplements remain unverified. The graphic/prose, aggregate-arithmetic, and actor-density inconsistencies are explicitly retained in the report and visual cautions.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 13, 14, 15, 16, 17. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1 Introduction
- 2 Related Work
- 3.1 Preliminaries: Video–Action Generation in WAMs
- 3.2 Per-Latent Timesteps as Information Gates
- 3.3 Learning the Per-Frame Schedule as a Policy
- 4.1 World Action Model and MoT Backbone
- 4.2 Training Substrate: Independent Per-Latent Timestep Sampling
- 4.3 Gating Policy Network (GPN)
- 4.4 Training with GRPO
- 5.1 Setup
- 5.2 Main Results
- 5.3 Ablation Studies
- 5.4 Qualitative Analysis: Gating, Schedules, and Noise-as-Masking
- 5.5 Case Study
- 6 Conclusion
- References
- A Limitations
- B Broader applicability
- C GPN Architecture and Inference Details, including Algorithm 1
- D GRPO Hyperparameters
- E Detailed RoboTwin Results
- F.1 Layer-stratified attention
- F.2 Cross-task structure of final-step residual noise
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The complete supplied 17-page text was read. The extraction’s image omission was addressed by direct PDF inspection; all original figures and tables were visually reviewed.
- Identity: the title page shows arXiv:2605.07794v1, 8 May 2026. Title and all eleven authors match the supplied catalog, allowing for name-order formatting. No revision or edition difference was observed; no other version was supplied for comparison.
- No code was inspected and no experiments were reproduced. References were read as bibliography entries, not as independently inspected research sources.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title block, author/affiliation lines, and arXiv margin
Observed title matches the supplied title. Authors are Wen Huang, Haoran Sun, Yongjian Guo, Yunxuan Ma, Haoran Li, Jing Long, Zhouying Mo, Zhong Guan, Yucheng Guo, Shuai Di, and Junwu Xiong. Affiliations are Tsinghua University, Peking University, JDT AI Infra, and Tianjin University. The margin identifies arXiv:2605.07794v1, 8 May 2026.
Go to primary source ↓e02PDF pp. 2–3, Introduction and §3.1–3.2
The paper motivates heterogeneous noise levels as task-dependent information gates, with clean observation and language conditioning joint future-video/action generation.
Go to primary source ↓e03PDF p. 4, Figure 2 and §4.1, Eq. (3); p. 5, opening paragraphs
A Wan 2.2-TI2V-5B video DiT and Action-Expert DiT share self-attention in a joint sequence with modality-specific feed-forward layers. Figure 2 separates demonstration training from scheduler-only GRPO and marks both backbone components frozen in Stage 2.
Go to primary source ↓e04PDF p. 5, §4.2
Stage 1 samples video times independently and a separate action time; it trains with a diffusion noise or flow-matching velocity target. A specific timestep distribution and complete implemented objective are not supplied here.
Go to primary source ↓e05PDF pp. 4–5, §3.3 Eq. (2) and §4.3 Eqs. (4)–(6)
The GPN observes video latents, observation, and current video times, controls only future-video decrements, and leaves a fixed global action schedule. Main-text scales lie in (0,2); positive decrements are subtracted and times floored at zero, without an ordering constraint across frames.
Go to primary source ↓e06PDF pp. 5–6, §4.4, Eqs. (7)–(9)
Only scheduler parameters are optimized. GRPO uses binary episodic success, group-relative standardized advantages, per-latent importance ratios, a clipped surrogate and decaying entropy bonus, without a KL term or learned value network.
Go to primary source ↓e07PDF p. 6, §5.1; p. 14, Appendix E
RoboTwin random-scene evaluation randomizes poses, colors, and backgrounds, using 100 episodes per task. Controlled studies use a Motus-derived, VLM-removed batch-128 configuration; main results use a scaled Fast-WAM-style batch-1024 configuration. Official RoboTwin data train Stage 1.
Go to primary source ↓e08PDF p. 7, Table 1, Average row; p. 16, Table 5, caption and Average row
Reported 50-task means are Stage-1 WAM 92.58 and NoiseGate 94.28 percent. Fast-WAM, LingBot-VA, π0.5, and Motus are 91.78, 91.50, 76.76, and 87.02. Table 5 says prior-method columns are adapted from the corresponding RoboTwin detail table; it reports no uncertainty intervals.
Go to primary source ↓e09PDF p. 16, Table 5, Hanging Mug, Open Microwave, and Place Object Scale rows; p. 7, §5.2 continuation
Stage-1/NoiseGate scores are 60/69 for Hanging Mug, 77/69 for Open Microwave, and 99/98 for Place Object Scale. The main text describes regressions as small.
Go to primary source ↓e10PDF p. 7, Table 2 and §5.3; p. 15, Table 4 caption and Average row
Printed controlled-study means are Shared-t 57.5%, Stage-1 WAM 61.3%, hand-crafted 63.4%, and NoiseGate 67.5%. Table 4 lists 15 tasks; Stage-1 uses independent noise training with shared-scalar inference, while the hand-crafted variant uses a fixed monotone per-latent schedule.
Go to primary source ↓e11PDF p. 15, Table 4, all 15 task rows versus Average row
Reader arithmetic on the displayed fractional rows gives sums 8.56, 9.14, 9.39, and 10.13 for Shared-t, Stage-1, hand-crafted, and NoiseGate. Dividing each by 15 yields 0.570667, 0.609333, 0.626000, and 0.675333. The first three differ from the printed 0.575, 0.613, and 0.634.
Go to primary source ↓e12PDF p. 8, Figure 3 and §5.4; p. 15, §F.1; p. 17, Figure 6 and caption
Action-to-video attention is binned against video noise. Figure 3’s caption calls the decay monotone, although its mean curve has local increases. Figure 6 separates early 0–9, middle 10–19, and late 20–29 layers, with mean ±1 standard deviation. Early/late trends decrease overall; middle layers are comparatively flat.
Go to primary source ↓e13PDF p. 8, Figure 4, caption and §5.4
The two panels, named only Task A and Task B, show single-chunk trajectories for conditioning latent 0 and predicted latents 1 and 2. Observation noise stays zero; future-frame trajectories and ordering differ between panels.
Go to primary source ↓e14PDF p. 9, Figure 5, caption and §5.5
The case study distinguishes actual observations following action execution from predicted frames. The authors attribute a baseline grasp failure and NoiseGate success to uncertainty in the critical predicted grasp frame; the shown case is qualitative.
Go to primary source ↓e15PDF p. 13, Appendices A–B
Sparse simulator rewards make rollout collection the dominant training cost. General large-scale multi-task schedule learning is left to future work; video prediction, driving, and navigation are suggested broader applications.
Go to primary source ↓e16PDF p. 13, Appendix C, Interface and Network architecture
The GPN excludes action time, uses two strided 3-D convolutions with GroupNorm/SiLU, three pooling summaries, D=256 tokens, observation-conditioned gated residual fusion, and a three-layer eight-head temporal Transformer. Time embeddings, attention pooling, and a three-layer actor MLP parameterize a squashed Gaussian.
Go to primary source ↓e17PDF p. 5, §4.3 Eq. (5); p. 13, Appendix C interface and Eq. (11); p. 14, Algorithm 1 lines 2–3 and 7
The main text uses r in (0,2) with Δt=δt r. Appendix C first states (0,1), then defines r=2σ(u) but writes u=logit(r) and an unscaled sigmoid log-Jacobian. Algorithm 1 instead uses r=σ(u), Δt=2δt r. These notational and density differences are present in the PDF.
Go to primary source ↓e18PDF p. 14, Algorithm 1 and immediately following paragraph
Each step encodes the current state, samples scales, updates video times with a zero floor, advances the fixed action clock, and invokes the frozen MoT. It stores the sum of per-latent log-probabilities for GRPO; both DiT backbones remain frozen.
Go to primary source ↓e19PDF p. 14, Appendix D, Table 3
GRPO settings: group 8; clip 0.2; inner epochs 4; AdamW; learning rate 10^{-4}; gradient clip 1.0; initial entropy coefficient 0.01; entropy decay 0.999 per step, floored at 0; ten denoising steps; 50 epochs; eight episodes per epoch; eight GPU rollout workers; seed 42. GPU models and wall-clock costs are not provided.
Go to primary source ↓e20PDF p. 15, §F.2 and Table 4; p. 17, Figure 7 and caption; p. 16, Table 5
Figure 7 gives per-task final-noise means and ±1 standard deviation for two future frames. Place A2B Right has frame 1 farther right than frame 2, despite being a prose example of retaining more farther-future noise. Its SR 90% label differs from Table 4’s 78% and Table 5’s 98%; Blocks Ranking RGB is labeled 100%, versus 82% and 99% respectively. The diagnostic evaluation subset is not reconciled with the tables.
Go to primary source ↓8.5 Primary sources
NoiseGate: Learning Per-Latent Timestep Schedules as Information Gating in World Action Models ↗
PDF · 8,559 extracted words
Source fingerprint
3038cd278120ca8e7978eadf5c0784fe1aed9c463f5b1a2127da2bb59bf986ea