One-step Diffusion with Distribution Matching Distillation
1. Paper overview
In one sentence: DMD trades expensive distillation training for one-pass image generation by combining a real-versus-generated score gradient with teacher-pair regression. e-probleme-architecturee-scorese-regressione-ablatione-coco-threee-coco-eight
| At a glance | What to know |
|---|---|
| Research problem | Source description Diffusion sampling is slow because it repeatedly evaluates a denoiser. Directly fitting the teacher’s entire noise-to-image mapping is expensive and difficult. DMD relaxes exact correspondence into distribution matching while retaining limited paired supervision to stabilize the student. e-problem |
| Core mechanism | |
| A key reported result | Zero-shot COCO-30k text-to-image generation at guidance 3: DMD: 11.49 FID; 0.09 s per image. FID (lower better); latency. Table 3; 512×512 SD v1.5 distillation on LAION-Aesthetics-6.25+; latency batch size 1. LCM-LoRA uses guidance 7.5. SD v1.5 teacher: 8.78, 2.59 s; InstaFlow-0.9B: 13.10, 0.09 s; UFOGen: 12.78, 0.09 s. The table implies about 28.8× lower latency than the teacher, with a 2.71 FID gap. These reader calculations preserve the table’s protocol; the heterogeneous baseline list is not a uniform retraining comparison. e-text-traininge-coco-three |
| Reading caution | Author claim The authors acknowledge remaining quality differences from finely sampled diffusion and substantial training memory from updating both generator and fake-score model. LoRA is suggested as a possible remedy, not evaluated here. e-limitations |
Core contributions
- Source description
LPIPS regression on precomputed teacher outputs complements the distribution objective; ablations support both components and the proposed gradient weighting. e-regressione-ablation
Figure 2. Two score estimators and an offline teacher-pair stream train the one-pass generator. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the random latent on the left and follow the black path through the generator to the fake image. The green branch compares that output with a precomputed teacher image for the same noise. The dashed diffusion arrow adds noise before the image reaches both score estimators. The lock marks the fixed real-data model; the blue fake-data model learns through its separate denoising loss. The red loop supplies the generator’s distribution gradient. Read its subtraction using Equation (7): the gradient contains fake minus real scores, and descent reverses that direction. Algorithm 1 specifies detached generator outputs when updating the fake model. e-architecturee-scorese-regressione-algorithme-ablation
What it supports. The training diagram contains three network roles, while the deployed generator makes its image in one pass. Regression supplies correspondence to offline teacher examples; the score difference supplies distribution-level feedback on generated samples. These two learning signals are complementary in the reported ablations.
Where the evidence stops. The denoised pictures visualize score information, not a second image-generation stage at inference. The red feedback arrow represents training. The drawing alone does not specify subtraction order or gradient detachment; Equations (4)–(7) and Algorithm 1 resolve those details.
2. Motivation
2.1 The problem and the proposed response
Diffusion sampling is slow because it repeatedly evaluates a denoiser. Directly fitting the teacher’s entire noise-to-image mapping is expensive and difficult. DMD relaxes exact correspondence into distribution matching while retaining limited paired supervision to stabilize the student. e-problem
2.2 What this reading follows
A diffusion teacher can draw a convincing image, but ordinarily needs repeated denoising evaluations to do so. DMD asks a student to learn the resulting distribution in one pass. Its training system contains more machinery than its inference path: a frozen teacher, a moving estimate of the student’s own image distribution, and a bank of teacher-generated examples. The following figures explain why those components have different jobs, then connect them to the loss ablations and two distinct text-to-image settings. The evidence supports fast image synthesis with a remaining teacher-quality gap; it does not establish action prediction or interactive control. e-probleme-architecturee-scorese-regressione-ablatione-coco-threee-coco-eight
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Foundational work |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The foundational distillation classification is supported. DMD learns image generation, with auxiliary score models used during training. It has no world-state rollout, action decoder, inverse dynamics or executed-control evaluation. The WAM architecture, prediction-paradigm and quadrant labels are therefore not applicable; one-pass inference is not evidence for a One Model world-action architecture. e-architecturee-algorithme-imagenete-coco-three
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Make the student distribution visible to its own teacher signal
Start with the generator’s current output rather than a full teacher trajectory. DMD adds noise to that output and asks two denoisers about the same perturbed sample. The frozen base model describes the target distribution, while the fake denoiser learns the distribution the student currently produces. Their difference becomes useful because the Gaussian perturbation gives the distributions overlapping support. Equations (4) and (5) express scores through mean predictions; Equation (7) then combines those scores with the generator derivative. This distinction prevents a sign error: fake minus real appears in the gradient, and the optimizer subtracts the gradient. The fake estimator is continually updated on detached generated images, so its denoising objective trains the estimator rather than directly moving the generator. e-scorese-algorithm
Figure 3. The mode illustration explains why a score difference still needs a regression anchor. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the panels from left to right, keeping the two blue target modes in view. The initial red samples are near one target mode. Panel (a) uses the real score alone and concentrates samples near that mode. Panel (b) adds the fake score, spreading samples locally but leaving the other target mode uncovered. Panel (c) also includes regression and places generated samples around both target regions. The figure is a schematic explanation of different objectives from the same initial configuration. It should be connected to the image experiments through Table 2 and Figure 5 rather than treated as an image benchmark itself. e-modee-ablation
What it supports. Repelling samples from their current concentration can improve local spread without recovering missing modes. The paper uses this example to motivate teacher-pair regression alongside distribution matching. Its image ablation then reports worse FID and visually repeated outputs when regression is removed.
Where the evidence stops. This illustration provides intuition, not a guarantee that regression preserves every mode of a high-dimensional image distribution. It contains no numerical coverage metric or uncertainty estimate.
5.2 Give distribution matching a memory of missing modes
The generator is not required to reproduce every teacher noise-to-image correspondence exactly. Yet discarding all correspondence introduces a different difficulty: generated samples can spread around an already occupied mode while leaving another mode unrepresented. Figure 3 illustrates this failure. DMD therefore keeps an offline collection of deterministic teacher outputs and occasionally uses the associated noise as input to the student. LPIPS penalizes disagreement between the resulting student image and its paired target. This provides a structural anchor alongside the score gradient, with a default coefficient of 0.25. Table 2 supports the combination empirically: removing either term worsens FID on both datasets. Reader interpretation: the pairing constrains where some noise samples should go, while the score term improves the distribution between those anchors; neither observation proves complete mode recovery. e-modee-regressione-ablation
5.3 Separate training expense, inference work and evaluation settings
The score estimators and teacher-pair generation are training machinery. Once distilled, the generator produces an image without running that alternating optimization or traversing the teacher’s denoising trajectory. This explains why one-pass inference can coexist with substantial training cost: the guidance-3 text model reportedly required about 36 hours on 72 A100 GPUs. Evaluate the resulting tradeoff using the correct columns. ImageNet supplies forward-pass counts, whereas the COCO tables supply batch-one latency. Also keep the two text students separate: guidance 3 uses LAION-Aesthetics-6.25+, and guidance 8 uses LAION-Aesthetics-6+. Reader interpretation: their quality differences cannot isolate the effect of guidance. The evidence establishes faster image synthesis under reported protocols, without measuring online action selection, environmental feedback or executed-control success. e-architecturee-algorithme-text-traininge-imagenete-coco-threee-coco-eight
5.4 Training and inference
During training
Offline pairs use deterministic teacher sampling: Heun with 18 steps for CIFAR-10 and 256 for ImageNet, and PNDM with 50 steps for LAION. The default regression coefficient is 0.25. The distribution-gradient time interval is [0.02T, 0.98T], with T = 1000. e-architecturee-scorese-regression
For classifier-free guidance, generate guided regression targets and replace only the real score with the guided score; leave the fake-score formulation unchanged. Each generator is trained at a fixed guidance scale. The SD v1.5 guidance-3 model uses LAION-Aesthetics-6.25+ and takes about 36 hours on 72 A100 GPUs. A separate guidance-8 model uses LAION-Aesthetics-6+. e-guidancee-text-training
During inference
A noise sample and conditioning produce an image through the distilled generator without iterative denoising or the two training score estimators. Fixed-scale training does not establish an adjustable guidance control for the same student. e-architecturee-guidance
The authors additionally report 20 FPS with FP16 inference. This is a separate configuration from the tables’ 0.09-second latency; the supplied main paper does not identify the inference GPU or complete timing configuration. e-coco-three
5.5 Implementation flow
- Initialize the three roles
Initialize generator G_theta and fake denoiser from the pretrained base model. The generator uses the base denoiser architecture without time conditioning, initialized to its terminal-time output. Keep a separate base-model copy fixed as the real-score estimator. e-architecturee-scores
- Compare scores on perturbed images
Generate x, then add Gaussian noise at a sampled diffusion time. Both denoisers process the same noisy image. The fixed teacher estimates the real-distribution score; the learned fake denoiser tracks the changing generator distribution. Diffusion makes the distributions overlap, allowing a usable approximate score difference. e-scores
- Anchor outputs and alternate updates
Update the generator using distribution matching plus LPIPS regression. Then update only the fake denoiser with a denoising objective on detached generated images. Algorithm 1 includes regression-stream images in the distribution-matching batch for LAION; its stop-gradient prevents the fake-denoiser update from training the generator. e-regressione-algorithm
6. Experiments & results
Distribution Matching Distillation (DMD) converts a pretrained diffusion denoiser into a one-pass image generator. A frozen teacher and a continually updated model of generated images supply a score-difference gradient; an offline teacher-pair regression loss anchors structure and diversity. The reported image-quality gains over earlier one-step methods retain a gap to the teacher and require substantial distillation training.
6.1 Read the original evidence
Table 1. DMD narrows the ImageNet quality gap while using one generator evaluation. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First separate the three row groups: established generators, diffusion-distillation alternatives, and the EDM teacher at the bottom. Both measured columns favor smaller values. Compare DMD with the one-pass rows to assess the distillation improvement, then compare it with its teacher to see the remaining quality cost. The Consistency Model row gives a useful same-pass-count reference. The table’s caption attributes baseline numbers to Song et al.; this is not a claim that this reading reran those systems. The forward-pass column is a computational count, so it cannot by itself supply a hardware-specific latency ratio. e-imagenet
What it supports. DMD reports FID 2.62 versus 6.20 for the one-pass Consistency Model. The EDM teacher remains better at 2.32 with 512 forward passes. Thus the table supports improved one-step distillation and a 0.30 FID teacher gap, rather than overall dominance across every listed generator.
Where the evidence stops. Section 4.1 describes a 512-fold speed increase, but this table directly documents a 512-to-1 forward-pass reduction. It supplies neither corresponding ImageNet wall-clock timings nor repeat-run uncertainty.
Table 3. The guidance-3 student improves the listed diffusion accelerators while retaining a teacher-quality gap. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the resolution column before comparing quality or latency, because the table combines several model families and image sizes. For the closest speed comparison, inspect the 512-resolution accelerated-diffusion block and the SD v1.5 teacher beneath it. DMD, InstaFlow and UFOGen share the reported 0.09-second latency, making their FID differences easy to locate. The attached protocol notes matter: batch size is one, daggers identify results reevaluated by the paper’s authors, and LCM-LoRA uses guidance 7.5 rather than 3. The DMD model for this table was trained on LAION-Aesthetics-6.25+, separately from the high-guidance model. e-text-traininge-coco-three
What it supports. DMD reports 11.49 FID, improving on InstaFlow’s 13.10 and UFOGen’s 12.78 at the same listed latency. The teacher reports 8.78 FID at 2.59 seconds. Dividing the tabulated times gives about 28.8× lower latency, while subtracting the FIDs gives a remaining 2.71 gap.
Where the evidence stops. The table mixes published and reevaluated baselines and includes a guidance mismatch for LCM-LoRA. Its 0.09-second measurement must not be equated with the separate FP16 claim of 20 FPS; inference hardware details are absent from the supplied main paper.
Table 4. At higher guidance, DMD retains strong text alignment with a residual FID gap. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read FID and CLIP-Score together: smaller FID and larger CLIP-Score are preferred, whereas latency favors smaller values. The multiple LCM-LoRA rows show different sampling step counts for that baseline. Compare DMD’s row with both the four-step alternatives and the teacher, keeping the footnote’s guidance exception visible. DMD and the other non-LCM guided comparisons use scale 8; LCM-LoRA uses 7.5. This table comes from a separate student trained on LAION-Aesthetics-6+, not from changing a runtime slider on the guidance-3 student. Consequently, differences from Table 3 mix guidance and training-data changes. e-text-traininge-coco-eighte-coco-three
What it supports. DMD reports FID 14.93 and CLIP-Score 0.320 at 0.09 seconds, approaching the teacher’s 13.45 and 0.322 at 2.59 seconds. Four-step LCM-LoRA reports 23.62 and 0.297 at 0.19 seconds. These results support a favorable speed–quality comparison within the reported settings.
Where the evidence stops. CLIP-Score measures aggregate text–image alignment, not comprehensive instruction following or physical correctness. The changed training subset prevents treating the two COCO tables as a controlled guidance-only ablation, and no uncertainty estimates accompany their differences.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Class-conditional ImageNet-64×64 generation Table 1; EDM-based distillation. Baseline numbers are attributed to Song et al.; complete evaluation details are deferred to absent appendices. | DMD: 2.62 FID, 1 pass. FID (lower better); forward passes | Consistency Model: 6.20, 1 pass; EDM teacher: 2.32, 512 passes. DMD substantially improves the listed one-step baselines but remains 0.30 FID above its teacher. The 512-to-1 comparison counts forward passes, not measured wall-clock speed. e-imagenet |
| Loss and weighting ablations on CIFAR-10 and ImageNet-64×64 Table 2; class-conditional generation. Left: remove one loss. Right: change distribution-gradient weighting on CIFAR-10. | Full DMD: 2.66 CIFAR / 2.62 ImageNet. FID (lower better) | Without distribution matching: 3.82 / 9.21; without regression: 5.58 / 5.61. CIFAR weighting alternatives sigma_t/alpha_t and sigma_t^3/alpha_t give 3.60 and 3.71, versus 2.66. Both losses contribute. Figure 5 illustrates structural degradation and repeated cars, but FID alone does not isolate mode coverage; repeat-run uncertainty is not provided. e-ablation |
| Zero-shot COCO-30k text-to-image generation at guidance 3 Table 3; 512×512 SD v1.5 distillation on LAION-Aesthetics-6.25+; latency batch size 1. LCM-LoRA uses guidance 7.5. | DMD: 11.49 FID; 0.09 s per image. FID (lower better); latency | SD v1.5 teacher: 8.78, 2.59 s; InstaFlow-0.9B: 13.10, 0.09 s; UFOGen: 12.78, 0.09 s. The table implies about 28.8× lower latency than the teacher, with a 2.71 FID gap. These reader calculations preserve the table’s protocol; the heterogeneous baseline list is not a uniform retraining comparison. e-text-traininge-coco-three |
| Zero-shot COCO-30k text-to-image generation at guidance 8 Table 4; separate SD v1.5 student trained on LAION-Aesthetics-6+; latency batch size 1. LCM-LoRA uses guidance 7.5. | DMD: 14.93 FID, 0.320 CLIP-Score, 0.09 s. FID (lower better); CLIP-Score (higher better); latency | Teacher: 13.45, 0.322, 2.59 s; four-step LCM-LoRA: 23.62, 0.297, 0.19 s. DMD approaches the teacher’s text alignment and exceeds the listed accelerated alternatives, while retaining a quality gap. Comparing Tables 3 and 4 does not isolate guidance because the training subset also changes. e-text-traininge-coco-eight |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2. Loss removal and weighting changes expose separate contributions to the reported FID. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The left panel changes the training objective, while the right panel changes the sample weighting. All entries are FID, with lower values preferred. Read each dataset column vertically: removing distribution matching retains regression, and removing regression retains distribution matching. The bold DMD row uses both. In the right panel, the two alternatives use the diffusion noise and signal coefficients; Equation (8) instead includes the teacher’s denoising error to normalize gradient magnitude. The repeated CIFAR value of 2.66 is the full-method reference in both panels. The right panel does not report an ImageNet weighting comparison. e-ablatione-scores
What it supports. The full method obtains 2.66 on CIFAR and 2.62 on ImageNet, compared with 3.82/9.21 without distribution matching and 5.58/5.61 without regression. On CIFAR, the proposed weighting gives 2.66 against 3.60 and 3.71. Each tested removal or substitution worsens the reported FID.
Where the evidence stops. FID cannot distinguish all causes of degradation. Figure 5 supports the authors’ structural-quality and mode-collapse interpretations qualitatively, but the supplied table provides no independent diversity statistic, repeated-run spread or weighting sweep on ImageNet.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors acknowledge remaining quality differences from finely sampled diffusion and substantial training memory from updating both generator and fake-score model. LoRA is suggested as a possible remedy, not evaluated here. e-limitations
The toy mode example motivates regression but does not prove preservation of every image mode. Aggregate FID, selected image grids and absent repeat-run uncertainty limit conclusions about diversity and robustness. e-modee-ablatione-coco-threee-coco-eight
7.2 Questions for discussion
- Does regression improve semantic mode coverage beyond what aggregate FID reveals?
- How sensitive is one-step quality to lag in the continually trained fake-score estimator?
- Can one student support multiple guidance scales without retraining?
8. Reproducibility audit
8.1 Requirements and known gaps
A faithful implementation needs the pretrained teacher, deterministic pair generation, LPIPS, noise schedule, two trainable networks, frozen real score and Algorithm 1’s detached fake-score targets. The supplied main paper defers optimizer, batch-size, schedule and fuller evaluation specifications to appendices that are absent. e-architecturee-scorese-regressione-algorithme-scope
Proposed checks: repeat the loss ablation while measuring diversity independently of FID, and compare dynamic versus frozen fake-score estimation under matched compute. These would test the roles suggested by the mode illustration and alternating update rule. e-modee-algorithme-ablation
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test whether regression restores diversity beyond lowering FID
Reader-proposed experiment, not performed: repeat the CIFAR-10 comparison with full DMD, no regression, and no distribution matching, using the same teacher, offline pairs, generator initialization, training budget and evaluation noise. Use several independent training seeds and report both FID and a prespecified within-class diversity measure, such as duplicate-neighbor frequency in a fixed feature space. Retain matched-seed image grids to inspect structural failures. The paper’s mechanism predicts that removing regression increases concentration or repetition, while removing distribution matching harms realism. If full DMD improves FID without improving the independent diversity measure, the stronger mode-coverage interpretation would remain unsupported. Unavailable training settings must be declared as reproduction choices. e-regressione-algorithme-modee-ablatione-scope
Check 2: Test whether tracking the moving fake distribution matters
Reader-proposed experiment, not performed: after a shared CIFAR-10 warm-up with both networks updating, clone the run into branches that update the fake denoiser every generator step, every fourth step, or never again. Preserve generator update counts, paired examples, random inputs and evaluation protocol. Match compute by calculating but discarding fake-denoiser updates in delayed or frozen branches. Monitor held-out generated-image denoising error, gradient magnitudes by diffusion time, FID and the same diversity diagnostic. Freezing after warm-up avoids the trivial initial equality of the two pretrained score estimators. If stale estimation consistently degrades quality, that supports the tracking mechanism; unchanged performance would qualify its importance at the tested scale. e-scorese-algorithme-ablatione-scope
8.3 Reading coverage
Visual audit: All 11 supplied PDF pages were rendered and visually inspected, including the title/author block and version watermark, Figures 1–6, Algorithm 1, Equations (1)–(9), Tables 1–4, limitations and references. Each of the six final original crops was viewed; the Table 2 crop was tightened to remove a fragment of surrounding caption text and viewed again. Table 3 and Table 4 retain their attached protocol notes because these define guidance exceptions, baseline provenance and timing batch size. Figure 2’s locks, feedback direction and subtraction were checked against Equations (4)–(7) and Algorithm 1; the equation determines gradient sign and the algorithm determines stop-gradient behavior. All pages supporting the reported training hardware, methods, numbers and proposed checks are included. Referenced appendices and separate supplementary material are absent from the supplied PDF and were not inspected. No external source or code was inspected and no experiment was run.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract (PDF p. 1)
- 1. Introduction (PDF pp. 1–2)
- 2. Related Work (PDF pp. 2–3)
- 3. Distribution Matching Distillation (PDF pp. 3–6), including Sections 3.1–3.4 and Equations (1)–(9)
- Algorithm 1 (PDF p. 6)
- 4. Experiments, including Sections 4.1–4.3 (PDF pp. 6–7)
- 5. Limitations (PDF pp. 7–8)
- Acknowledgements (PDF p. 8)
- References (PDF pp. 9–11)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The text-extraction image omission was addressed by visually inspecting all 11 original PDF pages, Figures 1–6, Algorithm 1 and Tables 1–4.
- The supplied PDF contains the complete main paper and references, printed pp. 6613–6623, but no appendices. Referenced Appendix B training details, Appendix F derivations, Appendix G full prompts, Appendix H parameterization conversion, and additional appendix evaluation material were not supplied or read.
- Identity/version: the observed title and all seven authors match the catalog. The title-page watermark identifies the CVPR open-access version and states that it is identical to the accepted version except for the watermark; it distinguishes the final IEEE proceedings version, which was not inspected. No arXiv revision identifier is established.
- Code, model weights, datasets and linked resources were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1 (printed p. 6613), title/author block and CVF watermark
The title and seven authors match the catalog. Affiliations are Massachusetts Institute of Technology and Adobe Research. The watermark identifies the open-access accepted version and distinguishes the final proceedings version.
Go to primary source ↓e-problemPDF pp. 1–3 (printed pp. 6613–6615), Abstract, Sections 1–2 and Section 3 opening
DMD targets diffusion sampling cost by training a one-step generator with distribution matching and regression. The related-work discussion distinguishes training a generator from VSD optimization of individual 3D objects.
Go to primary source ↓e-architecturePDF p. 3, Figure 2 and caption; PDF p. 4, Section 3.1
The diagram separates generator, frozen real-score model, learned fake-score model and offline pairs. Section 3.1 specifies pretrained initialization, no generator time conditioning, T = 1000, and EDM/Stable Diffusion base models.
Go to primary source ↓e-scoresPDF p. 4, Section 3.2, Equations (1)–(4); PDF p. 5, Figure 4 and Equations (5)–(8)
The KL gradient is approximated by weighted fake-minus-real scores after Gaussian perturbation. The real denoiser is fixed; the fake denoiser tracks generated samples. Equation (8) defines error-normalized weighting, and the time interval is 0.02T to 0.98T.
Go to primary source ↓e-modePDF p. 4, Figure 3 and caption; PDF p. 5, Section 3.3 opening
The illustration shows nearest-mode collapse with real score alone, local spreading with both scores, and both target modes covered after adding regression. Section 3.3 motivates regression through unreliable low-noise scores and mode dropping.
Go to primary source ↓e-regressionPDF p. 5, Section 3.3, Equation (9) and Final objective
Offline deterministic pairs use Heun with 18 CIFAR or 256 ImageNet steps and PNDM with 50 LAION steps. LPIPS regression is weighted by 0.25 by default; the fake denoiser is trained separately with its denoising objective.
Go to primary source ↓e-algorithmPDF p. 6, Algorithm 1, lines 2–19
Initialize generator and fake estimator from the teacher, update the generator with both losses, then train the fake estimator on detached generated images. LAION concatenates unpaired and regression-stream generated images for distribution matching.
Go to primary source ↓e-guidancePDF pp. 5–6, Section 3.4
Guided teacher sampling constructs regression pairs. Guidance replaces the real score, leaves the fake-score formulation unchanged, and is fixed when training each generator.
Go to primary source ↓e-text-trainingPDF p. 6, Section 4; PDF p. 7, Section 4.3 and High guidance-scale diffusion distillation
Guidance 3 distills SD v1.5 on LAION-Aesthetics-6.25+, with about 36 hours on 72 A100 GPUs. A separate guidance-8 model uses LAION-Aesthetics-6+.
Go to primary source ↓e-imagenetPDF p. 6, Section 4.1 and Table 1, DMD / Consistency Model / EDM teacher rows
ImageNet-64×64 FID and forward passes are 2.62/1 for DMD, 6.20/1 for Consistency Model and 2.32/512 for the EDM teacher. The caption attributes baseline numbers to Song et al.
Go to primary source ↓e-ablationPDF p. 6, Section 4.2; PDF p. 7, Table 2, both panels, and Figure 5 with captions
Table 2 reports full and loss-removed CIFAR/ImageNet FIDs and compares three CIFAR weightings. Figure 5 shows structural degradation without distribution matching and repeated grey cars without regression, using matched seeds for each comparison.
Go to primary source ↓e-coco-threePDF p. 7, Section 4.3, Table 3 and its protocol notes
COCO-30k reports DMD FID 11.49 at 0.09 s, teacher 8.78 at 2.59 s, InstaFlow 13.10 and UFOGen 12.78 at 0.09 s. Batch size is 1; LCM-LoRA guidance is 7.5 versus 3 for other guided comparisons. The prose separately reports 20 FPS with FP16.
Go to primary source ↓e-coco-eightPDF p. 8, Table 4 and its protocol notes; Figure 6 and caption
At guidance 8, DMD has FID 14.93, CLIP-Score 0.320 and latency 0.09 s; the teacher has 13.45, 0.322 and 2.59 s. Four-step LCM-LoRA has 23.62, 0.297 and 0.19 s, using guidance 7.5. Timing batch size is 1.
Go to primary source ↓e-limitationsPDF pp. 7–8, Section 5
The authors state a remaining quality gap and significant training-memory use from updating both fake score and generator. They suggest LoRA as a possible solution.
Go to primary source ↓e-scopePDF p. 1, Figure 1 caption; PDF p. 4, Section 3.1; PDF p. 5, Sections 3.2–3.3; PDF p. 6, Sections 4–4.1; PDF pp. 9–11, References
The main paper refers to Appendices B, F, G and H and additional evaluation material. The supplied 11-page PDF ends with references, without those appendices.
Go to primary source ↓8.5 Primary sources
One-step Diffusion with Distribution Matching Distillation ↗
PDF · 8,358 extracted words
Source fingerprint
26dfa2d56ffcbc058a4f36142530d0dbd07e0fabc99ff22b0800bba73295148d