Consistency Models
1. Paper overview
In one sentence: Learning a common endpoint for every point on a diffusion trajectory enables one-step image generation, while optional repeated denoising improves quality and enables editing. objectiveconsistencycdctcifarlarge-resultsediting-algorithmediting-tasks
| At a glance | What to know |
|---|---|
| Research problem | Source description How can a generative model retain diffusion-style refinement and inverse-problem editing while avoiding a long denoising chain? The paper replaces repeated numerical integration with a learned trajectory-to-endpoint map, evaluated once when speed is the priority. objectiveconsistencysampling |
| Core mechanism | Source description A consistency function shares one endpoint across all times on a probability-flow trajectory; an identity boundary condition anchors this mapping to data. consistency |
| A key reported result | CIFAR-10 few-step consistency distillation: CD: FID 3.55 / 2.93; IS 9.48 / 9.75 at one/two evaluations. FID ↓; Inception Score ↑. One/two network evaluations; CD and PD distill the same in-house EDM. Tables use LPIPS for CD and squared ℓ2 for PD. PD: FID 8.34 / 5.58; IS 8.69 / 9.05. EDM: FID 2.04 at 35 evaluations. CD improves on tabulated PD but remains worse than the slower teacher. This table mixes metric and algorithm effects; reference split and FID sample count are unstated. cifarcomparisonevaluation-scope |
| Reading caution |
Core contributions
- Source description
A consistency function shares one endpoint across all times on a probability-flow trajectory; an identity boundary condition anchors this mapping to data. consistency
- Source description
Few-step distillation and qualitative editing experiments demonstrate useful capabilities, with substantial dependence on the discrepancy metric, solver and training schedules. ablationcifarlarge-resultsediting
Figure 2. One learned map returns different noise levels on a trajectory to the same image endpoint. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the green arrows from the data distribution on the left toward noise on the right. Then follow the red arrows in the opposite direction: the marked intermediate states and final noisy state all return to the same point. These red arrows represent separate evaluations of the same time-conditioned function, not a sequence of solver updates. Section 3 makes the shared-output requirement explicit and implements its boundary using a skip connection. The drawing labels the origin x₀ at time zero, whereas the formal model stops at x_ε for numerical stability; the report follows that formal definition. consistencypf-odesamplingarchitecture
What it supports. The shortcut is learned across noise levels, so a trained model can accept terminal noise and output an image in one evaluation. The same time-conditioned interface remains usable at intermediate noise levels, which permits repeated denoising and editing. This schematic explains the design; sample quality is measured in the later tables.
Where the evidence stops. The image is a conceptual mechanism diagram, not a layer-by-layer architecture or proof of exact recovery. The x₀ versus x_ε notation is a schematic simplification. Section 3 does not enforce invertibility of the learned network.
2. Motivation
2.1 The problem and the proposed response
How can a generative model retain diffusion-style refinement and inverse-problem editing while avoiding a long denoising chain? The paper replaces repeated numerical integration with a learned trajectory-to-endpoint map, evaluated once when speed is the priority. objectiveconsistencysampling
2.2 What this reading follows
A diffusion sampler normally moves through many intermediate states before producing an image. Consistency Models asks the network to predict the endpoint directly from any noise level. The crucial training signal is agreement between predictions at neighboring times, anchored by an identity condition at the low-noise boundary. This reading separates distillation from a pretrained diffusion model (CD) from direct consistency training (CT), because their data dependencies and reported image quality differ. Five original visuals explain the endpoint map, the training choices, the benchmark comparisons and the editing procedure. The final checks focus on comparison fairness and an unresolved stroke-conditioning inconsistency in the printed algorithm. objectiveconsistencycdctcifarlarge-resultsediting-algorithmediting-tasks
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Foundational work |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The foundational diffusion/distillation classification is supported. This image endpoint map has no future-observation/action output architecture, inverse-dynamics action extraction or environmental feedback. Online/target networks implement training, not a world/action decomposition. The recorded architecture, prediction paradigm and quadrant remain not applicable; architectural flexibility alone cannot establish a One Model WAM. consistencycdctediting
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Anchor agreement to an actual image endpoint
Begin with an image diffused into progressively noisier states. A conventional score-based sampler follows the probability-flow ODE backward through many network evaluations. The consistency function instead answers the same question at every time: which low-noise endpoint belongs to this trajectory? Merely forcing different inputs to agree would allow an uninformative constant output. The boundary condition removes that solution by requiring the model to return its input at ε. Equation (5) implements the condition through skip and output coefficients, while retaining an image-shaped backbone. The learned function therefore has both a consistency requirement across time and a fixed anchor near the data distribution. After training, evaluating it on noise at T produces a sample without invoking the teacher's ODE solver. pf-odeconsistencycdsampling
5.2 Separate the two ways of building an agreement pair
In CD, a real image is first perturbed to a later noise time. A fixed pretrained score model and a numerical solver estimate a neighboring earlier state. The online consistency network processes the later state; its stop-gradient target processes the earlier estimate. CT removes the score-model call and constructs the pair from one image and one shared noise vector at two scales. Reader interpretation: the shared noise is essential to this local comparison, but it does not make every finite-step CT pair an exact probability-flow trajectory. The source's justification relates objectives asymptotically under ideal-score and smoothness assumptions. Keep that distinction when interpreting the results: CD's best FIDs depend on diffusion pretraining, while the main discrete CT results demonstrate learning without that teacher. cdctcifartraining-details
5.3 Use repeated denoising to impose observations
One-step generation and editing use the same endpoint predictor but different sampling procedures. Algorithm 1 adds fresh noise to a generated image before another denoising evaluation, allowing refinement at a lower noise level. Algorithm 4 additionally transforms an image, restores its observed components, and keeps generated components where the mask is one. This gives a concrete route to inpainting and to other constraints that become masking after a linear transformation. The source uses many steps for default bedroom editing, so zero-shot describes the lack of editing-specific training rather than a one-step runtime. A reproduction caveat appears for strokes: the stated all-ones mask makes the algorithm's initial reference-masking assignment erase the input. The demonstrated stroke images cannot resolve that printed implementation conflict. samplingeditingediting-algorithmediting-tasks
5.4 Training and inference
During training
Optimize online parameters while stopping target gradients. Target averaging uses coefficient μ, distinct from the EMA for final model weights. CD starts from EDM weights; main discrete CT starts randomly. LPIPS is the default discrepancy metric and loss weighting is one. cdcttraining-configtraining-details
CD uses a fixed time grid and Heun solver. CT increases discretization resolution and adjusts target averaging. Figure 3 supports the combined adaptive schedule, but jointly changing N and μ does not isolate their individual effects. ablationtraining-configtraining-details
During inference
One-step generation applies the consistency model to Gaussian noise at T. Algorithm 1 optionally injects fresh noise and denoises again at decreasing times. Greedy ternary search selects sampling times by FID, assuming unimodality. The teacher is not used for sampling. consistencysampling
Editing combines denoising with replacement of observed components in a transformed image space. Appendix D uses 40 steps by default for bedroom editing but specifies two for strokes; these are task-specific settings, not universal one-step editing. editing-algorithmediting-tasks
5.5 Implementation flow
- Define the noisy image family
Gaussian perturbations have variance t². The empirical probability-flow ODE uses the score model to define trajectories. Pixels are scaled to [−1, 1], with maximum time T = 80 and minimum time ε = 0.002. pf-ode
- Anchor the endpoint map
The network predicts an endpoint rather than a score. Skip/output coefficients enforce identity at ε. Experiments reuse NCSN++ on CIFAR-10 and Dhariwal–Nichol architectures on larger datasets, with modified EDM skip connections. consistencyarchitecture
- Construct training pairs
CD perturbs a real image at the later time and takes one teacher-driven solver step toward the earlier time. CT instead adds the same Gaussian vector to the same image at neighboring noise scales. cdct
6. Experiments & results
Consistency Models maps a noisy image at any diffusion time directly to the low-noise endpoint of its probability-flow trajectory. This supports one-network-evaluation generation and optional refinement. Consistency distillation (CD) learns from a pretrained diffusion teacher; consistency training (CT) learns from paired perturbations of real data. CD has the better reported one-step FIDs of these two training routes, while CT removes diffusion-teacher dependence. The experiments concern images, with no executed actions or world-model control.
6.1 Read the original evidence
Table 1. CIFAR-10 separates the quality of a distilled shortcut from learning without a diffusion teacher. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the NFE column to match one-evaluation and two-evaluation rows, then compare FID downward and Inception Score upward. CD appears in the diffusion-plus-distillation block; CT appears at the bottom of direct generation. The asterisk marks methods that construct synthetic data for distillation and does not mark significance. The PD rows use the original squared-ℓ2 discrepancy, while CD uses LPIPS; Figure 4 separately examines the metric choice. Finally, inspect the slower EDM row and the strongest GAN rows to see what the shortcut gives up. Blank cells are unreported values, not zero scores. cifarcomparisontraining-detailsevaluation-scopeappendix-discrepancies
What it supports. CD reports FID 3.55 with one evaluation and 2.93 with two, compared with PD's 8.34 and 5.58. CT reports 8.70 and 5.83 without a diffusion teacher. These are distinct achievements: teacher-free training is useful, but it does not inherit the distilled model's best FID.
Where the evidence stops. The table provides neither repeated-run uncertainty nor an explicit FID reference split/sample count. Figure 18 labels one-step CT 8.73 instead of Table 1's 8.70. The report preserves the table value and flags the disagreement.
Table 2. The gains extend across datasets, while the distilled and teacher-free models retain different quality levels. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Treat ImageNet, Bedroom and Cat as separate experiments. Within each block, compare rows at the same NFE before examining the much slower diffusion references. FID decreases with better reported quality; precision and recall increase. The dagger identifies distillation, separating CD and PD from CT. On ImageNet, the one- and two-evaluation CD rows improve FID while changing precision and recall only modestly. The Bedroom and Cat blocks show the same need to separate training route from inference budget. Their numbers should never be averaged into a single score without defining a new evaluation protocol. large-resultscomparisonappendix-discrepanciesevaluation-scope
What it supports. ImageNet CD improves from FID 6.20 to 4.70 with a second evaluation; PD reports 15.39 and 8.95. Bedroom CD improves from 7.80 to 5.22 and Cat CD from 11.0 to 8.84. These gains coexist with remaining gaps to slower diffusion references and with poorer CT FIDs.
Where the evidence stops. All claims remain specific to the listed image protocols. Appendix CT sample labels differ from the table for two-step Bedroom and Cat. Figure 4 also qualifies broad CD superiority: one-step Bedroom with ℓ2 is an exception.
Figure 6(a–c). The same bedroom CD model is demonstrated on several editing tasks without task-specific training. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each row horizontally. In (a), the grayscale reference is at the left, alternative colorizations occupy the middle, and ground truth is at the right. In (b), the left image is the low-resolution input, the middle images are generated high-resolution alternatives, and the last image is ground truth. Row (c) begins with a stroke painting and continues with generated bedrooms; it has no ground-truth endpoint. The examples come from the same bedroom CD model. Appendix D explains observed-component replacement and uses 40 editing steps by default, with a separately stated two-step stroke configuration. editingediting-algorithmediting-tasks
What it supports. The panels show that a time-conditioned endpoint map can be reused for qualitative editing, rather than serving only as a terminal-noise generator. Multiple outputs illustrate ambiguity in color and fine detail. These examples support the existence of the demonstrated capability, not a numerical reconstruction-accuracy or reliability claim.
Where the evidence stops. Appendix D sets Ω to all ones for strokes, but Algorithm 4 line 2 then erases the reference before initialization. This conflicts with the stated stroke-conditioning recipe. The faithful examples remain visible; the printed algorithm leaves their exact implementation unresolved.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| CIFAR-10 few-step consistency distillation One/two network evaluations; CD and PD distill the same in-house EDM. Tables use LPIPS for CD and squared ℓ2 for PD. | CD: FID 3.55 / 2.93; IS 9.48 / 9.75 at one/two evaluations. FID ↓; Inception Score ↑ | PD: FID 8.34 / 5.58; IS 8.69 / 9.05. EDM: FID 2.04 at 35 evaluations. CD improves on tabulated PD but remains worse than the slower teacher. This table mixes metric and algorithm effects; reference split and FID sample count are unstated. cifarcomparisonevaluation-scope |
| ImageNet 64 × 64 few-step consistency distillation Same-teacher CD/PD comparison, one/two evaluations. Reference split and FID sample count are not stated. | CD FID 6.20 / 4.70; precision 0.68 / 0.69; recall 0.63 / 0.64. FID ↓; precision and recall ↑ | PD FID 15.39 / 8.95; EDM FID 2.44 at 79 evaluations. A second CD evaluation improves FID, but compression retains a quality gap to the iterative teacher. large-resultscomparisonevaluation-scope |
| CIFAR-10 generation without a diffusion teacher Main discrete CT, randomly initialized; one/two evaluations. Values follow Table 1; evaluation split/sample count are unstated. | CT: 8.70 / 5.83. FID ↓ | One-step DC-VAE: 17.9; DenseFlow: 34.9; StyleGAN-XL: 1.85. CT beats these VAE/flow baselines, not all one-step generators. Figure 18 labels one-step CT 8.73, an unresolved discrepancy. cifartraining-detailsappendix-discrepanciesevaluation-scope |
| LSUN Bedroom and Cat 256 × 256 generation Separate Bedroom/Cat datasets; one/two evaluations, Table 2 configurations. Evaluation split/sample count are unstated. | Bedroom CD 7.80 / 5.22, CT 16.0 / 7.85; Cat CD 11.0 / 8.84, CT 20.7 / 11.7. FID ↓ | PD: Bedroom 16.92 / 8.47; Cat 29.6 / 15.5. Both routes benefit from a second evaluation; CD outperforms CT. Appendix CT labels differ: Bedroom two-step 7.80 and Cat two-step 11.76. large-resultscomparisonappendix-discrepanciesevaluation-scope |
| CIFAR-10 training ablations Figure 3 varies metrics, solver/grids and CT schedules across training iterations. | Selected CD configuration: LPIPS, Heun, N = 18. Adaptive N and μ improve CT convergence. FID ↓ | LPIPS beats ℓ1/ℓ2; Heun beats Euler at matched N; fixed CT schedules fare worse. These are plotted trends, without invented curve endpoints or uncertainty estimates. Adaptive N and μ are jointly varied. ablation |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 3(a–d). The endpoint objective needs careful choices of discrepancy, solver and discretization schedule. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each panel separately: the vertical axis is FID, where lower is better, and the horizontal axis is training iterations in units of ten thousand. Panel (a) compares ℓ2, ℓ1 and LPIPS for CD. Panel (b) pairs Euler and Heun at matching N values; panel (c) varies N with Heun held fixed. Panel (d) switches to CT and compares fixed N/μ settings with a jointly adaptive schedule. N is training discretization resolution, not the number of network evaluations used to generate one sample. Also notice that panel (d) spans a longer training horizon than the other panels. ablationcdcttraining-configtraining-details
What it supports. LPIPS gives the best CD curve among these metrics, and Heun improves on Euler at matched grids. The authors select N = 18 for CIFAR-10 CD. The adaptive CT curve reaches better FID sooner than the fixed settings shown, supporting schedule adaptation as a practical component of the method.
Where the evidence stops. Panel (d) varies N and μ together, so it cannot identify their separate contributions. Curves have no uncertainty bands. Matched grid size also does not establish matched teacher-evaluation cost between Euler and Heun.
7. Analysis & limitations
7.1 What the evidence leaves open
CD's asymptotic guarantee assumes zero loss, Lipschitz regularity and bounded solver error; the CT connection additionally assumes an exact score and smoothness. Neither proves finite-budget neural optimization will recover the exact endpoint map. cdct
The continuous-time CT comparison initializes both variants from pretrained EDM and changes learning rates and scheduling. Its favorable curve cannot establish better teacher-free training from scratch. continuous
FID reference splits, generated-sample counts, repeated-run uncertainty and software versions are not explicitly specified. NFE measures network evaluations, not end-to-end latency. Small table/appendix FID disagreements remain unresolved. evaluation-scopeappendix-discrepancies
Editing evidence is qualitative and bedroom-specific. Shared-noise similarity between CT and EDM is suggestive, not proof against mode collapse. Image synthesis and analogies to Q-learning do not establish physical control. editingcomparisonlarge-results
7.2 Questions for discussion
- How much of the CD–PD gap survives matching perceptual loss and training compute?
- Can continuous-time CT retain its advantage with random initialization and matched tuning?
- Which implementation detail preserves strokes despite Algorithm 4's masking step?
8. Reproducibility audit
8.1 Requirements and known gaps
CIFAR-10 uses NCSN++, batch 512, learning rate 4e−4, 800k iterations and eight A100 GPUs. CD uses Heun, N = 18 and target μ = 0; CT uses μ₀ = 0.9, s₀ = 2, s₁ = 150. Separate model-weight EMA is 0.9999. architecturetraining-configtraining-details
Use Rectified Adam without warm-up, learning-rate decay or weight decay, horizontal flips, and 224 × 224 bilinear resizing for CIFAR-10/ImageNet LPIPS. Architecture/teacher recipes partly defer to prior work. Resolve the prose's LSUN Bedroom zero-EMA exception against Table 3. training-configtraining-details
Stroke conditioning is unresolved: Appendix D sets A = I and Ω to all ones, but Algorithm 4 line 2 then zeros y before sampling around it. Determine how stroke information is retained; silently deleting that line would be an undocumented correction. editing-algorithmediting-tasks
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate loss-metric gains from the CD–PD algorithm
Reader-proposed check, not performed: on CIFAR-10, use the same fixed EDM teacher and backbone for a 2 × 2 comparison of CD versus PD and LPIPS versus squared ℓ2. Keep data preprocessing, initialization and evaluation seeds identical; report both training updates and actual teacher/network evaluation cost. Evaluate one- and two-step FID with a declared reference split, fixed sample count and repeated seeds. If CD's gap largely vanishes when the metric is matched, the tabulated improvement is partly attributable to perceptual loss; if it persists across both metrics, that supports an algorithm-specific gain. Report tuning effort separately. comparisoncifarablationtraining-detailsevaluation-scope
Check 2: Check whether stroke information survives initialization
Reader-proposed check, not performed: use one bedroom CD checkpoint, two visibly different stroke inputs and identical Gaussian draws. Run the printed Algorithm 4 with the stated identity transform, all-ones mask and times 5.38 then 2.24. Compare it with an explicitly hypothetical variant that preserves the original stroke reference when adding initial noise, while leaving subsequent sampling unchanged. Log the reference immediately before noise is added and compare output dependence on the two inputs. Literal line 2 predicts input erasure; any input-dependent output would indicate an additional implementation pathway. The alternative tests a possible correction and must not be reported as the authors' verified procedure. editingediting-algorithmediting-tasks
8.3 Reading coverage
Visual audit: Actually viewed the title/author page; all main method, algorithm, ablation and result pages; Appendix B.3 comparisons and Appendix C configuration pages; Appendix D editing procedures; and Appendix E pages needed to verify inconsistent CT labels. Figures 1–7, 18, 20 and 21 and Tables 1–3 were visually inspected. Each of the five final original crops was separately viewed, including table headers/footnotes and plot legends. Figure 2's red arrow directions were checked against Section 3; its x₀ label is distinguished from formal x_ε. Figure 6 was checked against Algorithm 4 and Appendix D, revealing the stroke-reference masking conflict. Images of Figures 8–17 and 19 were not inspected; their captions and all supplied appendix text were read. No code was inspected or experiments run.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 24, 25, 26, 27, 28, 39, 41, 42. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; Sections 1–2: Introduction and Diffusion Models
- Section 3: Consistency Models
- Section 4: Training Consistency Models via Distillation
- Section 5: Training Consistency Models in Isolation
- Sections 6.1–6.3: training diagnostics, generation and editing experiments
- Section 7, Acknowledgements and References
- Appendix A.1–A.3: notation and proofs
- Appendix B.1–B.3: continuous-time extensions, proofs and experimental verification
- Appendix C: architectures, parameterization, schedules and training details
- Appendix D: all editing procedures
- Appendix E: additional sample captions through PDF p. 42
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Identity/edition: the title page reads Consistency Models by Yang Song, Prafulla Dhariwal, Mark Chen and Ilya Sutskever, matching the catalog. It identifies the ICML 2023 proceedings edition, PMLR 202. No numbered revision or revision history is supplied; equivalence to other editions was not checked.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This limitation was addressed for the visually inspected pages and five crops declared in the illustrated edition.
- Separate supplemental material availability has not been fully verified.
- All nine supplied text chunks were read individually, including appendices. Appendix figure captions were read throughout; images of Figures 8–17 and 19 were not inspected. Proofs were read but not independently rederived.
- Code and external references were not inspected; no experiments were reproduced. Evaluation details absent from the supplied paper were not inferred from cited implementations.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title, author line and affiliation/proceedings footnotes
Consistency Models; Yang Song, Prafulla Dhariwal, Mark Chen, Ilya Sutskever; OpenAI, San Francisco, CA 94110, USA; ICML 2023, PMLR 202.
Go to primary source ↓objectivePDF pp. 1–2, Abstract and Section 1
The objective is one-step generation with optional multistep refinement and zero-shot editing; training is offered through distillation or directly from data.
Go to primary source ↓pf-odePDF p. 2, Section 2, Eqs. (1)–(3) and final paragraph
Gaussian perturbations have variance t²; the empirical PF ODE uses −t times the score model. Pixel range [−1,1], T = 80 and ε = 0.002 are specified.
Go to primary source ↓consistencyPDF p. 3, Figure 2 and Section 3, Definition, Parameterization and Sampling; Eqs. (4)–(5)
Points on one PF trajectory share endpoint x_ε. Skip coefficients impose identity at ε. The schematic labels the ideal origin x₀; generation evaluates the map at T. Invertibility is not enforced.
Go to primary source ↓samplingPDF p. 4, Algorithm 1 and sampling text immediately below
Multistep generation alternates denoising and fresh noise of scale sqrt(τ_n²−ε²). Greedy ternary search optimizes FID under an empirical unimodality assumption.
Go to primary source ↓cdPDF pp. 4–5, Section 4, Eqs. (6)–(8), Algorithm 2 and Theorem 1
A fixed teacher solver constructs adjacent inputs; online and stop-gradient target outputs are compared. EMA updates the target. Boundary anchoring excludes constant-zero output; the theorem requires zero loss, Lipschitz regularity and bounded solver error.
Go to primary source ↓ctPDF pp. 5–6, Algorithm 3, Section 5, Theorem 2 and Eqs. (9)–(10)
CT uses the same data image and Gaussian noise at neighboring scales without score-model parameters. The asymptotic CD relation assumes an exact score, Euler integration and smoothness. N and μ vary during training.
Go to primary source ↓ablationPDF pp. 6–7, Section 6.1 and Figure 3(a–d)
CIFAR-10 diagnostics favor LPIPS, Heun and N = 18 for CD. Adaptive N and μ improve CT learning relative to fixed settings.
Go to primary source ↓comparisonPDF p. 7, Figure 4 and Section 6.2; PDF p. 9, continuation of Section 6.2
CD and PD distill the same in-house EDM. Tables use original squared-ℓ2 PD; LPIPS improves PD in Figure 4. A one-step Bedroom ℓ2 exception qualifies the broad CD superiority claim. Shared-noise CT/EDM similarity motivates an author claim about mode collapse.
Go to primary source ↓cifarPDF p. 8, Table 1, CD/PD/CT, EDM, DC-VAE, DenseFlow and StyleGAN-XL rows
CD FID 3.55/2.93, IS 9.48/9.75 at one/two NFE; PD FID 8.34/5.58, IS 8.69/9.05. CT FID 8.70/5.83; EDM 2.04 at 35 NFE; one-NFE DC-VAE 17.9, DenseFlow 34.9, StyleGAN-XL 1.85.
Go to primary source ↓large-resultsPDF p. 8, Table 2, ImageNet, Bedroom and Cat blocks, CD/CT/PD/EDM rows
ImageNet CD FID 6.20/4.70, precision 0.68/0.69, recall 0.63/0.64; PD FID 15.39/8.95; EDM 2.44 at 79 NFE. Bedroom CD 7.80/5.22, CT 16.0/7.85; Cat CD 11.0/8.84, CT 20.7/11.7. PD Bedroom 16.92/8.47, Cat 29.6/15.5.
Go to primary source ↓editingPDF p. 9, Figure 6(a–c), Section 6.3 and Conclusion
A bedroom CD model demonstrates colorization, 32 × 32 to 256 × 256 super-resolution, and stroke-guided synthesis without training on those editing tasks. Examples are qualitative.
Go to primary source ↓continuousPDF pp. 24–25, Appendix B.3 and Figure 7
Continuous-time comparisons modify learning rates and settings. Both CT variants in this comparison start from pretrained EDM; discrete CT uses fixed N = 120 and target EMA 0.99. Random initialization works for the main discrete method.
Go to primary source ↓architecturePDF pp. 25–26, Appendix C, Model Architectures and Parameterization
CIFAR-10 uses NCSN++; larger datasets use Dhariwal–Nichol architectures. Modified EDM coefficients enforce the ε boundary, with σ_data = 0.5.
Go to primary source ↓training-configPDF p. 25, Table 3, CIFAR-10 CD/CT columns and LSUN columns
CIFAR-10 learning rate 4e−4, batch 512, 800k iterations, eight GPUs; CD target μ = 0, Heun, N = 18; CT μ₀ = 0.9, s₀ = 2, s₁ = 150. Both list separate EMA 0.9999. LSUN columns provide shared defaults.
Go to primary source ↓training-detailsPDF p. 26, Appendix C, Schedule Functions and Training Details
CD initializes from EDM, main CT randomly; Rectified Adam without warm-up, learning-rate decay or weight decay. Horizontal flips, LPIPS resizing, A100 GPUs and an LSUN Bedroom CD zero-EMA exception are stated. Teacher and architecture settings partly defer to prior work.
Go to primary source ↓editing-algorithmPDF pp. 26–27, Appendix D and Algorithm 4, lines 2–5 and 7–9
Reference y, binary mask Ω and invertible A guide replacement. Line 2 overwrites y with its unmasked transformed components; later steps restore these. Default bedroom editing uses N = 40. Inpainting marks missing pixels with Ω = 1.
Go to primary source ↓editing-tasksPDF p. 28, Appendix D, Super-resolution, Stroke-guided image generation, Denoising and Interpolation
Super-resolution assumes nonoverlapping patch averages. Strokes use A = I, Ω all ones, N = 2, t₁ = 5.38, t₂ = 2.24. Denoising evaluates at the input noise scale; interpolation uses spherical interpolation between initial noises.
Go to primary source ↓appendix-discrepanciesPDF p. 8, Tables 1–2 versus PDF p. 39, Figure 18(b); p. 41, Figure 20(c); p. 42, Figure 21(c)
Table/appendix CT FIDs differ: CIFAR-10 one-step 8.70/8.73, Bedroom two-step 7.85/7.80, Cat two-step 11.7/11.76. The source does not reconcile these labels.
Go to primary source ↓evaluation-scopePDF pp. 6–9, Section 6 and Tables 1–2; pp. 25–26, Appendix C and Table 3
These evaluation/implementation sections name datasets, metrics, NFE and training configurations, but do not explicitly specify FID sample counts, reference splits, repeated-run uncertainty or software versions.
Go to primary source ↓8.5 Primary sources
Consistency Models ↗
PDF · 17,747 extracted words
Source fingerprint
8f65127c770a8e8994777010ed36dfcf64fc38f840042778eef7fe8ab2c12e10