GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
1. Paper overview
In one sentence: Separate GAN learning rates can improve selected training outcomes, while FID measures image-distribution similarity through feature moments; both claims require careful attention to their assumptions and evaluation protocol. e-probleme-conditionse-fide-protocole-supplement
| At a glance | What to know |
|---|---|
| Research problem | Source description GAN training changes the objective seen by each network as the other learns, so ordinary optimization intuition does not guarantee convergence. The authors seek stochastic minibatch training that approaches a stationary local Nash equilibrium. Evaluation poses a second problem: Inception Score uses generated-image label statistics without explicitly comparing them with real-image statistics. e-probleme-fid |
| Core mechanism | Source description TTUR gives discriminator and generator separate learning rates and applies a two-time-scale stochastic-approximation theorem under explicit stability, smoothness, noise and boundedness assumptions. e-tture-conditions |
| A key reported result | WGAN-GP image generation on LSUN Bedrooms: 9.5 at 1,900 minutes FID (lower is better). Training-image FID reference; 50,000 generated images. Selected TTUR b=0.0003,a=0.0001 versus baseline b=a=0.0001. Original training: 20.5 at 2,010 minutes. A substantial selected FID improvement; schedules and selected times differ. Table 1 supplies no uncertainty estimate. e-protocole-wgane-table-wgan |
| Reading caution | Reader analysis Fixed experimental rates do not satisfy the asymptotic rate conditions as written. The Adam argument also requires stationary second moments and scheduled averaging parameters; it should not be read as an unconditional guarantee for ordinary Adam settings. e-conditionse-adame-table-dcgane-table-wgan |
Core contributions
- Source description
TTUR gives discriminator and generator separate learning rates and applies a two-time-scale stochastic-approximation theorem under explicit stability, smoothness, noise and boundedness assumptions. e-tture-conditions
- Author claim
The authors connect a constrained Adam formulation to heavy-ball-with-friction dynamics and extend their convergence argument. The associated preference for flat minima is an interpretation of those dynamics, not a direct measurement of GAN loss geometry. e-adame-adam-proof
- Source description
FID replaces generated-only label statistics with a distance between Gaussian approximations to real and generated Inception features, accompanied by corruption diagnostics and GAN comparisons. e-fide-fid-diagnostic
Figure 2. The paper's mechanical picture of gradient memory: momentum can carry an iterate beyond a narrow basin. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the vertical objective axis, then follow the arrows along the parameter landscape. The ball begins at the position marked \(\theta_0\), passes the narrow local minimum \(\theta^+\), and continues through \(\theta_n\) toward the wider basin at \(\theta^*\). The arrows around the final ball indicate settling in that basin. Section 2 ties this illustration to Adam's averaging of past gradients and to the heavy-ball-with-friction equation, whose acceleration term retains motion while damping dissipates it. The arrow directions agree with the caption's overshooting account. This diagram explains an optimizer analogy; it does not depict the GAN's network layers. e-adame-adam-proofe-protocol
What it supports. The illustration makes the authors' motivation for Adam concrete: accumulated gradient motion may resist being trapped in a small region associated with mode collapse. The mathematical connection is conditional, relying on the specified averaging schedules and stationary second moments. It supplies an intuition for the optimizer choice rather than measured evidence about a particular GAN's minima.
Where the evidence stops. The landscape is schematic. Neither its basin widths nor the ball trajectory are measured GAN quantities. The main PDF provides no quantitative Adam-versus-no-Adam mode-collapse ablation here, and the generality of the intuition exceeds what this picture establishes.
2. Motivation
2.1 The problem and the proposed response
GAN training changes the objective seen by each network as the other learns, so ordinary optimization intuition does not guarantee convergence. The authors seek stochastic minibatch training that approaches a stationary local Nash equilibrium. Evaluation poses a second problem: Inception Score uses generated-image label statistics without explicitly comparing them with real-image statistics. e-probleme-fid
2.2 What this reading follows
A GAN is a moving learning problem: every generator change alters the discriminator's task, and every discriminator change alters the generator's feedback. This paper addresses that interaction with separate update rates, then introduces FID to evaluate the resulting images against real data. Read its contributions in three layers: the conditional convergence argument, the practical training schedules, and the feature-based evaluation procedure. They reinforce one another without being equivalent. The original figures explain optimizer intuition, reveal metric sensitivities, and show why a selected best score is only part of the training story. The supplied edition contains the main paper and references; its separately cited supplement is unavailable. e-probleme-conditionse-fide-protocole-supplement
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Evaluation metrics |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded evaluation-metrics placement is supported by FID; training optimization is a second central contribution. Architecture, prediction paradigm and quadrant are appropriately not applicable to the world-action taxonomy: the paper specifies adversarial generation and feature-based evaluation, not joint future/action prediction or inverse dynamics. This does not mean its GANs lack neural architectures. e-probleme-tture-fid
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Let the discriminator track a slowly changing generator
The argument begins by temporarily holding the generator fixed. Under Assumption A4, the discriminator's differential equation has a locally stable attractor, written \(\lambda(\theta)\). The generator's slower equation then evolves using that discriminator response. Assumption A2 formalizes the separation through diminishing rates with \(a(n)/b(n)\to0\); bounded iterates, regular gradients and controlled stochastic errors complete the stated conditions. Theorem 1 supplies the convergence result under those assumptions. Practical experiments instead search for rate pairs that keep learning stable. The authors explicitly warn that the numerical rate alone does not determine how strongly one network perturbs the other, explaining why some DCGAN settings give the generator the larger rate. Keep the theorem and the empirical rate search distinct when interpreting the training plots. e-tture-conditionse-ratese-protocol
5.2 Turn image collections into comparable feature statistics
FID adds a real-data reference that Inception Score lacks. Images first become feature vectors from Inception-v3's last pooling layer. The paper then represents each collection by a mean and covariance and computes the distance between the corresponding Gaussian approximations. Its implementation estimates real statistics from all training images and generated statistics from 50,000 samples. This construction explains why the score can react both to individual-image degradation and to contamination by images from another distribution. Reader inference: the compression is also an evidence boundary, because matching those summaries cannot establish every property of the original image distribution. Figure 3 tests specific perturbations rather than proving general perceptual validity; its nearly saturated salt-and-pepper response is especially useful for separating the metric's practical sensitivity from the caption's stronger monotonicity language. e-fide-protocole-fid-diagnostic
5.3 Judge the whole training comparison before the best checkpoint
Table 1 answers which selected score each tuned configuration attained. The curves answer a different question: how performance and run-to-run spread evolved along the way. DCGAN can improve rapidly and later deteriorate, so a good early score does not establish sustained stability. WGAN-GP introduces another distinction: TTUR changes both the learning-rate pair and the number of discriminator updates, making minutes a useful comparison axis while leaving causal attribution mixed. The source reports eight image runs and ten language runs, with min–max envelopes around mean curves. Reader analysis: those envelopes describe observed spread but are not confidence intervals, and the selected table entries should not be substituted for the plotted means. Finally, language JSD and image FID evaluate different output statistics; neither result demonstrates executed actions or environmental control. e-dcgane-wgane-protocole-languagee-table-dcgane-table-wgane-table-language
5.4 Training and inference
During training
The theory additionally requires Lipschitz gradients, controlled martingale-difference noise, bounded parameters and stable fast/slow ODE attractors. The paper explicitly excludes ReLU networks from its stated smoothness example. Its conclusion is local and conditional. e-conditions
Experiments use Adam and tune rates to permit fast learning while FID or JSD decreases. Stopping points are selected from the same progress measures. DCGAN's reported generator rates can exceed discriminator rates; the authors distinguish numerical rates from effective perturbations. e-protocole-rates
For WGAN-GP, TTUR uses one discriminator update per generator step; the original image and language schedules use five and ten, respectively. Wall-clock comparisons therefore change update counts as well as rates. e-protocole-wgane-language
During inference
After training, the generator maps random inputs to synthetic samples. The language example uses a 1D CNN and selects characters from softmax outputs to form 32-character sequences. The discriminator provides training feedback; this paper defines no action extraction, environment feedback loop or robot execution. e-probleme-language
5.5 Implementation flow
- Form the coupled learning problem
The discriminator D has parameters w and the generator G has parameters theta. Each receives a stochastic gradient estimated from minibatches. Their objectives need not be a single shared min-max loss, so the proposed update framework is broader than that special case. e-ttur
- Separate adaptation speeds
The theoretical fast discriminator approaches a local attractor for a fixed generator; the slow generator then evolves relative to that attractor. Small cross-network perturbations motivate the design. Merely assigning unequal constant rates is not the theorem's complete condition. e-conditionse-rates
- Measure feature-distribution discrepancy
Propagate real and generated images through pretrained Inception-v3, retain its last pooling features, and compute separate means and covariances. FID compares these Gaussian summaries. This is an evaluation path, not an auxiliary GAN training loss in the reported experiments. e-fide-protocol
6. Experiments & results
This paper couples two contributions: separate generator/discriminator learning rates for GAN training, and FID for comparing generated images with real data through Inception features. It presents conditional local-equilibrium convergence arguments and selected improvements for DCGAN and WGAN-GP. The experiments support useful training and evaluation procedures; they do not certify that practical fixed-rate runs satisfy every theorem assumption.
The main PDF supplies a conceptual optimizer diagram rather than a GAN architecture schematic. Its toy rate comparison and FID corruption panels provide diagnostics, but it contains no controlled neural-component ablation table or quantitative Adam mode-collapse ablation. Those experiment types are not fabricated here. Expanded metric comparisons, implementation details and further experiments are assigned to the separate, unsupplied supplement. e-adame-toye-fid-diagnostice-protocole-supplement
6.1 Read the original evidence
Figure 5. DCGAN's training histories show stability differences that selected best scores alone cannot convey. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The panels are CelebA, CIFAR-10, SVHN and LSUN Bedrooms in reading order. Red denotes TTUR; other colors denote the original equal-rate runs. In a TTUR legend the first number is discriminator rate \(b\) and the second generator rate \(a\). The horizontal axis counts thousands of minibatch updates, and lower FID is better. Solid curves are means over eight runs; shaded envelopes run from the minimum to maximum, rather than expressing standard errors. CIFAR-10 begins at 10,000 updates for visibility. Follow both early descent and later drift, especially on CelebA and LSUN, before judging a rate setting. e-dcgane-protocole-ratese-conditionse-table-dcgan
What it supports. TTUR maintains lower late-training FID with narrower displayed run ranges in these comparisons. CelebA and LSUN particularly illustrate why an early improvement can be misleading: several original-rate curves subsequently worsen. CIFAR-10 presents a smaller difference, consistent with Table 1's selected FIDs of 36.9 for TTUR and 37.7 for original training.
Where the evidence stops. The plotted and tabulated results use tuned learning rates and selected stopping points. Some TTUR generator rates exceed discriminator rates; the authors discuss effective perturbations separately from numerical rates. These constant-rate histories do not verify the asymptotic schedule conditions in Assumption A2.
Figure 6. Wall-clock alignment accounts for the different numbers of discriminator updates, while retaining a mixed intervention. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the caption's explicit panel assignments: CIFAR-10 is on the left and LSUN Bedrooms on the right. The horizontal axis is minutes, unlike the update-count axis in Figure 5. This matters because original WGAN-GP uses five discriminator updates for each generator update, while TTUR uses one. The red legend lists discriminator/generator rates 0.0003/0.0001; the original curves use equal rates. Solid curves and shaded regions again represent eight-run means and min–max ranges. CIFAR-10 starts at minute 20. Look for quality at comparable elapsed time and for differences in the spread of the runs. e-wgane-protocole-table-wgane-supplement
What it supports. The plotted TTUR configuration attains lower FID with a relatively narrow run range, most clearly on LSUN Bedrooms. Table 1 selects LSUN FID 9.5 at 1,900 minutes for TTUR versus 20.5 at 2,010 minutes for the original schedule. This supports a useful practical configuration, while the simultaneous rate and update-count changes prevent attributing the entire gain to one factor.
Where the evidence stops. The caption's opening sentence also names CelebA and SVHN, but the actual two panels, explicit left/right labels and surrounding experiment text identify only CIFAR-10 and LSUN. The supplied main paper gives no hardware specification for transporting these minute values to another machine.
Table 1. The selected benchmark results favor TTUR, but each row carries its own rate and stopping-point comparison. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each row across the two method blocks. On the left, TTUR has separate discriminator and generator rates \(b,a\); on the right, original training uses \(b=a\). DCGAN reports minibatch updates, whereas WGAN-GP reports minutes. The image blocks use FID; the language block uses normalized JSD for different n-gram statistics. Both metrics favor smaller values, but their magnitudes are not interchangeable. The bold numbers are selected TTUR results, not confidence bounds. Keep the rate and time columns attached to each score: for example, CelebA's selected TTUR result uses substantially more updates than its baseline. e-protocole-table-dcgane-table-wgane-table-languagee-language
What it supports. Every displayed row favors TTUR on its selected metric. The gain varies: DCGAN CIFAR-10 changes from 37.7 to 36.9 FID, while WGAN-GP LSUN changes from 20.5 to 9.5. Language results improve from 0.38 to 0.35 for 4-grams and from 0.77 to 0.74 for 6-grams. Those language scores concern distribution matching, not semantic understanding.
Where the evidence stops. The authors optimize rates and stopping points; paired entries are not uniformly matched-compute endpoints. The table supplies no uncertainty columns or explicit across-run aggregation rule. Its selected values must not be relabeled as the mean curves in Figures 5–7.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| WGAN-GP image generation on LSUN Bedrooms Training-image FID reference; 50,000 generated images. Selected TTUR b=0.0003,a=0.0001 versus baseline b=a=0.0001. | 9.5 at 1,900 minutes FID (lower is better) | Original training: 20.5 at 2,010 minutes. A substantial selected FID improvement; schedules and selected times differ. Table 1 supplies no uncertainty estimate. e-protocole-wgane-table-wgan |
| DCGAN image generation on CelebA Same training-reference FID protocol; selected TTUR b=0.00001,a=0.0005 versus baseline b=a=0.0005. | 12.5 at 225,000 updates FID (lower is better) | Original training: 21.4 at 70,000 updates. TTUR improves the selected value but uses more updates; Figure 5 also shows later instability in original training. e-protocole-dcgane-table-dcgan |
| DCGAN image generation on CIFAR-10 Training-reference FID; selected TTUR b=0.0001,a=0.0005 versus baseline b=a=0.0001. | 36.9 at 75,000 updates FID (lower is better) | Original training: 37.7 at 100,000 updates. The selected improvement is modest; the paper does not report a significance test. e-protocole-table-dcgan |
| WGAN-GP image generation on CIFAR-10 Training-reference FID; selected TTUR b=0.0003,a=0.0001 versus baseline b=a=0.0001. | 24.8 at 700 minutes FID (lower is better) | Original training: 29.3 at 800 minutes. Selected quality improves under wall-clock reporting, with the changed discriminator update schedule still part of the comparison. e-protocole-wgane-table-wgan |
| WGAN-GP language generation on One Billion Word: 4-gram statistics Selected checkpoints; TTUR b=0.0003,a=0.0001 versus baseline b=a=0.0001. Main paper does not identify a held-out evaluation split. | 0.35 at 1,150 minutes Normalized JSD (lower is better) | Original training: 0.38 at 1,040 minutes. Better selected n-gram distribution matching; this is not FID or a demonstrated semantic-quality gain. e-languagee-table-language |
| WGAN-GP language generation on One Billion Word: 6-gram statistics Same language comparison; independently selected times for this statistic. | 0.74 at 1,120 minutes Normalized JSD (lower is better) | Original training: 0.77 at 1,070 minutes. The selected improvement extends to longer n-grams. Table entries are not identified as the plotted across-run means. e-languagee-table-language |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 3. FID responds to several ways of degrading or changing an image distribution, with markedly different sensitivities. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the top row left to right as Gaussian noise, Gaussian blur and implanted black rectangles; the bottom row contains swirl, salt-and-pepper noise and ImageNet contamination of CelebA. Each horizontal axis is disturbance level, while the vertical axis is FID. The image strips illustrate the corresponding changes. Compare the trend within each panel before comparing heights across panels, because the vertical scales differ. FID is computed from real and altered image feature distributions, not from the displayed example alone. Most curves rise visibly; the salt-and-pepper response rises sharply at first and then becomes nearly flat. e-fide-fid-diagnostic
What it supports. The panels support FID as a detector of these image-distribution changes. They also show that equal increments on a disturbance axis need not produce equal score changes. Blur and swirl have strongly nonlinear responses, while the salt-and-pepper panel is nearly saturated after its first increase. These are diagnostics of metric sensitivity, not GAN training results.
Where the evidence stops. The caption says FID increases monotonically, but the last salt-and-pepper marker is slightly below the preceding one. Preserve that discrepancy. Corruption parameter definitions and the detailed Inception Score comparison are delegated to the unavailable supplement; this figure alone does not establish universal perceptual calibration.
Figure 4. A two-variable saddle problem exposes why both update scale and the assignment of the faster variable matter. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The surface represents \(f(x,y)=(1+x^2)(100-y^2)\), with descent in \(x\) and ascent in \(y\). On the right, columns show objective value, distance from the origin and the parameter trajectory. Rows use equal rates 0.01, equal rates 0.001, then \(x/y\) rates 0.0001/0.01 and 0.01/0.0001. The target is the saddle at \((0,0)\), with objective 100 and norm zero. Read the norm together with the objective: an apparently settled objective can conceal continuing parameter movement. Compare rows three and four to see the effect of exchanging which coordinate receives the larger rate. e-toye-conditions
What it supports. In the displayed noisy example, the high equal-rate setting fluctuates strongly, while the smaller equal-rate setting approaches the target more slowly. The authors' preferred separation, slow \(x\) and fast \(y\), reaches the neighborhood quickly. Reversing that assignment is less effective in these traces. The comparison gives a mechanism diagnostic before the much larger GAN experiments.
Where the evidence stops. This is one constructed objective with additive Gaussian noise, not a neural architecture ablation. The main text does not specify the noise magnitude or provide an uncertainty summary for these trajectories. Finite noisy traces do not themselves prove almost-sure asymptotic convergence.
7. Analysis & limitations
7.1 What the evidence leaves open
Fixed experimental rates do not satisfy the asymptotic rate conditions as written. The Adam argument also requires stationary second moments and scheduled averaging parameters; it should not be read as an unconditional guarantee for ordinary Adam settings. e-conditionse-adame-table-dcgane-table-wgan
FID compresses distributions into two moments of one pretrained representation. Reader inference: identical summaries need not establish identical image distributions. Figure 3 supports corruption sensitivity, but its salt-and-pepper panel slightly dips at the end despite the caption's monotonicity claim. e-fide-fid-diagnostic
Learning-rate and checkpoint selection use the reported evaluation measures. Eight-run image and ten-run language curves show min–max ranges, not confidence intervals. The selected table values lack uncertainty and an explicit aggregation rule. e-protocole-table-dcgane-table-language
The supplied main paper does not expose the detailed FID-versus-Inception-Score comparison or expanded implementation and proof material assigned to its supplement. Its broad human-judgment claim cannot be fully assessed from the corruption panels alone. e-fid-diagnostice-supplement
7.2 Questions for discussion
- How much of WGAN-GP's gain survives when discriminator update count and learning rates are varied independently?
- How sensitive is the FID ranking to sampling variation and to the choice of real-image reference set?
8. Reproducibility audit
8.1 Requirements and known gaps
A faithful implementation needs the original model configurations, preprocessing, Inception weights and feature computation, Adam parameters, seeds and hardware. The main text supplies dataset names, rates, evaluation sample count and schedule, but not a complete software/hardware recipe; it refers implementation details to the supplement. e-protocole-supplemente-table-dcgane-table-wgan
Resolve the printed update-sign convention before implementation: Eq. (1) adds g and h while the preceding text defines them as loss gradients. The toy experiment explicitly specifies descent in x and ascent in y, providing an unambiguous small-scale check; its Gaussian noise magnitude is not stated here. e-tture-toy
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate WGAN-GP's learning-rate change from its update-count change
Reader-proposed check, not an experiment performed here: on CIFAR-10, cross discriminator update counts of one and five with the paper's equal-rate pair (0.0001, 0.0001) and TTUR pair (0.0003, 0.0001), in discriminator/generator order. Use the same model, data processing, evaluator and eight paired seeds in all four cells, and record both elapsed time and cumulative discriminator updates. Retain the source's 50,000 generated images per FID estimate, but predeclare comparison budgets and a checkpoint-selection rule. If unequal rates help at both update counts, that supports a rate-specific effect; if gains appear mainly when the update count changes, the original comparison's benefit cannot be assigned primarily to rate separation. Full configuration recovery still requires the missing implementation details. e-wgane-protocole-table-wgane-supplement
Check 2: Test FID's corruption ordering and the apparent saturation
Reader-proposed diagnostic: hold a clean CelebA reference set and the Inception-v3 evaluator fixed, then evaluate clean controls and progressively stronger Gaussian noise, blur and salt-and-pepper corruption using 50,000 sampled images per estimate. Because the main PDF omits the corruption parameters, define and disclose a severity grid instead of claiming exact reproduction of Figure 3. Repeat image sampling and corruption draws, reporting score distributions and each adjacent-level difference. Compare FID and Inception Score on identical image sets. Persistent negative adjacent differences or overlapping distributions at high salt-and-pepper severity would weaken strict monotonic ordering; consistently ordered changes across repeats would support a narrower robustness claim than universal agreement with human judgment. e-fide-fid-diagnostice-protocole-supplement
8.3 Reading coverage
Visual audit: The title/author page, all method and proof pages, FID definition and diagnostic, toy experiment, all GAN training curves, and every block of Table 1 were rendered and visually read. All six final original crops were individually inspected. Figure 2's arrows were checked against its caption and HBF formulation; Figure 3's salt-and-pepper trend and Figure 6's dataset-caption discrepancy are disclosed. Pages 10–12 contain references and were read as text. No supplemental pages were supplied or inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract (p. 1)
- 1 Introduction and Two Time-Scale Update Rule for GANs (pp. 1–4)
- 2 Adam Follows an HBF ODE and Ensures TTUR Convergence (pp. 4–5)
- 3 Experiments: Performance Measure; Model Selection and Evaluation; Simple Toy Data; DCGAN on Image Data; WGAN-GP on Image Data; WGAN-GP on Language Data (pp. 5–9)
- 4 Conclusion and Acknowledgment (p. 9)
- References (pp. 10–12)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The complete supplied proceedings PDF was read through all five text chunks. Pages 1–9 and all seven figures plus Table 1 were visually inspected; reference pages 10–12 were read as text.
- The verified edition is the NIPS 2017 proceedings paper. Title and authors match the catalog; no revision number or explicit revision date is supplied, and no comparison with another edition was possible.
- The acquisition notes that text extraction does not reconstruct figure images; the retained PDF supplied the visual evidence in this reading.
- Separate supplemental material availability has not been fully verified. No supplement was supplied or read, including the expanded proofs, implementation details and additional experiments repeatedly cited by the main paper. No appendix is contained in this PDF.
- Code was not inspected and experiments were not reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author block and conference footer
The title and five authors match the catalog. The shared affiliation is LIT AI Lab & Institute of Bioinformatics, Johannes Kepler University Linz, A-4040 Linz, Austria. The footer identifies NIPS 2017; no revision number is printed.
Go to primary source ↓e-problemPDF pp. 1–2, Abstract and Section 1, GAN game and contributions
The generator constructs synthetic data from random variables; the discriminator separates synthetic from real data. The paper proposes TTUR, conditional local-equilibrium convergence, an Adam interpretation and FID.
Go to primary source ↓e-tturPDF pp. 2–3, Two Time-Scale Update Rule for GANs, definitions and Eq. (1)
Discriminator parameters w and generator parameters theta use stochastic minibatch gradients and respective rates b(n) and a(n). Losses need not form a min-max pair. Eq. (1) prints plus signs although preceding definitions call g and h gradients of losses.
Go to primary source ↓e-conditionsPDF p. 3, assumptions A1–A5 and Theorem 1
The theorem requires Lipschitz gradients, nonsummable but square-summable rates with a(n)=o(b(n)), bounded martingale-difference noise, locally stable fast and reduced slow ODEs, and bounded iterates. Its limit is described as a stationary local Nash equilibrium.
Go to primary source ↓e-ratesPDF p. 4, paragraph preceding Section 2; p. 7, DCGAN on Image Data
The authors distinguish numerical learning rates from cross-network perturbation strength and discuss generator learning rates exceeding discriminator rates in DCGAN. Further justification is referred to Supplement Section 5.
Go to primary source ↓e-adamPDF p. 4, Section 2, Figure 2, Theorem 2 and Eqs. (2)–(4)
Figure 2 illustrates momentum passing the narrow minimum theta-plus and settling at theta-star. Theorem 2 specifies schedule-dependent beta parameters, a lower-bounded differentiable objective, stationary gradient second moments and Lipschitz gradients for its HBF account of Adam.
Go to primary source ↓e-adam-proofPDF p. 5, Section 2, Eq. (5) and proof continuation
The Adam argument linearizes second-moment normalization around stationary moments, treats deviations as noise and invokes a dissipative HBF energy to extend the two-time-scale argument.
Go to primary source ↓e-fidPDF pp. 5–6, Section 3, Performance Measure and unnumbered FID equation
FID compares Gaussian approximations to real and generated Inception feature distributions using their means and covariances. The paper motivates this by Inception Score's omission of real-sample statistics.
Go to primary source ↓e-fid-diagnosticPDF p. 6, Figure 3 and caption; Performance Measure continuation
Six panels test Gaussian noise, blur, black rectangles, swirl, salt-and-pepper noise and ImageNet contamination of CelebA. The caption claims monotonic increase, but the salt-and-pepper plot nearly plateaus and its final marker is slightly lower than the preceding marker. Detailed comparison with Inception Score is assigned to Supplement Section 1.
Go to primary source ↓e-protocolPDF pp. 6–7, Section 3, Model Selection and Evaluation
Real feature statistics use all training images; generated statistics use 50,000 images and Inception-v3's last pooling layer. Image runs are repeated eight times and language runs ten times. Rates and stopping points are selected using FID/JSD progress; curves show mean and min–max ranges. Evaluation intervals are 1,000 DCGAN updates, 5,000 WGAN-GP image outer iterations and 100 language outer iterations.
Go to primary source ↓e-toyPDF p. 7, Simple Toy Data and Figure 4
The toy objective is (1+x^2)(100-y^2), minimized in x and maximized in y with additive Gaussian noise. Its target is (0,0), objective 100. Figure rows use equal rates 0.01, equal rates 0.001, x/y rates 0.0001/0.01 and x/y rates 0.01/0.0001.
Go to primary source ↓e-dcganPDF p. 7, DCGAN on Image Data; p. 8, Figure 5 and caption
DCGAN is evaluated on CelebA, CIFAR-10, SVHN and LSUN Bedrooms. Figure 5 shows FID versus minibatch updates, eight-run means and min–max bands. TTUR is red; legends list discriminator rate b before generator rate a.
Go to primary source ↓e-wganPDF pp. 7–8, WGAN-GP on Image Data; p. 8, Figure 6 and caption
WGAN-GP image comparisons change discriminator updates per generator step from five to one and align progress by wall-clock time. Figure 6 contains CIFAR-10 left and LSUN Bedrooms right; its opening caption sentence also names CelebA and SVHN, inconsistent with its actual panels and surrounding experiment description.
Go to primary source ↓e-languagePDF p. 8, WGAN-GP on Language Data; p. 9, Figure 7 and caption
The character-level generator and discriminator are 1D CNNs; the output is a sequence of 32 characters. The language experiment uses normalized JSD of 4-gram and 6-gram statistics, ten runs, wall-clock alignment, and ten baseline discriminator updates versus one for TTUR.
Go to primary source ↓e-table-dcganPDF p. 9, Table 1, DCGAN Image block, all rows
Selected TTUR/original FIDs are CelebA 12.5/21.4 at 225k/70k updates, CIFAR-10 36.9/37.7 at 75k/100k, SVHN 12.5/21.4 at 165k/185k, and LSUN 57.5/70.4 at 340k/70k. Table headers identify b,a and baseline b=a; no uncertainty columns are supplied.
Go to primary source ↓e-table-wganPDF p. 9, Table 1, WGAN-GP Image block, CIFAR-10 and LSUN rows
TTUR uses b=0.0003,a=0.0001; baseline b=a=0.0001. CIFAR-10 FID is 24.8 at 700 minutes versus 29.3 at 800; LSUN FID is 9.5 at 1,900 minutes versus 20.5 at 2,010.
Go to primary source ↓e-table-languagePDF p. 9, Table 1, WGAN-GP Language block, 4-gram and 6-gram rows
With TTUR b=0.0003,a=0.0001 versus baseline b=a=0.0001, selected 4-gram JSD is 0.35 at 1,150 minutes versus 0.38 at 1,040; 6-gram JSD is 0.74 at 1,120 versus 0.77 at 1,070.
Go to primary source ↓e-supplementPDF p. 3, supplement references following Eq. (1) and Theorem 1; p. 5, final Section 2 paragraph; pp. 6–7, supplement references in Section 3
The main paper assigns expanded convergence arguments and rates, Adam analysis, metric comparisons, implementations and further experiments to a separate supplement. Those references do not supply the supplemental content within this 12-page PDF.
Go to primary source ↓8.5 Primary sources
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium ↗
PDF · 7,836 extracted words
Source fingerprint
cec84d8a98eac808e564262edc3100a4d4ace43fd9938df741f87575ed16671d