FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3083576952> ?p ?o ?g. }

• W3083576952 abstract "Spectral normalization (SN) is a widely-used technique for improving the stability of Generative Adversarial Networks (GANs) by forcing each layer of the discriminator to have unit spectral norm. This approach controls the Lipschitz constant of the discriminator, and is empirically known to improve sample quality in many GAN architectures. However, there is currently little understanding of why SN is so effective. In this work, we show that SN controls two important failure modes of GAN training: exploding and vanishing gradients. Our proofs illustrate a (perhaps unintentional) connection with the successful LeCun initialization technique, proposed over two decades ago to control gradients in the training of deep neural networks. This connection helps to explain why the most popular implementation of SN for GANs requires no hyperparameter tuning, whereas stricter implementations of SN have poor empirical performance out-of-the-box. Unlike LeCun initialization which only controls gradient vanishing at the beginning of training, we show that SN tends to preserve this property throughout training. Finally, building on this theoretical understanding, we propose Bidirectional Spectral Normalization (BSN), a modification of SN inspired by Xavier initialization, a later improvement to LeCun initialization. Theoretically, we show that BSN gives better gradient control than SN. Empirically, we demonstrate that BSN outperforms SN in sample quality on several benchmark datasets, while also exhibiting better training stability." @default.
• W3083576952 created "2020-09-11" @default.
• W3083576952 creator A5003733418 @default.
• W3083576952 creator A5076026636 @default.
• W3083576952 creator A5079175103 @default.
• W3083576952 date "2020-09-06" @default.
• W3083576952 modified "2023-09-22" @default.
• W3083576952 title "Why Spectral Normalization Stabilizes GANs: Analysis and Improvements" @default.
• W3083576952 cites W1533861849 @default.
• W3083576952 cites W1677182931 @default.
• W3083576952 cites W1815076433 @default.
• W3083576952 cites W1834627138 @default.
• W3083576952 cites W2058094241 @default.
• W3083576952 cites W2099471712 @default.
• W3083576952 cites W2107878631 @default.
• W3083576952 cites W2120316325 @default.
• W3083576952 cites W2125930537 @default.
• W3083576952 cites W2173520492 @default.
• W3083576952 cites W2402302915 @default.
• W3083576952 cites W2618017917 @default.
• W3083576952 cites W2751842161 @default.
• W3083576952 cites W2796303840 @default.
• W3083576952 cites W2796892552 @default.
• W3083576952 cites W2810518847 @default.
• W3083576952 cites W2893749619 @default.
• W3083576952 cites W2949117887 @default.
• W3083576952 cites W2962754210 @default.
• W3083576952 cites W2962760235 @default.
• W3083576952 cites W2962879692 @default.
• W3083576952 cites W2963285844 @default.
• W3083576952 cites W2963373786 @default.
• W3083576952 cites W2963399222 @default.
• W3083576952 cites W2963556353 @default.
• W3083576952 cites W2963570896 @default.
• W3083576952 cites W2963685250 @default.
• W3083576952 cites W2963836885 @default.
• W3083576952 cites W2963981733 @default.
• W3083576952 cites W2964057425 @default.
• W3083576952 cites W2964092448 @default.
• W3083576952 cites W2964120007 @default.
• W3083576952 cites W2964121744 @default.
• W3083576952 cites W2964144352 @default.
• W3083576952 cites W2964201867 @default.
• W3083576952 cites W2982763192 @default.
• W3083576952 cites W3004456744 @default.
• W3083576952 cites W3035626604 @default.
• W3083576952 cites W3118608800 @default.
• W3083576952 hasPublicationYear "2020" @default.
• W3083576952 type Work @default.
• W3083576952 sameAs 3083576952 @default.
• W3083576952 citedByCount "0" @default.
• W3083576952 crossrefType "posted-content" @default.
• W3083576952 hasAuthorship W3083576952A5003733418 @default.
• W3083576952 hasAuthorship W3083576952A5076026636 @default.
• W3083576952 hasAuthorship W3083576952A5079175103 @default.
• W3083576952 hasConcept C11413529 @default.
• W3083576952 hasConcept C114466953 @default.
• W3083576952 hasConcept C119857082 @default.
• W3083576952 hasConcept C13280743 @default.
• W3083576952 hasConcept C134306372 @default.
• W3083576952 hasConcept C136886441 @default.
• W3083576952 hasConcept C144024400 @default.
• W3083576952 hasConcept C154945302 @default.
• W3083576952 hasConcept C17744445 @default.
• W3083576952 hasConcept C185798385 @default.
• W3083576952 hasConcept C19165224 @default.
• W3083576952 hasConcept C191795146 @default.
• W3083576952 hasConcept C199360897 @default.
• W3083576952 hasConcept C199539241 @default.
• W3083576952 hasConcept C205649164 @default.
• W3083576952 hasConcept C22324862 @default.
• W3083576952 hasConcept C2779803651 @default.
• W3083576952 hasConcept C33923547 @default.
• W3083576952 hasConcept C39890363 @default.
• W3083576952 hasConcept C41008148 @default.
• W3083576952 hasConcept C76155785 @default.
• W3083576952 hasConcept C8642999 @default.
• W3083576952 hasConcept C94915269 @default.
• W3083576952 hasConceptScore W3083576952C11413529 @default.
• W3083576952 hasConceptScore W3083576952C114466953 @default.
• W3083576952 hasConceptScore W3083576952C119857082 @default.
• W3083576952 hasConceptScore W3083576952C13280743 @default.
• W3083576952 hasConceptScore W3083576952C134306372 @default.
• W3083576952 hasConceptScore W3083576952C136886441 @default.
• W3083576952 hasConceptScore W3083576952C144024400 @default.
• W3083576952 hasConceptScore W3083576952C154945302 @default.
• W3083576952 hasConceptScore W3083576952C17744445 @default.
• W3083576952 hasConceptScore W3083576952C185798385 @default.
• W3083576952 hasConceptScore W3083576952C19165224 @default.
• W3083576952 hasConceptScore W3083576952C191795146 @default.
• W3083576952 hasConceptScore W3083576952C199360897 @default.
• W3083576952 hasConceptScore W3083576952C199539241 @default.
• W3083576952 hasConceptScore W3083576952C205649164 @default.
• W3083576952 hasConceptScore W3083576952C22324862 @default.
• W3083576952 hasConceptScore W3083576952C2779803651 @default.
• W3083576952 hasConceptScore W3083576952C33923547 @default.
• W3083576952 hasConceptScore W3083576952C39890363 @default.
• W3083576952 hasConceptScore W3083576952C41008148 @default.
• W3083576952 hasConceptScore W3083576952C76155785 @default.
• W3083576952 hasConceptScore W3083576952C8642999 @default.

• next