Matches in SemOpenAlex for { <https://semopenalex.org/work/W3037648364> ?p ?o ?g. }
- W3037648364 abstract "Understanding the impact of data structure on the computational tractability of learning is a key challenge for the theory of neural networks. Many theoretical works do not explicitly model training data, or assume that inputs are drawn component-wise independently from some simple probability distribution. Here, we go beyond this simple paradigm by studying the performance of neural networks trained on data drawn from pre-trained generative models. This is possible due to a Gaussian equivalence stating that the key metrics of interest, such as the training and test errors, can be fully captured by an appropriately chosen Gaussian model. We provide three strands of rigorous, analytical and numerical evidence corroborating this equivalence. First, we establish rigorous conditions for the Gaussian equivalence to hold in the case of single-layer generative models, as well as deterministic rates for convergence in distribution. Second, we leverage this equivalence to derive a closed set of equations describing the generalisation performance of two widely studied machine learning problems: two-layer neural networks trained using one-pass stochastic gradient descent, and full-batch pre-learned features or kernel methods. Finally, we perform experiments demonstrating how our theory applies to deep, pre-trained generative models. These results open a viable path to the theoretical study of machine learning models with realistic data." @default.
- W3037648364 created "2020-07-02" @default.
- W3037648364 creator A5037000437 @default.
- W3037648364 creator A5057039281 @default.
- W3037648364 creator A5068236230 @default.
- W3037648364 creator A5084396713 @default.
- W3037648364 creator A5089268172 @default.
- W3037648364 date "2020-11-01" @default.
- W3037648364 modified "2023-10-17" @default.
- W3037648364 title "The Gaussian equivalence of generative models for learning with two-layer neural networks" @default.
- W3037648364 cites W1567512734 @default.
- W3037648364 cites W1583912456 @default.
- W3037648364 cites W1836465849 @default.
- W3037648364 cites W1891181203 @default.
- W3037648364 cites W1944672 @default.
- W3037648364 cites W1959608418 @default.
- W3037648364 cites W1964862779 @default.
- W3037648364 cites W1968104963 @default.
- W3037648364 cites W1985890860 @default.
- W3037648364 cites W1989730800 @default.
- W3037648364 cites W1992774725 @default.
- W3037648364 cites W1995842804 @default.
- W3037648364 cites W2006997461 @default.
- W3037648364 cites W2014557467 @default.
- W3037648364 cites W2037985840 @default.
- W3037648364 cites W2042318263 @default.
- W3037648364 cites W2066424095 @default.
- W3037648364 cites W2070792261 @default.
- W3037648364 cites W2072072671 @default.
- W3037648364 cites W2078626246 @default.
- W3037648364 cites W2081988173 @default.
- W3037648364 cites W2090614046 @default.
- W3037648364 cites W2091987367 @default.
- W3037648364 cites W2099471712 @default.
- W3037648364 cites W2104067967 @default.
- W3037648364 cites W2123395972 @default.
- W3037648364 cites W2125389028 @default.
- W3037648364 cites W2132657058 @default.
- W3037648364 cites W2135825502 @default.
- W3037648364 cites W2144902422 @default.
- W3037648364 cites W2150872430 @default.
- W3037648364 cites W2151781750 @default.
- W3037648364 cites W2154579312 @default.
- W3037648364 cites W2156909104 @default.
- W3037648364 cites W2161278885 @default.
- W3037648364 cites W2490183153 @default.
- W3037648364 cites W2553079770 @default.
- W3037648364 cites W2562573853 @default.
- W3037648364 cites W2610857016 @default.
- W3037648364 cites W2617020621 @default.
- W3037648364 cites W2752851182 @default.
- W3037648364 cites W2763894180 @default.
- W3037648364 cites W2766572821 @default.
- W3037648364 cites W2785885194 @default.
- W3037648364 cites W2803439868 @default.
- W3037648364 cites W2804589149 @default.
- W3037648364 cites W2809090039 @default.
- W3037648364 cites W2893749619 @default.
- W3037648364 cites W2903327037 @default.
- W3037648364 cites W2923228966 @default.
- W3037648364 cites W2924720492 @default.
- W3037648364 cites W2940025123 @default.
- W3037648364 cites W2952204734 @default.
- W3037648364 cites W2962685794 @default.
- W3037648364 cites W2962695743 @default.
- W3037648364 cites W2962767131 @default.
- W3037648364 cites W2963090522 @default.
- W3037648364 cites W2963095610 @default.
- W3037648364 cites W2963201159 @default.
- W3037648364 cites W2963341557 @default.
- W3037648364 cites W2963417959 @default.
- W3037648364 cites W2963504252 @default.
- W3037648364 cites W2963623651 @default.
- W3037648364 cites W2963650649 @default.
- W3037648364 cites W2963684088 @default.
- W3037648364 cites W2963744427 @default.
- W3037648364 cites W2963791871 @default.
- W3037648364 cites W2963885078 @default.
- W3037648364 cites W2963963733 @default.
- W3037648364 cites W2964052793 @default.
- W3037648364 cites W2964156139 @default.
- W3037648364 cites W2967536008 @default.
- W3037648364 cites W2970112944 @default.
- W3037648364 cites W2970452444 @default.
- W3037648364 cites W2970540478 @default.
- W3037648364 cites W2970721719 @default.
- W3037648364 cites W2970723196 @default.
- W3037648364 cites W2970935073 @default.
- W3037648364 cites W2970971581 @default.
- W3037648364 cites W2989169642 @default.
- W3037648364 cites W2992005611 @default.
- W3037648364 cites W2992035660 @default.
- W3037648364 cites W2996141621 @default.
- W3037648364 cites W3004639598 @default.
- W3037648364 cites W3029326706 @default.
- W3037648364 cites W3034418935 @default.
- W3037648364 cites W3034832718 @default.
- W3037648364 cites W3080149057 @default.
- W3037648364 cites W3083720136 @default.
- W3037648364 cites W3085596256 @default.