Matches in SemOpenAlex for { <https://semopenalex.org/work/W3124635189> ?p ?o ?g. }
Showing items 1 to 87 of
87
with 100 items per page.
- W3124635189 abstract "We study the multiple manifold problem, a binary classification task modeled on applications in machine vision, in which a deep fully-connected network is trained to separate two low-dimensional submanifolds of the unit sphere. We provide an analysis of the one-dimensional case, proving for a simple manifold configuration that when the network depth L is large relative to certain geometric and statistical properties of the data, the network width n grows as a sufficiently large polynomial in L, and the number of i.i.d. samples from the manifolds is polynomial in L, randomly-initialized gradient descent rapidly learns to classify the two manifolds perfectly with high probability. Our analysis demonstrates concrete benefits of depth and width in the context of a practically-motivated model problem: the depth acts as a fitting resource, with larger depths corresponding to smoother networks that can more readily separate the class manifolds, and the width acts as a statistical resource, enabling concentration of the randomly-initialized network and its gradients. The argument centers around the neural tangent of Jacot et al. and its role in the nonasymptotic analysis of training overparameterized networks; to this literature, we contribute essentially optimal rates of concentration for the tangent kernel of deep fully-connected ReLU networks, requiring width n≥Lpoly(d0) to achieve uniform concentration of the initial kernel over a d0-dimensional submanifold of the unit sphere Sn0−1, and a nonasymptotic framework for establishing generalization of networks trained in the NTK regime with structured data. The proof makes heavy use of martingale concentration to optimally treat statistical dependencies across layers of the initial random network. This approach should be of use in establishing similar results for other network architectures." @default.
- W3124635189 created "2021-02-01" @default.
- W3124635189 creator A5017296839 @default.
- W3124635189 creator A5019610791 @default.
- W3124635189 creator A5026684245 @default.
- W3124635189 date "2021-05-03" @default.
- W3124635189 modified "2023-09-24" @default.
- W3124635189 title "Deep Networks and the Multiple Manifold Problem" @default.
- W3124635189 hasPublicationYear "2021" @default.
- W3124635189 type Work @default.
- W3124635189 sameAs 3124635189 @default.
- W3124635189 citedByCount "9" @default.
- W3124635189 countsByYear W31246351892020 @default.
- W3124635189 countsByYear W31246351892021 @default.
- W3124635189 crossrefType "proceedings-article" @default.
- W3124635189 hasAuthorship W3124635189A5017296839 @default.
- W3124635189 hasAuthorship W3124635189A5019610791 @default.
- W3124635189 hasAuthorship W3124635189A5026684245 @default.
- W3124635189 hasConcept C11413529 @default.
- W3124635189 hasConcept C114614502 @default.
- W3124635189 hasConcept C127413603 @default.
- W3124635189 hasConcept C134306372 @default.
- W3124635189 hasConcept C138187205 @default.
- W3124635189 hasConcept C151300846 @default.
- W3124635189 hasConcept C151730666 @default.
- W3124635189 hasConcept C153258448 @default.
- W3124635189 hasConcept C154945302 @default.
- W3124635189 hasConcept C177148314 @default.
- W3124635189 hasConcept C184720557 @default.
- W3124635189 hasConcept C2524010 @default.
- W3124635189 hasConcept C2779343474 @default.
- W3124635189 hasConcept C33923547 @default.
- W3124635189 hasConcept C41008148 @default.
- W3124635189 hasConcept C50644808 @default.
- W3124635189 hasConcept C529865628 @default.
- W3124635189 hasConcept C74193536 @default.
- W3124635189 hasConcept C78519656 @default.
- W3124635189 hasConcept C86803240 @default.
- W3124635189 hasConcept C90119067 @default.
- W3124635189 hasConceptScore W3124635189C11413529 @default.
- W3124635189 hasConceptScore W3124635189C114614502 @default.
- W3124635189 hasConceptScore W3124635189C127413603 @default.
- W3124635189 hasConceptScore W3124635189C134306372 @default.
- W3124635189 hasConceptScore W3124635189C138187205 @default.
- W3124635189 hasConceptScore W3124635189C151300846 @default.
- W3124635189 hasConceptScore W3124635189C151730666 @default.
- W3124635189 hasConceptScore W3124635189C153258448 @default.
- W3124635189 hasConceptScore W3124635189C154945302 @default.
- W3124635189 hasConceptScore W3124635189C177148314 @default.
- W3124635189 hasConceptScore W3124635189C184720557 @default.
- W3124635189 hasConceptScore W3124635189C2524010 @default.
- W3124635189 hasConceptScore W3124635189C2779343474 @default.
- W3124635189 hasConceptScore W3124635189C33923547 @default.
- W3124635189 hasConceptScore W3124635189C41008148 @default.
- W3124635189 hasConceptScore W3124635189C50644808 @default.
- W3124635189 hasConceptScore W3124635189C529865628 @default.
- W3124635189 hasConceptScore W3124635189C74193536 @default.
- W3124635189 hasConceptScore W3124635189C78519656 @default.
- W3124635189 hasConceptScore W3124635189C86803240 @default.
- W3124635189 hasConceptScore W3124635189C90119067 @default.
- W3124635189 hasLocation W31246351891 @default.
- W3124635189 hasOpenAccess W3124635189 @default.
- W3124635189 hasPrimaryLocation W31246351891 @default.
- W3124635189 hasRelatedWork W2267573953 @default.
- W3124635189 hasRelatedWork W2517976956 @default.
- W3124635189 hasRelatedWork W2803568638 @default.
- W3124635189 hasRelatedWork W2809090039 @default.
- W3124635189 hasRelatedWork W2915105860 @default.
- W3124635189 hasRelatedWork W2952204734 @default.
- W3124635189 hasRelatedWork W2963146412 @default.
- W3124635189 hasRelatedWork W2965067322 @default.
- W3124635189 hasRelatedWork W2970004405 @default.
- W3124635189 hasRelatedWork W3006926186 @default.
- W3124635189 hasRelatedWork W3007103516 @default.
- W3124635189 hasRelatedWork W3033701124 @default.
- W3124635189 hasRelatedWork W3039653901 @default.
- W3124635189 hasRelatedWork W3081188487 @default.
- W3124635189 hasRelatedWork W3089846728 @default.
- W3124635189 hasRelatedWork W3131682106 @default.
- W3124635189 hasRelatedWork W3168039702 @default.
- W3124635189 hasRelatedWork W3181384422 @default.
- W3124635189 hasRelatedWork W3190957407 @default.
- W3124635189 hasRelatedWork W3202429688 @default.
- W3124635189 isParatext "false" @default.
- W3124635189 isRetracted "false" @default.
- W3124635189 magId "3124635189" @default.
- W3124635189 workType "article" @default.