FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3209329834> ?p ?o ?g. }

• W3209329834 abstract "Neural networks in the lazy training regime converge to kernel machines. Can neural networks in the rich feature learning regime learn a kernel machine with a data-dependent kernel? We demonstrate that this can indeed happen due to a phenomenon we term silent alignment, which requires that the tangent kernel of a network evolves in eigenstructure while small and before the loss appreciably decreases, and grows only in overall scale afterwards. We show that such an effect takes place in homogenous neural networks with small initialization and whitened data. We provide an analytical treatment of this effect in the linear network case. In general, we find that the kernel develops a low-rank contribution in the early phase of training, and then evolves in overall scale, yielding a function equivalent to a kernel regression solution with the final network's tangent kernel. The early spectral learning of the kernel depends on both depth and on relative learning rates in each layer. We also demonstrate that non-whitened data can weaken the silent alignment effect." @default.
• W3209329834 created "2021-11-08" @default.
• W3209329834 creator A5023195984 @default.
• W3209329834 creator A5039282308 @default.
• W3209329834 creator A5046607604 @default.
• W3209329834 date "2021-10-29" @default.
• W3209329834 modified "2023-09-27" @default.
• W3209329834 title "Neural Networks as Kernel Learners: The Silent Alignment Effect" @default.
• W3209329834 cites W115006267 @default.
• W3209329834 cites W164706946 @default.
• W3209329834 cites W1920328734 @default.
• W3209329834 cites W2125930537 @default.
• W3209329834 cites W2952204734 @default.
• W3209329834 cites W2952502547 @default.
• W3209329834 cites W2963376662 @default.
• W3209329834 cites W2964034630 @default.
• W3209329834 cites W2964121744 @default.
• W3209329834 cites W2964220724 @default.
• W3209329834 cites W2971043187 @default.
• W3209329834 cites W2994747787 @default.
• W3209329834 cites W2994922086 @default.
• W3209329834 cites W3004633050 @default.
• W3209329834 cites W3004639598 @default.
• W3209329834 cites W3034708079 @default.
• W3209329834 cites W3035216170 @default.
• W3209329834 cites W3046680711 @default.
• W3209329834 cites W3054558126 @default.
• W3209329834 cites W3101069636 @default.
• W3209329834 cites W3101581426 @default.
• W3209329834 cites W3102429474 @default.
• W3209329834 cites W3104969455 @default.
• W3209329834 cites W3106000452 @default.
• W3209329834 cites W3108435811 @default.
• W3209329834 cites W3130674728 @default.
• W3209329834 cites W3137695714 @default.
• W3209329834 cites W3153303803 @default.
• W3209329834 cites W3158469333 @default.
• W3209329834 cites W3159275413 @default.
• W3209329834 cites W3162003518 @default.
• W3209329834 cites W3169336656 @default.
• W3209329834 cites W3174815695 @default.
• W3209329834 cites W3175418406 @default.
• W3209329834 cites W3181384422 @default.
• W3209329834 hasPublicationYear "2021" @default.
• W3209329834 type Work @default.
• W3209329834 sameAs 3209329834 @default.
• W3209329834 citedByCount "1" @default.
• W3209329834 countsByYear W32093298342021 @default.
• W3209329834 crossrefType "posted-content" @default.
• W3209329834 hasAuthorship W3209329834A5023195984 @default.
• W3209329834 hasAuthorship W3209329834A5039282308 @default.
• W3209329834 hasAuthorship W3209329834A5046607604 @default.
• W3209329834 hasConcept C114466953 @default.
• W3209329834 hasConcept C114614502 @default.
• W3209329834 hasConcept C119857082 @default.
• W3209329834 hasConcept C122280245 @default.
• W3209329834 hasConcept C12267149 @default.
• W3209329834 hasConcept C134517425 @default.
• W3209329834 hasConcept C154945302 @default.
• W3209329834 hasConcept C160446489 @default.
• W3209329834 hasConcept C195699287 @default.
• W3209329834 hasConcept C199360897 @default.
• W3209329834 hasConcept C33923547 @default.
• W3209329834 hasConcept C41008148 @default.
• W3209329834 hasConcept C50644808 @default.
• W3209329834 hasConcept C55851704 @default.
• W3209329834 hasConcept C74193536 @default.
• W3209329834 hasConcept C75866337 @default.
• W3209329834 hasConceptScore W3209329834C114466953 @default.
• W3209329834 hasConceptScore W3209329834C114614502 @default.
• W3209329834 hasConceptScore W3209329834C119857082 @default.
• W3209329834 hasConceptScore W3209329834C122280245 @default.
• W3209329834 hasConceptScore W3209329834C12267149 @default.
• W3209329834 hasConceptScore W3209329834C134517425 @default.
• W3209329834 hasConceptScore W3209329834C154945302 @default.
• W3209329834 hasConceptScore W3209329834C160446489 @default.
• W3209329834 hasConceptScore W3209329834C195699287 @default.
• W3209329834 hasConceptScore W3209329834C199360897 @default.
• W3209329834 hasConceptScore W3209329834C33923547 @default.
• W3209329834 hasConceptScore W3209329834C41008148 @default.
• W3209329834 hasConceptScore W3209329834C50644808 @default.
• W3209329834 hasConceptScore W3209329834C55851704 @default.
• W3209329834 hasConceptScore W3209329834C74193536 @default.
• W3209329834 hasConceptScore W3209329834C75866337 @default.
• W3209329834 hasLocation W32093298341 @default.
• W3209329834 hasOpenAccess W3209329834 @default.
• W3209329834 hasPrimaryLocation W32093298341 @default.
• W3209329834 hasRelatedWork W2113517874 @default.
• W3209329834 hasRelatedWork W2593634001 @default.
• W3209329834 hasRelatedWork W2762534372 @default.
• W3209329834 hasRelatedWork W2785631679 @default.
• W3209329834 hasRelatedWork W2886979019 @default.
• W3209329834 hasRelatedWork W2899432656 @default.
• W3209329834 hasRelatedWork W2950850913 @default.
• W3209329834 hasRelatedWork W2970388773 @default.
• W3209329834 hasRelatedWork W2975491454 @default.
• W3209329834 hasRelatedWork W2979823801 @default.
• W3209329834 hasRelatedWork W2996318764 @default.
• W3209329834 hasRelatedWork W2996326647 @default.
• W3209329834 hasRelatedWork W3008234937 @default.

• next