Matches in SemOpenAlex for { <https://semopenalex.org/work/W4286889801> ?p ?o ?g. }
Showing items 1 to 67 of
67
with 100 items per page.
- W4286889801 abstract "Neural networks in the lazy training regime converge to kernel machines. Can neural networks in the rich feature learning regime learn a kernel machine with a data-dependent kernel? We demonstrate that this can indeed happen due to a phenomenon we term silent alignment, which requires that the tangent kernel of a network evolves in eigenstructure while small and before the loss appreciably decreases, and grows only in overall scale afterwards. We show that such an effect takes place in homogenous neural networks with small initialization and whitened data. We provide an analytical treatment of this effect in the linear network case. In general, we find that the kernel develops a low-rank contribution in the early phase of training, and then evolves in overall scale, yielding a function equivalent to a kernel regression solution with the final network's tangent kernel. The early spectral learning of the kernel depends on the depth. We also demonstrate that non-whitened data can weaken the silent alignment effect." @default.
- W4286889801 created "2022-07-25" @default.
- W4286889801 creator A5023195984 @default.
- W4286889801 creator A5039282308 @default.
- W4286889801 creator A5046607604 @default.
- W4286889801 date "2021-10-29" @default.
- W4286889801 modified "2023-10-17" @default.
- W4286889801 title "Neural Networks as Kernel Learners: The Silent Alignment Effect" @default.
- W4286889801 doi "https://doi.org/10.48550/arxiv.2111.00034" @default.
- W4286889801 hasPublicationYear "2021" @default.
- W4286889801 type Work @default.
- W4286889801 citedByCount "0" @default.
- W4286889801 crossrefType "posted-content" @default.
- W4286889801 hasAuthorship W4286889801A5023195984 @default.
- W4286889801 hasAuthorship W4286889801A5039282308 @default.
- W4286889801 hasAuthorship W4286889801A5046607604 @default.
- W4286889801 hasBestOaLocation W42868898011 @default.
- W4286889801 hasConcept C114466953 @default.
- W4286889801 hasConcept C119857082 @default.
- W4286889801 hasConcept C122280245 @default.
- W4286889801 hasConcept C12267149 @default.
- W4286889801 hasConcept C134517425 @default.
- W4286889801 hasConcept C153180895 @default.
- W4286889801 hasConcept C154945302 @default.
- W4286889801 hasConcept C160446489 @default.
- W4286889801 hasConcept C195699287 @default.
- W4286889801 hasConcept C199360897 @default.
- W4286889801 hasConcept C202444582 @default.
- W4286889801 hasConcept C27406209 @default.
- W4286889801 hasConcept C33923547 @default.
- W4286889801 hasConcept C41008148 @default.
- W4286889801 hasConcept C50644808 @default.
- W4286889801 hasConcept C74193536 @default.
- W4286889801 hasConcept C75866337 @default.
- W4286889801 hasConceptScore W4286889801C114466953 @default.
- W4286889801 hasConceptScore W4286889801C119857082 @default.
- W4286889801 hasConceptScore W4286889801C122280245 @default.
- W4286889801 hasConceptScore W4286889801C12267149 @default.
- W4286889801 hasConceptScore W4286889801C134517425 @default.
- W4286889801 hasConceptScore W4286889801C153180895 @default.
- W4286889801 hasConceptScore W4286889801C154945302 @default.
- W4286889801 hasConceptScore W4286889801C160446489 @default.
- W4286889801 hasConceptScore W4286889801C195699287 @default.
- W4286889801 hasConceptScore W4286889801C199360897 @default.
- W4286889801 hasConceptScore W4286889801C202444582 @default.
- W4286889801 hasConceptScore W4286889801C27406209 @default.
- W4286889801 hasConceptScore W4286889801C33923547 @default.
- W4286889801 hasConceptScore W4286889801C41008148 @default.
- W4286889801 hasConceptScore W4286889801C50644808 @default.
- W4286889801 hasConceptScore W4286889801C74193536 @default.
- W4286889801 hasConceptScore W4286889801C75866337 @default.
- W4286889801 hasLocation W42868898011 @default.
- W4286889801 hasOpenAccess W4286889801 @default.
- W4286889801 hasPrimaryLocation W42868898011 @default.
- W4286889801 hasRelatedWork W10975483 @default.
- W4286889801 hasRelatedWork W12366047 @default.
- W4286889801 hasRelatedWork W12729829 @default.
- W4286889801 hasRelatedWork W13771663 @default.
- W4286889801 hasRelatedWork W5557673 @default.
- W4286889801 hasRelatedWork W6229082 @default.
- W4286889801 hasRelatedWork W62477 @default.
- W4286889801 hasRelatedWork W6717794 @default.
- W4286889801 hasRelatedWork W7177339 @default.
- W4286889801 hasRelatedWork W6404606 @default.
- W4286889801 isParatext "false" @default.
- W4286889801 isRetracted "false" @default.
- W4286889801 workType "article" @default.