Matches in SemOpenAlex for { <https://semopenalex.org/work/W4306175576> ?p ?o ?g. }
Showing items 1 to 59 of
59
with 100 items per page.
- W4306175576 abstract "In humans and animals, curriculum learning -- presenting data in a curated order - is critical to rapid learning and effective pedagogy. Yet in machine learning, curricula are not widely used and empirically often yield only moderate benefits. This stark difference in the importance of curriculum raises a fundamental theoretical question: when and why does curriculum learning help? In this work, we analyse a prototypical neural network model of curriculum learning in the high-dimensional limit, employing statistical physics methods. Curricula could in principle change both the learning speed and asymptotic performance of a model. To study the former, we provide an exact description of the online learning setting, confirming the long-standing experimental observation that curricula can modestly speed up learning. To study the latter, we derive performance in a batch learning setting, in which a network trains to convergence in successive phases of learning on dataset slices of varying difficulty. With standard training losses, curriculum does not provide generalisation benefit, in line with empirical observations. However, we show that by connecting different learning phases through simple Gaussian priors, curriculum can yield a large improvement in test performance. Taken together, our reduced analytical descriptions help reconcile apparently conflicting empirical results and trace regimes where curriculum learning yields the largest gains. More broadly, our results suggest that fully exploiting a curriculum may require explicit changes to the loss function at curriculum boundaries." @default.
- W4306175576 created "2022-10-14" @default.
- W4306175576 creator A5005071277 @default.
- W4306175576 creator A5028694063 @default.
- W4306175576 creator A5083084348 @default.
- W4306175576 date "2021-06-15" @default.
- W4306175576 modified "2023-09-30" @default.
- W4306175576 title "An Analytical Theory of Curriculum Learning in Teacher-Student Networks" @default.
- W4306175576 doi "https://doi.org/10.48550/arxiv.2106.08068" @default.
- W4306175576 hasPublicationYear "2021" @default.
- W4306175576 type Work @default.
- W4306175576 citedByCount "0" @default.
- W4306175576 crossrefType "posted-content" @default.
- W4306175576 hasAuthorship W4306175576A5005071277 @default.
- W4306175576 hasAuthorship W4306175576A5028694063 @default.
- W4306175576 hasAuthorship W4306175576A5083084348 @default.
- W4306175576 hasBestOaLocation W43061755761 @default.
- W4306175576 hasConcept C119857082 @default.
- W4306175576 hasConcept C143266803 @default.
- W4306175576 hasConcept C145129785 @default.
- W4306175576 hasConcept C145420912 @default.
- W4306175576 hasConcept C154945302 @default.
- W4306175576 hasConcept C15744967 @default.
- W4306175576 hasConcept C185020186 @default.
- W4306175576 hasConcept C19417346 @default.
- W4306175576 hasConcept C2779011557 @default.
- W4306175576 hasConcept C33923547 @default.
- W4306175576 hasConcept C37228920 @default.
- W4306175576 hasConcept C41008148 @default.
- W4306175576 hasConcept C47177190 @default.
- W4306175576 hasConceptScore W4306175576C119857082 @default.
- W4306175576 hasConceptScore W4306175576C143266803 @default.
- W4306175576 hasConceptScore W4306175576C145129785 @default.
- W4306175576 hasConceptScore W4306175576C145420912 @default.
- W4306175576 hasConceptScore W4306175576C154945302 @default.
- W4306175576 hasConceptScore W4306175576C15744967 @default.
- W4306175576 hasConceptScore W4306175576C185020186 @default.
- W4306175576 hasConceptScore W4306175576C19417346 @default.
- W4306175576 hasConceptScore W4306175576C2779011557 @default.
- W4306175576 hasConceptScore W4306175576C33923547 @default.
- W4306175576 hasConceptScore W4306175576C37228920 @default.
- W4306175576 hasConceptScore W4306175576C41008148 @default.
- W4306175576 hasConceptScore W4306175576C47177190 @default.
- W4306175576 hasLocation W43061755761 @default.
- W4306175576 hasOpenAccess W4306175576 @default.
- W4306175576 hasPrimaryLocation W43061755761 @default.
- W4306175576 hasRelatedWork W1988683062 @default.
- W4306175576 hasRelatedWork W2015868954 @default.
- W4306175576 hasRelatedWork W2071049092 @default.
- W4306175576 hasRelatedWork W2300324000 @default.
- W4306175576 hasRelatedWork W2351915757 @default.
- W4306175576 hasRelatedWork W2369600742 @default.
- W4306175576 hasRelatedWork W2387670216 @default.
- W4306175576 hasRelatedWork W2392836370 @default.
- W4306175576 hasRelatedWork W329765334 @default.
- W4306175576 hasRelatedWork W2889622147 @default.
- W4306175576 isParatext "false" @default.
- W4306175576 isRetracted "false" @default.
- W4306175576 workType "article" @default.