FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3170383702> ?p ?o ?g. }

• W3170383702 endingPage "114014" @default.
• W3170383702 startingPage "114014" @default.
• W3170383702 abstract "In animals and humans, curriculum learning-presenting data in a curated order-is critical to rapid learning and effective pedagogy. A long history of experiments has demonstrated the impact of curricula in a variety of animals but, despite its ubiquitous presence, a theoretical understanding of the phenomenon is still lacking. Surprisingly, in contrast to animal learning, curricula strategies are not widely used in machine learning and recent simulation studies reach the conclusion that curricula are moderately effective or even ineffective in most cases. This stark difference in the importance of curriculum raises a fundamental theoretical question: when and why does curriculum learning help? In this work, we analyse a prototypical neural network model of curriculum learning in the high-dimensional limit, employing statistical physics methods. We study a task in which a sparse set of informative features are embedded amidst a large set of noisy features. We analytically derive average learning trajectories for simple neural networks on this task, which establish a clear speed benefit for curriculum learning in the online setting. However, when training experiences can be stored and replayed (for instance, during sleep), the advantage of curriculum in standard neural networks disappears, in line with observations from the deep learning literature. Inspired by synaptic consolidation techniques developed to combat catastrophic forgetting, we propose curriculum-aware algorithms that consolidate synapses at curriculum change points and investigate whether this can boost the benefits of curricula. We derive generalisation performance as a function of consolidation strength (implemented as an L2 regularisation/elastic coupling connecting learning phases), and show that curriculum-aware algorithms can yield a large improvement in test performance. Our reduced analytical descriptions help reconcile apparently conflicting empirical results, trace regimes where curriculum learning yields the largest gains, and provide experimentally-accessible predictions for the impact of task parameters on curriculum benefits. More broadly, our results suggest that fully exploiting a curriculum may require explicit adjustments in the loss." @default.
• W3170383702 created "2021-06-22" @default.
• W3170383702 creator A5005071277 @default.
• W3170383702 creator A5011428379 @default.
• W3170383702 creator A5028694063 @default.
• W3170383702 date "2022-11-01" @default.
• W3170383702 modified "2023-10-14" @default.
• W3170383702 title "An analytical theory of curriculum learning in teacher–student networks*" @default.
• W3170383702 cites W1966876874 @default.
• W3170383702 cites W1979344292 @default.
• W3170383702 cites W1984862412 @default.
• W3170383702 cites W2007347635 @default.
• W3170383702 cites W2009370030 @default.
• W3170383702 cites W2017479508 @default.
• W3170383702 cites W2025032307 @default.
• W3170383702 cites W2037985840 @default.
• W3170383702 cites W2042318263 @default.
• W3170383702 cites W2052007141 @default.
• W3170383702 cites W2058857604 @default.
• W3170383702 cites W2060062032 @default.
• W3170383702 cites W2084786824 @default.
• W3170383702 cites W2091150000 @default.
• W3170383702 cites W2116385713 @default.
• W3170383702 cites W2146200913 @default.
• W3170383702 cites W2151781750 @default.
• W3170383702 cites W2151834591 @default.
• W3170383702 cites W2553608650 @default.
• W3170383702 cites W2560647685 @default.
• W3170383702 cites W2737492962 @default.
• W3170383702 cites W2771424446 @default.
• W3170383702 cites W2893892086 @default.
• W3170383702 cites W2978241382 @default.
• W3170383702 cites W2988893955 @default.
• W3170383702 cites W2989931884 @default.
• W3170383702 cites W2996141621 @default.
• W3170383702 cites W3083720136 @default.
• W3170383702 cites W3162611316 @default.
• W3170383702 doi "https://doi.org/10.1088/1742-5468/ac9b3c" @default.
• W3170383702 hasPubMedId "https://pubmed.ncbi.nlm.nih.gov/37817944" @default.
• W3170383702 hasPublicationYear "2022" @default.
• W3170383702 type Work @default.
• W3170383702 sameAs 3170383702 @default.
• W3170383702 citedByCount "0" @default.
• W3170383702 crossrefType "journal-article" @default.
• W3170383702 hasAuthorship W3170383702A5005071277 @default.
• W3170383702 hasAuthorship W3170383702A5011428379 @default.
• W3170383702 hasAuthorship W3170383702A5028694063 @default.
• W3170383702 hasBestOaLocation W31703837021 @default.
• W3170383702 hasConcept C108583219 @default.
• W3170383702 hasConcept C119857082 @default.
• W3170383702 hasConcept C143266803 @default.
• W3170383702 hasConcept C145129785 @default.
• W3170383702 hasConcept C145420912 @default.
• W3170383702 hasConcept C154945302 @default.
• W3170383702 hasConcept C15744967 @default.
• W3170383702 hasConcept C177264268 @default.
• W3170383702 hasConcept C180747234 @default.
• W3170383702 hasConcept C185020186 @default.
• W3170383702 hasConcept C19417346 @default.
• W3170383702 hasConcept C199360897 @default.
• W3170383702 hasConcept C2779011557 @default.
• W3170383702 hasConcept C41008148 @default.
• W3170383702 hasConcept C47177190 @default.
• W3170383702 hasConcept C50644808 @default.
• W3170383702 hasConcept C7149132 @default.
• W3170383702 hasConceptScore W3170383702C108583219 @default.
• W3170383702 hasConceptScore W3170383702C119857082 @default.
• W3170383702 hasConceptScore W3170383702C143266803 @default.
• W3170383702 hasConceptScore W3170383702C145129785 @default.
• W3170383702 hasConceptScore W3170383702C145420912 @default.
• W3170383702 hasConceptScore W3170383702C154945302 @default.
• W3170383702 hasConceptScore W3170383702C15744967 @default.
• W3170383702 hasConceptScore W3170383702C177264268 @default.
• W3170383702 hasConceptScore W3170383702C180747234 @default.
• W3170383702 hasConceptScore W3170383702C185020186 @default.
• W3170383702 hasConceptScore W3170383702C19417346 @default.
• W3170383702 hasConceptScore W3170383702C199360897 @default.
• W3170383702 hasConceptScore W3170383702C2779011557 @default.
• W3170383702 hasConceptScore W3170383702C41008148 @default.
• W3170383702 hasConceptScore W3170383702C47177190 @default.
• W3170383702 hasConceptScore W3170383702C50644808 @default.
• W3170383702 hasConceptScore W3170383702C7149132 @default.
• W3170383702 hasFunder F4320307874 @default.
• W3170383702 hasIssue "11" @default.
• W3170383702 hasLocation W31703837021 @default.
• W3170383702 hasLocation W31703837022 @default.
• W3170383702 hasLocation W31703837023 @default.
• W3170383702 hasLocation W31703837024 @default.
• W3170383702 hasLocation W31703837025 @default.
• W3170383702 hasOpenAccess W3170383702 @default.
• W3170383702 hasPrimaryLocation W31703837021 @default.
• W3170383702 hasRelatedWork W2347879670 @default.
• W3170383702 hasRelatedWork W2363525455 @default.
• W3170383702 hasRelatedWork W2367341516 @default.
• W3170383702 hasRelatedWork W2373380871 @default.
• W3170383702 hasRelatedWork W2377218246 @default.
• W3170383702 hasRelatedWork W2800551416 @default.
• W3170383702 hasRelatedWork W2889128632 @default.

• next