Matches in SemOpenAlex for { <https://semopenalex.org/work/W3112617038> ?p ?o ?g. }
- W3112617038 abstract "Inspired by human learning, researchers have proposed ordering examples during training based on their difficulty. Both curriculum learning, exposing a network to easier examples early in training, and anti-curriculum learning, showing the most difficult examples first, have been suggested as improvements to the standard i.i.d. training. In this work, we set out to investigate the relative benefits of ordered learning. We first investigate the emph{implicit curricula} resulting from architectural and optimization bias and find that samples are learned in a highly consistent order. Next, to quantify the benefit of emph{explicit curricula}, we conduct extensive experiments over thousands of orderings spanning three kinds of learning: curriculum, anti-curriculum, and random-curriculum -- in which the size of the training dataset is dynamically increased over time, but the examples are randomly ordered. We find that for standard benchmark datasets, curricula have only marginal benefits, and that randomly ordered samples perform as well or better than curricula and anti-curricula, suggesting that any benefit is entirely due to the dynamic training set size. Inspired by common use cases of curriculum learning in practice, we investigate the role of limited training time budget and noisy data in the success of curriculum learning. Our experiments demonstrate that curriculum, but not anti-curriculum can indeed improve the performance either with limited training time budget or in existence of noisy data." @default.
- W3112617038 created "2020-12-21" @default.
- W3112617038 creator A5000701575 @default.
- W3112617038 creator A5010086803 @default.
- W3112617038 creator A5035369215 @default.
- W3112617038 date "2020-12-05" @default.
- W3112617038 modified "2023-09-27" @default.
- W3112617038 title "When Do Curricula Work" @default.
- W3112617038 cites W123476658 @default.
- W3112617038 cites W12634471 @default.
- W3112617038 cites W1581407678 @default.
- W3112617038 cites W1686810756 @default.
- W3112617038 cites W1836465849 @default.
- W3112617038 cites W1924762813 @default.
- W3112617038 cites W1966876874 @default.
- W3112617038 cites W2034368206 @default.
- W3112617038 cites W2060277733 @default.
- W3112617038 cites W2113290770 @default.
- W3112617038 cites W2132984949 @default.
- W3112617038 cites W2148330858 @default.
- W3112617038 cites W2151834591 @default.
- W3112617038 cites W2162287622 @default.
- W3112617038 cites W2166493072 @default.
- W3112617038 cites W2194775991 @default.
- W3112617038 cites W2256388387 @default.
- W3112617038 cites W2296073425 @default.
- W3112617038 cites W2554600978 @default.
- W3112617038 cites W2560647685 @default.
- W3112617038 cites W2743200750 @default.
- W3112617038 cites W2804071063 @default.
- W3112617038 cites W2887842788 @default.
- W3112617038 cites W2892283076 @default.
- W3112617038 cites W2897222461 @default.
- W3112617038 cites W2903996579 @default.
- W3112617038 cites W2923622379 @default.
- W3112617038 cites W2951954464 @default.
- W3112617038 cites W2955425717 @default.
- W3112617038 cites W2961199495 @default.
- W3112617038 cites W2962762068 @default.
- W3112617038 cites W2962800562 @default.
- W3112617038 cites W2963003887 @default.
- W3112617038 cites W2963028646 @default.
- W3112617038 cites W2963096987 @default.
- W3112617038 cites W2963262394 @default.
- W3112617038 cites W2963371670 @default.
- W3112617038 cites W2963433607 @default.
- W3112617038 cites W2963446712 @default.
- W3112617038 cites W2964121744 @default.
- W3112617038 cites W2964122237 @default.
- W3112617038 cites W2964137095 @default.
- W3112617038 cites W2964241437 @default.
- W3112617038 cites W2964327384 @default.
- W3112617038 cites W2964790801 @default.
- W3112617038 cites W2971120029 @default.
- W3112617038 cites W2972758308 @default.
- W3112617038 cites W2974929660 @default.
- W3112617038 cites W2978290191 @default.
- W3112617038 cites W2989581875 @default.
- W3112617038 cites W2991349609 @default.
- W3112617038 cites W2995812976 @default.
- W3112617038 cites W2996626261 @default.
- W3112617038 cites W3010512657 @default.
- W3112617038 cites W3030163527 @default.
- W3112617038 cites W3035693354 @default.
- W3112617038 cites W3035702670 @default.
- W3112617038 cites W3037299979 @default.
- W3112617038 cites W3038110170 @default.
- W3112617038 cites W3043058418 @default.
- W3112617038 cites W3082274269 @default.
- W3112617038 cites W3087450817 @default.
- W3112617038 cites W3118608800 @default.
- W3112617038 hasPublicationYear "2020" @default.
- W3112617038 type Work @default.
- W3112617038 sameAs 3112617038 @default.
- W3112617038 citedByCount "6" @default.
- W3112617038 countsByYear W31126170382021 @default.
- W3112617038 crossrefType "posted-content" @default.
- W3112617038 hasAuthorship W3112617038A5000701575 @default.
- W3112617038 hasAuthorship W3112617038A5010086803 @default.
- W3112617038 hasAuthorship W3112617038A5035369215 @default.
- W3112617038 hasConcept C119857082 @default.
- W3112617038 hasConcept C121332964 @default.
- W3112617038 hasConcept C127413603 @default.
- W3112617038 hasConcept C13280743 @default.
- W3112617038 hasConcept C143266803 @default.
- W3112617038 hasConcept C145129785 @default.
- W3112617038 hasConcept C145420912 @default.
- W3112617038 hasConcept C153294291 @default.
- W3112617038 hasConcept C154945302 @default.
- W3112617038 hasConcept C15744967 @default.
- W3112617038 hasConcept C177264268 @default.
- W3112617038 hasConcept C185020186 @default.
- W3112617038 hasConcept C185798385 @default.
- W3112617038 hasConcept C18762648 @default.
- W3112617038 hasConcept C19417346 @default.
- W3112617038 hasConcept C199360897 @default.
- W3112617038 hasConcept C205649164 @default.
- W3112617038 hasConcept C2777211547 @default.
- W3112617038 hasConcept C2779011557 @default.
- W3112617038 hasConcept C41008148 @default.