FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2995197005> ?p ?o ?g. }

• W2995197005 abstract "(Frankle & Carbin, 2019) shows that there exist winning tickets (small but critical subnetworks) for dense, randomly initialized networks, that can be trained alone to achieve comparable accuracies to the latter in a similar number of iterations. However, the identification of these winning tickets still requires the costly train-prune-retrain process, limiting their practical benefits. In this paper, we discover for the first time that the winning tickets can be identified at the very early training stage, which we term as early-bird (EB) tickets, via low-cost training schemes (e.g., early stopping and low-precision training) at large learning rates. Our finding of EB tickets is consistent with recently reported observations that the key connectivity patterns of neural networks emerge early. Furthermore, we propose a mask distance metric that can be used to identify EB tickets with low computational overhead, without needing to know the true winning tickets that emerge after the full training. Finally, we leverage the existence of EB tickets and the proposed mask distance to develop efficient training methods, which are achieved by first identifying EB tickets via low-cost schemes, and then continuing to train merely the EB tickets towards the target accuracy. Experiments based on various deep networks and datasets validate: 1) the existence of EB tickets, and the effectiveness of mask distance in efficiently identifying them; and 2) that the proposed efficient training via EB tickets can achieve up to 4.7x energy savings while maintaining comparable or even better accuracy, demonstrating a promising and easily adopted method for tackling cost-prohibitive deep network training." @default.
• W2995197005 created "2019-12-26" @default.
• W2995197005 creator A5000680423 @default.
• W2995197005 creator A5013113668 @default.
• W2995197005 creator A5019582323 @default.
• W2995197005 creator A5033632697 @default.
• W2995197005 creator A5048522863 @default.
• W2995197005 creator A5061572132 @default.
• W2995197005 creator A5065172226 @default.
• W2995197005 creator A5072713767 @default.
• W2995197005 creator A5080793133 @default.
• W2995197005 date "2020-04-30" @default.
• W2995197005 modified "2023-09-26" @default.
• W2995197005 title "Drawing Early-Bird Tickets: Toward More Efficient Training of Deep Networks" @default.
• W2995197005 cites W1686810756 @default.
• W2995197005 cites W2167215970 @default.
• W2995197005 cites W2194775991 @default.
• W2995197005 cites W2289252105 @default.
• W2995197005 cites W2302255633 @default.
• W2995197005 cites W2524428287 @default.
• W2995197005 cites W2622263826 @default.
• W2995197005 cites W2804193803 @default.
• W2995197005 cites W2808168148 @default.
• W2995197005 cites W2884711234 @default.
• W2995197005 cites W2889797931 @default.
• W2995197005 cites W2897097528 @default.
• W2995197005 cites W2911040189 @default.
• W2995197005 cites W2914483840 @default.
• W2995197005 cites W2940542208 @default.
• W2995197005 cites W2948253213 @default.
• W2995197005 cites W2951569836 @default.
• W2995197005 cites W2958636547 @default.
• W2995197005 cites W2962747323 @default.
• W2995197005 cites W2962851801 @default.
• W2995197005 cites W2962965870 @default.
• W2995197005 cites W2963000224 @default.
• W2995197005 cites W2963247446 @default.
• W2995197005 cites W2963374099 @default.
• W2995197005 cites W2963374479 @default.
• W2995197005 cites W2963709416 @default.
• W2995197005 cites W2963809228 @default.
• W2995197005 cites W2963813662 @default.
• W2995197005 cites W2964164125 @default.
• W2995197005 cites W2964233199 @default.
• W2995197005 cites W2964299589 @default.
• W2995197005 cites W2970277060 @default.
• W2995197005 hasPublicationYear "2020" @default.
• W2995197005 type Work @default.
• W2995197005 sameAs 2995197005 @default.
• W2995197005 citedByCount "56" @default.
• W2995197005 countsByYear W29951970052020 @default.
• W2995197005 countsByYear W29951970052021 @default.
• W2995197005 countsByYear W29951970052022 @default.
• W2995197005 crossrefType "proceedings-article" @default.
• W2995197005 hasAuthorship W2995197005A5000680423 @default.
• W2995197005 hasAuthorship W2995197005A5013113668 @default.
• W2995197005 hasAuthorship W2995197005A5019582323 @default.
• W2995197005 hasAuthorship W2995197005A5033632697 @default.
• W2995197005 hasAuthorship W2995197005A5048522863 @default.
• W2995197005 hasAuthorship W2995197005A5061572132 @default.
• W2995197005 hasAuthorship W2995197005A5065172226 @default.
• W2995197005 hasAuthorship W2995197005A5072713767 @default.
• W2995197005 hasAuthorship W2995197005A5080793133 @default.
• W2995197005 hasConcept C108583219 @default.
• W2995197005 hasConcept C111919701 @default.
• W2995197005 hasConcept C116834253 @default.
• W2995197005 hasConcept C119857082 @default.
• W2995197005 hasConcept C121332964 @default.
• W2995197005 hasConcept C127413603 @default.
• W2995197005 hasConcept C153083717 @default.
• W2995197005 hasConcept C153294291 @default.
• W2995197005 hasConcept C154945302 @default.
• W2995197005 hasConcept C176217482 @default.
• W2995197005 hasConcept C188198153 @default.
• W2995197005 hasConcept C21547014 @default.
• W2995197005 hasConcept C26517878 @default.
• W2995197005 hasConcept C2777211547 @default.
• W2995197005 hasConcept C2779960059 @default.
• W2995197005 hasConcept C38652104 @default.
• W2995197005 hasConcept C41008148 @default.
• W2995197005 hasConcept C59822182 @default.
• W2995197005 hasConcept C78519656 @default.
• W2995197005 hasConcept C86803240 @default.
• W2995197005 hasConcept C98045186 @default.
• W2995197005 hasConceptScore W2995197005C108583219 @default.
• W2995197005 hasConceptScore W2995197005C111919701 @default.
• W2995197005 hasConceptScore W2995197005C116834253 @default.
• W2995197005 hasConceptScore W2995197005C119857082 @default.
• W2995197005 hasConceptScore W2995197005C121332964 @default.
• W2995197005 hasConceptScore W2995197005C127413603 @default.
• W2995197005 hasConceptScore W2995197005C153083717 @default.
• W2995197005 hasConceptScore W2995197005C153294291 @default.
• W2995197005 hasConceptScore W2995197005C154945302 @default.
• W2995197005 hasConceptScore W2995197005C176217482 @default.
• W2995197005 hasConceptScore W2995197005C188198153 @default.
• W2995197005 hasConceptScore W2995197005C21547014 @default.
• W2995197005 hasConceptScore W2995197005C26517878 @default.
• W2995197005 hasConceptScore W2995197005C2777211547 @default.
• W2995197005 hasConceptScore W2995197005C2779960059 @default.
• W2995197005 hasConceptScore W2995197005C38652104 @default.

• next