FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3204181204> ?p ?o ?g. }

• W3204181204 abstract "Overparametrized Deep Neural Networks (DNNs) often achieve astounding performances, but may potentially result in severe generalization error. Recently, the relation between the sharpness of the loss landscape and the generalization error has been established by Foret et al. (2020), in which the Sharpness Aware Minimizer (SAM) was proposed to mitigate the degradation of the generalization. Unfortunately, SAM s computational cost is roughly double that of base optimizers, such as Stochastic Gradient Descent (SGD). This paper thus proposes Efficient Sharpness Aware Minimizer (ESAM), which boosts SAM s efficiency at no cost to its generalization performance. ESAM includes two novel and efficient training strategies-StochasticWeight Perturbation and Sharpness-Sensitive Data Selection. In the former, the sharpness measure is approximated by perturbing a stochastically chosen set of weights in each iteration; in the latter, the SAM loss is optimized using only a judiciously selected subset of data that is sensitive to the sharpness. We provide theoretical explanations as to why these strategies perform well. We also show, via extensive experiments on the CIFAR and ImageNet datasets, that ESAM enhances the efficiency over SAM from requiring 100% extra computations to 40% vis-a-vis base optimizers, while test accuracies are preserved or even improved." @default.
• W3204181204 created "2021-10-11" @default.
• W3204181204 creator A5001599295 @default.
• W3204181204 creator A5033884558 @default.
• W3204181204 creator A5035554947 @default.
• W3204181204 creator A5045125183 @default.
• W3204181204 creator A5058345431 @default.
• W3204181204 creator A5071743450 @default.
• W3204181204 creator A5086517854 @default.
• W3204181204 date "2021-10-06" @default.
• W3204181204 modified "2023-09-23" @default.
• W3204181204 title "Efficient Sharpness-aware Minimization for Improved Training of Neural Networks" @default.
• W3204181204 cites W1522301498 @default.
• W3204181204 cites W2108598243 @default.
• W3204181204 cites W2143163787 @default.
• W3204181204 cites W2194775991 @default.
• W3204181204 cites W2401231614 @default.
• W3204181204 cites W2518108298 @default.
• W3204181204 cites W2523060838 @default.
• W3204181204 cites W2552194003 @default.
• W3204181204 cites W2566079294 @default.
• W3204181204 cites W2604117713 @default.
• W3204181204 cites W2605372163 @default.
• W3204181204 cites W2732724430 @default.
• W3204181204 cites W2777662428 @default.
• W3204181204 cites W2962971773 @default.
• W3204181204 cites W2963026800 @default.
• W3204181204 cites W2963113370 @default.
• W3204181204 cites W2964153283 @default.
• W3204181204 cites W2992043355 @default.
• W3204181204 cites W3035226846 @default.
• W3204181204 cites W3091401866 @default.
• W3204181204 cites W3092873005 @default.
• W3204181204 cites W3126337491 @default.
• W3204181204 cites W3129626847 @default.
• W3204181204 cites W3137430387 @default.
• W3204181204 cites W3165478011 @default.
• W3204181204 cites W3166792421 @default.
• W3204181204 cites W3168489096 @default.
• W3204181204 cites W3169250519 @default.
• W3204181204 cites W3173631098 @default.
• W3204181204 cites W3180562345 @default.
• W3204181204 cites W3196660197 @default.
• W3204181204 doi "https://doi.org/10.48550/arxiv.2110.03141" @default.
• W3204181204 hasPublicationYear "2021" @default.
• W3204181204 type Work @default.
• W3204181204 sameAs 3204181204 @default.
• W3204181204 citedByCount "1" @default.
• W3204181204 countsByYear W32041812042021 @default.
• W3204181204 crossrefType "posted-content" @default.
• W3204181204 hasAuthorship W3204181204A5001599295 @default.
• W3204181204 hasAuthorship W3204181204A5033884558 @default.
• W3204181204 hasAuthorship W3204181204A5035554947 @default.
• W3204181204 hasAuthorship W3204181204A5045125183 @default.
• W3204181204 hasAuthorship W3204181204A5058345431 @default.
• W3204181204 hasAuthorship W3204181204A5071743450 @default.
• W3204181204 hasAuthorship W3204181204A5086517854 @default.
• W3204181204 hasBestOaLocation W32041812041 @default.
• W3204181204 hasConcept C11413529 @default.
• W3204181204 hasConcept C117765406 @default.
• W3204181204 hasConcept C134306372 @default.
• W3204181204 hasConcept C147764199 @default.
• W3204181204 hasConcept C153258448 @default.
• W3204181204 hasConcept C154945302 @default.
• W3204181204 hasConcept C169903167 @default.
• W3204181204 hasConcept C177148314 @default.
• W3204181204 hasConcept C177264268 @default.
• W3204181204 hasConcept C199360897 @default.
• W3204181204 hasConcept C206688291 @default.
• W3204181204 hasConcept C2984842247 @default.
• W3204181204 hasConcept C33923547 @default.
• W3204181204 hasConcept C41008148 @default.
• W3204181204 hasConcept C42058472 @default.
• W3204181204 hasConcept C45374587 @default.
• W3204181204 hasConcept C50644808 @default.
• W3204181204 hasConcept C51632099 @default.
• W3204181204 hasConcept C5465570 @default.
• W3204181204 hasConceptScore W3204181204C11413529 @default.
• W3204181204 hasConceptScore W3204181204C117765406 @default.
• W3204181204 hasConceptScore W3204181204C134306372 @default.
• W3204181204 hasConceptScore W3204181204C147764199 @default.
• W3204181204 hasConceptScore W3204181204C153258448 @default.
• W3204181204 hasConceptScore W3204181204C154945302 @default.
• W3204181204 hasConceptScore W3204181204C169903167 @default.
• W3204181204 hasConceptScore W3204181204C177148314 @default.
• W3204181204 hasConceptScore W3204181204C177264268 @default.
• W3204181204 hasConceptScore W3204181204C199360897 @default.
• W3204181204 hasConceptScore W3204181204C206688291 @default.
• W3204181204 hasConceptScore W3204181204C2984842247 @default.
• W3204181204 hasConceptScore W3204181204C33923547 @default.
• W3204181204 hasConceptScore W3204181204C41008148 @default.
• W3204181204 hasConceptScore W3204181204C42058472 @default.
• W3204181204 hasConceptScore W3204181204C45374587 @default.
• W3204181204 hasConceptScore W3204181204C50644808 @default.
• W3204181204 hasConceptScore W3204181204C51632099 @default.
• W3204181204 hasConceptScore W3204181204C5465570 @default.
• W3204181204 hasLocation W32041812041 @default.
• W3204181204 hasLocation W32041812042 @default.
• W3204181204 hasOpenAccess W3204181204 @default.
• W3204181204 hasPrimaryLocation W32041812041 @default.

• next