FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2785885246> ?p ?o ?g. }

• W2785885246 abstract "Contrary to most machine learning models, modern deep artificial neural networks typically include multiple components that contribute to regularization. Despite the fact that some (explicit) regularization techniques, such as weight decay and dropout, require costly fine-tuning of sensitive hyperparameters, the interplay between them and other elements that provide implicit regularization is not well understood yet. Shedding light upon these interactions is key to efficiently using computational resources and may contribute to solving the puzzle of generalization in deep learning. Here, we first provide formal definitions of explicit and implicit regularization that help understand essential differences between techniques. Second, we contrast data augmentation with weight decay and dropout. Our results show that visual object categorization models trained with data augmentation alone achieve the same performance or higher than models trained also with weight decay and dropout, as is common practice. We conclude that the contribution on generalization of weight decay and dropout is not only superfluous when sufficient implicit regularization is provided, but also such techniques can dramatically deteriorate the performance if the hyperparameters are not carefully tuned for the architecture and data set. In contrast, data augmentation systematically provides large generalization gains and does not require hyperparameter re-tuning. In view of our results, we suggest to optimize neural networks without weight decay and dropout to save computational resources, hence carbon emissions, and focus more on data augmentation and other inductive biases to improve performance and robustness." @default.
• W2785885246 created "2018-02-23" @default.
• W2785885246 creator A5024328042 @default.
• W2785885246 creator A5074140704 @default.
• W2785885246 date "2018-02-15" @default.
• W2785885246 modified "2023-09-22" @default.
• W2785885246 title "Data augmentation instead of explicit regularization" @default.
• W2785885246 cites W1521968289 @default.
• W2785885246 cites W1526055535 @default.
• W2785885246 cites W1533861849 @default.
• W2785885246 cites W1544532352 @default.
• W2785885246 cites W1677182931 @default.
• W2785885246 cites W1686810756 @default.
• W2785885246 cites W1836465849 @default.
• W2785885246 cites W1981025032 @default.
• W2785885246 cites W2029538739 @default.
• W2785885246 cites W2034978228 @default.
• W2785885246 cites W2095705004 @default.
• W2785885246 cites W2099621636 @default.
• W2785885246 cites W2111406701 @default.
• W2785885246 cites W2111494971 @default.
• W2785885246 cites W2117539524 @default.
• W2785885246 cites W2117897510 @default.
• W2785885246 cites W2120972216 @default.
• W2785885246 cites W2133319764 @default.
• W2785885246 cites W2138857742 @default.
• W2785885246 cites W2152722485 @default.
• W2785885246 cites W2154579312 @default.
• W2785885246 cites W2156387975 @default.
• W2785885246 cites W2163605009 @default.
• W2785885246 cites W2194775991 @default.
• W2785885246 cites W2331143823 @default.
• W2785885246 cites W2408279554 @default.
• W2785885246 cites W2419597278 @default.
• W2785885246 cites W2557283755 @default.
• W2785885246 cites W2579923771 @default.
• W2785885246 cites W2594477595 @default.
• W2785885246 cites W2604262106 @default.
• W2785885246 cites W2726367589 @default.
• W2785885246 cites W2746314669 @default.
• W2785885246 cites W2752860283 @default.
• W2785885246 cites W2765407302 @default.
• W2785885246 cites W2765861987 @default.
• W2785885246 cites W2766151966 @default.
• W2785885246 cites W2770173563 @default.
• W2785885246 cites W2775795276 @default.
• W2785885246 cites W2782476368 @default.
• W2785885246 cites W2787919999 @default.
• W2785885246 cites W2796902985 @default.
• W2785885246 cites W2891021639 @default.
• W2785885246 cites W2895616758 @default.
• W2785885246 cites W2912173254 @default.
• W2785885246 cites W2912916193 @default.
• W2785885246 cites W2945090955 @default.
• W2785885246 cites W2948223045 @default.
• W2785885246 cites W2955425717 @default.
• W2785885246 cites W2962821226 @default.
• W2785885246 cites W2963208657 @default.
• W2785885246 cites W2963341924 @default.
• W2785885246 cites W2963382180 @default.
• W2785885246 cites W2963399222 @default.
• W2785885246 cites W2963417959 @default.
• W2785885246 cites W2963446712 @default.
• W2785885246 cites W2963518130 @default.
• W2785885246 cites W2963649970 @default.
• W2785885246 cites W2963664410 @default.
• W2785885246 cites W2963695615 @default.
• W2785885246 cites W2963947040 @default.
• W2785885246 cites W2964137095 @default.
• W2785885246 cites W2964153729 @default.
• W2785885246 cites W2964313743 @default.
• W2785885246 cites W2971315489 @default.
• W2785885246 cites W2981540061 @default.
• W2785885246 cites W3004619146 @default.
• W2785885246 cites W3023215041 @default.
• W2785885246 cites W3037950864 @default.
• W2785885246 cites W3118608800 @default.
• W2785885246 cites W3137695714 @default.
• W2785885246 cites W79592478 @default.
• W2785885246 cites W2804900785 @default.
• W2785885246 hasPublicationYear "2018" @default.
• W2785885246 type Work @default.
• W2785885246 sameAs 2785885246 @default.
• W2785885246 citedByCount "39" @default.
• W2785885246 countsByYear W27858852462018 @default.
• W2785885246 countsByYear W27858852462019 @default.
• W2785885246 countsByYear W27858852462020 @default.
• W2785885246 countsByYear W27858852462021 @default.
• W2785885246 crossrefType "posted-content" @default.
• W2785885246 hasAuthorship W2785885246A5024328042 @default.
• W2785885246 hasAuthorship W2785885246A5074140704 @default.
• W2785885246 hasConcept C108583219 @default.
• W2785885246 hasConcept C119857082 @default.
• W2785885246 hasConcept C134306372 @default.
• W2785885246 hasConcept C154945302 @default.
• W2785885246 hasConcept C177148314 @default.
• W2785885246 hasConcept C2776135515 @default.
• W2785885246 hasConcept C2984842247 @default.
• W2785885246 hasConcept C33923547 @default.
• W2785885246 hasConcept C41008148 @default.

• next