Matches in SemOpenAlex for { <https://semopenalex.org/work/W3104518154> ?p ?o ?g. }
Showing items 1 to 96 of
96
with 100 items per page.
- W3104518154 abstract "The minibatch stochastic gradient descent method (SGD) is widely applied in deep learning due to its efficiency and scalability that enable training deep networks with a large volume of data. Particularly in the distributed setting, SGD is usually applied with large batch size. However, as opposed to small-batch SGD, neural network models trained with large-batch SGD can hardly generalize well, i.e., the validation accuracy is low. In this work, we introduce a novel regularization technique, namely distinctive regularization (DReg), which replicates a certain layer of the deep network and encourages the parameters of both layers to be diverse. The DReg technique introduces very little computation overhead. Moreover, we empirically show that optimizing the neural network with DReg using large-batch SGD achieves a significant boost in the convergence and improved generalization performance. We also demonstrate that DReg can boost the convergence of large-batch SGD with momentum. We believe that DReg can be used as a simple regularization trick to accelerate large-batch training in deep learning." @default.
- W3104518154 created "2020-11-23" @default.
- W3104518154 creator A5016676096 @default.
- W3104518154 creator A5025545085 @default.
- W3104518154 creator A5029321729 @default.
- W3104518154 creator A5066270537 @default.
- W3104518154 date "2020-11-17" @default.
- W3104518154 modified "2023-09-26" @default.
- W3104518154 title "Contrastive Weight Regularization for Large Minibatch SGD" @default.
- W3104518154 cites W104184427 @default.
- W3104518154 cites W1598866093 @default.
- W3104518154 cites W1686810756 @default.
- W3104518154 cites W1987371344 @default.
- W3104518154 cites W1994616650 @default.
- W3104518154 cites W2009941369 @default.
- W3104518154 cites W2119862467 @default.
- W3104518154 cites W2124608575 @default.
- W3104518154 cites W2138621090 @default.
- W3104518154 cites W2144513243 @default.
- W3104518154 cites W2167433878 @default.
- W3104518154 cites W2194775991 @default.
- W3104518154 cites W2339666411 @default.
- W3104518154 cites W2401231614 @default.
- W3104518154 cites W2405883473 @default.
- W3104518154 cites W2523060838 @default.
- W3104518154 cites W2622263826 @default.
- W3104518154 cites W2755682530 @default.
- W3104518154 cites W2769856846 @default.
- W3104518154 cites W2795783309 @default.
- W3104518154 cites W2799042347 @default.
- W3104518154 cites W2804386825 @default.
- W3104518154 cites W2879454547 @default.
- W3104518154 cites W2930786691 @default.
- W3104518154 cites W2945785363 @default.
- W3104518154 cites W2949704402 @default.
- W3104518154 cites W2963016543 @default.
- W3104518154 cites W2963069632 @default.
- W3104518154 cites W2963163009 @default.
- W3104518154 cites W2963655672 @default.
- W3104518154 cites W2963702144 @default.
- W3104518154 cites W2963803379 @default.
- W3104518154 cites W2981975608 @default.
- W3104518154 cites W3008424029 @default.
- W3104518154 cites W3035524453 @default.
- W3104518154 doi "https://doi.org/10.48550/arxiv.2011.08968" @default.
- W3104518154 hasPublicationYear "2020" @default.
- W3104518154 type Work @default.
- W3104518154 sameAs 3104518154 @default.
- W3104518154 citedByCount "0" @default.
- W3104518154 crossrefType "posted-content" @default.
- W3104518154 hasAuthorship W3104518154A5016676096 @default.
- W3104518154 hasAuthorship W3104518154A5025545085 @default.
- W3104518154 hasAuthorship W3104518154A5029321729 @default.
- W3104518154 hasAuthorship W3104518154A5066270537 @default.
- W3104518154 hasBestOaLocation W31045181541 @default.
- W3104518154 hasConcept C108583219 @default.
- W3104518154 hasConcept C11413529 @default.
- W3104518154 hasConcept C119857082 @default.
- W3104518154 hasConcept C154945302 @default.
- W3104518154 hasConcept C206688291 @default.
- W3104518154 hasConcept C2776135515 @default.
- W3104518154 hasConcept C2984842247 @default.
- W3104518154 hasConcept C41008148 @default.
- W3104518154 hasConcept C45374587 @default.
- W3104518154 hasConcept C48044578 @default.
- W3104518154 hasConcept C50644808 @default.
- W3104518154 hasConcept C77088390 @default.
- W3104518154 hasConceptScore W3104518154C108583219 @default.
- W3104518154 hasConceptScore W3104518154C11413529 @default.
- W3104518154 hasConceptScore W3104518154C119857082 @default.
- W3104518154 hasConceptScore W3104518154C154945302 @default.
- W3104518154 hasConceptScore W3104518154C206688291 @default.
- W3104518154 hasConceptScore W3104518154C2776135515 @default.
- W3104518154 hasConceptScore W3104518154C2984842247 @default.
- W3104518154 hasConceptScore W3104518154C41008148 @default.
- W3104518154 hasConceptScore W3104518154C45374587 @default.
- W3104518154 hasConceptScore W3104518154C48044578 @default.
- W3104518154 hasConceptScore W3104518154C50644808 @default.
- W3104518154 hasConceptScore W3104518154C77088390 @default.
- W3104518154 hasLocation W31045181541 @default.
- W3104518154 hasOpenAccess W3104518154 @default.
- W3104518154 hasPrimaryLocation W31045181541 @default.
- W3104518154 hasRelatedWork W2785875001 @default.
- W3104518154 hasRelatedWork W2791691546 @default.
- W3104518154 hasRelatedWork W2792987183 @default.
- W3104518154 hasRelatedWork W2909645158 @default.
- W3104518154 hasRelatedWork W2950066684 @default.
- W3104518154 hasRelatedWork W3083085261 @default.
- W3104518154 hasRelatedWork W3124943098 @default.
- W3104518154 hasRelatedWork W3162132941 @default.
- W3104518154 hasRelatedWork W3179488938 @default.
- W3104518154 hasRelatedWork W3195829100 @default.
- W3104518154 isParatext "false" @default.
- W3104518154 isRetracted "false" @default.
- W3104518154 magId "3104518154" @default.
- W3104518154 workType "article" @default.