Matches in SemOpenAlex for { <https://semopenalex.org/work/W4287285959> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W4287285959 abstract "Batch Normalization (BN) is a commonly used technique to accelerate and stabilize training of deep neural networks. Despite its empirical success, a full theoretical understanding of BN is yet to be developed. In this work, we analyze BN through the lens of convex optimization. We introduce an analytic framework based on convex duality to obtain exact convex representations of weight-decay regularized ReLU networks with BN, which can be trained in polynomial-time. Our analyses also show that optimal layer weights can be obtained as simple closed-form formulas in the high-dimensional and/or overparameterized regimes. Furthermore, we find that Gradient Descent provides an algorithmic bias effect on the standard non-convex BN network, and we design an approach to explicitly encode this implicit regularization into the convex objective. Experiments with CIFAR image classification highlight the effectiveness of this explicit regularization for mimicking and substantially improving the performance of standard BN networks." @default.
- W4287285959 created "2022-07-25" @default.
- W4287285959 creator A5001436196 @default.
- W4287285959 creator A5008348052 @default.
- W4287285959 creator A5030024620 @default.
- W4287285959 creator A5040173784 @default.
- W4287285959 creator A5060755739 @default.
- W4287285959 creator A5072391499 @default.
- W4287285959 date "2021-03-02" @default.
- W4287285959 modified "2023-09-27" @default.
- W4287285959 title "Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization" @default.
- W4287285959 doi "https://doi.org/10.48550/arxiv.2103.01499" @default.
- W4287285959 hasPublicationYear "2021" @default.
- W4287285959 type Work @default.
- W4287285959 citedByCount "0" @default.
- W4287285959 crossrefType "posted-content" @default.
- W4287285959 hasAuthorship W4287285959A5001436196 @default.
- W4287285959 hasAuthorship W4287285959A5008348052 @default.
- W4287285959 hasAuthorship W4287285959A5030024620 @default.
- W4287285959 hasAuthorship W4287285959A5040173784 @default.
- W4287285959 hasAuthorship W4287285959A5060755739 @default.
- W4287285959 hasAuthorship W4287285959A5072391499 @default.
- W4287285959 hasBestOaLocation W42872859591 @default.
- W4287285959 hasConcept C112680207 @default.
- W4287285959 hasConcept C11413529 @default.
- W4287285959 hasConcept C12108790 @default.
- W4287285959 hasConcept C126255220 @default.
- W4287285959 hasConcept C136886441 @default.
- W4287285959 hasConcept C144024400 @default.
- W4287285959 hasConcept C153258448 @default.
- W4287285959 hasConcept C154945302 @default.
- W4287285959 hasConcept C157972887 @default.
- W4287285959 hasConcept C19165224 @default.
- W4287285959 hasConcept C2524010 @default.
- W4287285959 hasConcept C2776135515 @default.
- W4287285959 hasConcept C28826006 @default.
- W4287285959 hasConcept C2984842247 @default.
- W4287285959 hasConcept C33923547 @default.
- W4287285959 hasConcept C41008148 @default.
- W4287285959 hasConcept C50644808 @default.
- W4287285959 hasConcept C79248915 @default.
- W4287285959 hasConceptScore W4287285959C112680207 @default.
- W4287285959 hasConceptScore W4287285959C11413529 @default.
- W4287285959 hasConceptScore W4287285959C12108790 @default.
- W4287285959 hasConceptScore W4287285959C126255220 @default.
- W4287285959 hasConceptScore W4287285959C136886441 @default.
- W4287285959 hasConceptScore W4287285959C144024400 @default.
- W4287285959 hasConceptScore W4287285959C153258448 @default.
- W4287285959 hasConceptScore W4287285959C154945302 @default.
- W4287285959 hasConceptScore W4287285959C157972887 @default.
- W4287285959 hasConceptScore W4287285959C19165224 @default.
- W4287285959 hasConceptScore W4287285959C2524010 @default.
- W4287285959 hasConceptScore W4287285959C2776135515 @default.
- W4287285959 hasConceptScore W4287285959C28826006 @default.
- W4287285959 hasConceptScore W4287285959C2984842247 @default.
- W4287285959 hasConceptScore W4287285959C33923547 @default.
- W4287285959 hasConceptScore W4287285959C41008148 @default.
- W4287285959 hasConceptScore W4287285959C50644808 @default.
- W4287285959 hasConceptScore W4287285959C79248915 @default.
- W4287285959 hasLocation W42872859591 @default.
- W4287285959 hasOpenAccess W4287285959 @default.
- W4287285959 hasPrimaryLocation W42872859591 @default.
- W4287285959 hasRelatedWork W2081395119 @default.
- W4287285959 hasRelatedWork W2089251650 @default.
- W4287285959 hasRelatedWork W2743758565 @default.
- W4287285959 hasRelatedWork W3036599379 @default.
- W4287285959 hasRelatedWork W3094945934 @default.
- W4287285959 hasRelatedWork W3135094205 @default.
- W4287285959 hasRelatedWork W3159351664 @default.
- W4287285959 hasRelatedWork W3217498676 @default.
- W4287285959 hasRelatedWork W4286906341 @default.
- W4287285959 hasRelatedWork W4287285959 @default.
- W4287285959 isParatext "false" @default.
- W4287285959 isRetracted "false" @default.
- W4287285959 workType "article" @default.