Matches in SemOpenAlex for { <https://semopenalex.org/work/W3211544218> ?p ?o ?g. }
Showing items 1 to 92 of
92
with 100 items per page.
- W3211544218 abstract "Optimization is a key component for training machine learning models and has a strong impact on their generalization. In this paper, we consider a particular optimization method -- the stochastic gradient Langevin dynamics (SGLD) algorithm -- and investigate the generalization of models trained by SGLD. We derive a new generalization bound by connecting SGLD with Gaussian channels found in information and communication theory. Our bound can be computed from the training data and incorporates the variance of gradients for quantifying a particular kind of sharpness of the loss landscape. We also consider a closely related algorithm with SGLD, namely differentially private SGD (DP-SGD). We prove that the generalization capability of DP-SGD can be amplified by iteration. Specifically, our bound can be sharpened by including a time-decaying factor if the DP-SGD algorithm outputs the last iterate while keeping other iterates hidden. This decay factor enables the contribution of early iterations to our bound to reduce with time and is established by strong data processing inequalities -- a fundamental tool in information theory. We demonstrate our bound through numerical experiments, showing that it can predict the behavior of the true generalization gap." @default.
- W3211544218 created "2021-11-22" @default.
- W3211544218 creator A5004972663 @default.
- W3211544218 creator A5028562033 @default.
- W3211544218 creator A5074697940 @default.
- W3211544218 creator A5080102032 @default.
- W3211544218 date "2021-12-06" @default.
- W3211544218 modified "2023-10-18" @default.
- W3211544218 title "Analyzing the Generalization Capability of SGLD Using Properties of Gaussian Channels" @default.
- W3211544218 hasPublicationYear "2021" @default.
- W3211544218 type Work @default.
- W3211544218 sameAs 3211544218 @default.
- W3211544218 citedByCount "0" @default.
- W3211544218 crossrefType "proceedings-article" @default.
- W3211544218 hasAuthorship W3211544218A5004972663 @default.
- W3211544218 hasAuthorship W3211544218A5028562033 @default.
- W3211544218 hasAuthorship W3211544218A5074697940 @default.
- W3211544218 hasAuthorship W3211544218A5080102032 @default.
- W3211544218 hasConcept C105795698 @default.
- W3211544218 hasConcept C11413529 @default.
- W3211544218 hasConcept C121332964 @default.
- W3211544218 hasConcept C121955636 @default.
- W3211544218 hasConcept C126255220 @default.
- W3211544218 hasConcept C134306372 @default.
- W3211544218 hasConcept C140479938 @default.
- W3211544218 hasConcept C144133560 @default.
- W3211544218 hasConcept C154945302 @default.
- W3211544218 hasConcept C162324750 @default.
- W3211544218 hasConcept C163716315 @default.
- W3211544218 hasConcept C177148314 @default.
- W3211544218 hasConcept C196083921 @default.
- W3211544218 hasConcept C206688291 @default.
- W3211544218 hasConcept C2777303404 @default.
- W3211544218 hasConcept C2780004032 @default.
- W3211544218 hasConcept C28826006 @default.
- W3211544218 hasConcept C33923547 @default.
- W3211544218 hasConcept C41008148 @default.
- W3211544218 hasConcept C50522688 @default.
- W3211544218 hasConcept C50644808 @default.
- W3211544218 hasConcept C62520636 @default.
- W3211544218 hasConcept C77553402 @default.
- W3211544218 hasConceptScore W3211544218C105795698 @default.
- W3211544218 hasConceptScore W3211544218C11413529 @default.
- W3211544218 hasConceptScore W3211544218C121332964 @default.
- W3211544218 hasConceptScore W3211544218C121955636 @default.
- W3211544218 hasConceptScore W3211544218C126255220 @default.
- W3211544218 hasConceptScore W3211544218C134306372 @default.
- W3211544218 hasConceptScore W3211544218C140479938 @default.
- W3211544218 hasConceptScore W3211544218C144133560 @default.
- W3211544218 hasConceptScore W3211544218C154945302 @default.
- W3211544218 hasConceptScore W3211544218C162324750 @default.
- W3211544218 hasConceptScore W3211544218C163716315 @default.
- W3211544218 hasConceptScore W3211544218C177148314 @default.
- W3211544218 hasConceptScore W3211544218C196083921 @default.
- W3211544218 hasConceptScore W3211544218C206688291 @default.
- W3211544218 hasConceptScore W3211544218C2777303404 @default.
- W3211544218 hasConceptScore W3211544218C2780004032 @default.
- W3211544218 hasConceptScore W3211544218C28826006 @default.
- W3211544218 hasConceptScore W3211544218C33923547 @default.
- W3211544218 hasConceptScore W3211544218C41008148 @default.
- W3211544218 hasConceptScore W3211544218C50522688 @default.
- W3211544218 hasConceptScore W3211544218C50644808 @default.
- W3211544218 hasConceptScore W3211544218C62520636 @default.
- W3211544218 hasConceptScore W3211544218C77553402 @default.
- W3211544218 hasLocation W32115442181 @default.
- W3211544218 hasOpenAccess W3211544218 @default.
- W3211544218 hasPrimaryLocation W32115442181 @default.
- W3211544218 hasRelatedWork W1919402970 @default.
- W3211544218 hasRelatedWork W2118543192 @default.
- W3211544218 hasRelatedWork W2147036700 @default.
- W3211544218 hasRelatedWork W2282795067 @default.
- W3211544218 hasRelatedWork W2310999276 @default.
- W3211544218 hasRelatedWork W2890054640 @default.
- W3211544218 hasRelatedWork W2918745211 @default.
- W3211544218 hasRelatedWork W2952228663 @default.
- W3211544218 hasRelatedWork W2963161247 @default.
- W3211544218 hasRelatedWork W2974801445 @default.
- W3211544218 hasRelatedWork W2982286341 @default.
- W3211544218 hasRelatedWork W3004510491 @default.
- W3211544218 hasRelatedWork W3012806192 @default.
- W3211544218 hasRelatedWork W3034334241 @default.
- W3211544218 hasRelatedWork W3035158267 @default.
- W3211544218 hasRelatedWork W3104688295 @default.
- W3211544218 hasRelatedWork W3106413525 @default.
- W3211544218 hasRelatedWork W3117844572 @default.
- W3211544218 hasRelatedWork W3157298807 @default.
- W3211544218 hasRelatedWork W3181674477 @default.
- W3211544218 hasVolume "34" @default.
- W3211544218 isParatext "false" @default.
- W3211544218 isRetracted "false" @default.
- W3211544218 magId "3211544218" @default.
- W3211544218 workType "article" @default.