FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3208382257> ?p ?o ?g. }

• W3208382257 abstract "Optimization is a key component for training machine learning models and has a strong impact on their generalization. In this paper, we consider a particular optimization method -- the stochastic gradient Langevin dynamics (SGLD) algorithm -- and investigate the generalization of models trained by SGLD. We derive a new generalization bound by connecting SGLD with Gaussian channels found in information and communication theory. Our bound can be computed from the training data and incorporates the variance of gradients for quantifying a particular kind of sharpness of the loss landscape. We also consider a closely related algorithm with SGLD, namely differentially private SGD (DP-SGD). We prove that the generalization capability of DP-SGD can be amplified by iteration. Specifically, our bound can be sharpened by including a time-decaying factor if the DP-SGD algorithm outputs the last iterate while keeping other iterates hidden. This decay factor enables the contribution of early iterations to our bound to reduce with time and is established by strong data processing inequalities -- a fundamental tool in information theory. We demonstrate our bound through numerical experiments, showing that it can predict the behavior of the true generalization gap." @default.
• W3208382257 created "2021-11-08" @default.
• W3208382257 creator A5004972663 @default.
• W3208382257 creator A5028562033 @default.
• W3208382257 creator A5074697940 @default.
• W3208382257 creator A5080102032 @default.
• W3208382257 date "2021-11-01" @default.
• W3208382257 modified "2023-09-28" @default.
• W3208382257 title "Analyzing the Generalization Capability of SGLD Using Properties of Gaussian Channels." @default.
• W3208382257 cites W1568555062 @default.
• W3208382257 cites W1873763122 @default.
• W3208382257 cites W1985511977 @default.
• W3208382257 cites W1985514943 @default.
• W3208382257 cites W1994908596 @default.
• W3208382257 cites W2003776291 @default.
• W3208382257 cites W2019363670 @default.
• W3208382257 cites W2029538739 @default.
• W3208382257 cites W2099111195 @default.
• W3208382257 cites W2111616148 @default.
• W3208382257 cites W2118439011 @default.
• W3208382257 cites W2120456162 @default.
• W3208382257 cites W2129774678 @default.
• W3208382257 cites W2130401121 @default.
• W3208382257 cites W2167433878 @default.
• W3208382257 cites W2225981128 @default.
• W3208382257 cites W2258658829 @default.
• W3208382257 cites W2473418344 @default.
• W3208382257 cites W2902964282 @default.
• W3208382257 cites W2914671377 @default.
• W3208382257 cites W2962702650 @default.
• W3208382257 cites W2962725299 @default.
• W3208382257 cites W2962839956 @default.
• W3208382257 cites W2963423396 @default.
• W3208382257 cites W2963664410 @default.
• W3208382257 cites W2963695615 @default.
• W3208382257 cites W2963739978 @default.
• W3208382257 cites W2963794891 @default.
• W3208382257 cites W2963862692 @default.
• W3208382257 cites W2963874210 @default.
• W3208382257 cites W2963898870 @default.
• W3208382257 cites W2964310974 @default.
• W3208382257 cites W2970638103 @default.
• W3208382257 cites W2972203720 @default.
• W3208382257 cites W3100231902 @default.
• W3208382257 cites W3116262828 @default.
• W3208382257 cites W3118608800 @default.
• W3208382257 cites W3137695714 @default.
• W3208382257 hasPublicationYear "2021" @default.
• W3208382257 type Work @default.
• W3208382257 sameAs 3208382257 @default.
• W3208382257 citedByCount "0" @default.
• W3208382257 crossrefType "posted-content" @default.
• W3208382257 hasAuthorship W3208382257A5004972663 @default.
• W3208382257 hasAuthorship W3208382257A5028562033 @default.
• W3208382257 hasAuthorship W3208382257A5074697940 @default.
• W3208382257 hasAuthorship W3208382257A5080102032 @default.
• W3208382257 hasConcept C105795698 @default.
• W3208382257 hasConcept C11413529 @default.
• W3208382257 hasConcept C121332964 @default.
• W3208382257 hasConcept C121955636 @default.
• W3208382257 hasConcept C134306372 @default.
• W3208382257 hasConcept C140479938 @default.
• W3208382257 hasConcept C144133560 @default.
• W3208382257 hasConcept C154945302 @default.
• W3208382257 hasConcept C162324750 @default.
• W3208382257 hasConcept C163716315 @default.
• W3208382257 hasConcept C177148314 @default.
• W3208382257 hasConcept C196083921 @default.
• W3208382257 hasConcept C206688291 @default.
• W3208382257 hasConcept C2777303404 @default.
• W3208382257 hasConcept C2780004032 @default.
• W3208382257 hasConcept C28826006 @default.
• W3208382257 hasConcept C33923547 @default.
• W3208382257 hasConcept C41008148 @default.
• W3208382257 hasConcept C50522688 @default.
• W3208382257 hasConcept C50644808 @default.
• W3208382257 hasConcept C62520636 @default.
• W3208382257 hasConcept C77553402 @default.
• W3208382257 hasConceptScore W3208382257C105795698 @default.
• W3208382257 hasConceptScore W3208382257C11413529 @default.
• W3208382257 hasConceptScore W3208382257C121332964 @default.
• W3208382257 hasConceptScore W3208382257C121955636 @default.
• W3208382257 hasConceptScore W3208382257C134306372 @default.
• W3208382257 hasConceptScore W3208382257C140479938 @default.
• W3208382257 hasConceptScore W3208382257C144133560 @default.
• W3208382257 hasConceptScore W3208382257C154945302 @default.
• W3208382257 hasConceptScore W3208382257C162324750 @default.
• W3208382257 hasConceptScore W3208382257C163716315 @default.
• W3208382257 hasConceptScore W3208382257C177148314 @default.
• W3208382257 hasConceptScore W3208382257C196083921 @default.
• W3208382257 hasConceptScore W3208382257C206688291 @default.
• W3208382257 hasConceptScore W3208382257C2777303404 @default.
• W3208382257 hasConceptScore W3208382257C2780004032 @default.
• W3208382257 hasConceptScore W3208382257C28826006 @default.
• W3208382257 hasConceptScore W3208382257C33923547 @default.
• W3208382257 hasConceptScore W3208382257C41008148 @default.
• W3208382257 hasConceptScore W3208382257C50522688 @default.
• W3208382257 hasConceptScore W3208382257C50644808 @default.
• W3208382257 hasConceptScore W3208382257C62520636 @default.
• W3208382257 hasConceptScore W3208382257C77553402 @default.

• next