Matches in SemOpenAlex for { <https://semopenalex.org/work/W3094116343> ?p ?o ?g. }
- W3094116343 endingPage "3150" @default.
- W3094116343 startingPage "3105" @default.
- W3094116343 abstract "Overfitting data is a well-known phenomenon related with the generation of a model that mimics too closely (or exactly) a particular instance of data, and may therefore fail to predict future observations reliably. In practice, this behaviour is controlled by various—sometimes based on heuristics—regularization techniques, which are motivated by upper bounds to the generalization error. In this work, we study the generalization error of classifiers relying on stochastic encodings which are trained on the cross-entropy loss, which is often used in deep learning for classification problems. We derive bounds to the generalization error showing that there exists a regime where the generalization error is bounded by the mutual information between input features and the corresponding representations in the latent space, which are randomly generated according to the encoding distribution. Our bounds provide an information-theoretic understanding of generalization in the so-called class of variational classifiers, which are regularized by a Kullback–Leibler (KL) divergence term. These results give theoretical grounds for the highly popular KL term in variational inference methods that was already recognized to act effectively as a regularization penalty. We further observe connections with well studied notions such as Variational Autoencoders, Information Dropout, Information Bottleneck and Boltzmann Machines. Finally, we perform numerical experiments on MNIST, CIFAR and other datasets and show that mutual information is indeed highly representative of the behaviour of the generalization error." @default.
- W3094116343 created "2020-10-29" @default.
- W3094116343 creator A5035529329 @default.
- W3094116343 creator A5052132831 @default.
- W3094116343 creator A5071189599 @default.
- W3094116343 date "2023-05-08" @default.
- W3094116343 modified "2023-10-16" @default.
- W3094116343 title "The role of mutual information in variational classifiers" @default.
- W3094116343 cites W1892947258 @default.
- W3094116343 cites W2015627422 @default.
- W3094116343 cites W2111049014 @default.
- W3094116343 cites W2116064496 @default.
- W3094116343 cites W2123469175 @default.
- W3094116343 cites W2136922672 @default.
- W3094116343 cites W2139338362 @default.
- W3094116343 cites W2148143831 @default.
- W3094116343 cites W2529714286 @default.
- W3094116343 cites W2683470288 @default.
- W3094116343 cites W2744885465 @default.
- W3094116343 cites W2770992186 @default.
- W3094116343 cites W2795414739 @default.
- W3094116343 cites W2964184826 @default.
- W3094116343 cites W2996320484 @default.
- W3094116343 cites W3007586552 @default.
- W3094116343 cites W3153113045 @default.
- W3094116343 cites W4232613155 @default.
- W3094116343 cites W4235825416 @default.
- W3094116343 cites W44815768 @default.
- W3094116343 doi "https://doi.org/10.1007/s10994-023-06337-6" @default.
- W3094116343 hasPublicationYear "2023" @default.
- W3094116343 type Work @default.
- W3094116343 sameAs 3094116343 @default.
- W3094116343 citedByCount "0" @default.
- W3094116343 crossrefType "journal-article" @default.
- W3094116343 hasAuthorship W3094116343A5035529329 @default.
- W3094116343 hasAuthorship W3094116343A5052132831 @default.
- W3094116343 hasAuthorship W3094116343A5071189599 @default.
- W3094116343 hasBestOaLocation W30941163432 @default.
- W3094116343 hasConcept C106301342 @default.
- W3094116343 hasConcept C11413529 @default.
- W3094116343 hasConcept C119857082 @default.
- W3094116343 hasConcept C121332964 @default.
- W3094116343 hasConcept C134306372 @default.
- W3094116343 hasConcept C152139883 @default.
- W3094116343 hasConcept C154945302 @default.
- W3094116343 hasConcept C171752962 @default.
- W3094116343 hasConcept C177148314 @default.
- W3094116343 hasConcept C190502265 @default.
- W3094116343 hasConcept C22019652 @default.
- W3094116343 hasConcept C2776135515 @default.
- W3094116343 hasConcept C2776214188 @default.
- W3094116343 hasConcept C33923547 @default.
- W3094116343 hasConcept C41008148 @default.
- W3094116343 hasConcept C50644808 @default.
- W3094116343 hasConcept C5465570 @default.
- W3094116343 hasConcept C60008888 @default.
- W3094116343 hasConcept C62520636 @default.
- W3094116343 hasConceptScore W3094116343C106301342 @default.
- W3094116343 hasConceptScore W3094116343C11413529 @default.
- W3094116343 hasConceptScore W3094116343C119857082 @default.
- W3094116343 hasConceptScore W3094116343C121332964 @default.
- W3094116343 hasConceptScore W3094116343C134306372 @default.
- W3094116343 hasConceptScore W3094116343C152139883 @default.
- W3094116343 hasConceptScore W3094116343C154945302 @default.
- W3094116343 hasConceptScore W3094116343C171752962 @default.
- W3094116343 hasConceptScore W3094116343C177148314 @default.
- W3094116343 hasConceptScore W3094116343C190502265 @default.
- W3094116343 hasConceptScore W3094116343C22019652 @default.
- W3094116343 hasConceptScore W3094116343C2776135515 @default.
- W3094116343 hasConceptScore W3094116343C2776214188 @default.
- W3094116343 hasConceptScore W3094116343C33923547 @default.
- W3094116343 hasConceptScore W3094116343C41008148 @default.
- W3094116343 hasConceptScore W3094116343C50644808 @default.
- W3094116343 hasConceptScore W3094116343C5465570 @default.
- W3094116343 hasConceptScore W3094116343C60008888 @default.
- W3094116343 hasConceptScore W3094116343C62520636 @default.
- W3094116343 hasFunder F4320321594 @default.
- W3094116343 hasFunder F4320323257 @default.
- W3094116343 hasFunder F4320338337 @default.
- W3094116343 hasIssue "9" @default.
- W3094116343 hasLocation W30941163431 @default.
- W3094116343 hasLocation W30941163432 @default.
- W3094116343 hasOpenAccess W3094116343 @default.
- W3094116343 hasPrimaryLocation W30941163431 @default.
- W3094116343 hasRelatedWork W2038951629 @default.
- W3094116343 hasRelatedWork W2795435272 @default.
- W3094116343 hasRelatedWork W2951851447 @default.
- W3094116343 hasRelatedWork W2989932438 @default.
- W3094116343 hasRelatedWork W3094116343 @default.
- W3094116343 hasRelatedWork W3099765033 @default.
- W3094116343 hasRelatedWork W3167660944 @default.
- W3094116343 hasRelatedWork W3177409857 @default.
- W3094116343 hasRelatedWork W4301016710 @default.
- W3094116343 hasRelatedWork W4312225749 @default.
- W3094116343 hasVolume "112" @default.
- W3094116343 isParatext "false" @default.
- W3094116343 isRetracted "false" @default.
- W3094116343 magId "3094116343" @default.