Matches in SemOpenAlex for { <https://semopenalex.org/work/W3178129852> ?p ?o ?g. }
Showing items 1 to 93 of
93
with 100 items per page.
- W3178129852 endingPage "862" @default.
- W3178129852 startingPage "862" @default.
- W3178129852 abstract "Deep learning methods have had outstanding performances in various fields. A fundamental query is why they are so effective. Information theory provides a potential answer by interpreting the learning process as the information transmission and compression of data. The information flows can be visualized on the information plane of the mutual information among the input, hidden, and output layers. In this study, we examine how the information flows are shaped by the network parameters, such as depth, sparsity, weight constraints, and hidden representations. Here, we adopt autoencoders as models of deep learning, because (i) they have clear guidelines for their information flows, and (ii) they have various species, such as vanilla, sparse, tied, variational, and label autoencoders. We measured their information flows using Rényi’s matrix-based α-order entropy functional. As learning progresses, they show a typical fitting phase where the amounts of input-to-hidden and hidden-to-output mutual information both increase. In the last stage of learning, however, some autoencoders show a simplifying phase, previously called the “compression phase”, where input-to-hidden mutual information diminishes. In particular, the sparsity regularization of hidden activities amplifies the simplifying phase. However, tied, variational, and label autoencoders do not have a simplifying phase. Nevertheless, all autoencoders have similar reconstruction errors for training and test data. Thus, the simplifying phase does not seem to be necessary for the generalization of learning." @default.
- W3178129852 created "2021-07-19" @default.
- W3178129852 creator A5002854119 @default.
- W3178129852 creator A5056386000 @default.
- W3178129852 date "2021-07-05" @default.
- W3178129852 modified "2023-10-11" @default.
- W3178129852 title "Information Flows of Diverse Autoencoders" @default.
- W3178129852 cites W1995875735 @default.
- W3178129852 cites W2017257315 @default.
- W3178129852 cites W2029236670 @default.
- W3178129852 cites W2032558547 @default.
- W3178129852 cites W2060038468 @default.
- W3178129852 cites W2078626246 @default.
- W3178129852 cites W2112796928 @default.
- W3178129852 cites W2148986322 @default.
- W3178129852 cites W2962783375 @default.
- W3178129852 cites W2963386266 @default.
- W3178129852 cites W2963490109 @default.
- W3178129852 cites W2966030703 @default.
- W3178129852 cites W2996320484 @default.
- W3178129852 cites W2997574889 @default.
- W3178129852 cites W3004645319 @default.
- W3178129852 cites W3106022083 @default.
- W3178129852 cites W3106311775 @default.
- W3178129852 doi "https://doi.org/10.3390/e23070862" @default.
- W3178129852 hasPubMedCentralId "https://www.ncbi.nlm.nih.gov/pmc/articles/8303402" @default.
- W3178129852 hasPubMedId "https://pubmed.ncbi.nlm.nih.gov/34356403" @default.
- W3178129852 hasPublicationYear "2021" @default.
- W3178129852 type Work @default.
- W3178129852 sameAs 3178129852 @default.
- W3178129852 citedByCount "4" @default.
- W3178129852 countsByYear W31781298522022 @default.
- W3178129852 countsByYear W31781298522023 @default.
- W3178129852 crossrefType "journal-article" @default.
- W3178129852 hasAuthorship W3178129852A5002854119 @default.
- W3178129852 hasAuthorship W3178129852A5056386000 @default.
- W3178129852 hasBestOaLocation W31781298521 @default.
- W3178129852 hasConcept C105795698 @default.
- W3178129852 hasConcept C106301342 @default.
- W3178129852 hasConcept C119857082 @default.
- W3178129852 hasConcept C121332964 @default.
- W3178129852 hasConcept C134306372 @default.
- W3178129852 hasConcept C152139883 @default.
- W3178129852 hasConcept C153180895 @default.
- W3178129852 hasConcept C154945302 @default.
- W3178129852 hasConcept C177148314 @default.
- W3178129852 hasConcept C2776135515 @default.
- W3178129852 hasConcept C33923547 @default.
- W3178129852 hasConcept C41008148 @default.
- W3178129852 hasConcept C50644808 @default.
- W3178129852 hasConcept C52622258 @default.
- W3178129852 hasConcept C62520636 @default.
- W3178129852 hasConceptScore W3178129852C105795698 @default.
- W3178129852 hasConceptScore W3178129852C106301342 @default.
- W3178129852 hasConceptScore W3178129852C119857082 @default.
- W3178129852 hasConceptScore W3178129852C121332964 @default.
- W3178129852 hasConceptScore W3178129852C134306372 @default.
- W3178129852 hasConceptScore W3178129852C152139883 @default.
- W3178129852 hasConceptScore W3178129852C153180895 @default.
- W3178129852 hasConceptScore W3178129852C154945302 @default.
- W3178129852 hasConceptScore W3178129852C177148314 @default.
- W3178129852 hasConceptScore W3178129852C2776135515 @default.
- W3178129852 hasConceptScore W3178129852C33923547 @default.
- W3178129852 hasConceptScore W3178129852C41008148 @default.
- W3178129852 hasConceptScore W3178129852C50644808 @default.
- W3178129852 hasConceptScore W3178129852C52622258 @default.
- W3178129852 hasConceptScore W3178129852C62520636 @default.
- W3178129852 hasFunder F4320328359 @default.
- W3178129852 hasIssue "7" @default.
- W3178129852 hasLocation W31781298521 @default.
- W3178129852 hasLocation W31781298522 @default.
- W3178129852 hasLocation W31781298523 @default.
- W3178129852 hasLocation W31781298524 @default.
- W3178129852 hasLocation W31781298525 @default.
- W3178129852 hasOpenAccess W3178129852 @default.
- W3178129852 hasPrimaryLocation W31781298521 @default.
- W3178129852 hasRelatedWork W2036846997 @default.
- W3178129852 hasRelatedWork W2112223184 @default.
- W3178129852 hasRelatedWork W2115702840 @default.
- W3178129852 hasRelatedWork W2161963661 @default.
- W3178129852 hasRelatedWork W2165061339 @default.
- W3178129852 hasRelatedWork W2347586617 @default.
- W3178129852 hasRelatedWork W2889544313 @default.
- W3178129852 hasRelatedWork W2920676536 @default.
- W3178129852 hasRelatedWork W3124771927 @default.
- W3178129852 hasRelatedWork W3212925274 @default.
- W3178129852 hasVolume "23" @default.
- W3178129852 isParatext "false" @default.
- W3178129852 isRetracted "false" @default.
- W3178129852 magId "3178129852" @default.
- W3178129852 workType "article" @default.