Matches in SemOpenAlex for { <https://semopenalex.org/work/W3034922011> ?p ?o ?g. }
Showing items 1 to 88 of
88
with 100 items per page.
- W3034922011 abstract "Compact neural networks are essential for affordable and power efficient deep learning solutions. Binary Neural Networks (BNNs) take compactification to the extreme by constraining both weights and activations to two levels, ${+1, -1}$. However, training BNNs are not easy due to the discontinuity in activation functions, and the training dynamics of BNNs is not well understood. In this paper, we present an information-theoretic perspective of BNN training. We analyze BNNs through the Information Bottleneck principle and observe that the training dynamics of BNNs is considerably different from that of Deep Neural Networks (DNNs). While DNNs have a separate empirical risk minimization and representation compression phases, our numerical experiments show that in BNNs, both these phases are simultaneous. Since BNNs have a less expressive capacity, they tend to find efficient hidden representations concurrently with label fitting. Experiments in multiple datasets support these observations, and we see a consistent behavior across different activation functions in BNNs." @default.
- W3034922011 created "2020-06-19" @default.
- W3034922011 creator A5043597908 @default.
- W3034922011 creator A5046290878 @default.
- W3034922011 creator A5077545936 @default.
- W3034922011 date "2020-06-13" @default.
- W3034922011 modified "2023-09-27" @default.
- W3034922011 title "Understanding Learning Dynamics of Binary Neural Networks via Information Bottleneck." @default.
- W3034922011 cites W1821462560 @default.
- W3034922011 cites W2099111195 @default.
- W3034922011 cites W2319920447 @default.
- W3034922011 cites W2593634001 @default.
- W3034922011 cites W2828552710 @default.
- W3034922011 cites W3010156474 @default.
- W3034922011 cites W3106022083 @default.
- W3034922011 hasPublicationYear "2020" @default.
- W3034922011 type Work @default.
- W3034922011 sameAs 3034922011 @default.
- W3034922011 citedByCount "2" @default.
- W3034922011 countsByYear W30349220112020 @default.
- W3034922011 countsByYear W30349220112021 @default.
- W3034922011 crossrefType "posted-content" @default.
- W3034922011 hasAuthorship W3034922011A5043597908 @default.
- W3034922011 hasAuthorship W3034922011A5046290878 @default.
- W3034922011 hasAuthorship W3034922011A5077545936 @default.
- W3034922011 hasConcept C119857082 @default.
- W3034922011 hasConcept C12713177 @default.
- W3034922011 hasConcept C149635348 @default.
- W3034922011 hasConcept C152139883 @default.
- W3034922011 hasConcept C154945302 @default.
- W3034922011 hasConcept C17744445 @default.
- W3034922011 hasConcept C199539241 @default.
- W3034922011 hasConcept C2776359362 @default.
- W3034922011 hasConcept C2780513914 @default.
- W3034922011 hasConcept C2984842247 @default.
- W3034922011 hasConcept C33923547 @default.
- W3034922011 hasConcept C41008148 @default.
- W3034922011 hasConcept C48372109 @default.
- W3034922011 hasConcept C50644808 @default.
- W3034922011 hasConcept C60008888 @default.
- W3034922011 hasConcept C80444323 @default.
- W3034922011 hasConcept C94375191 @default.
- W3034922011 hasConcept C94625758 @default.
- W3034922011 hasConceptScore W3034922011C119857082 @default.
- W3034922011 hasConceptScore W3034922011C12713177 @default.
- W3034922011 hasConceptScore W3034922011C149635348 @default.
- W3034922011 hasConceptScore W3034922011C152139883 @default.
- W3034922011 hasConceptScore W3034922011C154945302 @default.
- W3034922011 hasConceptScore W3034922011C17744445 @default.
- W3034922011 hasConceptScore W3034922011C199539241 @default.
- W3034922011 hasConceptScore W3034922011C2776359362 @default.
- W3034922011 hasConceptScore W3034922011C2780513914 @default.
- W3034922011 hasConceptScore W3034922011C2984842247 @default.
- W3034922011 hasConceptScore W3034922011C33923547 @default.
- W3034922011 hasConceptScore W3034922011C41008148 @default.
- W3034922011 hasConceptScore W3034922011C48372109 @default.
- W3034922011 hasConceptScore W3034922011C50644808 @default.
- W3034922011 hasConceptScore W3034922011C60008888 @default.
- W3034922011 hasConceptScore W3034922011C80444323 @default.
- W3034922011 hasConceptScore W3034922011C94375191 @default.
- W3034922011 hasConceptScore W3034922011C94625758 @default.
- W3034922011 hasLocation W30349220111 @default.
- W3034922011 hasOpenAccess W3034922011 @default.
- W3034922011 hasPrimaryLocation W30349220111 @default.
- W3034922011 hasRelatedWork W2890494546 @default.
- W3034922011 hasRelatedWork W2912277744 @default.
- W3034922011 hasRelatedWork W2920993265 @default.
- W3034922011 hasRelatedWork W2922317475 @default.
- W3034922011 hasRelatedWork W2963451067 @default.
- W3034922011 hasRelatedWork W2970691381 @default.
- W3034922011 hasRelatedWork W2978975428 @default.
- W3034922011 hasRelatedWork W2989588881 @default.
- W3034922011 hasRelatedWork W2996798951 @default.
- W3034922011 hasRelatedWork W2998977000 @default.
- W3034922011 hasRelatedWork W3007926765 @default.
- W3034922011 hasRelatedWork W3016853811 @default.
- W3034922011 hasRelatedWork W3035178967 @default.
- W3034922011 hasRelatedWork W3036407970 @default.
- W3034922011 hasRelatedWork W3089733125 @default.
- W3034922011 hasRelatedWork W3131103899 @default.
- W3034922011 hasRelatedWork W3132590279 @default.
- W3034922011 hasRelatedWork W3153138627 @default.
- W3034922011 hasRelatedWork W3170336723 @default.
- W3034922011 hasRelatedWork W3173623482 @default.
- W3034922011 isParatext "false" @default.
- W3034922011 isRetracted "false" @default.
- W3034922011 magId "3034922011" @default.
- W3034922011 workType "article" @default.