Matches in SemOpenAlex for { <https://semopenalex.org/work/W3198207789> ?p ?o ?g. }
Showing items 1 to 94 of
94
with 100 items per page.
- W3198207789 abstract "Neural networks have gained importance as the machine learning models that achieve state-of-the-art performance on large-scale image classification, object detection and natural language processing tasks. In this paper, we consider noisy binary neural networks, where each neuron has a non-zero probability of producing an incorrect output. These noisy models may arise from biological, physical and electronic contexts and constitute an important class of models that are relevant to the physical world. Intuitively, the number of neurons in such systems has to grow to compensate for the noise while maintaining the same level of expressive power and computation reliability. Our key finding is a lower bound for the required number of neurons in noisy neural networks, which is first of its kind. To prove this lower bound, we take an information theoretic approach and obtain a strong data processing inequality (SDPI), which not only generalizes the Evans-Schulman results for binary channels to general channels but also improves the tightness drastically when applied to estimate end-to-end information contraction. Applying the SDPI in noisy binary neural networks, we obtain our key lower bound and investigate its implications on network depth-width trade-offs, our results suggest a depth-width trade-off for noisy neural networks that is very different from the established understanding regarding noiseless neural networks. This paper offers new understanding of noisy information processing systems through the lens of information theory." @default.
- W3198207789 created "2021-09-13" @default.
- W3198207789 creator A5006106735 @default.
- W3198207789 creator A5051610883 @default.
- W3198207789 creator A5088853105 @default.
- W3198207789 creator A5090422974 @default.
- W3198207789 date "2021-07-12" @default.
- W3198207789 modified "2023-10-16" @default.
- W3198207789 title "Strong data processing inequality in neural networks with noisy neurons and its implications" @default.
- W3198207789 cites W2001187993 @default.
- W3198207789 cites W2003776291 @default.
- W3198207789 cites W2018168021 @default.
- W3198207789 cites W2053801139 @default.
- W3198207789 cites W2107810558 @default.
- W3198207789 cites W2137906383 @default.
- W3198207789 cites W2141432878 @default.
- W3198207789 cites W2166524562 @default.
- W3198207789 cites W2242818861 @default.
- W3198207789 cites W2290247654 @default.
- W3198207789 cites W2497735908 @default.
- W3198207789 cites W2752849906 @default.
- W3198207789 cites W2798701005 @default.
- W3198207789 cites W2952032520 @default.
- W3198207789 cites W3003821665 @default.
- W3198207789 cites W3011209639 @default.
- W3198207789 cites W3091482820 @default.
- W3198207789 cites W3098895744 @default.
- W3198207789 cites W3105432754 @default.
- W3198207789 cites W3122917808 @default.
- W3198207789 cites W2165025072 @default.
- W3198207789 doi "https://doi.org/10.1109/isit45174.2021.9517787" @default.
- W3198207789 hasPublicationYear "2021" @default.
- W3198207789 type Work @default.
- W3198207789 sameAs 3198207789 @default.
- W3198207789 citedByCount "1" @default.
- W3198207789 countsByYear W31982077892021 @default.
- W3198207789 crossrefType "proceedings-article" @default.
- W3198207789 hasAuthorship W3198207789A5006106735 @default.
- W3198207789 hasAuthorship W3198207789A5051610883 @default.
- W3198207789 hasAuthorship W3198207789A5088853105 @default.
- W3198207789 hasAuthorship W3198207789A5090422974 @default.
- W3198207789 hasConcept C11413529 @default.
- W3198207789 hasConcept C115961682 @default.
- W3198207789 hasConcept C119857082 @default.
- W3198207789 hasConcept C134306372 @default.
- W3198207789 hasConcept C154945302 @default.
- W3198207789 hasConcept C175202392 @default.
- W3198207789 hasConcept C26517878 @default.
- W3198207789 hasConcept C33923547 @default.
- W3198207789 hasConcept C38652104 @default.
- W3198207789 hasConcept C41008148 @default.
- W3198207789 hasConcept C45374587 @default.
- W3198207789 hasConcept C48372109 @default.
- W3198207789 hasConcept C50644808 @default.
- W3198207789 hasConcept C77553402 @default.
- W3198207789 hasConcept C80444323 @default.
- W3198207789 hasConcept C86582703 @default.
- W3198207789 hasConcept C94375191 @default.
- W3198207789 hasConcept C99498987 @default.
- W3198207789 hasConceptScore W3198207789C11413529 @default.
- W3198207789 hasConceptScore W3198207789C115961682 @default.
- W3198207789 hasConceptScore W3198207789C119857082 @default.
- W3198207789 hasConceptScore W3198207789C134306372 @default.
- W3198207789 hasConceptScore W3198207789C154945302 @default.
- W3198207789 hasConceptScore W3198207789C175202392 @default.
- W3198207789 hasConceptScore W3198207789C26517878 @default.
- W3198207789 hasConceptScore W3198207789C33923547 @default.
- W3198207789 hasConceptScore W3198207789C38652104 @default.
- W3198207789 hasConceptScore W3198207789C41008148 @default.
- W3198207789 hasConceptScore W3198207789C45374587 @default.
- W3198207789 hasConceptScore W3198207789C48372109 @default.
- W3198207789 hasConceptScore W3198207789C50644808 @default.
- W3198207789 hasConceptScore W3198207789C77553402 @default.
- W3198207789 hasConceptScore W3198207789C80444323 @default.
- W3198207789 hasConceptScore W3198207789C86582703 @default.
- W3198207789 hasConceptScore W3198207789C94375191 @default.
- W3198207789 hasConceptScore W3198207789C99498987 @default.
- W3198207789 hasLocation W31982077891 @default.
- W3198207789 hasOpenAccess W3198207789 @default.
- W3198207789 hasPrimaryLocation W31982077891 @default.
- W3198207789 hasRelatedWork W2007647063 @default.
- W3198207789 hasRelatedWork W2133996617 @default.
- W3198207789 hasRelatedWork W2360829920 @default.
- W3198207789 hasRelatedWork W2377319029 @default.
- W3198207789 hasRelatedWork W2386387936 @default.
- W3198207789 hasRelatedWork W2509089808 @default.
- W3198207789 hasRelatedWork W2592098600 @default.
- W3198207789 hasRelatedWork W3029113222 @default.
- W3198207789 hasRelatedWork W3150126121 @default.
- W3198207789 hasRelatedWork W1629725936 @default.
- W3198207789 isParatext "false" @default.
- W3198207789 isRetracted "false" @default.
- W3198207789 magId "3198207789" @default.
- W3198207789 workType "article" @default.