Matches in SemOpenAlex for { <https://semopenalex.org/work/W3084020051> ?p ?o ?g. }
Showing items 1 to 92 of
92
with 100 items per page.
- W3084020051 abstract "Inspired by the adaptation phenomenon of neuronal firing, we propose the regularity normalization (RN) as an unsupervised attention mechanism (UAM) which computes the statistical regularity in the implicit space of neural networks under the Minimum Description Length (MDL) principle. Treating the neural network optimization process as a partially observable model selection problem, UAM constrains the implicit space by a normalization factor, the universal code length. We compute this universal code incrementally across neural network layers and demonstrated the flexibility to include data priors such as top-down attention and other oracle information. Empirically, our approach outperforms existing normalization methods in tackling limited, imbalanced and non-stationary input distribution in image classification, classic control, procedurally-generated reinforcement learning, generative modeling, handwriting generation and question answering tasks with various neural network architectures. Lastly, UAM tracks dependency and critical learning stages across layers and recurrent time steps of deep networks." @default.
- W3084020051 created "2020-09-14" @default.
- W3084020051 creator A5018612055 @default.
- W3084020051 date "2019-02-27" @default.
- W3084020051 modified "2023-09-25" @default.
- W3084020051 title "Constraining Implicit Space with Minimum Description Length: An Unsupervised Attention Mechanism across Neural Network Layers" @default.
- W3084020051 cites W1522301498 @default.
- W3084020051 cites W1525961042 @default.
- W3084020051 cites W1533169541 @default.
- W3084020051 cites W1599016936 @default.
- W3084020051 cites W1757796397 @default.
- W3084020051 cites W1850742715 @default.
- W3084020051 cites W1999228575 @default.
- W3084020051 cites W2007431958 @default.
- W3084020051 cites W2029538739 @default.
- W3084020051 cites W2031342017 @default.
- W3084020051 cites W2047229728 @default.
- W3084020051 cites W2054658115 @default.
- W3084020051 cites W2101460669 @default.
- W3084020051 cites W2114771311 @default.
- W3084020051 cites W2118781169 @default.
- W3084020051 cites W2138200515 @default.
- W3084020051 cites W2141559645 @default.
- W3084020051 cites W2142963090 @default.
- W3084020051 cites W2153158097 @default.
- W3084020051 cites W2156737235 @default.
- W3084020051 cites W2244807774 @default.
- W3084020051 cites W2290658005 @default.
- W3084020051 cites W2337199865 @default.
- W3084020051 cites W2401823607 @default.
- W3084020051 cites W2417982661 @default.
- W3084020051 cites W2484168561 @default.
- W3084020051 cites W2593634001 @default.
- W3084020051 cites W2736601468 @default.
- W3084020051 cites W2783047733 @default.
- W3084020051 cites W2805003733 @default.
- W3084020051 cites W2912457762 @default.
- W3084020051 cites W2913289884 @default.
- W3084020051 cites W2921713804 @default.
- W3084020051 cites W2949117887 @default.
- W3084020051 cites W2950220847 @default.
- W3084020051 cites W2963275611 @default.
- W3084020051 cites W2963403868 @default.
- W3084020051 cites W2963674932 @default.
- W3084020051 cites W2963685250 @default.
- W3084020051 cites W2964153729 @default.
- W3084020051 cites W2994081359 @default.
- W3084020051 cites W2996320484 @default.
- W3084020051 cites W3037207827 @default.
- W3084020051 cites W3172100757 @default.
- W3084020051 cites W3181414820 @default.
- W3084020051 hasPublicationYear "2019" @default.
- W3084020051 type Work @default.
- W3084020051 sameAs 3084020051 @default.
- W3084020051 citedByCount "0" @default.
- W3084020051 crossrefType "posted-content" @default.
- W3084020051 hasAuthorship W3084020051A5018612055 @default.
- W3084020051 hasBestOaLocation W30840200511 @default.
- W3084020051 hasConcept C111919701 @default.
- W3084020051 hasConcept C121332964 @default.
- W3084020051 hasConcept C154945302 @default.
- W3084020051 hasConcept C2778572836 @default.
- W3084020051 hasConcept C41008148 @default.
- W3084020051 hasConcept C50644808 @default.
- W3084020051 hasConcept C62520636 @default.
- W3084020051 hasConcept C89611455 @default.
- W3084020051 hasConceptScore W3084020051C111919701 @default.
- W3084020051 hasConceptScore W3084020051C121332964 @default.
- W3084020051 hasConceptScore W3084020051C154945302 @default.
- W3084020051 hasConceptScore W3084020051C2778572836 @default.
- W3084020051 hasConceptScore W3084020051C41008148 @default.
- W3084020051 hasConceptScore W3084020051C50644808 @default.
- W3084020051 hasConceptScore W3084020051C62520636 @default.
- W3084020051 hasConceptScore W3084020051C89611455 @default.
- W3084020051 hasLocation W30840200511 @default.
- W3084020051 hasLocation W30840200512 @default.
- W3084020051 hasOpenAccess W3084020051 @default.
- W3084020051 hasPrimaryLocation W30840200511 @default.
- W3084020051 hasRelatedWork W1987137857 @default.
- W3084020051 hasRelatedWork W2326662918 @default.
- W3084020051 hasRelatedWork W2359007671 @default.
- W3084020051 hasRelatedWork W2386387936 @default.
- W3084020051 hasRelatedWork W2393858554 @default.
- W3084020051 hasRelatedWork W2394463640 @default.
- W3084020051 hasRelatedWork W2583183695 @default.
- W3084020051 hasRelatedWork W4362499384 @default.
- W3084020051 hasRelatedWork W644753246 @default.
- W3084020051 hasRelatedWork W1629725936 @default.
- W3084020051 isParatext "false" @default.
- W3084020051 isRetracted "false" @default.
- W3084020051 magId "3084020051" @default.
- W3084020051 workType "article" @default.