Matches in SemOpenAlex for { <https://semopenalex.org/work/W1500861687> ?p ?o ?g. }
- W1500861687 abstract "Recent experimental studies indicate that recurrent networks initialized with ‘small’ weights are inherently biased towards finite memory machines [37]. This paper establishes a theoretical counterpart: we prove that recurrent networks with small weights or contractive transition function, respectively, can be approximated arbitrarily well on input sequences of unbounded length by a finite memory machine. Conversely, every finite memory machine can be simulated by a recurrent network with contractive transition function. Hence initialization with small weights induces an architectural bias into learning with recurrent neural networks. This bias has benefits from the point of view of statistical learning theory: it emphasizes regions of the weight space where good generalization can be expected. It is well known that standard recurrent neural networks are not distribution independent learnable in the PAC sense. We prove that recurrent networks with contractive transition function with a fixed parameter of the contraction fulfill the so-called distribution independent UCED property and hence are distribution independent PAC-learnable unlike general recurrent networks." @default.
- W1500861687 created "2016-06-24" @default.
- W1500861687 creator A5051182859 @default.
- W1500861687 creator A5091180862 @default.
- W1500861687 date "2002-01-01" @default.
- W1500861687 modified "2023-10-18" @default.
- W1500861687 title "Neural networks with small weights implement finite memory machines" @default.
- W1500861687 cites W1485243506 @default.
- W1500861687 cites W1508510875 @default.
- W1500861687 cites W1542886316 @default.
- W1500861687 cites W1580142630 @default.
- W1500861687 cites W1583702242 @default.
- W1500861687 cites W1585638233 @default.
- W1500861687 cites W1606770439 @default.
- W1500861687 cites W1679913846 @default.
- W1500861687 cites W1934057878 @default.
- W1500861687 cites W1964826896 @default.
- W1500861687 cites W1984367183 @default.
- W1500861687 cites W1994888168 @default.
- W1500861687 cites W2011485152 @default.
- W1500861687 cites W2031585281 @default.
- W1500861687 cites W2050778826 @default.
- W1500861687 cites W2053955420 @default.
- W1500861687 cites W2058580716 @default.
- W1500861687 cites W2059811159 @default.
- W1500861687 cites W2064675550 @default.
- W1500861687 cites W2067619114 @default.
- W1500861687 cites W2076332735 @default.
- W1500861687 cites W2102834748 @default.
- W1500861687 cites W2106491486 @default.
- W1500861687 cites W2107878631 @default.
- W1500861687 cites W2134702682 @default.
- W1500861687 cites W2135208096 @default.
- W1500861687 cites W2150039113 @default.
- W1500861687 cites W2157140289 @default.
- W1500861687 cites W2159782014 @default.
- W1500861687 cites W2165758113 @default.
- W1500861687 cites W2165959773 @default.
- W1500861687 cites W2567948266 @default.
- W1500861687 cites W3146803896 @default.
- W1500861687 cites W92982764 @default.
- W1500861687 cites W1592122239 @default.
- W1500861687 hasPublicationYear "2002" @default.
- W1500861687 type Work @default.
- W1500861687 sameAs 1500861687 @default.
- W1500861687 citedByCount "2" @default.
- W1500861687 crossrefType "journal-article" @default.
- W1500861687 hasAuthorship W1500861687A5051182859 @default.
- W1500861687 hasAuthorship W1500861687A5091180862 @default.
- W1500861687 hasConcept C110121322 @default.
- W1500861687 hasConcept C11413529 @default.
- W1500861687 hasConcept C114466953 @default.
- W1500861687 hasConcept C118615104 @default.
- W1500861687 hasConcept C127413603 @default.
- W1500861687 hasConcept C134306372 @default.
- W1500861687 hasConcept C14036430 @default.
- W1500861687 hasConcept C146978453 @default.
- W1500861687 hasConcept C147168706 @default.
- W1500861687 hasConcept C154945302 @default.
- W1500861687 hasConcept C177148314 @default.
- W1500861687 hasConcept C199360897 @default.
- W1500861687 hasConcept C25432639 @default.
- W1500861687 hasConcept C33923547 @default.
- W1500861687 hasConcept C41008148 @default.
- W1500861687 hasConcept C50644808 @default.
- W1500861687 hasConcept C78458016 @default.
- W1500861687 hasConcept C86803240 @default.
- W1500861687 hasConceptScore W1500861687C110121322 @default.
- W1500861687 hasConceptScore W1500861687C11413529 @default.
- W1500861687 hasConceptScore W1500861687C114466953 @default.
- W1500861687 hasConceptScore W1500861687C118615104 @default.
- W1500861687 hasConceptScore W1500861687C127413603 @default.
- W1500861687 hasConceptScore W1500861687C134306372 @default.
- W1500861687 hasConceptScore W1500861687C14036430 @default.
- W1500861687 hasConceptScore W1500861687C146978453 @default.
- W1500861687 hasConceptScore W1500861687C147168706 @default.
- W1500861687 hasConceptScore W1500861687C154945302 @default.
- W1500861687 hasConceptScore W1500861687C177148314 @default.
- W1500861687 hasConceptScore W1500861687C199360897 @default.
- W1500861687 hasConceptScore W1500861687C25432639 @default.
- W1500861687 hasConceptScore W1500861687C33923547 @default.
- W1500861687 hasConceptScore W1500861687C41008148 @default.
- W1500861687 hasConceptScore W1500861687C50644808 @default.
- W1500861687 hasConceptScore W1500861687C78458016 @default.
- W1500861687 hasConceptScore W1500861687C86803240 @default.
- W1500861687 hasLocation W15008616871 @default.
- W1500861687 hasOpenAccess W1500861687 @default.
- W1500861687 hasPrimaryLocation W15008616871 @default.
- W1500861687 hasRelatedWork W149320829 @default.
- W1500861687 hasRelatedWork W1527857787 @default.
- W1500861687 hasRelatedWork W1563120954 @default.
- W1500861687 hasRelatedWork W1579435142 @default.
- W1500861687 hasRelatedWork W1601254443 @default.
- W1500861687 hasRelatedWork W1892276215 @default.
- W1500861687 hasRelatedWork W1953884456 @default.
- W1500861687 hasRelatedWork W2043644192 @default.
- W1500861687 hasRelatedWork W2100311509 @default.
- W1500861687 hasRelatedWork W2107119116 @default.
- W1500861687 hasRelatedWork W2109916967 @default.
- W1500861687 hasRelatedWork W2116000277 @default.