Matches in SemOpenAlex for { <https://semopenalex.org/work/W3185853715> ?p ?o ?g. }
- W3185853715 abstract "A common lens to theoretically study neural net architectures is to analyze the functions they can approximate. However, constructions from approximation theory may be unrealistic and therefore less meaningful. For example, a common unrealistic trick is to encode target function values using infinite precision. To address these issues, this work proposes a formal definition of statistically meaningful (SM) approximation which requires the approximating network to exhibit good statistical learnability. We study SM approximation for two function classes: boolean circuits and Turing machines. We show that overparameterized feedforward neural nets can SM approximate boolean circuits with sample complexity depending only polynomially on the circuit size, not the size of the network. In addition, we show that transformers can SM approximate Turing machines with computation time bounded by $T$ with sample complexity polynomial in the alphabet size, state space size, and $log (T)$. We also introduce new tools for analyzing generalization which provide much tighter sample complexities than the typical VC-dimension or norm-based bounds, which may be of independent interest." @default.
- W3185853715 created "2021-08-02" @default.
- W3185853715 creator A5055799536 @default.
- W3185853715 creator A5061905935 @default.
- W3185853715 creator A5066483084 @default.
- W3185853715 date "2021-07-28" @default.
- W3185853715 modified "2023-09-24" @default.
- W3185853715 title "Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers" @default.
- W3185853715 cites W1564514837 @default.
- W3185853715 cites W1994584977 @default.
- W3185853715 cites W2024450772 @default.
- W3185853715 cites W2053955420 @default.
- W3185853715 cites W2067619114 @default.
- W3185853715 cites W2080745194 @default.
- W3185853715 cites W2103496339 @default.
- W3185853715 cites W2113442785 @default.
- W3185853715 cites W2150120952 @default.
- W3185853715 cites W2166116275 @default.
- W3185853715 cites W2555745781 @default.
- W3185853715 cites W2584401907 @default.
- W3185853715 cites W2593965610 @default.
- W3185853715 cites W2596625124 @default.
- W3185853715 cites W2626778328 @default.
- W3185853715 cites W2709553318 @default.
- W3185853715 cites W2798579706 @default.
- W3185853715 cites W2800415562 @default.
- W3185853715 cites W2804822090 @default.
- W3185853715 cites W2810898455 @default.
- W3185853715 cites W2908802752 @default.
- W3185853715 cites W2920448302 @default.
- W3185853715 cites W2944619585 @default.
- W3185853715 cites W2947075492 @default.
- W3185853715 cites W2949146054 @default.
- W3185853715 cites W2949292434 @default.
- W3185853715 cites W2950220847 @default.
- W3185853715 cites W2952574409 @default.
- W3185853715 cites W2962742960 @default.
- W3185853715 cites W2962845550 @default.
- W3185853715 cites W2962857907 @default.
- W3185853715 cites W2963095610 @default.
- W3185853715 cites W2963285844 @default.
- W3185853715 cites W2963706817 @default.
- W3185853715 cites W2964153674 @default.
- W3185853715 cites W2964290344 @default.
- W3185853715 cites W2979484367 @default.
- W3185853715 cites W2995744795 @default.
- W3185853715 cites W2995975522 @default.
- W3185853715 cites W2996492157 @default.
- W3185853715 cites W3001822488 @default.
- W3185853715 cites W3034855424 @default.
- W3185853715 cites W3034933105 @default.
- W3185853715 cites W3037052212 @default.
- W3185853715 cites W3094397926 @default.
- W3185853715 cites W3102041518 @default.
- W3185853715 cites W3104298168 @default.
- W3185853715 cites W3110996603 @default.
- W3185853715 cites W3119586787 @default.
- W3185853715 cites W3125537303 @default.
- W3185853715 cites W3126988966 @default.
- W3185853715 cites W3146803896 @default.
- W3185853715 cites W3157282745 @default.
- W3185853715 doi "https://doi.org/10.48550/arxiv.2107.13163" @default.
- W3185853715 hasPublicationYear "2021" @default.
- W3185853715 type Work @default.
- W3185853715 sameAs 3185853715 @default.
- W3185853715 citedByCount "1" @default.
- W3185853715 countsByYear W31858537152021 @default.
- W3185853715 crossrefType "posted-content" @default.
- W3185853715 hasAuthorship W3185853715A5055799536 @default.
- W3185853715 hasAuthorship W3185853715A5061905935 @default.
- W3185853715 hasAuthorship W3185853715A5066483084 @default.
- W3185853715 hasBestOaLocation W31858537151 @default.
- W3185853715 hasConcept C105795698 @default.
- W3185853715 hasConcept C11413529 @default.
- W3185853715 hasConcept C118615104 @default.
- W3185853715 hasConcept C119322782 @default.
- W3185853715 hasConcept C129848803 @default.
- W3185853715 hasConcept C131671149 @default.
- W3185853715 hasConcept C134306372 @default.
- W3185853715 hasConcept C141796577 @default.
- W3185853715 hasConcept C144271403 @default.
- W3185853715 hasConcept C15190523 @default.
- W3185853715 hasConcept C154945302 @default.
- W3185853715 hasConcept C187455244 @default.
- W3185853715 hasConcept C2777723229 @default.
- W3185853715 hasConcept C29248071 @default.
- W3185853715 hasConcept C33923547 @default.
- W3185853715 hasConcept C34388435 @default.
- W3185853715 hasConcept C41008148 @default.
- W3185853715 hasConcept C45374587 @default.
- W3185853715 hasConcept C48415503 @default.
- W3185853715 hasConcept C50644808 @default.
- W3185853715 hasConcept C80444323 @default.
- W3185853715 hasConcept C91873725 @default.
- W3185853715 hasConceptScore W3185853715C105795698 @default.
- W3185853715 hasConceptScore W3185853715C11413529 @default.
- W3185853715 hasConceptScore W3185853715C118615104 @default.
- W3185853715 hasConceptScore W3185853715C119322782 @default.
- W3185853715 hasConceptScore W3185853715C129848803 @default.
- W3185853715 hasConceptScore W3185853715C131671149 @default.