Matches in SemOpenAlex for { <https://semopenalex.org/work/W3117575511> ?p ?o ?g. }
Showing items 1 to 84 of
84
with 100 items per page.
- W3117575511 endingPage "124001" @default.
- W3117575511 startingPage "124001" @default.
- W3117575511 abstract "How many training data are needed to learn a supervised task? It is often observed that the generalization error decreases as $n^{-beta}$ where $n$ is the number of training examples and $beta$ an exponent that depends on both data and algorithm. In this work we measure $beta$ when applying kernel methods to real datasets. For MNIST we find $betaapprox 0.4$ and for CIFAR10 $betaapprox 0.1$, for both regression and classification tasks, and for Gaussian or Laplace kernels. To rationalize the existence of non-trivial exponents that can be independent of the specific kernel used, we study the Teacher-Student framework for kernels. In this scheme, a Teacher generates data according to a Gaussian random field, and a Student learns them via kernel regression. With a simplifying assumption -- namely that the data are sampled from a regular lattice -- we derive analytically $beta$ for translation invariant kernels, using previous results from the kriging literature. Provided that the Student is not too sensitive to high frequencies, $beta$ depends only on the smoothness and dimension of the training data. We confirm numerically that these predictions hold when the training points are sampled at random on a hypersphere. Overall, the test error is found to be controlled by the magnitude of the projection of the true function on the kernel eigenvectors whose rank is larger than $n$. Using this idea we predict relate the exponent $beta$ to an exponent $a$ describing how the coefficients of the true function in the eigenbasis of the kernel decay with rank. We extract $a$ from real data by performing kernel PCA, leading to $betaapprox0.36$ for MNIST and $betaapprox0.07$ for CIFAR10, in good agreement with observations. We argue that these rather large exponents are possible due to the small effective dimension of the data." @default.
- W3117575511 created "2021-01-05" @default.
- W3117575511 creator A5002843500 @default.
- W3117575511 creator A5019813807 @default.
- W3117575511 creator A5074827720 @default.
- W3117575511 date "2020-12-01" @default.
- W3117575511 modified "2023-10-16" @default.
- W3117575511 title "Asymptotic learning curves of kernel methods: empirical data versus teacher–student paradigm" @default.
- W3117575511 cites W1980906795 @default.
- W3117575511 cites W1995842804 @default.
- W3117575511 cites W2007154098 @default.
- W3117575511 cites W2007832013 @default.
- W3117575511 cites W2012501405 @default.
- W3117575511 cites W2022845879 @default.
- W3117575511 cites W2024697317 @default.
- W3117575511 cites W2029401646 @default.
- W3117575511 cites W2072072671 @default.
- W3117575511 cites W2754478492 @default.
- W3117575511 cites W2918745211 @default.
- W3117575511 cites W3020315344 @default.
- W3117575511 doi "https://doi.org/10.1088/1742-5468/abc61d" @default.
- W3117575511 hasPublicationYear "2020" @default.
- W3117575511 type Work @default.
- W3117575511 sameAs 3117575511 @default.
- W3117575511 citedByCount "27" @default.
- W3117575511 countsByYear W31175755112020 @default.
- W3117575511 countsByYear W31175755112021 @default.
- W3117575511 countsByYear W31175755112022 @default.
- W3117575511 countsByYear W31175755112023 @default.
- W3117575511 crossrefType "journal-article" @default.
- W3117575511 hasAuthorship W3117575511A5002843500 @default.
- W3117575511 hasAuthorship W3117575511A5019813807 @default.
- W3117575511 hasAuthorship W3117575511A5074827720 @default.
- W3117575511 hasBestOaLocation W31175755112 @default.
- W3117575511 hasConcept C118615104 @default.
- W3117575511 hasConcept C121332964 @default.
- W3117575511 hasConcept C122280245 @default.
- W3117575511 hasConcept C12267149 @default.
- W3117575511 hasConcept C154945302 @default.
- W3117575511 hasConcept C163716315 @default.
- W3117575511 hasConcept C190502265 @default.
- W3117575511 hasConcept C195699287 @default.
- W3117575511 hasConcept C33923547 @default.
- W3117575511 hasConcept C41008148 @default.
- W3117575511 hasConcept C50644808 @default.
- W3117575511 hasConcept C62520636 @default.
- W3117575511 hasConcept C7218915 @default.
- W3117575511 hasConcept C74193536 @default.
- W3117575511 hasConceptScore W3117575511C118615104 @default.
- W3117575511 hasConceptScore W3117575511C121332964 @default.
- W3117575511 hasConceptScore W3117575511C122280245 @default.
- W3117575511 hasConceptScore W3117575511C12267149 @default.
- W3117575511 hasConceptScore W3117575511C154945302 @default.
- W3117575511 hasConceptScore W3117575511C163716315 @default.
- W3117575511 hasConceptScore W3117575511C190502265 @default.
- W3117575511 hasConceptScore W3117575511C195699287 @default.
- W3117575511 hasConceptScore W3117575511C33923547 @default.
- W3117575511 hasConceptScore W3117575511C41008148 @default.
- W3117575511 hasConceptScore W3117575511C50644808 @default.
- W3117575511 hasConceptScore W3117575511C62520636 @default.
- W3117575511 hasConceptScore W3117575511C7218915 @default.
- W3117575511 hasConceptScore W3117575511C74193536 @default.
- W3117575511 hasIssue "12" @default.
- W3117575511 hasLocation W31175755111 @default.
- W3117575511 hasLocation W31175755112 @default.
- W3117575511 hasOpenAccess W3117575511 @default.
- W3117575511 hasPrimaryLocation W31175755111 @default.
- W3117575511 hasRelatedWork W145098650 @default.
- W3117575511 hasRelatedWork W1995926157 @default.
- W3117575511 hasRelatedWork W2098028537 @default.
- W3117575511 hasRelatedWork W2099577980 @default.
- W3117575511 hasRelatedWork W2113751036 @default.
- W3117575511 hasRelatedWork W2155899303 @default.
- W3117575511 hasRelatedWork W2372482000 @default.
- W3117575511 hasRelatedWork W2388787028 @default.
- W3117575511 hasRelatedWork W2417556355 @default.
- W3117575511 hasRelatedWork W4225083764 @default.
- W3117575511 hasVolume "2020" @default.
- W3117575511 isParatext "false" @default.
- W3117575511 isRetracted "false" @default.
- W3117575511 magId "3117575511" @default.
- W3117575511 workType "article" @default.