FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3095890476> ?p ?o ?g. }

• W3095890476 abstract "In this paper, we consider the problem of empirical risk minimization (ERM) of smooth, strongly convex loss functions using iterative gradient-based methods. A major goal of this literature has been to compare different algorithms, such as gradient descent (GD) or stochastic gradient descent (SGD), by analyzing their rates of convergence to $epsilon$-approximate solutions. For example, the oracle complexity of GD is $O(nlog(epsilon^{-1}))$, where $n$ is the number of training samples. When $n$ is large, this can be expensive in practice, and SGD is preferred due to its oracle complexity of $O(epsilon^{-1})$. Such standard analyses only utilize the smoothness of the loss function in the parameter being optimized. In contrast, we demonstrate that when the loss function is smooth in the data, we can learn the oracle at every iteration and beat the oracle complexities of both GD and SGD in important regimes. Specifically, at every iteration, our proposed algorithm performs local polynomial regression to learn the gradient of the loss function, and then estimates the true gradient of the ERM objective function. We establish that the oracle complexity of our algorithm scales like $tilde{O}((p epsilon^{-1})^{d/(2eta)})$ (neglecting sub-dominant factors), where $d$ and $p$ are the data and parameter space dimensions, respectively, and the gradient of the loss function belongs to a $eta$-Holder class with respect to the data. Our proof extends the analysis of local polynomial regression in non-parametric statistics to provide interpolation guarantees in multivariate settings, and also exploits tools from the inexact GD literature. Unlike GD and SGD, the complexity of our method depends on $d$ and $p$. However, when $d$ is small and the loss function exhibits modest smoothness in the data, our algorithm beats GD and SGD in oracle complexity for a very broad range of $p$ and $epsilon$." @default.
• W3095890476 created "2020-11-09" @default.
• W3095890476 creator A5029499294 @default.
• W3095890476 creator A5073197040 @default.
• W3095890476 creator A5078288116 @default.
• W3095890476 date "2020-11-04" @default.
• W3095890476 modified "2023-09-27" @default.
• W3095890476 title "Gradient-Based Empirical Risk Minimization using Local Polynomial Regression." @default.
• W3095890476 cites W1490488263 @default.
• W3095890476 cites W1502001434 @default.
• W3095890476 cites W1505731132 @default.
• W3095890476 cites W1511694993 @default.
• W3095890476 cites W1554944419 @default.
• W3095890476 cites W1663973292 @default.
• W3095890476 cites W1966268321 @default.
• W3095890476 cites W1988078417 @default.
• W3095890476 cites W1992208280 @default.
• W3095890476 cites W2014497688 @default.
• W3095890476 cites W2025341678 @default.
• W3095890476 cites W2071128523 @default.
• W3095890476 cites W2088658556 @default.
• W3095890476 cites W2095209741 @default.
• W3095890476 cites W2097462699 @default.
• W3095890476 cites W2100675398 @default.
• W3095890476 cites W2107438106 @default.
• W3095890476 cites W2122678453 @default.
• W3095890476 cites W2130062883 @default.
• W3095890476 cites W2144366961 @default.
• W3095890476 cites W2156718197 @default.
• W3095890476 cites W2166903773 @default.
• W3095890476 cites W2294798173 @default.
• W3095890476 cites W2411801785 @default.
• W3095890476 cites W2610857016 @default.
• W3095890476 cites W2751862591 @default.
• W3095890476 cites W2913715341 @default.
• W3095890476 cites W2962834995 @default.
• W3095890476 cites W2963208729 @default.
• W3095890476 cites W2963483869 @default.
• W3095890476 cites W2974641549 @default.
• W3095890476 cites W2991489140 @default.
• W3095890476 cites W2995896684 @default.
• W3095890476 cites W3034344439 @default.
• W3095890476 cites W3103657382 @default.
• W3095890476 cites W3141595720 @default.
• W3095890476 cites W3157614448 @default.
• W3095890476 hasPublicationYear "2020" @default.
• W3095890476 type Work @default.
• W3095890476 sameAs 3095890476 @default.
• W3095890476 citedByCount "0" @default.
• W3095890476 crossrefType "posted-content" @default.
• W3095890476 hasAuthorship W3095890476A5029499294 @default.
• W3095890476 hasAuthorship W3095890476A5073197040 @default.
• W3095890476 hasAuthorship W3095890476A5078288116 @default.
• W3095890476 hasConcept C102634674 @default.
• W3095890476 hasConcept C105795698 @default.
• W3095890476 hasConcept C107321475 @default.
• W3095890476 hasConcept C112680207 @default.
• W3095890476 hasConcept C11413529 @default.
• W3095890476 hasConcept C115903868 @default.
• W3095890476 hasConcept C117251300 @default.
• W3095890476 hasConcept C126255220 @default.
• W3095890476 hasConcept C134306372 @default.
• W3095890476 hasConcept C14036430 @default.
• W3095890476 hasConcept C145446738 @default.
• W3095890476 hasConcept C153258448 @default.
• W3095890476 hasConcept C154945302 @default.
• W3095890476 hasConcept C206688291 @default.
• W3095890476 hasConcept C22324862 @default.
• W3095890476 hasConcept C2524010 @default.
• W3095890476 hasConcept C28826006 @default.
• W3095890476 hasConcept C33923547 @default.
• W3095890476 hasConcept C41008148 @default.
• W3095890476 hasConcept C50644808 @default.
• W3095890476 hasConcept C55166926 @default.
• W3095890476 hasConcept C78458016 @default.
• W3095890476 hasConcept C83546350 @default.
• W3095890476 hasConcept C86803240 @default.
• W3095890476 hasConcept C90119067 @default.
• W3095890476 hasConceptScore W3095890476C102634674 @default.
• W3095890476 hasConceptScore W3095890476C105795698 @default.
• W3095890476 hasConceptScore W3095890476C107321475 @default.
• W3095890476 hasConceptScore W3095890476C112680207 @default.
• W3095890476 hasConceptScore W3095890476C11413529 @default.
• W3095890476 hasConceptScore W3095890476C115903868 @default.
• W3095890476 hasConceptScore W3095890476C117251300 @default.
• W3095890476 hasConceptScore W3095890476C126255220 @default.
• W3095890476 hasConceptScore W3095890476C134306372 @default.
• W3095890476 hasConceptScore W3095890476C14036430 @default.
• W3095890476 hasConceptScore W3095890476C145446738 @default.
• W3095890476 hasConceptScore W3095890476C153258448 @default.
• W3095890476 hasConceptScore W3095890476C154945302 @default.
• W3095890476 hasConceptScore W3095890476C206688291 @default.
• W3095890476 hasConceptScore W3095890476C22324862 @default.
• W3095890476 hasConceptScore W3095890476C2524010 @default.
• W3095890476 hasConceptScore W3095890476C28826006 @default.
• W3095890476 hasConceptScore W3095890476C33923547 @default.
• W3095890476 hasConceptScore W3095890476C41008148 @default.
• W3095890476 hasConceptScore W3095890476C50644808 @default.
• W3095890476 hasConceptScore W3095890476C55166926 @default.
• W3095890476 hasConceptScore W3095890476C78458016 @default.

• next