Matches in SemOpenAlex for { <https://semopenalex.org/work/W4287417336> ?p ?o ?g. }
Showing items 1 to 81 of
81
with 100 items per page.
- W4287417336 abstract "Stochastic gradient descent (SGD) is a cornerstone of machine learning. When the number N of data items is large, SGD relies on constructing an unbiased estimator of the gradient of the empirical risk using a small subset of the original dataset, called a minibatch. Default minibatch construction involves uniformly sampling a subset of the desired size, but alternatives have been explored for variance reduction. In particular, experimental evidence suggests drawing minibatches from determinantal point processes (DPPs), distributions over minibatches that favour diversity among selected items. However, like in recent work on DPPs for coresets, providing a systematic and principled understanding of how and why DPPs help has been difficult. In this work, we contribute an orthogonal polynomial-based DPP paradigm for minibatch sampling in SGD. Our approach leverages the specific data distribution at hand, which endows it with greater sensitivity and power over existing data-agnostic methods. We substantiate our method via a detailed theoretical analysis of its convergence properties, interweaving between the discrete data set and the underlying continuous domain. In particular, we show how specific DPPs and a string of controlled approximations can lead to gradient estimators with a variance that decays faster with the batchsize than under uniform sampling. Coupled with existing finite-time guarantees for SGD on convex objectives, this entails that, DPP minibatches lead to a smaller bound on the mean square approximation error than uniform minibatches. Moreover, our estimators are amenable to a recent algorithm that directly samples linear statistics of DPPs (i.e., the gradient estimator) without sampling the underlying DPP (i.e., the minibatch), thereby reducing computational overhead. We provide detailed synthetic as well as real data experiments to substantiate our theoretical claims." @default.
- W4287417336 created "2022-07-25" @default.
- W4287417336 creator A5036123893 @default.
- W4287417336 creator A5042756211 @default.
- W4287417336 creator A5062461687 @default.
- W4287417336 date "2021-12-11" @default.
- W4287417336 modified "2023-10-09" @default.
- W4287417336 title "Determinantal point processes based on orthogonal polynomials for sampling minibatches in SGD" @default.
- W4287417336 doi "https://doi.org/10.48550/arxiv.2112.06007" @default.
- W4287417336 hasPublicationYear "2021" @default.
- W4287417336 type Work @default.
- W4287417336 citedByCount "0" @default.
- W4287417336 crossrefType "posted-content" @default.
- W4287417336 hasAuthorship W4287417336A5036123893 @default.
- W4287417336 hasAuthorship W4287417336A5042756211 @default.
- W4287417336 hasAuthorship W4287417336A5062461687 @default.
- W4287417336 hasBestOaLocation W42874173361 @default.
- W4287417336 hasConcept C105795698 @default.
- W4287417336 hasConcept C106131492 @default.
- W4287417336 hasConcept C11413529 @default.
- W4287417336 hasConcept C121955636 @default.
- W4287417336 hasConcept C126255220 @default.
- W4287417336 hasConcept C134306372 @default.
- W4287417336 hasConcept C140779682 @default.
- W4287417336 hasConcept C144133560 @default.
- W4287417336 hasConcept C154945302 @default.
- W4287417336 hasConcept C162324750 @default.
- W4287417336 hasConcept C185429906 @default.
- W4287417336 hasConcept C19499675 @default.
- W4287417336 hasConcept C196083921 @default.
- W4287417336 hasConcept C206688291 @default.
- W4287417336 hasConcept C2777303404 @default.
- W4287417336 hasConcept C28826006 @default.
- W4287417336 hasConcept C31972630 @default.
- W4287417336 hasConcept C33923547 @default.
- W4287417336 hasConcept C41008148 @default.
- W4287417336 hasConcept C50522688 @default.
- W4287417336 hasConcept C50644808 @default.
- W4287417336 hasConcept C62644790 @default.
- W4287417336 hasConcept C90119067 @default.
- W4287417336 hasConceptScore W4287417336C105795698 @default.
- W4287417336 hasConceptScore W4287417336C106131492 @default.
- W4287417336 hasConceptScore W4287417336C11413529 @default.
- W4287417336 hasConceptScore W4287417336C121955636 @default.
- W4287417336 hasConceptScore W4287417336C126255220 @default.
- W4287417336 hasConceptScore W4287417336C134306372 @default.
- W4287417336 hasConceptScore W4287417336C140779682 @default.
- W4287417336 hasConceptScore W4287417336C144133560 @default.
- W4287417336 hasConceptScore W4287417336C154945302 @default.
- W4287417336 hasConceptScore W4287417336C162324750 @default.
- W4287417336 hasConceptScore W4287417336C185429906 @default.
- W4287417336 hasConceptScore W4287417336C19499675 @default.
- W4287417336 hasConceptScore W4287417336C196083921 @default.
- W4287417336 hasConceptScore W4287417336C206688291 @default.
- W4287417336 hasConceptScore W4287417336C2777303404 @default.
- W4287417336 hasConceptScore W4287417336C28826006 @default.
- W4287417336 hasConceptScore W4287417336C31972630 @default.
- W4287417336 hasConceptScore W4287417336C33923547 @default.
- W4287417336 hasConceptScore W4287417336C41008148 @default.
- W4287417336 hasConceptScore W4287417336C50522688 @default.
- W4287417336 hasConceptScore W4287417336C50644808 @default.
- W4287417336 hasConceptScore W4287417336C62644790 @default.
- W4287417336 hasConceptScore W4287417336C90119067 @default.
- W4287417336 hasLocation W42874173361 @default.
- W4287417336 hasLocation W42874173362 @default.
- W4287417336 hasLocation W42874173363 @default.
- W4287417336 hasOpenAccess W4287417336 @default.
- W4287417336 hasPrimaryLocation W42874173361 @default.
- W4287417336 hasRelatedWork W1544940847 @default.
- W4287417336 hasRelatedWork W1977112355 @default.
- W4287417336 hasRelatedWork W2107438106 @default.
- W4287417336 hasRelatedWork W2169367269 @default.
- W4287417336 hasRelatedWork W2289285490 @default.
- W4287417336 hasRelatedWork W2298254442 @default.
- W4287417336 hasRelatedWork W2950038056 @default.
- W4287417336 hasRelatedWork W3031816500 @default.
- W4287417336 hasRelatedWork W3150274214 @default.
- W4287417336 hasRelatedWork W4283750944 @default.
- W4287417336 isParatext "false" @default.
- W4287417336 isRetracted "false" @default.
- W4287417336 workType "article" @default.