FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3040308834> ?p ?o ?g. }

• W3040308834 abstract "In distributed second order optimization, a standard strategy is to average many local estimates, each of which is based on a small sketch or batch of the data. However, the local estimates on each machine are typically biased, relative to the full solution on all of the data, and this can limit the effectiveness of averaging. Here, we introduce a new technique for debiasing the local estimates, which leads to both theoretical and empirical improvements in the convergence rate of distributed second order methods. Our technique has two novel components: (1) modifying standard sketching techniques to obtain what we call a surrogate sketch; and (2) carefully scaling the global regularization parameter for local computations. Our surrogate sketches are based on determinantal point processes, a family of distributions for which the bias of an estimate of the inverse Hessian can be computed exactly. Based on this computation, we show that when the objective being minimized is $l_2$-regularized with parameter $lambda$ and individual machines are each given a sketch of size $m$, then to eliminate the bias, local estimates should be computed using a shrunk regularization parameter given by $lambda^{prime}=lambdacdot(1-frac{d_{lambda}}{m})$, where $d_{lambda}$ is the $lambda$-effective dimension of the Hessian (or, for quadratic problems, the data matrix)." @default.
• W3040308834 created "2020-07-10" @default.
• W3040308834 creator A5001436196 @default.
• W3040308834 creator A5013422880 @default.
• W3040308834 creator A5033006662 @default.
• W3040308834 creator A5062073113 @default.
• W3040308834 date "2020-07-02" @default.
• W3040308834 modified "2023-09-27" @default.
• W3040308834 title "Debiasing Distributed Second Order Optimization with Surrogate Sketching and Scaled Regularization" @default.
• W3040308834 cites W2008095352 @default.
• W3040308834 cites W2144164089 @default.
• W3040308834 cites W2185932763 @default.
• W3040308834 cites W2510078447 @default.
• W3040308834 cites W2537046506 @default.
• W3040308834 cites W2580753685 @default.
• W3040308834 cites W2615267611 @default.
• W3040308834 cites W2616345629 @default.
• W3040308834 cites W2888492004 @default.
• W3040308834 cites W2947648688 @default.
• W3040308834 cites W2948501930 @default.
• W3040308834 cites W2962937842 @default.
• W3040308834 cites W2963002486 @default.
• W3040308834 cites W2963060476 @default.
• W3040308834 cites W2963172825 @default.
• W3040308834 cites W2963214110 @default.
• W3040308834 cites W2963335515 @default.
• W3040308834 cites W2963459305 @default.
• W3040308834 cites W2963693254 @default.
• W3040308834 cites W2963992805 @default.
• W3040308834 cites W2964507278 @default.
• W3040308834 cites W2965683504 @default.
• W3040308834 cites W2970449350 @default.
• W3040308834 cites W2971199077 @default.
• W3040308834 cites W2988331362 @default.
• W3040308834 cites W2993182292 @default.
• W3040308834 cites W2996279083 @default.
• W3040308834 cites W3004280950 @default.
• W3040308834 cites W3005983886 @default.
• W3040308834 cites W3006441070 @default.
• W3040308834 cites W3034580602 @default.
• W3040308834 cites W3034919836 @default.
• W3040308834 cites W3103014337 @default.
• W3040308834 cites W3103183085 @default.
• W3040308834 cites W3104005230 @default.
• W3040308834 cites W3139361274 @default.
• W3040308834 cites W316798226 @default.
• W3040308834 hasPublicationYear "2020" @default.
• W3040308834 type Work @default.
• W3040308834 sameAs 3040308834 @default.
• W3040308834 citedByCount "0" @default.
• W3040308834 crossrefType "posted-content" @default.
• W3040308834 hasAuthorship W3040308834A5001436196 @default.
• W3040308834 hasAuthorship W3040308834A5013422880 @default.
• W3040308834 hasAuthorship W3040308834A5033006662 @default.
• W3040308834 hasAuthorship W3040308834A5062073113 @default.
• W3040308834 hasConcept C11413529 @default.
• W3040308834 hasConcept C120665830 @default.
• W3040308834 hasConcept C121332964 @default.
• W3040308834 hasConcept C126255220 @default.
• W3040308834 hasConcept C129844170 @default.
• W3040308834 hasConcept C154945302 @default.
• W3040308834 hasConcept C203616005 @default.
• W3040308834 hasConcept C2524010 @default.
• W3040308834 hasConcept C2776135515 @default.
• W3040308834 hasConcept C2778113609 @default.
• W3040308834 hasConcept C28826006 @default.
• W3040308834 hasConcept C33923547 @default.
• W3040308834 hasConcept C41008148 @default.
• W3040308834 hasConcept C45374587 @default.
• W3040308834 hasConceptScore W3040308834C11413529 @default.
• W3040308834 hasConceptScore W3040308834C120665830 @default.
• W3040308834 hasConceptScore W3040308834C121332964 @default.
• W3040308834 hasConceptScore W3040308834C126255220 @default.
• W3040308834 hasConceptScore W3040308834C129844170 @default.
• W3040308834 hasConceptScore W3040308834C154945302 @default.
• W3040308834 hasConceptScore W3040308834C203616005 @default.
• W3040308834 hasConceptScore W3040308834C2524010 @default.
• W3040308834 hasConceptScore W3040308834C2776135515 @default.
• W3040308834 hasConceptScore W3040308834C2778113609 @default.
• W3040308834 hasConceptScore W3040308834C28826006 @default.
• W3040308834 hasConceptScore W3040308834C33923547 @default.
• W3040308834 hasConceptScore W3040308834C41008148 @default.
• W3040308834 hasConceptScore W3040308834C45374587 @default.
• W3040308834 hasLocation W30403088341 @default.
• W3040308834 hasOpenAccess W3040308834 @default.
• W3040308834 hasPrimaryLocation W30403088341 @default.
• W3040308834 hasRelatedWork W1562852809 @default.
• W3040308834 hasRelatedWork W1799913961 @default.
• W3040308834 hasRelatedWork W2158469249 @default.
• W3040308834 hasRelatedWork W2549640852 @default.
• W3040308834 hasRelatedWork W2852105405 @default.
• W3040308834 hasRelatedWork W2884027952 @default.
• W3040308834 hasRelatedWork W2898060137 @default.
• W3040308834 hasRelatedWork W2949742625 @default.
• W3040308834 hasRelatedWork W2950409937 @default.
• W3040308834 hasRelatedWork W2962736036 @default.
• W3040308834 hasRelatedWork W2963483869 @default.
• W3040308834 hasRelatedWork W2994974560 @default.
• W3040308834 hasRelatedWork W3084415865 @default.
• W3040308834 hasRelatedWork W3085857580 @default.

• next