Matches in SemOpenAlex for { <https://semopenalex.org/work/W3136297071> ?p ?o ?g. }
- W3136297071 endingPage "76" @default.
- W3136297071 startingPage "35" @default.
- W3136297071 abstract "In this paper, we propose and analyze zeroth-order stochastic approximation algorithms for nonconvex and convex optimization, with a focus on addressing constrained optimization, high-dimensional setting, and saddle point avoiding. To handle constrained optimization, we first propose generalizations of the conditional gradient algorithm achieving rates similar to the standard stochastic gradient algorithm using only zeroth-order information. To facilitate zeroth-order optimization in high dimensions, we explore the advantages of structural sparsity assumptions. Specifically, (i) we highlight an implicit regularization phenomenon where the standard stochastic gradient algorithm with zeroth-order information adapts to the sparsity of the problem at hand by just varying the step size and (ii) propose a truncated stochastic gradient algorithm with zeroth-order information, whose rate of convergence depends only poly-logarithmically on the dimensionality. We next focus on avoiding saddle points in nonconvex setting. Toward that, we interpret the Gaussian smoothing technique for estimating gradient based on zeroth-order information as an instantiation of first-order Stein’s identity. Based on this, we provide a novel linear-(in dimension) time estimator of the Hessian matrix of a function using only zeroth-order information, which is based on second-order Stein’s identity. We then provide a zeroth-order variant of cubic regularized Newton method for avoiding saddle points and discuss its rate of convergence to local minima." @default.
- W3136297071 created "2021-03-29" @default.
- W3136297071 creator A5014500490 @default.
- W3136297071 creator A5063746021 @default.
- W3136297071 date "2021-03-19" @default.
- W3136297071 modified "2023-10-01" @default.
- W3136297071 title "Zeroth-Order Nonconvex Stochastic Optimization: Handling Constraints, High Dimensionality, and Saddle Points" @default.
- W3136297071 cites W1568307856 @default.
- W3136297071 cites W181733065 @default.
- W3136297071 cites W1985768560 @default.
- W3136297071 cites W1994974865 @default.
- W3136297071 cites W1996674925 @default.
- W3136297071 cites W2009941369 @default.
- W3136297071 cites W2054640142 @default.
- W3136297071 cites W2079135578 @default.
- W3136297071 cites W2124541940 @default.
- W3136297071 cites W2136128169 @default.
- W3136297071 cites W2136885855 @default.
- W3136297071 cites W2147632348 @default.
- W3136297071 cites W2149479912 @default.
- W3136297071 cites W2156005216 @default.
- W3136297071 cites W2171830216 @default.
- W3136297071 cites W2188195356 @default.
- W3136297071 cites W2264518955 @default.
- W3136297071 cites W2546420264 @default.
- W3136297071 cites W2586930666 @default.
- W3136297071 cites W2746600820 @default.
- W3136297071 cites W2753896574 @default.
- W3136297071 cites W2763081248 @default.
- W3136297071 cites W2772283936 @default.
- W3136297071 cites W2808862920 @default.
- W3136297071 cites W2950929549 @default.
- W3136297071 cites W2962867868 @default.
- W3136297071 cites W2963142706 @default.
- W3136297071 cites W2963307318 @default.
- W3136297071 cites W2963368181 @default.
- W3136297071 cites W2963470657 @default.
- W3136297071 cites W2963831922 @default.
- W3136297071 cites W2985213689 @default.
- W3136297071 cites W3113425034 @default.
- W3136297071 cites W4250589301 @default.
- W3136297071 cites W4250955649 @default.
- W3136297071 cites W599176452 @default.
- W3136297071 doi "https://doi.org/10.1007/s10208-021-09499-8" @default.
- W3136297071 hasPublicationYear "2021" @default.
- W3136297071 type Work @default.
- W3136297071 sameAs 3136297071 @default.
- W3136297071 citedByCount "14" @default.
- W3136297071 countsByYear W31362970712021 @default.
- W3136297071 countsByYear W31362970712022 @default.
- W3136297071 countsByYear W31362970712023 @default.
- W3136297071 crossrefType "journal-article" @default.
- W3136297071 hasAuthorship W3136297071A5014500490 @default.
- W3136297071 hasAuthorship W3136297071A5063746021 @default.
- W3136297071 hasBestOaLocation W31362970712 @default.
- W3136297071 hasConcept C105795698 @default.
- W3136297071 hasConcept C111030470 @default.
- W3136297071 hasConcept C11413529 @default.
- W3136297071 hasConcept C121332964 @default.
- W3136297071 hasConcept C126255220 @default.
- W3136297071 hasConcept C194387892 @default.
- W3136297071 hasConcept C203616005 @default.
- W3136297071 hasConcept C2524010 @default.
- W3136297071 hasConcept C26517878 @default.
- W3136297071 hasConcept C2681867 @default.
- W3136297071 hasConcept C28826006 @default.
- W3136297071 hasConcept C33923547 @default.
- W3136297071 hasConcept C38652104 @default.
- W3136297071 hasConcept C41008148 @default.
- W3136297071 hasConcept C57869625 @default.
- W3136297071 hasConcept C62520636 @default.
- W3136297071 hasConcept C68755742 @default.
- W3136297071 hasConceptScore W3136297071C105795698 @default.
- W3136297071 hasConceptScore W3136297071C111030470 @default.
- W3136297071 hasConceptScore W3136297071C11413529 @default.
- W3136297071 hasConceptScore W3136297071C121332964 @default.
- W3136297071 hasConceptScore W3136297071C126255220 @default.
- W3136297071 hasConceptScore W3136297071C194387892 @default.
- W3136297071 hasConceptScore W3136297071C203616005 @default.
- W3136297071 hasConceptScore W3136297071C2524010 @default.
- W3136297071 hasConceptScore W3136297071C26517878 @default.
- W3136297071 hasConceptScore W3136297071C2681867 @default.
- W3136297071 hasConceptScore W3136297071C28826006 @default.
- W3136297071 hasConceptScore W3136297071C33923547 @default.
- W3136297071 hasConceptScore W3136297071C38652104 @default.
- W3136297071 hasConceptScore W3136297071C41008148 @default.
- W3136297071 hasConceptScore W3136297071C57869625 @default.
- W3136297071 hasConceptScore W3136297071C62520636 @default.
- W3136297071 hasConceptScore W3136297071C68755742 @default.
- W3136297071 hasIssue "1" @default.
- W3136297071 hasLocation W31362970711 @default.
- W3136297071 hasLocation W31362970712 @default.
- W3136297071 hasOpenAccess W3136297071 @default.
- W3136297071 hasPrimaryLocation W31362970711 @default.
- W3136297071 hasRelatedWork W2025554373 @default.
- W3136297071 hasRelatedWork W2095374904 @default.
- W3136297071 hasRelatedWork W2152074179 @default.
- W3136297071 hasRelatedWork W2169345436 @default.