Matches in SemOpenAlex for { <https://semopenalex.org/work/W4313304467> ?p ?o ?g. }
Showing items 1 to 55 of
55
with 100 items per page.
- W4313304467 abstract "Many problems in machine learning involve bilevel optimization (BLO), including hyperparameter optimization, meta-learning, and dataset distillation. Bilevel problems consist of two nested sub-problems, called the outer and inner problems, respectively. In practice, often at least one of these sub-problems is overparameterized. In this case, there are many ways to choose among optima that achieve equivalent objective values. Inspired by recent studies of the implicit bias induced by optimization algorithms in single-level optimization, we investigate the implicit bias of gradient-based algorithms for bilevel optimization. We delineate two standard BLO methods -- cold-start and warm-start -- and show that the converged solution or long-run behavior depends to a large degree on these and other algorithmic choices, such as the hypergradient approximation. We also show that the inner solutions obtained by warm-start BLO can encode a surprising amount of information about the outer objective, even when the outer parameters are low-dimensional. We believe that implicit bias deserves as central a role in the study of bilevel optimization as it has attained in the study of single-level neural net optimization." @default.
- W4313304467 created "2023-01-06" @default.
- W4313304467 creator A5008480143 @default.
- W4313304467 creator A5030409494 @default.
- W4313304467 creator A5030776397 @default.
- W4313304467 creator A5066572762 @default.
- W4313304467 creator A5067036768 @default.
- W4313304467 date "2022-12-28" @default.
- W4313304467 modified "2023-09-23" @default.
- W4313304467 title "On Implicit Bias in Overparameterized Bilevel Optimization" @default.
- W4313304467 doi "https://doi.org/10.48550/arxiv.2212.14032" @default.
- W4313304467 hasPublicationYear "2022" @default.
- W4313304467 type Work @default.
- W4313304467 citedByCount "0" @default.
- W4313304467 crossrefType "posted-content" @default.
- W4313304467 hasAuthorship W4313304467A5008480143 @default.
- W4313304467 hasAuthorship W4313304467A5030409494 @default.
- W4313304467 hasAuthorship W4313304467A5030776397 @default.
- W4313304467 hasAuthorship W4313304467A5066572762 @default.
- W4313304467 hasAuthorship W4313304467A5067036768 @default.
- W4313304467 hasBestOaLocation W43133044671 @default.
- W4313304467 hasConcept C11413529 @default.
- W4313304467 hasConcept C126255220 @default.
- W4313304467 hasConcept C137836250 @default.
- W4313304467 hasConcept C154945302 @default.
- W4313304467 hasConcept C2987595161 @default.
- W4313304467 hasConcept C3309286 @default.
- W4313304467 hasConcept C33923547 @default.
- W4313304467 hasConcept C41008148 @default.
- W4313304467 hasConcept C8642999 @default.
- W4313304467 hasConceptScore W4313304467C11413529 @default.
- W4313304467 hasConceptScore W4313304467C126255220 @default.
- W4313304467 hasConceptScore W4313304467C137836250 @default.
- W4313304467 hasConceptScore W4313304467C154945302 @default.
- W4313304467 hasConceptScore W4313304467C2987595161 @default.
- W4313304467 hasConceptScore W4313304467C3309286 @default.
- W4313304467 hasConceptScore W4313304467C33923547 @default.
- W4313304467 hasConceptScore W4313304467C41008148 @default.
- W4313304467 hasConceptScore W4313304467C8642999 @default.
- W4313304467 hasLocation W43133044671 @default.
- W4313304467 hasOpenAccess W4313304467 @default.
- W4313304467 hasPrimaryLocation W43133044671 @default.
- W4313304467 hasRelatedWork W1994436307 @default.
- W4313304467 hasRelatedWork W2000976787 @default.
- W4313304467 hasRelatedWork W2340378315 @default.
- W4313304467 hasRelatedWork W2745612259 @default.
- W4313304467 hasRelatedWork W2795565301 @default.
- W4313304467 hasRelatedWork W2896140431 @default.
- W4313304467 hasRelatedWork W3044651776 @default.
- W4313304467 hasRelatedWork W3213825583 @default.
- W4313304467 hasRelatedWork W4281711591 @default.
- W4313304467 hasRelatedWork W4225852813 @default.
- W4313304467 isParatext "false" @default.
- W4313304467 isRetracted "false" @default.
- W4313304467 workType "article" @default.