Matches in SemOpenAlex for { <https://semopenalex.org/work/W3209347846> ?p ?o ?g. }
Showing items 1 to 84 of
84
with 100 items per page.
- W3209347846 abstract "The goal of multi-task learning is to enable more efficient learning than single task learning by sharing model structures for a diverse set of tasks. A standard multi-task learning objective is to minimize the average loss across all tasks. While straightforward, using this objective often results in much worse final performance for each task than learning them independently. A major challenge in optimizing a multi-task model is the conflicting gradients, where gradients of different task objectives are not well aligned so that following the average gradient direction can be detrimental to specific tasks' performance. Previous work has proposed several heuristics to manipulate the task gradients for mitigating this problem. But most of them lack convergence guarantee and/or could converge to any Pareto-stationary point. In this paper, we introduce Conflict-Averse Gradient descent (CAGrad) which minimizes the average loss function, while leveraging the worst local improvement of individual tasks to regularize the algorithm trajectory. CAGrad balances the objectives automatically and still provably converges to a minimum over the average loss. It includes the regular gradient descent (GD) and the multiple gradient descent algorithm (MGDA) in the multi-objective optimization (MOO) literature as special cases. On a series of challenging multi-task supervised learning and reinforcement learning tasks, CAGrad achieves improved performance over prior state-of-the-art multi-objective gradient manipulation methods." @default.
- W3209347846 created "2021-11-08" @default.
- W3209347846 creator A5001594330 @default.
- W3209347846 creator A5024864770 @default.
- W3209347846 creator A5057055806 @default.
- W3209347846 creator A5062901935 @default.
- W3209347846 creator A5090815103 @default.
- W3209347846 date "2021-12-06" @default.
- W3209347846 modified "2023-09-27" @default.
- W3209347846 title "Conflict-Averse Gradient Descent for Multi-task learning" @default.
- W3209347846 hasPublicationYear "2021" @default.
- W3209347846 type Work @default.
- W3209347846 sameAs 3209347846 @default.
- W3209347846 citedByCount "0" @default.
- W3209347846 crossrefType "proceedings-article" @default.
- W3209347846 hasAuthorship W3209347846A5001594330 @default.
- W3209347846 hasAuthorship W3209347846A5024864770 @default.
- W3209347846 hasAuthorship W3209347846A5057055806 @default.
- W3209347846 hasAuthorship W3209347846A5062901935 @default.
- W3209347846 hasAuthorship W3209347846A5090815103 @default.
- W3209347846 hasConcept C111919701 @default.
- W3209347846 hasConcept C119857082 @default.
- W3209347846 hasConcept C126255220 @default.
- W3209347846 hasConcept C127705205 @default.
- W3209347846 hasConcept C153258448 @default.
- W3209347846 hasConcept C154945302 @default.
- W3209347846 hasConcept C162324750 @default.
- W3209347846 hasConcept C177264268 @default.
- W3209347846 hasConcept C187736073 @default.
- W3209347846 hasConcept C199360897 @default.
- W3209347846 hasConcept C2777303404 @default.
- W3209347846 hasConcept C2780451532 @default.
- W3209347846 hasConcept C28006648 @default.
- W3209347846 hasConcept C33923547 @default.
- W3209347846 hasConcept C41008148 @default.
- W3209347846 hasConcept C50522688 @default.
- W3209347846 hasConcept C50644808 @default.
- W3209347846 hasConcept C97541855 @default.
- W3209347846 hasConceptScore W3209347846C111919701 @default.
- W3209347846 hasConceptScore W3209347846C119857082 @default.
- W3209347846 hasConceptScore W3209347846C126255220 @default.
- W3209347846 hasConceptScore W3209347846C127705205 @default.
- W3209347846 hasConceptScore W3209347846C153258448 @default.
- W3209347846 hasConceptScore W3209347846C154945302 @default.
- W3209347846 hasConceptScore W3209347846C162324750 @default.
- W3209347846 hasConceptScore W3209347846C177264268 @default.
- W3209347846 hasConceptScore W3209347846C187736073 @default.
- W3209347846 hasConceptScore W3209347846C199360897 @default.
- W3209347846 hasConceptScore W3209347846C2777303404 @default.
- W3209347846 hasConceptScore W3209347846C2780451532 @default.
- W3209347846 hasConceptScore W3209347846C28006648 @default.
- W3209347846 hasConceptScore W3209347846C33923547 @default.
- W3209347846 hasConceptScore W3209347846C41008148 @default.
- W3209347846 hasConceptScore W3209347846C50522688 @default.
- W3209347846 hasConceptScore W3209347846C50644808 @default.
- W3209347846 hasConceptScore W3209347846C97541855 @default.
- W3209347846 hasLocation W32093478461 @default.
- W3209347846 hasOpenAccess W3209347846 @default.
- W3209347846 hasPrimaryLocation W32093478461 @default.
- W3209347846 hasRelatedWork W1517383877 @default.
- W3209347846 hasRelatedWork W1569756368 @default.
- W3209347846 hasRelatedWork W1696410204 @default.
- W3209347846 hasRelatedWork W2392699888 @default.
- W3209347846 hasRelatedWork W2398490066 @default.
- W3209347846 hasRelatedWork W2787387965 @default.
- W3209347846 hasRelatedWork W2800391888 @default.
- W3209347846 hasRelatedWork W2891645069 @default.
- W3209347846 hasRelatedWork W2896914049 @default.
- W3209347846 hasRelatedWork W2953335040 @default.
- W3209347846 hasRelatedWork W2963179943 @default.
- W3209347846 hasRelatedWork W2967645217 @default.
- W3209347846 hasRelatedWork W2975199294 @default.
- W3209347846 hasRelatedWork W3104364497 @default.
- W3209347846 hasRelatedWork W3108562470 @default.
- W3209347846 hasRelatedWork W3118694267 @default.
- W3209347846 hasRelatedWork W3160027457 @default.
- W3209347846 hasRelatedWork W3185109031 @default.
- W3209347846 hasRelatedWork W3208761439 @default.
- W3209347846 hasRelatedWork W3209208698 @default.
- W3209347846 hasVolume "34" @default.
- W3209347846 isParatext "false" @default.
- W3209347846 isRetracted "false" @default.
- W3209347846 magId "3209347846" @default.
- W3209347846 workType "article" @default.