FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W4386076670> ?p ?o ?g. }

• W4386076670 abstract "Optimization in multi-task learning (MTL) is more challenging than single-task learning (STL), as the gradient from different tasks can be contradictory. When tasks are related, it can be beneficial to share some parameters among them (cooperation). However, some tasks require additional parameters with expertise in a specific type of data or discrimination (specialization). To address the MTL challenge, we propose Mod-Squad, a new model that is Modularized into groups of experts (a ‘Squad’). This structure allows us to formalize cooperation and specialization as the process of matching experts and tasks. We optimize this matching process during the training of a single model. Specifically, we incorporate mixture of experts (MoE) layers into a transformer model, with a new loss that incorporates the mutual dependence between tasks and experts. As a result, only a small set of experts are activated for each task. This prevents the sharing of the entire backbone model between all tasks, which strengthens the model, especially when the training set size and the number of tasks scale up. More interestingly, for each task, we can extract the small set of experts as a standalone model that maintains the same performance as the large model. Extensive experiments on the Taskonomy dataset with 13 vision tasks and the PASCAL-Context dataset with 5 vision tasks show the superiority of our approach. The project page can be accessed at https://vis-www.cs.umass.edu/Mod-Squad." @default.
• W4386076670 created "2023-08-23" @default.
• W4386076670 creator A5017218416 @default.
• W4386076670 creator A5040877128 @default.
• W4386076670 creator A5042636572 @default.
• W4386076670 creator A5045674062 @default.
• W4386076670 creator A5073742611 @default.
• W4386076670 creator A5078109015 @default.
• W4386076670 creator A5081201135 @default.
• W4386076670 date "2023-06-01" @default.
• W4386076670 modified "2023-10-02" @default.
• W4386076670 title "Mod-Squad: Designing Mixtures of Experts As Modular Multi-Task Learners" @default.
• W4386076670 cites W2125215748 @default.
• W4386076670 cites W2150884987 @default.
• W4386076670 cites W2808168148 @default.
• W4386076670 cites W2913340405 @default.
• W4386076670 cites W2962864421 @default.
• W4386076670 cites W2963430933 @default.
• W4386076670 cites W2963498646 @default.
• W4386076670 cites W2963877604 @default.
• W4386076670 cites W2964185501 @default.
• W4386076670 cites W2964247799 @default.
• W4386076670 cites W2981468122 @default.
• W4386076670 cites W2984039063 @default.
• W4386076670 cites W2989808579 @default.
• W4386076670 cites W4313166855 @default.
• W4386076670 cites W4385573095 @default.
• W4386076670 doi "https://doi.org/10.1109/cvpr52729.2023.01138" @default.
• W4386076670 hasPublicationYear "2023" @default.
• W4386076670 type Work @default.
• W4386076670 citedByCount "0" @default.
• W4386076670 crossrefType "proceedings-article" @default.
• W4386076670 hasAuthorship W4386076670A5017218416 @default.
• W4386076670 hasAuthorship W4386076670A5040877128 @default.
• W4386076670 hasAuthorship W4386076670A5042636572 @default.
• W4386076670 hasAuthorship W4386076670A5045674062 @default.
• W4386076670 hasAuthorship W4386076670A5073742611 @default.
• W4386076670 hasAuthorship W4386076670A5078109015 @default.
• W4386076670 hasAuthorship W4386076670A5081201135 @default.
• W4386076670 hasConcept C101468663 @default.
• W4386076670 hasConcept C105795698 @default.
• W4386076670 hasConcept C107457646 @default.
• W4386076670 hasConcept C119857082 @default.
• W4386076670 hasConcept C121332964 @default.
• W4386076670 hasConcept C151730666 @default.
• W4386076670 hasConcept C154945302 @default.
• W4386076670 hasConcept C162324750 @default.
• W4386076670 hasConcept C165064840 @default.
• W4386076670 hasConcept C165801399 @default.
• W4386076670 hasConcept C177264268 @default.
• W4386076670 hasConcept C187736073 @default.
• W4386076670 hasConcept C199360897 @default.
• W4386076670 hasConcept C2779343474 @default.
• W4386076670 hasConcept C2780451532 @default.
• W4386076670 hasConcept C33923547 @default.
• W4386076670 hasConcept C41008148 @default.
• W4386076670 hasConcept C62520636 @default.
• W4386076670 hasConcept C66322947 @default.
• W4386076670 hasConcept C75608658 @default.
• W4386076670 hasConcept C86803240 @default.
• W4386076670 hasConcept C98045186 @default.
• W4386076670 hasConceptScore W4386076670C101468663 @default.
• W4386076670 hasConceptScore W4386076670C105795698 @default.
• W4386076670 hasConceptScore W4386076670C107457646 @default.
• W4386076670 hasConceptScore W4386076670C119857082 @default.
• W4386076670 hasConceptScore W4386076670C121332964 @default.
• W4386076670 hasConceptScore W4386076670C151730666 @default.
• W4386076670 hasConceptScore W4386076670C154945302 @default.
• W4386076670 hasConceptScore W4386076670C162324750 @default.
• W4386076670 hasConceptScore W4386076670C165064840 @default.
• W4386076670 hasConceptScore W4386076670C165801399 @default.
• W4386076670 hasConceptScore W4386076670C177264268 @default.
• W4386076670 hasConceptScore W4386076670C187736073 @default.
• W4386076670 hasConceptScore W4386076670C199360897 @default.
• W4386076670 hasConceptScore W4386076670C2779343474 @default.
• W4386076670 hasConceptScore W4386076670C2780451532 @default.
• W4386076670 hasConceptScore W4386076670C33923547 @default.
• W4386076670 hasConceptScore W4386076670C41008148 @default.
• W4386076670 hasConceptScore W4386076670C62520636 @default.
• W4386076670 hasConceptScore W4386076670C66322947 @default.
• W4386076670 hasConceptScore W4386076670C75608658 @default.
• W4386076670 hasConceptScore W4386076670C86803240 @default.
• W4386076670 hasConceptScore W4386076670C98045186 @default.
• W4386076670 hasFunder F4320307791 @default.
• W4386076670 hasFunder F4320315072 @default.
• W4386076670 hasFunder F4320320741 @default.
• W4386076670 hasLocation W43860766701 @default.
• W4386076670 hasOpenAccess W4386076670 @default.
• W4386076670 hasPrimaryLocation W43860766701 @default.
• W4386076670 hasRelatedWork W1504101963 @default.
• W4386076670 hasRelatedWork W1987706094 @default.
• W4386076670 hasRelatedWork W2140468882 @default.
• W4386076670 hasRelatedWork W2365088826 @default.
• W4386076670 hasRelatedWork W2961085424 @default.
• W4386076670 hasRelatedWork W2963260880 @default.
• W4386076670 hasRelatedWork W4231350079 @default.
• W4386076670 hasRelatedWork W4244612519 @default.
• W4386076670 hasRelatedWork W4298819607 @default.
• W4386076670 hasRelatedWork W88370528 @default.
• W4386076670 isParatext "false" @default.

• next