Matches in SemOpenAlex for { <https://semopenalex.org/work/W3166152691> ?p ?o ?g. }
Showing items 1 to 99 of
99
with 100 items per page.
- W3166152691 abstract "Language is an interface to the outside world. In order for embodied agents to use it, language must be grounded in other, sensorimotor modalities. While there is an extended literature studying how machines can learn grounded language, the topic of how to learn spatio-temporal linguistic concepts is still largely uncharted. To make progress in this direction, we here introduce a novel spatio-temporal language grounding task where the goal is to learn the meaning of spatio-temporal descriptions of behavioral traces of an embodied agent. This is achieved by training a truth function that predicts if a description matches a given history of observations. The descriptions involve time-extended predicates in past and present tense as well as spatio-temporal references to objects in the scene. To study the role of architectural biases in this task, we train several models including multimodal Transformer architectures; the latter implement different attention computations between words and objects across space and time. We test models on two classes of generalization: 1) generalization to randomly held-out sentences; 2) generalization to grammar primitives. We observe that maintaining object identity in the attention computation of our Transformers is instrumental to achieving good performance on generalization overall, and that summarizing object traces in a single token has little influence on performance. We then discuss how this opens new perspectives for language-guided autonomous embodied agents. We also release our code under open-source license as well as pretrained models and datasets to encourage the wider community to build upon and extend our work in the future." @default.
- W3166152691 created "2021-06-22" @default.
- W3166152691 creator A5018010486 @default.
- W3166152691 creator A5019224674 @default.
- W3166152691 creator A5027441885 @default.
- W3166152691 creator A5055305084 @default.
- W3166152691 creator A5085903884 @default.
- W3166152691 date "2021-12-07" @default.
- W3166152691 modified "2023-10-17" @default.
- W3166152691 title "Grounding Spatio-Temporal Language with Transformers" @default.
- W3166152691 cites W182277498 @default.
- W3166152691 cites W2020931258 @default.
- W3166152691 cites W2022778969 @default.
- W3166152691 cites W2026660464 @default.
- W3166152691 cites W2033945912 @default.
- W3166152691 cites W2039170454 @default.
- W3166152691 cites W2089967673 @default.
- W3166152691 cites W2109094612 @default.
- W3166152691 cites W2109529157 @default.
- W3166152691 cites W2114739781 @default.
- W3166152691 cites W2118781169 @default.
- W3166152691 cites W2122540544 @default.
- W3166152691 cites W2143623899 @default.
- W3166152691 cites W2162457740 @default.
- W3166152691 cites W2167293745 @default.
- W3166152691 cites W2220867547 @default.
- W3166152691 cites W2317464915 @default.
- W3166152691 cites W2474390746 @default.
- W3166152691 cites W2553496799 @default.
- W3166152691 cites W2626778328 @default.
- W3166152691 cites W2627585944 @default.
- W3166152691 cites W2805516822 @default.
- W3166152691 cites W2895560838 @default.
- W3166152691 cites W2911448865 @default.
- W3166152691 cites W2919112510 @default.
- W3166152691 cites W2938030785 @default.
- W3166152691 cites W2948380112 @default.
- W3166152691 cites W2953212746 @default.
- W3166152691 cites W2963267799 @default.
- W3166152691 cites W2963907629 @default.
- W3166152691 cites W2963921132 @default.
- W3166152691 cites W2964121744 @default.
- W3166152691 cites W2994971263 @default.
- W3166152691 cites W2995993311 @default.
- W3166152691 cites W3005188920 @default.
- W3166152691 cites W3007295596 @default.
- W3166152691 cites W3017374003 @default.
- W3166152691 cites W3023309694 @default.
- W3166152691 cites W3043208707 @default.
- W3166152691 cites W3049574859 @default.
- W3166152691 cites W3096609285 @default.
- W3166152691 cites W3098741467 @default.
- W3166152691 cites W3102696055 @default.
- W3166152691 cites W3119978476 @default.
- W3166152691 cites W3120048909 @default.
- W3166152691 cites W3122114180 @default.
- W3166152691 cites W3133416941 @default.
- W3166152691 hasPublicationYear "2021" @default.
- W3166152691 type Work @default.
- W3166152691 sameAs 3166152691 @default.
- W3166152691 citedByCount "0" @default.
- W3166152691 crossrefType "proceedings-article" @default.
- W3166152691 hasAuthorship W3166152691A5018010486 @default.
- W3166152691 hasAuthorship W3166152691A5019224674 @default.
- W3166152691 hasAuthorship W3166152691A5027441885 @default.
- W3166152691 hasAuthorship W3166152691A5055305084 @default.
- W3166152691 hasAuthorship W3166152691A5085903884 @default.
- W3166152691 hasBestOaLocation W31661526911 @default.
- W3166152691 hasConcept C119599485 @default.
- W3166152691 hasConcept C127413603 @default.
- W3166152691 hasConcept C165801399 @default.
- W3166152691 hasConcept C168993435 @default.
- W3166152691 hasConcept C41008148 @default.
- W3166152691 hasConcept C66322947 @default.
- W3166152691 hasConceptScore W3166152691C119599485 @default.
- W3166152691 hasConceptScore W3166152691C127413603 @default.
- W3166152691 hasConceptScore W3166152691C165801399 @default.
- W3166152691 hasConceptScore W3166152691C168993435 @default.
- W3166152691 hasConceptScore W3166152691C41008148 @default.
- W3166152691 hasConceptScore W3166152691C66322947 @default.
- W3166152691 hasLocation W31661526911 @default.
- W3166152691 hasLocation W31661526912 @default.
- W3166152691 hasLocation W31661526913 @default.
- W3166152691 hasOpenAccess W3166152691 @default.
- W3166152691 hasPrimaryLocation W31661526911 @default.
- W3166152691 hasRelatedWork W2158290499 @default.
- W3166152691 hasRelatedWork W2166205028 @default.
- W3166152691 hasRelatedWork W2362900165 @default.
- W3166152691 hasRelatedWork W2364549715 @default.
- W3166152691 hasRelatedWork W2382063497 @default.
- W3166152691 hasRelatedWork W2386735231 @default.
- W3166152691 hasRelatedWork W3128369928 @default.
- W3166152691 hasRelatedWork W3157910026 @default.
- W3166152691 hasRelatedWork W4205754550 @default.
- W3166152691 hasRelatedWork W4322775922 @default.
- W3166152691 isParatext "false" @default.
- W3166152691 isRetracted "false" @default.
- W3166152691 magId "3166152691" @default.
- W3166152691 workType "article" @default.