Matches in SemOpenAlex for { <https://semopenalex.org/work/W3125166688> ?p ?o ?g. }
- W3125166688 abstract "While designing inductive bias in neural architectures has been widely studied, we hypothesize that transformer networks are flexible enough to learn inductive bias from suitable generic tasks. Here, we replace architecture engineering by encoding inductive bias in the form of datasets. Inspired by Peirce's view that deduction, induction, and abduction form an irreducible set of reasoning primitives, we design three synthetic tasks that are intended to require the model to have these three abilities. We specifically design these synthetic tasks in a way that they are devoid of mathematical knowledge to ensure that only the fundamental reasoning biases can be learned from these tasks. This defines a new pre-training methodology called (Learning Inductive bias for Mathematical rEasoning). Models trained with LIME significantly outperform vanilla transformers on three very different large mathematical reasoning benchmarks. Unlike dominating the computation cost as traditional pre-training approaches, LIME requires only a small fraction of the computation cost of the typical downstream task." @default.
- W3125166688 created "2021-02-01" @default.
- W3125166688 creator A5002183320 @default.
- W3125166688 creator A5012276327 @default.
- W3125166688 creator A5024901763 @default.
- W3125166688 creator A5067036768 @default.
- W3125166688 creator A5072217103 @default.
- W3125166688 creator A5079665734 @default.
- W3125166688 date "2021-01-15" @default.
- W3125166688 modified "2023-09-27" @default.
- W3125166688 title "LIME: Learning Inductive Bias for Primitives of Mathematical Reasoning." @default.
- W3125166688 cites W1902237438 @default.
- W3125166688 cites W2020830132 @default.
- W3125166688 cites W2064675550 @default.
- W3125166688 cites W2116341502 @default.
- W3125166688 cites W2183341477 @default.
- W3125166688 cites W2921992864 @default.
- W3125166688 cites W2933138175 @default.
- W3125166688 cites W2944815030 @default.
- W3125166688 cites W2945576559 @default.
- W3125166688 cites W2946011656 @default.
- W3125166688 cites W2946559657 @default.
- W3125166688 cites W2962765587 @default.
- W3125166688 cites W2963005248 @default.
- W3125166688 cites W2963117013 @default.
- W3125166688 cites W2963341956 @default.
- W3125166688 cites W2963403868 @default.
- W3125166688 cites W2963596026 @default.
- W3125166688 cites W2963938535 @default.
- W3125166688 cites W2964121744 @default.
- W3125166688 cites W2965373594 @default.
- W3125166688 cites W2970049541 @default.
- W3125166688 cites W2970597249 @default.
- W3125166688 cites W2971274815 @default.
- W3125166688 cites W2981037730 @default.
- W3125166688 cites W2986920492 @default.
- W3125166688 cites W2994892931 @default.
- W3125166688 cites W2995359496 @default.
- W3125166688 cites W2995889583 @default.
- W3125166688 cites W2996086147 @default.
- W3125166688 cites W2997319416 @default.
- W3125166688 cites W3009562733 @default.
- W3125166688 cites W3011411500 @default.
- W3125166688 cites W3029284261 @default.
- W3125166688 cites W3030163527 @default.
- W3125166688 cites W3034144525 @default.
- W3125166688 cites W3034186971 @default.
- W3125166688 cites W3034199594 @default.
- W3125166688 cites W3034715004 @default.
- W3125166688 cites W3039090944 @default.
- W3125166688 cites W3040224948 @default.
- W3125166688 cites W3040628294 @default.
- W3125166688 cites W3042960190 @default.
- W3125166688 cites W3080426653 @default.
- W3125166688 cites W3082274269 @default.
- W3125166688 cites W3083835029 @default.
- W3125166688 cites W3104570641 @default.
- W3125166688 cites W3126568801 @default.
- W3125166688 cites W3133204645 @default.
- W3125166688 cites W3134522972 @default.
- W3125166688 cites W3134976210 @default.
- W3125166688 hasPublicationYear "2021" @default.
- W3125166688 type Work @default.
- W3125166688 sameAs 3125166688 @default.
- W3125166688 citedByCount "4" @default.
- W3125166688 countsByYear W31251666882021 @default.
- W3125166688 crossrefType "posted-content" @default.
- W3125166688 hasAuthorship W3125166688A5002183320 @default.
- W3125166688 hasAuthorship W3125166688A5012276327 @default.
- W3125166688 hasAuthorship W3125166688A5024901763 @default.
- W3125166688 hasAuthorship W3125166688A5067036768 @default.
- W3125166688 hasAuthorship W3125166688A5072217103 @default.
- W3125166688 hasAuthorship W3125166688A5079665734 @default.
- W3125166688 hasConcept C11413529 @default.
- W3125166688 hasConcept C119599485 @default.
- W3125166688 hasConcept C119857082 @default.
- W3125166688 hasConcept C127413603 @default.
- W3125166688 hasConcept C154945302 @default.
- W3125166688 hasConcept C165801399 @default.
- W3125166688 hasConcept C197352929 @default.
- W3125166688 hasConcept C201995342 @default.
- W3125166688 hasConcept C21563000 @default.
- W3125166688 hasConcept C2779382394 @default.
- W3125166688 hasConcept C2780451532 @default.
- W3125166688 hasConcept C28006648 @default.
- W3125166688 hasConcept C41008148 @default.
- W3125166688 hasConcept C45374587 @default.
- W3125166688 hasConcept C50644808 @default.
- W3125166688 hasConcept C66322947 @default.
- W3125166688 hasConceptScore W3125166688C11413529 @default.
- W3125166688 hasConceptScore W3125166688C119599485 @default.
- W3125166688 hasConceptScore W3125166688C119857082 @default.
- W3125166688 hasConceptScore W3125166688C127413603 @default.
- W3125166688 hasConceptScore W3125166688C154945302 @default.
- W3125166688 hasConceptScore W3125166688C165801399 @default.
- W3125166688 hasConceptScore W3125166688C197352929 @default.
- W3125166688 hasConceptScore W3125166688C201995342 @default.
- W3125166688 hasConceptScore W3125166688C21563000 @default.
- W3125166688 hasConceptScore W3125166688C2779382394 @default.
- W3125166688 hasConceptScore W3125166688C2780451532 @default.