Matches in SemOpenAlex for { <https://semopenalex.org/work/W3212461880> ?p ?o ?g. }
Showing items 1 to 69 of
69
with 100 items per page.
- W3212461880 abstract "Our goal is to deliver a new task and leaderboard to stimulate research on question answering and pre-trained language models (PTLMs) to understand a significant instructional document, e.g., an introductory college textbook or a manual. PTLMs have shown great success in many question-answering tasks, given significant supervised training, but much less so in zero-shot settings. We propose a new task that includes two college-level introductory texts in the social sciences (American Government 2e) and humanities (U.S. History), hundreds of true/false statements based on review questions written by the textbook authors, validation/development tests based on the first eight chapters of the textbooks, blind tests based on the remaining textbook chapters, and baseline results given state-of-the-art PTLMs. Since the questions are balanced, random performance should be ~50%. T5, fine-tuned with BoolQ achieves the same performance, suggesting that the textbook’s content is not pre-represented in the PTLM. Taking the exam closed book, but having read the textbook (i.e., adding the textbook to T5’s pre-training), yields at best minor improvement (56%), suggesting that the PTLM may not have “understood” the textbook (or perhaps misunderstood the questions). Performance is better (~60%) when the exam is taken open-book (i.e., allowing the machine to automatically retrieve a paragraph and use it to answer the question)." @default.
- W3212461880 created "2021-11-22" @default.
- W3212461880 creator A5005074475 @default.
- W3212461880 creator A5011618765 @default.
- W3212461880 creator A5023531655 @default.
- W3212461880 creator A5027489318 @default.
- W3212461880 creator A5063784054 @default.
- W3212461880 creator A5083339822 @default.
- W3212461880 date "2021-09-03" @default.
- W3212461880 modified "2023-09-26" @default.
- W3212461880 title "Perhaps PTLMs should go to School – A Task to Assess Open Book and Closed Book QA" @default.
- W3212461880 hasPublicationYear "2021" @default.
- W3212461880 type Work @default.
- W3212461880 sameAs 3212461880 @default.
- W3212461880 citedByCount "0" @default.
- W3212461880 crossrefType "journal-article" @default.
- W3212461880 hasAuthorship W3212461880A5005074475 @default.
- W3212461880 hasAuthorship W3212461880A5011618765 @default.
- W3212461880 hasAuthorship W3212461880A5023531655 @default.
- W3212461880 hasAuthorship W3212461880A5027489318 @default.
- W3212461880 hasAuthorship W3212461880A5063784054 @default.
- W3212461880 hasAuthorship W3212461880A5083339822 @default.
- W3212461880 hasConcept C136764020 @default.
- W3212461880 hasConcept C145420912 @default.
- W3212461880 hasConcept C154945302 @default.
- W3212461880 hasConcept C15744967 @default.
- W3212461880 hasConcept C162324750 @default.
- W3212461880 hasConcept C187736073 @default.
- W3212461880 hasConcept C2777206241 @default.
- W3212461880 hasConcept C2780451532 @default.
- W3212461880 hasConcept C41008148 @default.
- W3212461880 hasConcept C44291984 @default.
- W3212461880 hasConceptScore W3212461880C136764020 @default.
- W3212461880 hasConceptScore W3212461880C145420912 @default.
- W3212461880 hasConceptScore W3212461880C154945302 @default.
- W3212461880 hasConceptScore W3212461880C15744967 @default.
- W3212461880 hasConceptScore W3212461880C162324750 @default.
- W3212461880 hasConceptScore W3212461880C187736073 @default.
- W3212461880 hasConceptScore W3212461880C2777206241 @default.
- W3212461880 hasConceptScore W3212461880C2780451532 @default.
- W3212461880 hasConceptScore W3212461880C41008148 @default.
- W3212461880 hasConceptScore W3212461880C44291984 @default.
- W3212461880 hasLocation W32124618801 @default.
- W3212461880 hasOpenAccess W3212461880 @default.
- W3212461880 hasPrimaryLocation W32124618801 @default.
- W3212461880 hasRelatedWork W1521447173 @default.
- W3212461880 hasRelatedWork W1572289071 @default.
- W3212461880 hasRelatedWork W1691016530 @default.
- W3212461880 hasRelatedWork W1737203358 @default.
- W3212461880 hasRelatedWork W1982033646 @default.
- W3212461880 hasRelatedWork W2025241635 @default.
- W3212461880 hasRelatedWork W2042527984 @default.
- W3212461880 hasRelatedWork W2097058761 @default.
- W3212461880 hasRelatedWork W2261113228 @default.
- W3212461880 hasRelatedWork W2468881433 @default.
- W3212461880 hasRelatedWork W2501489889 @default.
- W3212461880 hasRelatedWork W2901704395 @default.
- W3212461880 hasRelatedWork W313873573 @default.
- W3212461880 hasRelatedWork W3211157621 @default.
- W3212461880 hasRelatedWork W571083543 @default.
- W3212461880 hasRelatedWork W609973301 @default.
- W3212461880 hasRelatedWork W628187815 @default.
- W3212461880 hasRelatedWork W657672007 @default.
- W3212461880 hasRelatedWork W95507102 @default.
- W3212461880 hasRelatedWork W1950729494 @default.
- W3212461880 isParatext "false" @default.
- W3212461880 isRetracted "false" @default.
- W3212461880 magId "3212461880" @default.
- W3212461880 workType "article" @default.