Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385574202> ?p ?o ?g. }
Showing items 1 to 71 of
71
with 100 items per page.
- W4385574202 abstract "While research on scientific claim verification has led to the development of powerful systems that appear to approach human performance, these approaches have yet to be tested in a realistic setting against large corpora of scientific literature. Moving to this open-domain evaluation setting, however, poses unique challenges; in particular, it is infeasible to exhaustively annotate all evidence documents. In this work, we present SciFact-Open, a new test collection designed to evaluate the performance of scientific claim verification systems on a corpus of 500K research abstracts. Drawing upon pooling techniques from information retrieval, we collect evidence for scientific claims by pooling and annotating the top predictions of four state-of-the-art scientific claim verification models. We find that systems developed on smaller corpora struggle to generalize to SciFact-Open, exhibiting performance drops of at least 15 F1. In addition, analysis of the evidence in SciFact-Open reveals interesting phenomena likely to appear when claim verification systems are deployed in practice, e.g., cases where the evidence supports only a special case of the claim. Our dataset is available at https://github.com/dwadden/scifact-open." @default.
- W4385574202 created "2023-08-05" @default.
- W4385574202 creator A5001778694 @default.
- W4385574202 creator A5037115259 @default.
- W4385574202 creator A5037862554 @default.
- W4385574202 creator A5064858748 @default.
- W4385574202 creator A5066588555 @default.
- W4385574202 creator A5082305994 @default.
- W4385574202 creator A5090038537 @default.
- W4385574202 date "2022-01-01" @default.
- W4385574202 modified "2023-09-24" @default.
- W4385574202 title "SciFact-Open: Towards open-domain scientific claim verification" @default.
- W4385574202 doi "https://doi.org/10.18653/v1/2022.findings-emnlp.347" @default.
- W4385574202 hasPublicationYear "2022" @default.
- W4385574202 type Work @default.
- W4385574202 citedByCount "0" @default.
- W4385574202 crossrefType "proceedings-article" @default.
- W4385574202 hasAuthorship W4385574202A5001778694 @default.
- W4385574202 hasAuthorship W4385574202A5037115259 @default.
- W4385574202 hasAuthorship W4385574202A5037862554 @default.
- W4385574202 hasAuthorship W4385574202A5064858748 @default.
- W4385574202 hasAuthorship W4385574202A5066588555 @default.
- W4385574202 hasAuthorship W4385574202A5082305994 @default.
- W4385574202 hasAuthorship W4385574202A5090038537 @default.
- W4385574202 hasBestOaLocation W43855742021 @default.
- W4385574202 hasConcept C134306372 @default.
- W4385574202 hasConcept C136764020 @default.
- W4385574202 hasConcept C154945302 @default.
- W4385574202 hasConcept C199360897 @default.
- W4385574202 hasConcept C23123220 @default.
- W4385574202 hasConcept C2522767166 @default.
- W4385574202 hasConcept C2777904410 @default.
- W4385574202 hasConcept C2778464652 @default.
- W4385574202 hasConcept C2993776861 @default.
- W4385574202 hasConcept C3018397939 @default.
- W4385574202 hasConcept C33923547 @default.
- W4385574202 hasConcept C36503486 @default.
- W4385574202 hasConcept C41008148 @default.
- W4385574202 hasConcept C44291984 @default.
- W4385574202 hasConcept C70437156 @default.
- W4385574202 hasConceptScore W4385574202C134306372 @default.
- W4385574202 hasConceptScore W4385574202C136764020 @default.
- W4385574202 hasConceptScore W4385574202C154945302 @default.
- W4385574202 hasConceptScore W4385574202C199360897 @default.
- W4385574202 hasConceptScore W4385574202C23123220 @default.
- W4385574202 hasConceptScore W4385574202C2522767166 @default.
- W4385574202 hasConceptScore W4385574202C2777904410 @default.
- W4385574202 hasConceptScore W4385574202C2778464652 @default.
- W4385574202 hasConceptScore W4385574202C2993776861 @default.
- W4385574202 hasConceptScore W4385574202C3018397939 @default.
- W4385574202 hasConceptScore W4385574202C33923547 @default.
- W4385574202 hasConceptScore W4385574202C36503486 @default.
- W4385574202 hasConceptScore W4385574202C41008148 @default.
- W4385574202 hasConceptScore W4385574202C44291984 @default.
- W4385574202 hasConceptScore W4385574202C70437156 @default.
- W4385574202 hasLocation W43855742021 @default.
- W4385574202 hasOpenAccess W4385574202 @default.
- W4385574202 hasPrimaryLocation W43855742021 @default.
- W4385574202 hasRelatedWork W1935138864 @default.
- W4385574202 hasRelatedWork W2368388617 @default.
- W4385574202 hasRelatedWork W2811459222 @default.
- W4385574202 hasRelatedWork W3036846138 @default.
- W4385574202 hasRelatedWork W3198952405 @default.
- W4385574202 hasRelatedWork W3214164160 @default.
- W4385574202 hasRelatedWork W4226302158 @default.
- W4385574202 hasRelatedWork W4286978879 @default.
- W4385574202 hasRelatedWork W4287864530 @default.
- W4385574202 hasRelatedWork W4307416624 @default.
- W4385574202 isParatext "false" @default.
- W4385574202 isRetracted "false" @default.
- W4385574202 workType "article" @default.