Matches in SemOpenAlex for { <https://semopenalex.org/work/W2897412906> ?p ?o ?g. }
- W2897412906 abstract "Because it is expensive to construct test collections for Cranfield-based evaluation of information retrieval systems, a variety of lower-cost methods have been proposed. The reliability of these methods is often validated by measuring rank correlation (e.g., Kendall's tau) between known system rankings on the full test collection vs. observed system rankings on the lower-cost one. However, existing rank correlation measures do not consider the statistical significance of score differences between systems in the observed rankings. To address this, we propose two statistical-significance-aware rank correlation measures, one of which is a head-weighted version of the other. We first show empirical differences between our proposed measures and existing ones. We then compare the measures while benchmarking four system evaluation methods: pooling, crowdsourcing, evaluation with incomplete judgments, and automatic system ranking. We show that use of our measures can lead to different experimental conclusions regarding reliability of alternative low-cost evaluation methods." @default.
- W2897412906 created "2018-10-26" @default.
- W2897412906 creator A5023397430 @default.
- W2897412906 creator A5032141720 @default.
- W2897412906 creator A5035521035 @default.
- W2897412906 creator A5086826049 @default.
- W2897412906 date "2018-10-17" @default.
- W2897412906 modified "2023-10-18" @default.
- W2897412906 title "When Rank Order Isn't Enough" @default.
- W2897412906 cites W1553751323 @default.
- W2897412906 cites W1616993132 @default.
- W2897412906 cites W1707848225 @default.
- W2897412906 cites W1966835268 @default.
- W2897412906 cites W1967879792 @default.
- W2897412906 cites W1972827298 @default.
- W2897412906 cites W1975409770 @default.
- W2897412906 cites W1983595289 @default.
- W2897412906 cites W1985514943 @default.
- W2897412906 cites W1989220366 @default.
- W2897412906 cites W1990190154 @default.
- W2897412906 cites W2010181950 @default.
- W2897412906 cites W2017833987 @default.
- W2897412906 cites W2028119648 @default.
- W2897412906 cites W2036887984 @default.
- W2897412906 cites W2047909334 @default.
- W2897412906 cites W2052569738 @default.
- W2897412906 cites W2057495142 @default.
- W2897412906 cites W2068098598 @default.
- W2897412906 cites W2073913493 @default.
- W2897412906 cites W2093156799 @default.
- W2897412906 cites W2102661342 @default.
- W2897412906 cites W2104679989 @default.
- W2897412906 cites W2104822358 @default.
- W2897412906 cites W2109244020 @default.
- W2897412906 cites W2120308175 @default.
- W2897412906 cites W2124504084 @default.
- W2897412906 cites W2132232498 @default.
- W2897412906 cites W2148944229 @default.
- W2897412906 cites W2160266360 @default.
- W2897412906 cites W2218551734 @default.
- W2897412906 cites W2469178887 @default.
- W2897412906 cites W2609795610 @default.
- W2897412906 cites W2740387847 @default.
- W2897412906 cites W2809759799 @default.
- W2897412906 cites W4256046779 @default.
- W2897412906 doi "https://doi.org/10.1145/3269206.3271751" @default.
- W2897412906 hasPublicationYear "2018" @default.
- W2897412906 type Work @default.
- W2897412906 sameAs 2897412906 @default.
- W2897412906 citedByCount "3" @default.
- W2897412906 countsByYear W28974129062019 @default.
- W2897412906 countsByYear W28974129062021 @default.
- W2897412906 crossrefType "proceedings-article" @default.
- W2897412906 hasAuthorship W2897412906A5023397430 @default.
- W2897412906 hasAuthorship W2897412906A5032141720 @default.
- W2897412906 hasAuthorship W2897412906A5035521035 @default.
- W2897412906 hasAuthorship W2897412906A5086826049 @default.
- W2897412906 hasConcept C101601086 @default.
- W2897412906 hasConcept C105795698 @default.
- W2897412906 hasConcept C114614502 @default.
- W2897412906 hasConcept C117220453 @default.
- W2897412906 hasConcept C119857082 @default.
- W2897412906 hasConcept C120936955 @default.
- W2897412906 hasConcept C121332964 @default.
- W2897412906 hasConcept C124101348 @default.
- W2897412906 hasConcept C136764020 @default.
- W2897412906 hasConcept C144133560 @default.
- W2897412906 hasConcept C154945302 @default.
- W2897412906 hasConcept C162853370 @default.
- W2897412906 hasConcept C163258240 @default.
- W2897412906 hasConcept C164226766 @default.
- W2897412906 hasConcept C189430467 @default.
- W2897412906 hasConcept C2524010 @default.
- W2897412906 hasConcept C33923547 @default.
- W2897412906 hasConcept C41008148 @default.
- W2897412906 hasConcept C43214815 @default.
- W2897412906 hasConcept C49937458 @default.
- W2897412906 hasConcept C62230096 @default.
- W2897412906 hasConcept C62520636 @default.
- W2897412906 hasConcept C70437156 @default.
- W2897412906 hasConcept C86251818 @default.
- W2897412906 hasConceptScore W2897412906C101601086 @default.
- W2897412906 hasConceptScore W2897412906C105795698 @default.
- W2897412906 hasConceptScore W2897412906C114614502 @default.
- W2897412906 hasConceptScore W2897412906C117220453 @default.
- W2897412906 hasConceptScore W2897412906C119857082 @default.
- W2897412906 hasConceptScore W2897412906C120936955 @default.
- W2897412906 hasConceptScore W2897412906C121332964 @default.
- W2897412906 hasConceptScore W2897412906C124101348 @default.
- W2897412906 hasConceptScore W2897412906C136764020 @default.
- W2897412906 hasConceptScore W2897412906C144133560 @default.
- W2897412906 hasConceptScore W2897412906C154945302 @default.
- W2897412906 hasConceptScore W2897412906C162853370 @default.
- W2897412906 hasConceptScore W2897412906C163258240 @default.
- W2897412906 hasConceptScore W2897412906C164226766 @default.
- W2897412906 hasConceptScore W2897412906C189430467 @default.
- W2897412906 hasConceptScore W2897412906C2524010 @default.
- W2897412906 hasConceptScore W2897412906C33923547 @default.
- W2897412906 hasConceptScore W2897412906C41008148 @default.
- W2897412906 hasConceptScore W2897412906C43214815 @default.