Matches in SemOpenAlex for { <https://semopenalex.org/work/W2006996936> ?p ?o ?g. }
- W2006996936 abstract "The existence and use of standard test collections in information retrieval experimentation allows results to be compared between research groups and over time. Such comparisons, however, are rarely made. Most researchers only report results from their own experiments, a practice that allows lack of overall improvement to go unnoticed. In this paper, we analyze results achieved on the TREC Ad-Hoc, Web, Terabyte, and Robust collections as reported in SIGIR (1998--2008) and CIKM (2004--2008). Dozens of individual published experiments report effectiveness improvements, and often claim statistical significance. However, there is little evidence of improvement in ad-hoc retrieval technology over the past decade. Baselines are generally weak, often being below the median original TREC system. And in only a handful of experiments is the score of the best TREC automatic run exceeded. Given this finding, we question the value of achieving even a statistically significant result over a weak baseline. We propose that the community adopt a practice of regular longitudinal comparison to ensure measurable progress, or at least prevent the lack of it from going unnoticed. We describe an online database of retrieval runs that facilitates such a practice." @default.
- W2006996936 created "2016-06-24" @default.
- W2006996936 creator A5024719664 @default.
- W2006996936 creator A5041495909 @default.
- W2006996936 creator A5042916733 @default.
- W2006996936 creator A5081861848 @default.
- W2006996936 date "2009-01-01" @default.
- W2006996936 modified "2023-10-01" @default.
- W2006996936 title "Improvements that don't add up" @default.
- W2006996936 cites W135190683 @default.
- W2006996936 cites W1965690069 @default.
- W2006996936 cites W1968927634 @default.
- W2006996936 cites W1972594981 @default.
- W2006996936 cites W1982433219 @default.
- W2006996936 cites W1990190154 @default.
- W2006996936 cites W1999817920 @default.
- W2006996936 cites W2002157036 @default.
- W2006996936 cites W2015338694 @default.
- W2006996936 cites W2032459394 @default.
- W2006996936 cites W2037140704 @default.
- W2006996936 cites W2067073114 @default.
- W2006996936 cites W2070740689 @default.
- W2006996936 cites W2075893676 @default.
- W2006996936 cites W2076227143 @default.
- W2006996936 cites W2152784831 @default.
- W2006996936 cites W2154873880 @default.
- W2006996936 cites W2162083862 @default.
- W2006996936 cites W2168469656 @default.
- W2006996936 cites W2169213601 @default.
- W2006996936 cites W2169583766 @default.
- W2006996936 cites W2915170155 @default.
- W2006996936 cites W2916723970 @default.
- W2006996936 doi "https://doi.org/10.1145/1645953.1646031" @default.
- W2006996936 hasPublicationYear "2009" @default.
- W2006996936 type Work @default.
- W2006996936 sameAs 2006996936 @default.
- W2006996936 citedByCount "184" @default.
- W2006996936 countsByYear W20069969362012 @default.
- W2006996936 countsByYear W20069969362013 @default.
- W2006996936 countsByYear W20069969362014 @default.
- W2006996936 countsByYear W20069969362015 @default.
- W2006996936 countsByYear W20069969362016 @default.
- W2006996936 countsByYear W20069969362017 @default.
- W2006996936 countsByYear W20069969362018 @default.
- W2006996936 countsByYear W20069969362019 @default.
- W2006996936 countsByYear W20069969362020 @default.
- W2006996936 countsByYear W20069969362021 @default.
- W2006996936 countsByYear W20069969362022 @default.
- W2006996936 countsByYear W20069969362023 @default.
- W2006996936 crossrefType "proceedings-article" @default.
- W2006996936 hasAuthorship W2006996936A5024719664 @default.
- W2006996936 hasAuthorship W2006996936A5041495909 @default.
- W2006996936 hasAuthorship W2006996936A5042916733 @default.
- W2006996936 hasAuthorship W2006996936A5081861848 @default.
- W2006996936 hasConcept C111368507 @default.
- W2006996936 hasConcept C111919701 @default.
- W2006996936 hasConcept C12725497 @default.
- W2006996936 hasConcept C127313418 @default.
- W2006996936 hasConcept C136764020 @default.
- W2006996936 hasConcept C151730666 @default.
- W2006996936 hasConcept C199343813 @default.
- W2006996936 hasConcept C199683683 @default.
- W2006996936 hasConcept C23123220 @default.
- W2006996936 hasConcept C2777267654 @default.
- W2006996936 hasConcept C2992886853 @default.
- W2006996936 hasConcept C41008148 @default.
- W2006996936 hasConcept C44291984 @default.
- W2006996936 hasConcept C71924100 @default.
- W2006996936 hasConcept C86803240 @default.
- W2006996936 hasConceptScore W2006996936C111368507 @default.
- W2006996936 hasConceptScore W2006996936C111919701 @default.
- W2006996936 hasConceptScore W2006996936C12725497 @default.
- W2006996936 hasConceptScore W2006996936C127313418 @default.
- W2006996936 hasConceptScore W2006996936C136764020 @default.
- W2006996936 hasConceptScore W2006996936C151730666 @default.
- W2006996936 hasConceptScore W2006996936C199343813 @default.
- W2006996936 hasConceptScore W2006996936C199683683 @default.
- W2006996936 hasConceptScore W2006996936C23123220 @default.
- W2006996936 hasConceptScore W2006996936C2777267654 @default.
- W2006996936 hasConceptScore W2006996936C2992886853 @default.
- W2006996936 hasConceptScore W2006996936C41008148 @default.
- W2006996936 hasConceptScore W2006996936C44291984 @default.
- W2006996936 hasConceptScore W2006996936C71924100 @default.
- W2006996936 hasConceptScore W2006996936C86803240 @default.
- W2006996936 hasLocation W20069969361 @default.
- W2006996936 hasOpenAccess W2006996936 @default.
- W2006996936 hasPrimaryLocation W20069969361 @default.
- W2006996936 hasRelatedWork W1532762289 @default.
- W2006996936 hasRelatedWork W1596801655 @default.
- W2006996936 hasRelatedWork W2358668433 @default.
- W2006996936 hasRelatedWork W2373011076 @default.
- W2006996936 hasRelatedWork W2376932109 @default.
- W2006996936 hasRelatedWork W2387147530 @default.
- W2006996936 hasRelatedWork W2390279801 @default.
- W2006996936 hasRelatedWork W2688602461 @default.
- W2006996936 hasRelatedWork W2748952813 @default.
- W2006996936 hasRelatedWork W2899084033 @default.
- W2006996936 isParatext "false" @default.
- W2006996936 isRetracted "false" @default.
- W2006996936 magId "2006996936" @default.