Matches in SemOpenAlex for { <https://semopenalex.org/work/W1973836822> ?p ?o ?g. }
- W1973836822 endingPage "104" @default.
- W1973836822 startingPage "104" @default.
- W1973836822 abstract "Research in Information Retrieval has progressed against a background of rapidly increasing corpus size and heterogeneity, with every advance in technology quickly followed by a desire to organize and search more unstructured, more heterogeneous, and even bigger corpora. But as retrieval problems get larger and more complicated, evaluating the ranking performance of a retrieval engine gets harder: evaluation requires human judgments of the relevance of documents to queries, and for very large corpora the cost of acquiring these judgments may be insurmountable. This cost limits the types of problems researchers can study as well as the data they can be studied on. We present methods for understanding performance differences between retrieval engines in the presence of missing and noisy relevance judgments. The work introduces a model of the cost of experimentation that incorporates the cost of human judgments as well as the cost of drawing incorrect conclusions about differences between engines in both the training and testing phases of engine development. Through adopting a view of evaluation that is more concerned with distributions over performance differences rather than estimates of absolute performance, the expected cost can be minimized so as to reliably differentiate between engines with less than 1% of the human effort that has been used in past experiments." @default.
- W1973836822 created "2016-06-24" @default.
- W1973836822 creator A5027282342 @default.
- W1973836822 date "2008-11-30" @default.
- W1973836822 modified "2023-09-25" @default.
- W1973836822 title "Low-cost and robust evaluation of information retrieval systems" @default.
- W1973836822 cites W115453883 @default.
- W1973836822 cites W1493008453 @default.
- W1973836822 cites W1528762139 @default.
- W1973836822 cites W1553682320 @default.
- W1973836822 cites W1554394657 @default.
- W1973836822 cites W1561035524 @default.
- W1973836822 cites W1594174131 @default.
- W1973836822 cites W1606874436 @default.
- W1973836822 cites W168954559 @default.
- W1973836822 cites W1928657940 @default.
- W1973836822 cites W193708675 @default.
- W1973836822 cites W1979204324 @default.
- W1973836822 cites W1980036597 @default.
- W1973836822 cites W1988686126 @default.
- W1973836822 cites W1994960885 @default.
- W1973836822 cites W1995875735 @default.
- W1973836822 cites W2004737172 @default.
- W1973836822 cites W2009954908 @default.
- W1973836822 cites W2011798237 @default.
- W1973836822 cites W2015338694 @default.
- W1973836822 cites W2015629112 @default.
- W1973836822 cites W2021856948 @default.
- W1973836822 cites W2026937586 @default.
- W1973836822 cites W2027752285 @default.
- W1973836822 cites W2029071516 @default.
- W1973836822 cites W2029507263 @default.
- W1973836822 cites W2031536661 @default.
- W1973836822 cites W2032459394 @default.
- W1973836822 cites W2036448375 @default.
- W1973836822 cites W2040337753 @default.
- W1973836822 cites W2041098697 @default.
- W1973836822 cites W2043457403 @default.
- W1973836822 cites W2045638068 @default.
- W1973836822 cites W2045656233 @default.
- W1973836822 cites W2047221353 @default.
- W1973836822 cites W2053584122 @default.
- W1973836822 cites W2057495142 @default.
- W1973836822 cites W2058896506 @default.
- W1973836822 cites W2060494110 @default.
- W1973836822 cites W2061750459 @default.
- W1973836822 cites W2068098598 @default.
- W1973836822 cites W2068632118 @default.
- W1973836822 cites W2069870183 @default.
- W1973836822 cites W2072615624 @default.
- W1973836822 cites W2073722401 @default.
- W1973836822 cites W2075893676 @default.
- W1973836822 cites W2081983182 @default.
- W1973836822 cites W2083305840 @default.
- W1973836822 cites W2093432797 @default.
- W1973836822 cites W2099111195 @default.
- W1973836822 cites W2099944991 @default.
- W1973836822 cites W2101105183 @default.
- W1973836822 cites W2102693936 @default.
- W1973836822 cites W2107031757 @default.
- W1973836822 cites W2111310810 @default.
- W1973836822 cites W2120308175 @default.
- W1973836822 cites W2124379907 @default.
- W1973836822 cites W2124504084 @default.
- W1973836822 cites W2130145422 @default.
- W1973836822 cites W2133104961 @default.
- W1973836822 cites W2136349869 @default.
- W1973836822 cites W2141282920 @default.
- W1973836822 cites W2141623655 @default.
- W1973836822 cites W2143331230 @default.
- W1973836822 cites W2144200917 @default.
- W1973836822 cites W2151592910 @default.
- W1973836822 cites W2154884126 @default.
- W1973836822 cites W2158201212 @default.
- W1973836822 cites W2159048649 @default.
- W1973836822 cites W2162736173 @default.
- W1973836822 cites W2165387257 @default.
- W1973836822 cites W2165903295 @default.
- W1973836822 cites W2166183437 @default.
- W1973836822 cites W2169783907 @default.
- W1973836822 cites W2189144086 @default.
- W1973836822 cites W2295428206 @default.
- W1973836822 cites W2326231806 @default.
- W1973836822 cites W2407295445 @default.
- W1973836822 cites W3015241628 @default.
- W1973836822 cites W3126588943 @default.
- W1973836822 cites W3145128584 @default.
- W1973836822 cites W60779023 @default.
- W1973836822 cites W63837240 @default.
- W1973836822 cites W2195027018 @default.
- W1973836822 cites W91101475 @default.
- W1973836822 doi "https://doi.org/10.1145/1480506.1480527" @default.
- W1973836822 hasPublicationYear "2008" @default.
- W1973836822 type Work @default.
- W1973836822 sameAs 1973836822 @default.
- W1973836822 citedByCount "8" @default.
- W1973836822 countsByYear W19738368222012 @default.
- W1973836822 countsByYear W19738368222013 @default.