Matches in SemOpenAlex for { <https://semopenalex.org/work/W1488210821> ?p ?o ?g. }
Showing items 1 to 76 of
76
with 100 items per page.
- W1488210821 endingPage "38" @default.
- W1488210821 startingPage "26" @default.
- W1488210821 abstract "Traditional ad hoc IR test collections were built using a relatively large pool depth (e.g. 100), and are usually assumed to be reusable. Moreover, when they are reused to compare a new system with another or with systems that contributed to the pools (“contributors”), an even larger measurement depth (e.g. 1,000) is often used for computing evaluation metrics. In contrast, the web diversity test collections that have been created in the past few years at TREC and NTCIR use a much smaller pool depth (e.g. 20). The measurement depth is also small (e.g. 10-30), as search result diversification is primarily intended for the first result page. In this study, we examine the reusability of a typical web diversity test collection, namely, one from the NTCIR-9 INTENT-1 Chinese Document Ranking task, which used a pool depth of 20 and official measurement depths of 10, 20 and 30. First, we conducted additional relevance assessments to expand the official INTENT-1 collection to achieve a pool depth of 40. Using the expanded relevance assessments, we show that run rankings at the measurement depth of 30 are too unreliable, given that the pool depth is 20. Second, we conduct a leave-one-out experiment for every participating team of the INTENT-1 Chinese task, to examine how (un)fairly new runs are evaluated with the INTENT-1 collection. We show that, for the purpose of comparing new systems with the contributors of the test collection being used, condensed-list versions of existing diversity evaluation metrics are more reliable than the raw metrics. However, even the condensed-list metrics may be unreliable if the new systems are not competitive compared to the contributors." @default.
- W1488210821 created "2016-06-24" @default.
- W1488210821 creator A5010558184 @default.
- W1488210821 creator A5015800533 @default.
- W1488210821 creator A5023595778 @default.
- W1488210821 creator A5045875579 @default.
- W1488210821 date "2012-01-01" @default.
- W1488210821 modified "2023-09-27" @default.
- W1488210821 title "The Reusability of a Diversified Search Test Collection" @default.
- W1488210821 cites W1554394657 @default.
- W1488210821 cites W1966835268 @default.
- W1488210821 cites W1967879792 @default.
- W1488210821 cites W1986572161 @default.
- W1488210821 cites W2001142465 @default.
- W1488210821 cites W2004918107 @default.
- W1488210821 cites W2017292914 @default.
- W1488210821 cites W2021856948 @default.
- W1488210821 cites W2053100920 @default.
- W1488210821 cites W2058624977 @default.
- W1488210821 cites W2068098598 @default.
- W1488210821 cites W2075893676 @default.
- W1488210821 cites W2081033951 @default.
- W1488210821 cites W2102698119 @default.
- W1488210821 cites W2106927287 @default.
- W1488210821 cites W2109244020 @default.
- W1488210821 cites W2124108136 @default.
- W1488210821 doi "https://doi.org/10.1007/978-3-642-35341-3_3" @default.
- W1488210821 hasPublicationYear "2012" @default.
- W1488210821 type Work @default.
- W1488210821 sameAs 1488210821 @default.
- W1488210821 citedByCount "10" @default.
- W1488210821 countsByYear W14882108212012 @default.
- W1488210821 countsByYear W14882108212013 @default.
- W1488210821 countsByYear W14882108212014 @default.
- W1488210821 countsByYear W14882108212017 @default.
- W1488210821 countsByYear W14882108212022 @default.
- W1488210821 crossrefType "book-chapter" @default.
- W1488210821 hasAuthorship W1488210821A5010558184 @default.
- W1488210821 hasAuthorship W1488210821A5015800533 @default.
- W1488210821 hasAuthorship W1488210821A5023595778 @default.
- W1488210821 hasAuthorship W1488210821A5045875579 @default.
- W1488210821 hasBestOaLocation W14882108212 @default.
- W1488210821 hasConcept C111919701 @default.
- W1488210821 hasConcept C137981799 @default.
- W1488210821 hasConcept C18903297 @default.
- W1488210821 hasConcept C2777267654 @default.
- W1488210821 hasConcept C2777904410 @default.
- W1488210821 hasConcept C41008148 @default.
- W1488210821 hasConcept C86803240 @default.
- W1488210821 hasConceptScore W1488210821C111919701 @default.
- W1488210821 hasConceptScore W1488210821C137981799 @default.
- W1488210821 hasConceptScore W1488210821C18903297 @default.
- W1488210821 hasConceptScore W1488210821C2777267654 @default.
- W1488210821 hasConceptScore W1488210821C2777904410 @default.
- W1488210821 hasConceptScore W1488210821C41008148 @default.
- W1488210821 hasConceptScore W1488210821C86803240 @default.
- W1488210821 hasLocation W14882108211 @default.
- W1488210821 hasLocation W14882108212 @default.
- W1488210821 hasOpenAccess W1488210821 @default.
- W1488210821 hasPrimaryLocation W14882108211 @default.
- W1488210821 hasRelatedWork W1594229218 @default.
- W1488210821 hasRelatedWork W2120994104 @default.
- W1488210821 hasRelatedWork W2215606090 @default.
- W1488210821 hasRelatedWork W2278796325 @default.
- W1488210821 hasRelatedWork W2372886617 @default.
- W1488210821 hasRelatedWork W2394142092 @default.
- W1488210821 hasRelatedWork W2780416612 @default.
- W1488210821 hasRelatedWork W4236749683 @default.
- W1488210821 hasRelatedWork W61565270 @default.
- W1488210821 hasRelatedWork W2131591406 @default.
- W1488210821 isParatext "false" @default.
- W1488210821 isRetracted "false" @default.
- W1488210821 magId "1488210821" @default.
- W1488210821 workType "book-chapter" @default.