Matches in SemOpenAlex for { <https://semopenalex.org/work/W3155728719> ?p ?o ?g. }
Showing items 1 to 73 of
73
with 100 items per page.
- W3155728719 abstract "In IR evaluation based on depth-k pooling, there are several strategies to order the pooled documents for relevance assessors. Among them, the simplest approach is to completely randomise the order so assessors cannot tell if a document was highly ranked by some system or how many systems (or which systems) retrieved the document. An approach that is in sharp contrast to the above is the prioritisation approach taken by NTCIRPOOL, a tool widely used at NTCIR. NTCIRPOOL sorts the pooled documents by pseudorelevance, a statistic that reflects the popularity of each document within the depth-k pools. Although these two strategies have coexisted for over two decades, the IR research community has yet to reach a consensus as to what advantages each of these two strategies actually offer. To help researchers directly address this question using their favourite methods of analysis, we have released a large-scale data set called WWW3E8. It comprises eight independent sets of qrels for the 160 English topics of the NTCIR-15 WWW-3 task: four qrels files constructed using the randomisation approach, and another four constructed using the prioritisation approach of NTCIRPOOL. Each qrels file covers 32,375 topic-document pairs; hence, WWW3E8 contains a total of 259,000 relevance labels. Moreover, the data set contains the raw English subtask run files from the WWW-3 task, the randomised and prioritised pool files, and topic-by-run score matrices of the official measures used in the task. Hence, researchers interested in the above research question regarding document ordering can utilise WWW3E8 as a common ground to directly compare the two strategies." @default.
- W3155728719 created "2021-04-26" @default.
- W3155728719 creator A5023595778 @default.
- W3155728719 creator A5074501708 @default.
- W3155728719 creator A5091105421 @default.
- W3155728719 date "2021-07-11" @default.
- W3155728719 modified "2023-09-24" @default.
- W3155728719 title "WWW3E8: 259,000 Relevance Labels for Studying the Effect of Document Presentation Order for Relevance Assessors" @default.
- W3155728719 cites W1554394657 @default.
- W3155728719 cites W1824124613 @default.
- W3155728719 cites W1968927634 @default.
- W3155728719 cites W1991774506 @default.
- W3155728719 cites W2008285050 @default.
- W3155728719 cites W2075893676 @default.
- W3155728719 cites W2134210222 @default.
- W3155728719 cites W2137274315 @default.
- W3155728719 cites W2137843624 @default.
- W3155728719 cites W2141653139 @default.
- W3155728719 cites W2154884126 @default.
- W3155728719 cites W2336806308 @default.
- W3155728719 cites W2464171116 @default.
- W3155728719 cites W2609795610 @default.
- W3155728719 cites W2896022497 @default.
- W3155728719 cites W3035278698 @default.
- W3155728719 doi "https://doi.org/10.1145/3404835.3463236" @default.
- W3155728719 hasPublicationYear "2021" @default.
- W3155728719 type Work @default.
- W3155728719 sameAs 3155728719 @default.
- W3155728719 citedByCount "1" @default.
- W3155728719 countsByYear W31557287192022 @default.
- W3155728719 crossrefType "proceedings-article" @default.
- W3155728719 hasAuthorship W3155728719A5023595778 @default.
- W3155728719 hasAuthorship W3155728719A5074501708 @default.
- W3155728719 hasAuthorship W3155728719A5091105421 @default.
- W3155728719 hasConcept C10138342 @default.
- W3155728719 hasConcept C126838900 @default.
- W3155728719 hasConcept C144133560 @default.
- W3155728719 hasConcept C158154518 @default.
- W3155728719 hasConcept C17744445 @default.
- W3155728719 hasConcept C182306322 @default.
- W3155728719 hasConcept C199539241 @default.
- W3155728719 hasConcept C23123220 @default.
- W3155728719 hasConcept C2777601897 @default.
- W3155728719 hasConcept C41008148 @default.
- W3155728719 hasConcept C71924100 @default.
- W3155728719 hasConceptScore W3155728719C10138342 @default.
- W3155728719 hasConceptScore W3155728719C126838900 @default.
- W3155728719 hasConceptScore W3155728719C144133560 @default.
- W3155728719 hasConceptScore W3155728719C158154518 @default.
- W3155728719 hasConceptScore W3155728719C17744445 @default.
- W3155728719 hasConceptScore W3155728719C182306322 @default.
- W3155728719 hasConceptScore W3155728719C199539241 @default.
- W3155728719 hasConceptScore W3155728719C23123220 @default.
- W3155728719 hasConceptScore W3155728719C2777601897 @default.
- W3155728719 hasConceptScore W3155728719C41008148 @default.
- W3155728719 hasConceptScore W3155728719C71924100 @default.
- W3155728719 hasLocation W31557287191 @default.
- W3155728719 hasOpenAccess W3155728719 @default.
- W3155728719 hasPrimaryLocation W31557287191 @default.
- W3155728719 hasRelatedWork W1584628001 @default.
- W3155728719 hasRelatedWork W1843462531 @default.
- W3155728719 hasRelatedWork W1867797766 @default.
- W3155728719 hasRelatedWork W1967370444 @default.
- W3155728719 hasRelatedWork W2036641180 @default.
- W3155728719 hasRelatedWork W2071255384 @default.
- W3155728719 hasRelatedWork W2144190808 @default.
- W3155728719 hasRelatedWork W2366644548 @default.
- W3155728719 hasRelatedWork W2376314740 @default.
- W3155728719 hasRelatedWork W2384888906 @default.
- W3155728719 isParatext "false" @default.
- W3155728719 isRetracted "false" @default.
- W3155728719 magId "3155728719" @default.
- W3155728719 workType "article" @default.