Matches in SemOpenAlex for { <https://semopenalex.org/work/W4287710008> ?p ?o ?g. }
Showing items 1 to 77 of
77
with 100 items per page.
- W4287710008 abstract "Assessors make preference judgments faster and more consistently than graded judgments. Preference judgments can also recognize distinctions between items that appear equivalent under graded judgments. Unfortunately, preference judgments can require more than linear effort to fully order a pool of items, and evaluation measures for preference judgments are not as well established as those for graded judgments, such as NDCG. In this paper, we explore the assessment process for partial preference judgments, with the aim of identifying and ordering the top items in the pool, rather than fully ordering the entire pool. To measure the performance of a ranker, we compare its output to this preferred ordering by applying a rank similarity measure.We demonstrate the practical feasibility of this approach by crowdsourcing partial preferences for the TREC 2019 Conversational Assistance Track, replacing NDCG with a new measure named compatibility. This new measure has its most striking impact when comparing modern neural rankers, where it is able to recognize significant improvements in quality that would otherwise be missed by NDCG." @default.
- W4287710008 created "2022-07-26" @default.
- W4287710008 creator A5037737168 @default.
- W4287710008 creator A5049222552 @default.
- W4287710008 creator A5082095937 @default.
- W4287710008 date "2020-07-22" @default.
- W4287710008 modified "2023-09-25" @default.
- W4287710008 title "Assessing top-$k$ preferences" @default.
- W4287710008 doi "https://doi.org/10.48550/arxiv.2007.11682" @default.
- W4287710008 hasPublicationYear "2020" @default.
- W4287710008 type Work @default.
- W4287710008 citedByCount "0" @default.
- W4287710008 crossrefType "posted-content" @default.
- W4287710008 hasAuthorship W4287710008A5037737168 @default.
- W4287710008 hasAuthorship W4287710008A5049222552 @default.
- W4287710008 hasAuthorship W4287710008A5082095937 @default.
- W4287710008 hasBestOaLocation W42877100081 @default.
- W4287710008 hasConcept C103278499 @default.
- W4287710008 hasConcept C105795698 @default.
- W4287710008 hasConcept C111472728 @default.
- W4287710008 hasConcept C114614502 @default.
- W4287710008 hasConcept C115961682 @default.
- W4287710008 hasConcept C119857082 @default.
- W4287710008 hasConcept C124101348 @default.
- W4287710008 hasConcept C136764020 @default.
- W4287710008 hasConcept C138885662 @default.
- W4287710008 hasConcept C154945302 @default.
- W4287710008 hasConcept C164226766 @default.
- W4287710008 hasConcept C189430467 @default.
- W4287710008 hasConcept C204321447 @default.
- W4287710008 hasConcept C23123220 @default.
- W4287710008 hasConcept C2777868144 @default.
- W4287710008 hasConcept C2779530757 @default.
- W4287710008 hasConcept C2780009758 @default.
- W4287710008 hasConcept C2781249084 @default.
- W4287710008 hasConcept C33923547 @default.
- W4287710008 hasConcept C41008148 @default.
- W4287710008 hasConcept C62230096 @default.
- W4287710008 hasConcept C86037889 @default.
- W4287710008 hasConceptScore W4287710008C103278499 @default.
- W4287710008 hasConceptScore W4287710008C105795698 @default.
- W4287710008 hasConceptScore W4287710008C111472728 @default.
- W4287710008 hasConceptScore W4287710008C114614502 @default.
- W4287710008 hasConceptScore W4287710008C115961682 @default.
- W4287710008 hasConceptScore W4287710008C119857082 @default.
- W4287710008 hasConceptScore W4287710008C124101348 @default.
- W4287710008 hasConceptScore W4287710008C136764020 @default.
- W4287710008 hasConceptScore W4287710008C138885662 @default.
- W4287710008 hasConceptScore W4287710008C154945302 @default.
- W4287710008 hasConceptScore W4287710008C164226766 @default.
- W4287710008 hasConceptScore W4287710008C189430467 @default.
- W4287710008 hasConceptScore W4287710008C204321447 @default.
- W4287710008 hasConceptScore W4287710008C23123220 @default.
- W4287710008 hasConceptScore W4287710008C2777868144 @default.
- W4287710008 hasConceptScore W4287710008C2779530757 @default.
- W4287710008 hasConceptScore W4287710008C2780009758 @default.
- W4287710008 hasConceptScore W4287710008C2781249084 @default.
- W4287710008 hasConceptScore W4287710008C33923547 @default.
- W4287710008 hasConceptScore W4287710008C41008148 @default.
- W4287710008 hasConceptScore W4287710008C62230096 @default.
- W4287710008 hasConceptScore W4287710008C86037889 @default.
- W4287710008 hasLocation W42877100081 @default.
- W4287710008 hasOpenAccess W4287710008 @default.
- W4287710008 hasPrimaryLocation W42877100081 @default.
- W4287710008 hasRelatedWork W2186284405 @default.
- W4287710008 hasRelatedWork W2237756989 @default.
- W4287710008 hasRelatedWork W2293317945 @default.
- W4287710008 hasRelatedWork W2349125667 @default.
- W4287710008 hasRelatedWork W2499321295 @default.
- W4287710008 hasRelatedWork W2572749905 @default.
- W4287710008 hasRelatedWork W2954428433 @default.
- W4287710008 hasRelatedWork W4287710008 @default.
- W4287710008 hasRelatedWork W4297665600 @default.
- W4287710008 hasRelatedWork W4300415226 @default.
- W4287710008 isParatext "false" @default.
- W4287710008 isRetracted "false" @default.
- W4287710008 workType "article" @default.