Matches in SemOpenAlex for { <https://semopenalex.org/work/W2798839717> ?p ?o ?g. }
Showing items 1 to 94 of
94
with 100 items per page.
- W2798839717 abstract "The development of automatic systems that can produce timeline summaries by filtering high-volume streams of text documents, retaining only those that are relevant to a particular information need (e.g. topic or event), remains a very challenging task. To advance the field of automatic timeline generation, robust and reproducible evaluation methodologies are needed. To this end, several evaluation metrics and labeling methodologies have recently been developed - focusing on information nugget or cluster-based ground truth representations, respectively. These methodologies rely on human assessors manually mapping timeline items (e.g. tweets) to an explicit representation of what information a 'good' summary should contain. However, while these evaluation methodologies produce reusable ground truth labels, prior works have reported cases where such labels fail to accurately estimate the performance of new timeline generation systems due to label incompleteness. In this paper, we first quantify the extent to which timeline summary ground truth labels fail to generalize to new summarization systems, then we propose and evaluate new automatic solutions to this issue. In particular, using a depooling methodology over 21 systems and across three high-volume datasets, we quantify the degree of system ranking error caused by excluding those systems when labeling. We show that when considering lower-effectiveness systems, the test collections are robust (the likelihood of systems being miss-ranked is low). However, we show that the risk of systems being miss-ranked increases as the effectiveness of systems held-out from the pool increases. To reduce the risk of miss-ranking systems, we also propose two different automatic ground truth label expansion techniques. Our results show that our proposed expansion techniques can be effective for increasing the robustness of the TREC-TS test collections, markedly reducing the number of miss-rankings by up to 50% on average among the scenarios tested." @default.
- W2798839717 created "2018-05-07" @default.
- W2798839717 creator A5057643560 @default.
- W2798839717 creator A5057657785 @default.
- W2798839717 creator A5079046603 @default.
- W2798839717 date "2018-06-27" @default.
- W2798839717 modified "2023-09-23" @default.
- W2798839717 title "Automatic Ground Truth Expansion for Timeline Evaluation" @default.
- W2798839717 cites W1966835268 @default.
- W2798839717 cites W1977884995 @default.
- W2798839717 cites W1983018376 @default.
- W2798839717 cites W2005589065 @default.
- W2798839717 cites W2028742638 @default.
- W2798839717 cites W2073722401 @default.
- W2798839717 cites W2075893676 @default.
- W2798839717 cites W2081033951 @default.
- W2798839717 cites W2094661073 @default.
- W2798839717 cites W2109244020 @default.
- W2798839717 cites W2153256230 @default.
- W2798839717 cites W2160892561 @default.
- W2798839717 cites W2169783907 @default.
- W2798839717 cites W2463112553 @default.
- W2798839717 cites W2535880480 @default.
- W2798839717 cites W2757329051 @default.
- W2798839717 cites W2781872642 @default.
- W2798839717 cites W4206671592 @default.
- W2798839717 cites W4230624213 @default.
- W2798839717 cites W4243009725 @default.
- W2798839717 doi "https://doi.org/10.1145/3209978.3210034" @default.
- W2798839717 hasPublicationYear "2018" @default.
- W2798839717 type Work @default.
- W2798839717 sameAs 2798839717 @default.
- W2798839717 citedByCount "2" @default.
- W2798839717 countsByYear W27988397172020 @default.
- W2798839717 countsByYear W27988397172022 @default.
- W2798839717 crossrefType "proceedings-article" @default.
- W2798839717 hasAuthorship W2798839717A5057643560 @default.
- W2798839717 hasAuthorship W2798839717A5057657785 @default.
- W2798839717 hasAuthorship W2798839717A5079046603 @default.
- W2798839717 hasBestOaLocation W27988397172 @default.
- W2798839717 hasConcept C105795698 @default.
- W2798839717 hasConcept C119857082 @default.
- W2798839717 hasConcept C121332964 @default.
- W2798839717 hasConcept C124101348 @default.
- W2798839717 hasConcept C146849305 @default.
- W2798839717 hasConcept C154945302 @default.
- W2798839717 hasConcept C170858558 @default.
- W2798839717 hasConcept C17744445 @default.
- W2798839717 hasConcept C189430467 @default.
- W2798839717 hasConcept C199539241 @default.
- W2798839717 hasConcept C23123220 @default.
- W2798839717 hasConcept C2776359362 @default.
- W2798839717 hasConcept C2779662365 @default.
- W2798839717 hasConcept C33923547 @default.
- W2798839717 hasConcept C41008148 @default.
- W2798839717 hasConcept C4438859 @default.
- W2798839717 hasConcept C62520636 @default.
- W2798839717 hasConcept C94625758 @default.
- W2798839717 hasConceptScore W2798839717C105795698 @default.
- W2798839717 hasConceptScore W2798839717C119857082 @default.
- W2798839717 hasConceptScore W2798839717C121332964 @default.
- W2798839717 hasConceptScore W2798839717C124101348 @default.
- W2798839717 hasConceptScore W2798839717C146849305 @default.
- W2798839717 hasConceptScore W2798839717C154945302 @default.
- W2798839717 hasConceptScore W2798839717C170858558 @default.
- W2798839717 hasConceptScore W2798839717C17744445 @default.
- W2798839717 hasConceptScore W2798839717C189430467 @default.
- W2798839717 hasConceptScore W2798839717C199539241 @default.
- W2798839717 hasConceptScore W2798839717C23123220 @default.
- W2798839717 hasConceptScore W2798839717C2776359362 @default.
- W2798839717 hasConceptScore W2798839717C2779662365 @default.
- W2798839717 hasConceptScore W2798839717C33923547 @default.
- W2798839717 hasConceptScore W2798839717C41008148 @default.
- W2798839717 hasConceptScore W2798839717C4438859 @default.
- W2798839717 hasConceptScore W2798839717C62520636 @default.
- W2798839717 hasConceptScore W2798839717C94625758 @default.
- W2798839717 hasLocation W27988397171 @default.
- W2798839717 hasLocation W27988397172 @default.
- W2798839717 hasOpenAccess W2798839717 @default.
- W2798839717 hasPrimaryLocation W27988397171 @default.
- W2798839717 hasRelatedWork W132250100 @default.
- W2798839717 hasRelatedWork W2093597205 @default.
- W2798839717 hasRelatedWork W2251955598 @default.
- W2798839717 hasRelatedWork W2347919119 @default.
- W2798839717 hasRelatedWork W2389846579 @default.
- W2798839717 hasRelatedWork W2392495745 @default.
- W2798839717 hasRelatedWork W2832206490 @default.
- W2798839717 hasRelatedWork W2895798794 @default.
- W2798839717 hasRelatedWork W2981651290 @default.
- W2798839717 hasRelatedWork W68538572 @default.
- W2798839717 isParatext "false" @default.
- W2798839717 isRetracted "false" @default.
- W2798839717 magId "2798839717" @default.
- W2798839717 workType "article" @default.