Matches in SemOpenAlex for { <https://semopenalex.org/work/W2122045550> ?p ?o ?g. }
- W2122045550 abstract "Gathering annotations from non-expert online raters is an attractive method for quickly completing large-scale annotation tasks, but the increased possibility of unreliable annotators and diminished work quality remains a cause for concern. In the context of information retrieval, where human-encoded relevance judgments underlie the evaluation of new systems and methods, the ability to quickly and reliably collect trustworthy annotations allows for quicker development and iteration of research.In the context of paid online workers, this study evaluates indicators of non-expert performance along three lines: temporality, experience, and agreement. It is found that user performance is a key indicator for future performance. Additionally, the time spent by raters familiarizing themselves with a new set of tasks is important for rater quality, as is long-term familiarity with a topic being rated.These findings may inform large-scale digital collections' use of non-expert raters for performing more purposive and affordable online annotation activities." @default.
- W2122045550 created "2016-06-24" @default.
- W2122045550 creator A5001038669 @default.
- W2122045550 creator A5002257700 @default.
- W2122045550 creator A5031860314 @default.
- W2122045550 creator A5065007271 @default.
- W2122045550 date "2012-01-01" @default.
- W2122045550 modified "2023-09-26" @default.
- W2122045550 title "Evaluating rater quality and rating difficulty in online annotation activities" @default.
- W2122045550 cites W1541505110 @default.
- W2122045550 cites W1970381522 @default.
- W2122045550 cites W1992730580 @default.
- W2122045550 cites W2024166834 @default.
- W2122045550 cites W2048978851 @default.
- W2122045550 cites W2062538491 @default.
- W2122045550 cites W2125943921 @default.
- W2122045550 cites W2141381966 @default.
- W2122045550 cites W2158880898 @default.
- W2122045550 cites W2164124780 @default.
- W2122045550 cites W2404374285 @default.
- W2122045550 doi "https://doi.org/10.1002/meet.14504901166" @default.
- W2122045550 hasPublicationYear "2012" @default.
- W2122045550 type Work @default.
- W2122045550 sameAs 2122045550 @default.
- W2122045550 citedByCount "1" @default.
- W2122045550 countsByYear W21220455502014 @default.
- W2122045550 crossrefType "journal-article" @default.
- W2122045550 hasAuthorship W2122045550A5001038669 @default.
- W2122045550 hasAuthorship W2122045550A5002257700 @default.
- W2122045550 hasAuthorship W2122045550A5031860314 @default.
- W2122045550 hasAuthorship W2122045550A5065007271 @default.
- W2122045550 hasConcept C108827166 @default.
- W2122045550 hasConcept C111472728 @default.
- W2122045550 hasConcept C121332964 @default.
- W2122045550 hasConcept C136764020 @default.
- W2122045550 hasConcept C138885662 @default.
- W2122045550 hasConcept C141679102 @default.
- W2122045550 hasConcept C151730666 @default.
- W2122045550 hasConcept C153701036 @default.
- W2122045550 hasConcept C154945302 @default.
- W2122045550 hasConcept C158154518 @default.
- W2122045550 hasConcept C177264268 @default.
- W2122045550 hasConcept C17744445 @default.
- W2122045550 hasConcept C199360897 @default.
- W2122045550 hasConcept C199539241 @default.
- W2122045550 hasConcept C23123220 @default.
- W2122045550 hasConcept C2522767166 @default.
- W2122045550 hasConcept C2776321320 @default.
- W2122045550 hasConcept C2778755073 @default.
- W2122045550 hasConcept C2779343474 @default.
- W2122045550 hasConcept C2779530757 @default.
- W2122045550 hasConcept C41008148 @default.
- W2122045550 hasConcept C62230096 @default.
- W2122045550 hasConcept C62520636 @default.
- W2122045550 hasConcept C86803240 @default.
- W2122045550 hasConceptScore W2122045550C108827166 @default.
- W2122045550 hasConceptScore W2122045550C111472728 @default.
- W2122045550 hasConceptScore W2122045550C121332964 @default.
- W2122045550 hasConceptScore W2122045550C136764020 @default.
- W2122045550 hasConceptScore W2122045550C138885662 @default.
- W2122045550 hasConceptScore W2122045550C141679102 @default.
- W2122045550 hasConceptScore W2122045550C151730666 @default.
- W2122045550 hasConceptScore W2122045550C153701036 @default.
- W2122045550 hasConceptScore W2122045550C154945302 @default.
- W2122045550 hasConceptScore W2122045550C158154518 @default.
- W2122045550 hasConceptScore W2122045550C177264268 @default.
- W2122045550 hasConceptScore W2122045550C17744445 @default.
- W2122045550 hasConceptScore W2122045550C199360897 @default.
- W2122045550 hasConceptScore W2122045550C199539241 @default.
- W2122045550 hasConceptScore W2122045550C23123220 @default.
- W2122045550 hasConceptScore W2122045550C2522767166 @default.
- W2122045550 hasConceptScore W2122045550C2776321320 @default.
- W2122045550 hasConceptScore W2122045550C2778755073 @default.
- W2122045550 hasConceptScore W2122045550C2779343474 @default.
- W2122045550 hasConceptScore W2122045550C2779530757 @default.
- W2122045550 hasConceptScore W2122045550C41008148 @default.
- W2122045550 hasConceptScore W2122045550C62230096 @default.
- W2122045550 hasConceptScore W2122045550C62520636 @default.
- W2122045550 hasConceptScore W2122045550C86803240 @default.
- W2122045550 hasLocation W21220455501 @default.
- W2122045550 hasOpenAccess W2122045550 @default.
- W2122045550 hasPrimaryLocation W21220455501 @default.
- W2122045550 hasRelatedWork W1969235627 @default.
- W2122045550 hasRelatedWork W1970909954 @default.
- W2122045550 hasRelatedWork W2005176141 @default.
- W2122045550 hasRelatedWork W2036483307 @default.
- W2122045550 hasRelatedWork W2048601276 @default.
- W2122045550 hasRelatedWork W2058880306 @default.
- W2122045550 hasRelatedWork W2474333055 @default.
- W2122045550 hasRelatedWork W2588706215 @default.
- W2122045550 hasRelatedWork W2592213114 @default.
- W2122045550 hasRelatedWork W2778469527 @default.
- W2122045550 hasRelatedWork W2889634735 @default.
- W2122045550 hasRelatedWork W2902985622 @default.
- W2122045550 hasRelatedWork W2963352676 @default.
- W2122045550 hasRelatedWork W2979968844 @default.
- W2122045550 hasRelatedWork W2991440563 @default.
- W2122045550 hasRelatedWork W3167007835 @default.
- W2122045550 hasRelatedWork W69918181 @default.
- W2122045550 hasRelatedWork W1572289399 @default.