Matches in SemOpenAlex for { <https://semopenalex.org/work/W2290496639> ?p ?o ?g. }
- W2290496639 abstract "Crowdsourcing, due to its inexpensive and timely nature, has become a popular method of collecting data that is difficult for computers to generate. We focus on using this method of human computation to gather labels for classification tasks, to be used for machine learning. However, data gathered this way may be of varying quality, ranging from spam to perfect. We aim to maintain the cost-effective property of crowdsourcing, while also obtaining quality results. Towards a solution, we have multiple workers label the same problem instance, aggregating the responses into one label afterwards. We study what aggregation method to use, and what guarantees we can provide on its estimates. Different crowdsourcing models call for different techniques – we outline and organize various directions taken in the literature, and focus on the Dawid-Skene model. In this setting each instance has a true label, workers are independent, and the performance of each individual is assumed to be uniform over all instances, in the sense that she has an inherent skill that governs the probability with which she labels correctly. Her skill is unknown to us. Aggregation methods aim to find the true label of each task based solely on the labels the workers reported. We measure the performance of these methods by the probability with which the estimates they output match the true label. In practice, a popular procedure is to run the EM algorithm to find estimates of the skills and labels. However, this method is not directly guaranteed to perform well in our measure. We collect and evaluate theoretical results that bound the error of various aggregation methods, including specific variants of EM. Finally, we prove a guarantee on the error suffered by the maximum likelihood estimator, the global optima of the function that EM aims to numerically optimize." @default.
- W2290496639 created "2016-06-24" @default.
- W2290496639 creator A5040427757 @default.
- W2290496639 date "2015-10-29" @default.
- W2290496639 modified "2023-09-23" @default.
- W2290496639 title "A Statistical Analysis of the Aggregation of Crowdsourced Labels" @default.
- W2290496639 cites W108763474 @default.
- W2290496639 cites W1497882404 @default.
- W2290496639 cites W1508306565 @default.
- W2290496639 cites W1522318289 @default.
- W2290496639 cites W1522548332 @default.
- W2290496639 cites W1540526841 @default.
- W2290496639 cites W1581692152 @default.
- W2290496639 cites W1693667 @default.
- W2290496639 cites W1963492191 @default.
- W2290496639 cites W1988560072 @default.
- W2290496639 cites W2010029425 @default.
- W2290496639 cites W2074154707 @default.
- W2290496639 cites W2111296615 @default.
- W2290496639 cites W2129345386 @default.
- W2290496639 cites W2140890285 @default.
- W2290496639 cites W2141761557 @default.
- W2290496639 cites W2142518823 @default.
- W2290496639 cites W2149273804 @default.
- W2290496639 cites W2163522723 @default.
- W2290496639 cites W2554839354 @default.
- W2290496639 cites W2949312134 @default.
- W2290496639 cites W2963761635 @default.
- W2290496639 cites W2964305490 @default.
- W2290496639 cites W587554298 @default.
- W2290496639 cites W9014458 @default.
- W2290496639 cites W2152887173 @default.
- W2290496639 hasPublicationYear "2015" @default.
- W2290496639 type Work @default.
- W2290496639 sameAs 2290496639 @default.
- W2290496639 citedByCount "3" @default.
- W2290496639 countsByYear W22904966392017 @default.
- W2290496639 countsByYear W22904966392018 @default.
- W2290496639 countsByYear W22904966392019 @default.
- W2290496639 crossrefType "dissertation" @default.
- W2290496639 hasAuthorship W2290496639A5040427757 @default.
- W2290496639 hasConcept C111472728 @default.
- W2290496639 hasConcept C119857082 @default.
- W2290496639 hasConcept C120665830 @default.
- W2290496639 hasConcept C121332964 @default.
- W2290496639 hasConcept C124101348 @default.
- W2290496639 hasConcept C127413603 @default.
- W2290496639 hasConcept C136764020 @default.
- W2290496639 hasConcept C138885662 @default.
- W2290496639 hasConcept C154945302 @default.
- W2290496639 hasConcept C189950617 @default.
- W2290496639 hasConcept C192209626 @default.
- W2290496639 hasConcept C201995342 @default.
- W2290496639 hasConcept C2522767166 @default.
- W2290496639 hasConcept C2779530757 @default.
- W2290496639 hasConcept C2780009758 @default.
- W2290496639 hasConcept C2780451532 @default.
- W2290496639 hasConcept C41008148 @default.
- W2290496639 hasConcept C62230096 @default.
- W2290496639 hasConceptScore W2290496639C111472728 @default.
- W2290496639 hasConceptScore W2290496639C119857082 @default.
- W2290496639 hasConceptScore W2290496639C120665830 @default.
- W2290496639 hasConceptScore W2290496639C121332964 @default.
- W2290496639 hasConceptScore W2290496639C124101348 @default.
- W2290496639 hasConceptScore W2290496639C127413603 @default.
- W2290496639 hasConceptScore W2290496639C136764020 @default.
- W2290496639 hasConceptScore W2290496639C138885662 @default.
- W2290496639 hasConceptScore W2290496639C154945302 @default.
- W2290496639 hasConceptScore W2290496639C189950617 @default.
- W2290496639 hasConceptScore W2290496639C192209626 @default.
- W2290496639 hasConceptScore W2290496639C201995342 @default.
- W2290496639 hasConceptScore W2290496639C2522767166 @default.
- W2290496639 hasConceptScore W2290496639C2779530757 @default.
- W2290496639 hasConceptScore W2290496639C2780009758 @default.
- W2290496639 hasConceptScore W2290496639C2780451532 @default.
- W2290496639 hasConceptScore W2290496639C41008148 @default.
- W2290496639 hasConceptScore W2290496639C62230096 @default.
- W2290496639 hasLocation W22904966391 @default.
- W2290496639 hasOpenAccess W2290496639 @default.
- W2290496639 hasPrimaryLocation W22904966391 @default.
- W2290496639 hasRelatedWork W2252211741 @default.
- W2290496639 hasRelatedWork W250721181 @default.
- W2290496639 hasRelatedWork W2602938505 @default.
- W2290496639 hasRelatedWork W2735919294 @default.
- W2290496639 hasRelatedWork W2742099644 @default.
- W2290496639 hasRelatedWork W2799153978 @default.
- W2290496639 hasRelatedWork W2909469631 @default.
- W2290496639 hasRelatedWork W2945456403 @default.
- W2290496639 hasRelatedWork W2947088031 @default.
- W2290496639 hasRelatedWork W2947963870 @default.
- W2290496639 hasRelatedWork W2950902462 @default.
- W2290496639 hasRelatedWork W2951729010 @default.
- W2290496639 hasRelatedWork W2963443341 @default.
- W2290496639 hasRelatedWork W2968034890 @default.
- W2290496639 hasRelatedWork W2971962903 @default.
- W2290496639 hasRelatedWork W3033229230 @default.
- W2290496639 hasRelatedWork W3034457456 @default.
- W2290496639 hasRelatedWork W3048324731 @default.
- W2290496639 hasRelatedWork W3156091046 @default.
- W2290496639 hasRelatedWork W3177164493 @default.