Matches in SemOpenAlex for { <https://semopenalex.org/work/W2184059664> ?p ?o ?g. }
Showing items 1 to 72 of
72
with 100 items per page.
- W2184059664 endingPage "7" @default.
- W2184059664 startingPage "2" @default.
- W2184059664 abstract "We present techniques for gathering data that expose errors of automatic predictive models. In certain common settings, traditional methods for evaluating predictive models tend to miss rare-but-important errors--most importantly, rare cases for which the model is confident of its prediction (but wrong). In this paper we present a system that, in a game-like setting, asks humans to identify cases that will cause the predictive-model-based system to fail. Such techniques are valuable in discovering problematic cases that do not reveal themselves during the normal operation of the system, and may include cases that are rare but catastrophic. We describe the design of the system, including design iterations that did not quite work. In particular, the system incentivizes humans to provide examples that are difficult for the model to handle, by providing a reward proportional to the magnitude of the predictive model's error. The humans are asked to the and find cases where the automatic model (the Machine) is wrong. Experiments show that the humans using Beat the Machine identify more errors than traditional techniques for discovering errors in from predictive models, and indeed, they identify many more errors where the machine is confident it is correct. Further, the cases the humans identify seem to be not simply outliers, but coherent areas missed completely by the model. Beat the machine identifies the unknown unknowns." @default.
- W2184059664 created "2016-06-24" @default.
- W2184059664 creator A5010731709 @default.
- W2184059664 creator A5037283651 @default.
- W2184059664 creator A5064221602 @default.
- W2184059664 date "2011-01-01" @default.
- W2184059664 modified "2023-09-23" @default.
- W2184059664 title "Beat the machine: challenging workers to find the unknown unknowns" @default.
- W2184059664 cites W1488252886 @default.
- W2184059664 cites W1965262974 @default.
- W2184059664 cites W2010135967 @default.
- W2184059664 cites W2108736173 @default.
- W2184059664 cites W2116917894 @default.
- W2184059664 cites W2125943921 @default.
- W2184059664 cites W2145073242 @default.
- W2184059664 cites W2903158431 @default.
- W2184059664 hasPublicationYear "2011" @default.
- W2184059664 type Work @default.
- W2184059664 sameAs 2184059664 @default.
- W2184059664 citedByCount "27" @default.
- W2184059664 countsByYear W21840596642012 @default.
- W2184059664 countsByYear W21840596642013 @default.
- W2184059664 countsByYear W21840596642014 @default.
- W2184059664 countsByYear W21840596642016 @default.
- W2184059664 countsByYear W21840596642017 @default.
- W2184059664 countsByYear W21840596642018 @default.
- W2184059664 countsByYear W21840596642019 @default.
- W2184059664 countsByYear W21840596642020 @default.
- W2184059664 countsByYear W21840596642021 @default.
- W2184059664 crossrefType "proceedings-article" @default.
- W2184059664 hasAuthorship W2184059664A5010731709 @default.
- W2184059664 hasAuthorship W2184059664A5037283651 @default.
- W2184059664 hasAuthorship W2184059664A5064221602 @default.
- W2184059664 hasConcept C119857082 @default.
- W2184059664 hasConcept C124101348 @default.
- W2184059664 hasConcept C154945302 @default.
- W2184059664 hasConcept C41008148 @default.
- W2184059664 hasConcept C45804977 @default.
- W2184059664 hasConcept C79337645 @default.
- W2184059664 hasConceptScore W2184059664C119857082 @default.
- W2184059664 hasConceptScore W2184059664C124101348 @default.
- W2184059664 hasConceptScore W2184059664C154945302 @default.
- W2184059664 hasConceptScore W2184059664C41008148 @default.
- W2184059664 hasConceptScore W2184059664C45804977 @default.
- W2184059664 hasConceptScore W2184059664C79337645 @default.
- W2184059664 hasOpenAccess W2184059664 @default.
- W2184059664 hasRelatedWork W1480715147 @default.
- W2184059664 hasRelatedWork W1576620340 @default.
- W2184059664 hasRelatedWork W1861492603 @default.
- W2184059664 hasRelatedWork W2009681737 @default.
- W2184059664 hasRelatedWork W2036625304 @default.
- W2184059664 hasRelatedWork W2053075547 @default.
- W2184059664 hasRelatedWork W2058556535 @default.
- W2184059664 hasRelatedWork W2059362837 @default.
- W2184059664 hasRelatedWork W2092594752 @default.
- W2184059664 hasRelatedWork W2094485533 @default.
- W2184059664 hasRelatedWork W2106897586 @default.
- W2184059664 hasRelatedWork W2109021302 @default.
- W2184059664 hasRelatedWork W2125943921 @default.
- W2184059664 hasRelatedWork W2127008633 @default.
- W2184059664 hasRelatedWork W2141282920 @default.
- W2184059664 hasRelatedWork W2561664829 @default.
- W2184059664 hasRelatedWork W2583689529 @default.
- W2184059664 hasRelatedWork W2788428929 @default.
- W2184059664 hasRelatedWork W2788481061 @default.
- W2184059664 hasRelatedWork W2903158431 @default.
- W2184059664 isParatext "false" @default.
- W2184059664 isRetracted "false" @default.
- W2184059664 magId "2184059664" @default.
- W2184059664 workType "article" @default.