Matches in SemOpenAlex for { <https://semopenalex.org/work/W2466887399> ?p ?o ?g. }
- W2466887399 abstract "In many settings, an effective way of evaluating objects of interest is to collect evaluations from dispersed individuals and to aggregate these evaluations together. Some examples are categorizing online content and evaluating student assignments via peer grading. For this data science problem, one challenge is to motivate participants to conduct such evaluations carefully and to report them honestly, particularly when doing so is costly. Existing approaches, notably peer-prediction mechanisms, can incentivize truth telling in equilibrium. However, they also give rise to equilibria in which agents do not pay the costs required to evaluate accurately, and hence fail to elicit useful information. We show that this problem is unavoidable whenever agents are able to coordinate using low-cost signals about the items being evaluated (e.g., text labels or pictures). We then consider ways of circumventing this problem by comparing agents' reports to ground truth, which is available in practice when there exist trusted evaluators---such as teaching assistants in the peer grading scenario---who can perform a limited number of unbiased (but noisy) evaluations. Of course, when such ground truth is available, a simpler approach is also possible: rewarding each agent based on agreement with ground truth with some probability, and unconditionally rewarding the agent otherwise. Surprisingly, we show that the simpler mechanism achieves stronger incentive guarantees given less access to ground truth than a large set of peer-prediction mechanisms." @default.
- W2466887399 created "2016-07-22" @default.
- W2466887399 creator A5022280555 @default.
- W2466887399 creator A5088909874 @default.
- W2466887399 creator A5089807699 @default.
- W2466887399 date "2016-06-22" @default.
- W2466887399 modified "2023-09-27" @default.
- W2466887399 title "Incentivizing Evaluation via Limited Access to Ground Truth: Peer-Prediction Makes Things Worse" @default.
- W2466887399 cites W1499884756 @default.
- W2466887399 cites W1595524969 @default.
- W2466887399 cites W1595685357 @default.
- W2466887399 cites W1979430517 @default.
- W2466887399 cites W2025720061 @default.
- W2466887399 cites W2057283903 @default.
- W2466887399 cites W2063782051 @default.
- W2466887399 cites W2070693116 @default.
- W2466887399 cites W2098992709 @default.
- W2466887399 cites W2100267603 @default.
- W2466887399 cites W2109304399 @default.
- W2466887399 cites W2147498513 @default.
- W2466887399 cites W2148189517 @default.
- W2466887399 cites W2153209865 @default.
- W2466887399 cites W2166164009 @default.
- W2466887399 cites W2182568424 @default.
- W2466887399 cites W2184706947 @default.
- W2466887399 cites W2288655875 @default.
- W2466887399 cites W2788116530 @default.
- W2466887399 cites W2952068623 @default.
- W2466887399 cites W2949539581 @default.
- W2466887399 hasPublicationYear "2016" @default.
- W2466887399 type Work @default.
- W2466887399 sameAs 2466887399 @default.
- W2466887399 citedByCount "21" @default.
- W2466887399 countsByYear W24668873992015 @default.
- W2466887399 countsByYear W24668873992016 @default.
- W2466887399 countsByYear W24668873992017 @default.
- W2466887399 countsByYear W24668873992018 @default.
- W2466887399 countsByYear W24668873992019 @default.
- W2466887399 countsByYear W24668873992020 @default.
- W2466887399 countsByYear W24668873992021 @default.
- W2466887399 crossrefType "posted-content" @default.
- W2466887399 hasAuthorship W2466887399A5022280555 @default.
- W2466887399 hasAuthorship W2466887399A5088909874 @default.
- W2466887399 hasAuthorship W2466887399A5089807699 @default.
- W2466887399 hasConcept C11171543 @default.
- W2466887399 hasConcept C127413603 @default.
- W2466887399 hasConcept C146849305 @default.
- W2466887399 hasConcept C147176958 @default.
- W2466887399 hasConcept C154945302 @default.
- W2466887399 hasConcept C15744967 @default.
- W2466887399 hasConcept C159985019 @default.
- W2466887399 hasConcept C162324750 @default.
- W2466887399 hasConcept C175444787 @default.
- W2466887399 hasConcept C177264268 @default.
- W2466887399 hasConcept C192562407 @default.
- W2466887399 hasConcept C199360897 @default.
- W2466887399 hasConcept C2777286243 @default.
- W2466887399 hasConcept C2777877512 @default.
- W2466887399 hasConcept C2910433854 @default.
- W2466887399 hasConcept C29122968 @default.
- W2466887399 hasConcept C41008148 @default.
- W2466887399 hasConcept C4679612 @default.
- W2466887399 hasConcept C77805123 @default.
- W2466887399 hasConceptScore W2466887399C11171543 @default.
- W2466887399 hasConceptScore W2466887399C127413603 @default.
- W2466887399 hasConceptScore W2466887399C146849305 @default.
- W2466887399 hasConceptScore W2466887399C147176958 @default.
- W2466887399 hasConceptScore W2466887399C154945302 @default.
- W2466887399 hasConceptScore W2466887399C15744967 @default.
- W2466887399 hasConceptScore W2466887399C159985019 @default.
- W2466887399 hasConceptScore W2466887399C162324750 @default.
- W2466887399 hasConceptScore W2466887399C175444787 @default.
- W2466887399 hasConceptScore W2466887399C177264268 @default.
- W2466887399 hasConceptScore W2466887399C192562407 @default.
- W2466887399 hasConceptScore W2466887399C199360897 @default.
- W2466887399 hasConceptScore W2466887399C2777286243 @default.
- W2466887399 hasConceptScore W2466887399C2777877512 @default.
- W2466887399 hasConceptScore W2466887399C2910433854 @default.
- W2466887399 hasConceptScore W2466887399C29122968 @default.
- W2466887399 hasConceptScore W2466887399C41008148 @default.
- W2466887399 hasConceptScore W2466887399C4679612 @default.
- W2466887399 hasConceptScore W2466887399C77805123 @default.
- W2466887399 hasLocation W24668873991 @default.
- W2466887399 hasOpenAccess W2466887399 @default.
- W2466887399 hasPrimaryLocation W24668873991 @default.
- W2466887399 hasRelatedWork W1499884756 @default.
- W2466887399 hasRelatedWork W1506509256 @default.
- W2466887399 hasRelatedWork W1968072657 @default.
- W2466887399 hasRelatedWork W1969883013 @default.
- W2466887399 hasRelatedWork W2025720061 @default.
- W2466887399 hasRelatedWork W2063782051 @default.
- W2466887399 hasRelatedWork W2109304399 @default.
- W2466887399 hasRelatedWork W2120898858 @default.
- W2466887399 hasRelatedWork W2138524027 @default.
- W2466887399 hasRelatedWork W2147498513 @default.
- W2466887399 hasRelatedWork W2153209865 @default.
- W2466887399 hasRelatedWork W2219308446 @default.
- W2466887399 hasRelatedWork W2267283990 @default.
- W2466887399 hasRelatedWork W2295734738 @default.
- W2466887399 hasRelatedWork W2560051878 @default.