Matches in SemOpenAlex for { <https://semopenalex.org/work/W2013130010> ?p ?o ?g. }
- W2013130010 abstract "Motivated by the high cost of human curation of biological databases, there is an increasing interest in using computational approaches to assist human curators and accelerate the manual curation process. Towards the goal of cataloging drug indications from FDA drug labels, we recently developed LabeledIn, a human-curated drug indication resource for 250 clinical drugs. Its development required over 40 h of human effort across 20 weeks, despite using well-defined annotation guidelines. In this study, we aim to investigate the feasibility of scaling drug indication annotation through a crowdsourcing technique where an unknown network of workers can be recruited through the technical environment of Amazon Mechanical Turk (MTurk). To translate the expert-curation task of cataloging indications into human intelligence tasks (HITs) suitable for the average workers on MTurk, we first simplify the complex task such that each HIT only involves a worker making a binary judgment of whether a highlighted disease, in context of a given drug label, is an indication. In addition, this study is novel in the crowdsourcing interface design where the annotation guidelines are encoded into user options. For evaluation, we assess the ability of our proposed method to achieve high-quality annotations in a time-efficient and cost-effective manner. We posted over 3000 HITs drawn from 706 drug labels on MTurk. Within 8 h of posting, we collected 18 775 judgments from 74 workers, and achieved an aggregated accuracy of 96% on 450 control HITs (where gold-standard answers are known), at a cost of $1.75 per drug label. On the basis of these results, we conclude that our crowdsourcing approach not only results in significant cost and time saving, but also leads to accuracy comparable to that of domain experts. Database URL: ftp://ftp.ncbi.nlm.nih.gov/pub/lu/LabeledIn/Crowdsourcing/ ." @default.
- W2013130010 created "2016-06-24" @default.
- W2013130010 creator A5002815374 @default.
- W2013130010 creator A5015172675 @default.
- W2013130010 creator A5045869273 @default.
- W2013130010 creator A5054221875 @default.
- W2013130010 creator A5075669751 @default.
- W2013130010 creator A5077990968 @default.
- W2013130010 creator A5083081872 @default.
- W2013130010 date "2015-01-01" @default.
- W2013130010 modified "2023-10-11" @default.
- W2013130010 title "Scaling drug indication curation through crowdsourcing" @default.
- W2013130010 cites W1550258693 @default.
- W2013130010 cites W1690976696 @default.
- W2013130010 cites W19550711 @default.
- W2013130010 cites W1964670939 @default.
- W2013130010 cites W1970381522 @default.
- W2013130010 cites W1978700375 @default.
- W2013130010 cites W1984314602 @default.
- W2013130010 cites W2016914833 @default.
- W2013130010 cites W2031614194 @default.
- W2013130010 cites W2048157132 @default.
- W2013130010 cites W2049825704 @default.
- W2013130010 cites W2056119436 @default.
- W2013130010 cites W2061167720 @default.
- W2013130010 cites W2074640468 @default.
- W2013130010 cites W2093044076 @default.
- W2013130010 cites W2102575587 @default.
- W2013130010 cites W2104400119 @default.
- W2013130010 cites W2105217514 @default.
- W2013130010 cites W2105481534 @default.
- W2013130010 cites W2112227057 @default.
- W2013130010 cites W2116705992 @default.
- W2013130010 cites W2116947789 @default.
- W2013130010 cites W2119002393 @default.
- W2013130010 cites W2121111666 @default.
- W2013130010 cites W2123579157 @default.
- W2013130010 cites W2124798372 @default.
- W2013130010 cites W2127014487 @default.
- W2013130010 cites W2130332503 @default.
- W2013130010 cites W2142016317 @default.
- W2013130010 cites W2145383760 @default.
- W2013130010 cites W2147285089 @default.
- W2013130010 cites W2150850055 @default.
- W2013130010 cites W2156467546 @default.
- W2013130010 cites W2163303745 @default.
- W2013130010 cites W2166240318 @default.
- W2013130010 cites W2166817943 @default.
- W2013130010 cites W2168644196 @default.
- W2013130010 cites W2169997311 @default.
- W2013130010 cites W2171913066 @default.
- W2013130010 cites W2579555219 @default.
- W2013130010 doi "https://doi.org/10.1093/database/bav016" @default.
- W2013130010 hasPubMedCentralId "https://www.ncbi.nlm.nih.gov/pmc/articles/4369375" @default.
- W2013130010 hasPubMedId "https://pubmed.ncbi.nlm.nih.gov/25797061" @default.
- W2013130010 hasPublicationYear "2015" @default.
- W2013130010 type Work @default.
- W2013130010 sameAs 2013130010 @default.
- W2013130010 citedByCount "34" @default.
- W2013130010 countsByYear W20131300102015 @default.
- W2013130010 countsByYear W20131300102016 @default.
- W2013130010 countsByYear W20131300102017 @default.
- W2013130010 countsByYear W20131300102018 @default.
- W2013130010 countsByYear W20131300102019 @default.
- W2013130010 countsByYear W20131300102020 @default.
- W2013130010 countsByYear W20131300102021 @default.
- W2013130010 countsByYear W20131300102022 @default.
- W2013130010 countsByYear W20131300102023 @default.
- W2013130010 crossrefType "journal-article" @default.
- W2013130010 hasAuthorship W2013130010A5002815374 @default.
- W2013130010 hasAuthorship W2013130010A5015172675 @default.
- W2013130010 hasAuthorship W2013130010A5045869273 @default.
- W2013130010 hasAuthorship W2013130010A5054221875 @default.
- W2013130010 hasAuthorship W2013130010A5075669751 @default.
- W2013130010 hasAuthorship W2013130010A5077990968 @default.
- W2013130010 hasAuthorship W2013130010A5083081872 @default.
- W2013130010 hasBestOaLocation W20131300101 @default.
- W2013130010 hasConcept C111919701 @default.
- W2013130010 hasConcept C127413603 @default.
- W2013130010 hasConcept C136764020 @default.
- W2013130010 hasConcept C151730666 @default.
- W2013130010 hasConcept C154945302 @default.
- W2013130010 hasConcept C201995342 @default.
- W2013130010 hasConcept C206345919 @default.
- W2013130010 hasConcept C23123220 @default.
- W2013130010 hasConcept C2522767166 @default.
- W2013130010 hasConcept C2776321320 @default.
- W2013130010 hasConcept C2777922441 @default.
- W2013130010 hasConcept C2779343474 @default.
- W2013130010 hasConcept C2780451532 @default.
- W2013130010 hasConcept C31258907 @default.
- W2013130010 hasConcept C41008148 @default.
- W2013130010 hasConcept C62230096 @default.
- W2013130010 hasConcept C86803240 @default.
- W2013130010 hasConcept C91632574 @default.
- W2013130010 hasConcept C98045186 @default.
- W2013130010 hasConceptScore W2013130010C111919701 @default.
- W2013130010 hasConceptScore W2013130010C127413603 @default.
- W2013130010 hasConceptScore W2013130010C136764020 @default.
- W2013130010 hasConceptScore W2013130010C151730666 @default.