Matches in SemOpenAlex for { <https://semopenalex.org/work/W1598493612> ?p ?o ?g. }
Showing items 1 to 92 of
92
with 100 items per page.
- W1598493612 endingPage "49" @default.
- W1598493612 startingPage "41" @default.
- W1598493612 abstract "The emergence of online crowdsourcing sites, online work platforms, and evenMassive Open Online Courses (MOOCs), has created an increasing need for reliably evaluating the skills of the participating users in a scalable way.Many platforms already allow users to take online tests and verify their skills, but the existing approaches face many problems. First of all, cheating is very common in online testing without supervision, as the test questions often leak and become easily available online together with the answers.Second, technical skills, such as programming, require the tests to be frequently updated in order to reflect the current state-of-the-art. Third,there is very limited evaluation of the tests themselves, and how effectively they measure the skill that the users are tested for. In this paper, we present a Scalable Testing and Evaluation Platform (STEP),that allows continuous generation and evaluation of test questions. STEP leverages already available content, on Question Answering sites such as StackOverflow and re-purposes these questions to generate tests. The system utilizes a crowdsourcing component for the editing of the questions, while it uses automated techniques for identifying promising QA threads that can be successfully re-purposed for testing. This continuous question generation decreases the impact of cheating and also creates questions that are closer to the real problems that the skill holder is expected to solve in real life.STEP also leverages the use of Item Response Theory to evaluate the quality of the questions. We also use external signals about the quality of the workers.These identify the questions that have the strongest predictive ability in distinguishing workers that have the potential to succeed in the online job marketplaces. Existing approaches contrast in using only internal consistency metrics to evaluate the questions. Finally, our system employs an automatic leakage detector that queries the Internet to identify leaked versions of our questions. We then mark these questions as practice only, effectively removing them from the pool of questions used for evaluation. Our experimental evaluation shows that our system generates questions of comparable or higher quality compared to existing tests, with a cost of approximately 3-5 dollars per question, which is lower than the cost of licensing questions from existing test banks." @default.
- W1598493612 created "2016-06-24" @default.
- W1598493612 creator A5010731709 @default.
- W1598493612 creator A5066703585 @default.
- W1598493612 date "2014-09-05" @default.
- W1598493612 modified "2023-10-12" @default.
- W1598493612 title "STEP: A Scalable Testing and Evaluation Platform" @default.
- W1598493612 cites W1485504657 @default.
- W1598493612 cites W189319093 @default.
- W1598493612 cites W1986021671 @default.
- W1598493612 cites W2006818965 @default.
- W1598493612 cites W2032955274 @default.
- W1598493612 cites W2057769766 @default.
- W1598493612 cites W2066625870 @default.
- W1598493612 cites W2088700588 @default.
- W1598493612 cites W2111593814 @default.
- W1598493612 cites W2115747715 @default.
- W1598493612 cites W2131723483 @default.
- W1598493612 cites W2156109180 @default.
- W1598493612 cites W2168765606 @default.
- W1598493612 cites W2185761757 @default.
- W1598493612 cites W2337430557 @default.
- W1598493612 cites W2487802744 @default.
- W1598493612 cites W2911761300 @default.
- W1598493612 cites W3123006871 @default.
- W1598493612 cites W3124686198 @default.
- W1598493612 cites W99720618 @default.
- W1598493612 doi "https://doi.org/10.1609/hcomp.v2i1.13159" @default.
- W1598493612 hasPublicationYear "2014" @default.
- W1598493612 type Work @default.
- W1598493612 sameAs 1598493612 @default.
- W1598493612 citedByCount "4" @default.
- W1598493612 countsByYear W15984936122015 @default.
- W1598493612 countsByYear W15984936122017 @default.
- W1598493612 countsByYear W15984936122019 @default.
- W1598493612 countsByYear W15984936122021 @default.
- W1598493612 crossrefType "journal-article" @default.
- W1598493612 hasAuthorship W1598493612A5010731709 @default.
- W1598493612 hasAuthorship W1598493612A5066703585 @default.
- W1598493612 hasBestOaLocation W15984936121 @default.
- W1598493612 hasConcept C111472728 @default.
- W1598493612 hasConcept C136764020 @default.
- W1598493612 hasConcept C138885662 @default.
- W1598493612 hasConcept C151730666 @default.
- W1598493612 hasConcept C15744967 @default.
- W1598493612 hasConcept C2522767166 @default.
- W1598493612 hasConcept C2777267654 @default.
- W1598493612 hasConcept C2778024590 @default.
- W1598493612 hasConcept C2779530757 @default.
- W1598493612 hasConcept C41008148 @default.
- W1598493612 hasConcept C48044578 @default.
- W1598493612 hasConcept C49774154 @default.
- W1598493612 hasConcept C62230096 @default.
- W1598493612 hasConcept C77088390 @default.
- W1598493612 hasConcept C77805123 @default.
- W1598493612 hasConcept C86803240 @default.
- W1598493612 hasConceptScore W1598493612C111472728 @default.
- W1598493612 hasConceptScore W1598493612C136764020 @default.
- W1598493612 hasConceptScore W1598493612C138885662 @default.
- W1598493612 hasConceptScore W1598493612C151730666 @default.
- W1598493612 hasConceptScore W1598493612C15744967 @default.
- W1598493612 hasConceptScore W1598493612C2522767166 @default.
- W1598493612 hasConceptScore W1598493612C2777267654 @default.
- W1598493612 hasConceptScore W1598493612C2778024590 @default.
- W1598493612 hasConceptScore W1598493612C2779530757 @default.
- W1598493612 hasConceptScore W1598493612C41008148 @default.
- W1598493612 hasConceptScore W1598493612C48044578 @default.
- W1598493612 hasConceptScore W1598493612C49774154 @default.
- W1598493612 hasConceptScore W1598493612C62230096 @default.
- W1598493612 hasConceptScore W1598493612C77088390 @default.
- W1598493612 hasConceptScore W1598493612C77805123 @default.
- W1598493612 hasConceptScore W1598493612C86803240 @default.
- W1598493612 hasLocation W15984936121 @default.
- W1598493612 hasOpenAccess W1598493612 @default.
- W1598493612 hasPrimaryLocation W15984936121 @default.
- W1598493612 hasRelatedWork W1525643724 @default.
- W1598493612 hasRelatedWork W2024787338 @default.
- W1598493612 hasRelatedWork W2292984643 @default.
- W1598493612 hasRelatedWork W2302028273 @default.
- W1598493612 hasRelatedWork W2364921833 @default.
- W1598493612 hasRelatedWork W2379285345 @default.
- W1598493612 hasRelatedWork W2388030554 @default.
- W1598493612 hasRelatedWork W2461970972 @default.
- W1598493612 hasRelatedWork W2748952813 @default.
- W1598493612 hasRelatedWork W2827818489 @default.
- W1598493612 hasVolume "2" @default.
- W1598493612 isParatext "false" @default.
- W1598493612 isRetracted "false" @default.
- W1598493612 magId "1598493612" @default.
- W1598493612 workType "article" @default.