Matches in SemOpenAlex for { <https://semopenalex.org/work/W3202564776> ?p ?o ?g. }
- W3202564776 endingPage "482" @default.
- W3202564776 startingPage "467" @default.
- W3202564776 abstract "Abstract It is common practice to rely on a convenience sample of subject matter experts (SMEs) when developing scoring keys for situational judgment tests (SJTs). However, the defining characteristics of what constitutes a SME are often ambiguous and inconsistent. Sampling SMEs can also impose considerable costs. Other research fields have adopted crowdsourcing methods to replace or reproduce judgments thought to require subject matter expertise. Therefore, we conducted the current study to compare crowdsourced scoring keys to SME‐based scoring keys for three SJTs designed for three different job domains: Medicine, Communication, and Military. Our results indicate that scoring keys derived from crowdsourced samples are likely to converge with keys based on SME judgment, regardless of test content ( r = .88 to .94 between keys). We observed the weakest agreement among individual MTurk and SME ratings for the Medical SJT (classification consistency = 61%) relative to the Military and Communication SJTs (80% and 85%). Although general mental ability and conscientiousness were each related to greater expert similarity among MTurk raters, the average crowd rating outperformed nearly all individual MTurk raters. Using randomly‐drawn bootstrapped samples of MTurk ratings in each of the three samples, we found that as few as 30–40 raters may provide adequate estimates of SME judgments of most SJT items. These findings suggest the potential usefulness of crowdsourcing as an alternative or supplement to SME‐generated scoring keys." @default.
- W3202564776 created "2021-10-11" @default.
- W3202564776 creator A5004017544 @default.
- W3202564776 creator A5025620724 @default.
- W3202564776 creator A5039456504 @default.
- W3202564776 creator A5050036429 @default.
- W3202564776 creator A5078544149 @default.
- W3202564776 date "2021-10-11" @default.
- W3202564776 modified "2023-10-02" @default.
- W3202564776 title "Can you crowdsource expertise? Comparing expert and crowd‐based scoring keys for three situational judgment tests" @default.
- W3202564776 cites W1758566279 @default.
- W3202564776 cites W1822626517 @default.
- W3202564776 cites W1856762660 @default.
- W3202564776 cites W1941288239 @default.
- W3202564776 cites W1965194012 @default.
- W3202564776 cites W1968423387 @default.
- W3202564776 cites W1968956327 @default.
- W3202564776 cites W1969719348 @default.
- W3202564776 cites W1975482862 @default.
- W3202564776 cites W1979012461 @default.
- W3202564776 cites W1996646136 @default.
- W3202564776 cites W1999081431 @default.
- W3202564776 cites W2011399107 @default.
- W3202564776 cites W2015736731 @default.
- W3202564776 cites W2025261074 @default.
- W3202564776 cites W2027459380 @default.
- W3202564776 cites W2031805793 @default.
- W3202564776 cites W2034284819 @default.
- W3202564776 cites W2036814273 @default.
- W3202564776 cites W2037180041 @default.
- W3202564776 cites W2039347217 @default.
- W3202564776 cites W2042962409 @default.
- W3202564776 cites W2051130243 @default.
- W3202564776 cites W2054295201 @default.
- W3202564776 cites W2055718999 @default.
- W3202564776 cites W2063207484 @default.
- W3202564776 cites W2066559452 @default.
- W3202564776 cites W2067166634 @default.
- W3202564776 cites W2073729033 @default.
- W3202564776 cites W2079072929 @default.
- W3202564776 cites W2082064682 @default.
- W3202564776 cites W2084675134 @default.
- W3202564776 cites W2091611028 @default.
- W3202564776 cites W2092968198 @default.
- W3202564776 cites W2096812562 @default.
- W3202564776 cites W2101798695 @default.
- W3202564776 cites W2106510587 @default.
- W3202564776 cites W2113569975 @default.
- W3202564776 cites W2119460547 @default.
- W3202564776 cites W2119478623 @default.
- W3202564776 cites W2130969541 @default.
- W3202564776 cites W2137522119 @default.
- W3202564776 cites W2145409614 @default.
- W3202564776 cites W2146362431 @default.
- W3202564776 cites W2146790495 @default.
- W3202564776 cites W2148353238 @default.
- W3202564776 cites W2156556824 @default.
- W3202564776 cites W2165949980 @default.
- W3202564776 cites W2170939095 @default.
- W3202564776 cites W2171321543 @default.
- W3202564776 cites W2310109152 @default.
- W3202564776 cites W2357189319 @default.
- W3202564776 cites W2426350037 @default.
- W3202564776 cites W2462835675 @default.
- W3202564776 cites W2610943221 @default.
- W3202564776 cites W2770330190 @default.
- W3202564776 cites W2779905829 @default.
- W3202564776 cites W2790762940 @default.
- W3202564776 cites W2793888596 @default.
- W3202564776 cites W2799517186 @default.
- W3202564776 cites W2801462134 @default.
- W3202564776 cites W2887878767 @default.
- W3202564776 cites W2895349339 @default.
- W3202564776 cites W2896652031 @default.
- W3202564776 cites W2900441342 @default.
- W3202564776 cites W2904860679 @default.
- W3202564776 cites W2925879505 @default.
- W3202564776 cites W2983628064 @default.
- W3202564776 cites W2988048534 @default.
- W3202564776 cites W2989656881 @default.
- W3202564776 cites W3009748967 @default.
- W3202564776 cites W3028578692 @default.
- W3202564776 cites W3039187287 @default.
- W3202564776 cites W3080884520 @default.
- W3202564776 cites W3121311368 @default.
- W3202564776 cites W3121850723 @default.
- W3202564776 cites W3123547150 @default.
- W3202564776 cites W3125284435 @default.
- W3202564776 cites W3131739164 @default.
- W3202564776 cites W3143385792 @default.
- W3202564776 cites W3162672265 @default.
- W3202564776 cites W4241004881 @default.
- W3202564776 cites W4249247059 @default.
- W3202564776 cites W4313429106 @default.
- W3202564776 doi "https://doi.org/10.1111/ijsa.12353" @default.
- W3202564776 hasPublicationYear "2021" @default.
- W3202564776 type Work @default.
- W3202564776 sameAs 3202564776 @default.