FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W4236067475> ?p ?o ?g. }

• W4236067475 endingPage "4" @default.
• W4236067475 startingPage "3" @default.
• W4236067475 abstract "This issue's cover features the graphic entitled Combined Response Option and Response Time Curves for Selected-Response Items, designed by Peter van Rijn and Usama Ali of Educational Testing Service. It encourages the joint inspection of the proportion of students choosing each response option and the time to respond to each option conditional on estimated student ability. For its utility as a tool to use in standard psychometric analysis, it was among the four winning submissions of the 2016 EM:IP Cover Graphic/Data Visualization competition. Response times of test items can be informative in several ways, but are not often used in routine item analysis. For example, response times of selected-response (SR) items can be used for “obtaining information about construct representation and developing a better understanding of examinee cognition” (Meyer & Wise, 2006, p. 2). Van Rijn and Ali make use of kernel smoothing for nonparametric estimation of response option and time curves of SR items following the approach of Ramsay (1991) (see also Livingston & Dorans, 2004; Guo & Sinharay, 2011). The response option curves “allow the test developer to see, at a glance, the most important statistical characteristics of the item” (Livingston & Dorans, 2004, p. 1). Van Rijn and Ali promote the combination of the response option and time curves to provide a better understanding of the joint processes at play. The cover figure shows the response option and time curves of an SR item using the corrected total score (the total score minus the score on the item under scrutiny) as the criterion score. The top left panel shows the response option curves with a confidence interval for the correct answer only. Smoothing is only performed for the interval of criterion scores in which the response option is observed. The top right panel shows the item response curves for correct (option A) and incorrect answers (options B, C, and D). Similarly, the bottom left panel shows the response time curves for each response option, while the bottom right panel shows the response time curves only for the correct and incorrect answers. In all the panels, the vertical dashed lines indicate the 20th, 40th, 60th, and 80th percentiles of the criterion score distribution. For the response option curves, the proportion choosing response A is increasing with the criterion score, whereas these proportions are decreasing for the other options. This means that the item is functioning well in terms of discrimination. For the response time curves, we observe a slight decrease in the smoothed response time of response A for higher criterion scores. This means that test-takers with higher criterion scores tend to find the correct answer somewhat faster than test-takers with lower criterion scores. The provided graphical approach provides a useful graphical summary of relevant item features. The approach can be extended to more complicated SR items without too much effort (e.g., where test-takers have to select two out of five response options). In addition, van Rijn and Ali created these graphs using R (R Core Team, 2016) scripts, which they adapted from Ramsay (1991, p. 628). These R scripts make it relatively easy to automate the procedure so that the user can create and/or modify the graphs for multiple items at once. Van Rijn and Ali's Combined Response Curves plots seek to augment a well used tool in the pscyhometrician's toolbox. Reviewing empirical item response curves is standard practice in item analysis for many testing programs. With more and more standardized tests administered on the computer, it is easier to obtain response time data as well. The joint consideration of the proportion of students at each student ability level selecting a response option and the time it generally took students to arrive at that selection can provide insight into the functioning of the item across the ability distribution that can in turn inform item revisions. The left panels reveal interesting patterns in response selection by ability group that can augment distractor analyses. For instance, although only a small proportion of high-ability students selected the incorrect response option C, those that did took a much longer time to come to that decision than high-ability students who selected any other response option. Thus, response option C is a time-consuming distractor. Such information could be used in considerations of reducing testing time. Inspection of response option curves could also help identify if students of certain ability levels are generally guessing at or just speeding through their selections instead of taking time to think through the item. In such cases, we may expect to see very low response times. Abnormally low response times for correct answers could also help identify a test security breach. In addition, these curves could be easily extended to polytomous constructed-response items by plotting proportion receiving each score level and likewise the smoothed response time for students who earned each score level. Comparisons of several Combined Response Curves could also extend the utility of these plots. Psychometricians could compare such plots for several items to identify which items are more time-consuming than others or to determine if new field tested items perform similarly as established items in the pool. In addition, supplemental summary plots could be made of proportion selecting an option by response time with separate curves for different ability groups to determine the relationship between item difficulty and response time. Alternatively, these response curves could be produced for different subgroups to compare response times by response selection by demographic subgroups, which may aid in differential item functioning analyses. Accordingly, Combined Response Curves can be easily extended to enhance their utility in item analysis and the test development process. The graphical features themselves are not necessarily original, but the familiarity of the response curves is precisely what helps aid in their interpretation. If response time curves are adopted as part of standard item reviews, more work will need to be done to determine patterns or thresholds for flagging aberrant items and, as van Rijn and Ali recommend, such guidelines should jointly take into account response option selection and response time as opposed to being developed in isolation. Let us know what you think by emailing Katherine Furgol Castellano (KEcastellano@ets.org) or Howard Everson (howard.everson@sri.com)." @default.
• W4236067475 created "2022-05-12" @default.
• W4236067475 creator A5074581073 @default.
• W4236067475 date "2017-03-01" @default.
• W4236067475 modified "2023-10-18" @default.
• W4236067475 title "On the Cover" @default.
• W4236067475 cites W1979603005 @default.
• W4236067475 cites W2076807184 @default.
• W4236067475 cites W2108431703 @default.
• W4236067475 doi "https://doi.org/10.1111/emip.12147" @default.
• W4236067475 hasPublicationYear "2017" @default.
• W4236067475 type Work @default.
• W4236067475 citedByCount "0" @default.
• W4236067475 crossrefType "journal-article" @default.
• W4236067475 hasAuthorship W4236067475A5074581073 @default.
• W4236067475 hasBestOaLocation W42360674751 @default.
• W4236067475 hasConcept C127313418 @default.
• W4236067475 hasConcept C127413603 @default.
• W4236067475 hasConcept C2780428219 @default.
• W4236067475 hasConcept C39432304 @default.
• W4236067475 hasConcept C78519656 @default.
• W4236067475 hasConceptScore W4236067475C127313418 @default.
• W4236067475 hasConceptScore W4236067475C127413603 @default.
• W4236067475 hasConceptScore W4236067475C2780428219 @default.
• W4236067475 hasConceptScore W4236067475C39432304 @default.
• W4236067475 hasConceptScore W4236067475C78519656 @default.
• W4236067475 hasIssue "1" @default.
• W4236067475 hasLocation W42360674751 @default.
• W4236067475 hasOpenAccess W4236067475 @default.
• W4236067475 hasPrimaryLocation W42360674751 @default.
• W4236067475 hasRelatedWork W1999913942 @default.
• W4236067475 hasRelatedWork W2004200960 @default.
• W4236067475 hasRelatedWork W2086120259 @default.
• W4236067475 hasRelatedWork W2137941439 @default.
• W4236067475 hasRelatedWork W2151614148 @default.
• W4236067475 hasRelatedWork W2245170124 @default.
• W4236067475 hasRelatedWork W2324615561 @default.
• W4236067475 hasRelatedWork W2899084033 @default.
• W4236067475 hasRelatedWork W3186982001 @default.
• W4236067475 hasRelatedWork W623607250 @default.
• W4236067475 hasVolume "36" @default.
• W4236067475 isParatext "false" @default.
• W4236067475 isRetracted "false" @default.
• W4236067475 workType "article" @default.