FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W196161193> ?p ?o ?g. }

• W196161193 abstract "Performance in education and credentialing are becoming popular. At the same time, there do not exist any well established and validated methods for setting standards on performance assessments. This paper describes several of the new standard-setting methods that are emerging for use with performance and considers their strengths and weaknesses. Methods described that are being applied to performance are: (1) contrasting groups; (2) extended Angoff; (3) estimated mean, expected score distribution; (4) paper selection; (5) holistic or booklet; (6) dominant profile; and (7) policy capturing. A special problem is that of compensatory versus conjunctive standard setting. A compensatory standard is one in which any candidate who achieves a defined total score will pass. In conjunctive standard setting, raters set a conjunctive standard by stressing the most important parts of the assessment or making performance on a given item decisive rather than relying on overall score. A simulation study that compared the reliability and validity of these two approaches found surprising results for the conjunctive standard that suggest that increasing numbers of candidates will fail as the assessment length increases, and validity will actually decline. More research is needed to find better ways to set standards on performance assessments, although substantial evidence shows that defensible standards can be set for achievement and credentialing performance examinations. (Contains 4 figures and 12 references.) (SLD) *********************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. *********************************************************************** U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) This document has been reproduced as received from the person or organization originating it. Minor changes have been made to improve reproduction quality. Points of view or opinions stated in this document do not necessarily represent official OERI position or policy. Setting Standards on Performance Assessments: Promising New Methods and Technical Issues Ronald K. Hambleton University of Massachusetts at Amherst Abstract PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS BEEN GRANTED BY ,p K 14/4gogtz-e.,tiPERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS BEEN GRANTED BY ,p K 14/4gogtz-e.,ti TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) Performance in education and credentialing are becoming popular. At the same time, there do not exist any well established and validated methods for setting standards on performance assessments. The purposes of this paper are (1) to describe several of the new standard-setting methods which are emerging for use with performance and consider their strengths and weaknesses, and (2) to consider a special problem, that of compensatory versus conjunctive standard-setting methods. The main conclusions are that there is plenty of room for new ideas, creativity, and research in standard-setting methodology, and more effort is needed to document and validate standards for intended uses. 2 BEST COPY AVAILABLE . Setting Standards on Performance Assessments: Promising New Methods and Technical Issues12 Ronald K. Hambleton University of Massachusetts at Amherst 1. Introductory Remarks Whenever important individual decisions are made with examinations, there will be challenges to how the performance standards were set and who set them. Performance standards are the Achilles heal of credentialing exams. The fact that different standardsetting methods, in general, lead to different results (i.e. standards) is just one of many troublesome features of performance standards for credentialing agencies to explain (Zieky, 1995). Fortunately, there have been many successful efforts to set standards on multiplechoice credentialing exams. The defense of performance standards for a particular credentialing examination is based on (1) the credibility of the process used to set them (e.g. selection of appropriate panelists, excellent panelist training, and a well-planned and systematic process which provides ample opportunity for discussion and deliberations among the panelists), (2) the reliability of the performance standards (that is, there needs to be agreement among panelists about the performance standards), and (3) the reasonableness of the performance standards (i.e., the passing rate is not too far out of line with expectations about the quality of persons entering the profession). Current standard-setting methods can produce defensible results. bf course it is also true that not all credentialing agencies commit the resources and time to set standards in a defensible manner. Now, there is a new challenge to performance standard-setting methods: performance assessments. In both educational testing and credentialing exams, there has been an emergence of performance assessments. Kentucky has moved to a total performance assessment system for student and school accountability. Nearly all other states are using some form of performance assessment in student accountability. Most of my own standard setting research in the last two years has been with Dick Jaeger and Barbara Plake in developing new methods for setting standards on the performance-based of teachers (see, for example, Jaeger, Hambleton, & Plake, 1995). I have also been working on performance such as standardized patient assessments in the medical examination area with the Education Commission for Foreign Medical Graduates (Vu & Barrows, 1991). 1Laboratory of Psychometric and Evaluative Research Report No. 278. Amherst, MA: University of Massachusetts, School of Education. 2Paper presented at the meeting of APA, New York, August, 1995." @default.
• W196161193 created "2016-06-24" @default.
• W196161193 creator A5049408101 @default.
• W196161193 date "1995-08-01" @default.
• W196161193 modified "2023-09-24" @default.
• W196161193 title "Setting Standards on Performance Assessments: Promising New Methods and Technical Issues." @default.
• W196161193 cites W2079773609 @default.
• W196161193 cites W2107229891 @default.
• W196161193 cites W2994316891 @default.
• W196161193 hasPublicationYear "1995" @default.
• W196161193 type Work @default.
• W196161193 sameAs 196161193 @default.
• W196161193 citedByCount "2" @default.
• W196161193 crossrefType "journal-article" @default.
• W196161193 hasAuthorship W196161193A5049408101 @default.
• W196161193 hasConcept C121332964 @default.
• W196161193 hasConcept C145420912 @default.
• W196161193 hasConcept C15744967 @default.
• W196161193 hasConcept C163258240 @default.
• W196161193 hasConcept C177264268 @default.
• W196161193 hasConcept C199360897 @default.
• W196161193 hasConcept C203151758 @default.
• W196161193 hasConcept C2780530410 @default.
• W196161193 hasConcept C41008148 @default.
• W196161193 hasConcept C43214815 @default.
• W196161193 hasConcept C509550671 @default.
• W196161193 hasConcept C62520636 @default.
• W196161193 hasConcept C63882131 @default.
• W196161193 hasConcept C71924100 @default.
• W196161193 hasConcept C77805123 @default.
• W196161193 hasConceptScore W196161193C121332964 @default.
• W196161193 hasConceptScore W196161193C145420912 @default.
• W196161193 hasConceptScore W196161193C15744967 @default.
• W196161193 hasConceptScore W196161193C163258240 @default.
• W196161193 hasConceptScore W196161193C177264268 @default.
• W196161193 hasConceptScore W196161193C199360897 @default.
• W196161193 hasConceptScore W196161193C203151758 @default.
• W196161193 hasConceptScore W196161193C2780530410 @default.
• W196161193 hasConceptScore W196161193C41008148 @default.
• W196161193 hasConceptScore W196161193C43214815 @default.
• W196161193 hasConceptScore W196161193C509550671 @default.
• W196161193 hasConceptScore W196161193C62520636 @default.
• W196161193 hasConceptScore W196161193C63882131 @default.
• W196161193 hasConceptScore W196161193C71924100 @default.
• W196161193 hasConceptScore W196161193C77805123 @default.
• W196161193 hasLocation W1961611931 @default.
• W196161193 hasOpenAccess W196161193 @default.
• W196161193 hasPrimaryLocation W1961611931 @default.
• W196161193 hasRelatedWork W141701509 @default.
• W196161193 hasRelatedWork W1994107221 @default.
• W196161193 hasRelatedWork W2001255505 @default.
• W196161193 hasRelatedWork W2011635831 @default.
• W196161193 hasRelatedWork W2041199083 @default.
• W196161193 hasRelatedWork W204438660 @default.
• W196161193 hasRelatedWork W2079383196 @default.
• W196161193 hasRelatedWork W2085873101 @default.
• W196161193 hasRelatedWork W2137239648 @default.
• W196161193 hasRelatedWork W2158584042 @default.
• W196161193 hasRelatedWork W221208016 @default.
• W196161193 hasRelatedWork W226079512 @default.
• W196161193 hasRelatedWork W263492455 @default.
• W196161193 hasRelatedWork W270068858 @default.
• W196161193 hasRelatedWork W2727489474 @default.
• W196161193 hasRelatedWork W2733216520 @default.
• W196161193 hasRelatedWork W282590206 @default.
• W196161193 hasRelatedWork W297651866 @default.
• W196161193 hasRelatedWork W337921325 @default.
• W196161193 hasRelatedWork W48989779 @default.
• W196161193 isParatext "false" @default.
• W196161193 isRetracted "false" @default.
• W196161193 magId "196161193" @default.
• W196161193 workType "article" @default.