• Version
• Download 1308
• File Size 372.01 KB
• File Count 1
• Create Date October 28, 2019
• Last Updated October 28, 2019

A Standard Setting Method Using the D-scoring Method: Procedures and Application to Assessment for Teacher Certification

This paper presents an approach to deriving cutting scores for standard setting for large-scale standardized assessments. The approach is developed in the framework of the D-scoring method (DSM; Dimitrov, 2018, 2019) which is adopted at the National Center for Assessment (NCA) in Saudi Arabia and gaining attention in the field of educational assessment. Under the DSM, the test score of an examinee is based on his/her response pattern on dichotomously (1/0) scored items weighted by their expected difficulty for the target population of examinees. The DSM is a classical framework, providing transparency and computational simplicity, but it also has some measurement features analogous to those in item response theory (e.g., models of item response functions). The D-score of an examinee indicates what percent of the ability required for total success on the test is demonstrated by that examinee. The idea behind the proposed approach is to directly align the outcome of expert judgments with the computation of cutting scores on the D-scale. A panel of experts on the test content and targeted standards are required to identify response patterns of binary (1/0) item scores that reflect mastery of targeted standards measured by the test. To facilitate the experts in their judgments on this task, they are provided with the content of the test items and their expected difficulty precalibrated for a norm population of examinees. Based on such response patterns, cutting scores for standards setting are computed via a simple DSM procedure. In another scenario, when expert judgments are not available, cutting scores are identified based on a norm distribution of scores on the D-scale. The proposed approaches are illustrated with real data from assessments for teacher certification at the NCA.

Attached Files