• Version
• Download 209
• File Size 248.97 KB
• File Count 1
• Create Date August 2, 2018
• Last Updated August 2, 2018

Abolishing marksism and rescuing validity

What could be more valid than judging that one piece of work is more creative than another? Or more effective? Or just better? And if many judges agree that the same one is better, isn’,t that the best evidence for validity we could ask for?This paper describes progress in applying comparative judgement (first reported to IAEA in 2004) to the assessment of holistic traits like overall achievement, including effectivenes, quality and creativity. Marking was invented (in Cambridge) during the 18th century enlightment, not in pursuit of validity or even reliability but to overcome serious problems of bias and prejudice in the examinations of the day. Its unintended consequence has been a most serious loss of validity in most of our formal assessments.Some progress has been made in abolishing marksism in UK assessment. A web-based system has been developed for presenting pairs of ‘,scripts’, and collecting judgements, and the estimation procedure has been shown to be remarkably robust with the extremely sparse data ‘,matrices’, that result. A simple initiation algorithm has been used developed. Some technical aspects of the procedure are described, and a procedure for qualitative description of the scalel for public use id described.

Attached Files