• Version
• Download 131
• File Size 95.47 KB
• File Count 1
• Create Date August 2, 2018
• Last Updated August 2, 2018

General Speaking Ability: An Assessable Construct?

In a study on large-scale oral assessment in Hong Kong, high correlations emerged between ratings on the various rating criteria and verifiable quantitative measures (VQM) for these criteria. The initial conclusion was that the raters were paying attention to the textual features measured by the VQM. However an alternative hypothesis (AltH) was suggested i.e. ‘,raters were simply assessing ‘,general speaking ability’, which would naturally correlate with VQM. To investigate AltH, Rasch Fair Averages for rated criteria were correlated against each other. Ratings on all criteria correlated with each other, 0.963 to 0.979, initially indicating that AltH was correct. However, upon investigating ‘,raw score’, correlations between criteria for our best rater (clearly acceptable Rasch fit values and good correlations with both expert panel and VQM), it was found that correlations between the various criteria were only 0.787 to 0.871, indicating that there was a ‘,smoothing ,function’, in the Rasch fair average which exaggerated correlations between different criteria. Without this function, correlations dropped, indicating that various rating criteria may well be separate entities, reflecting attention to separate axiomatically-related textual phenomena rather than showing AltH was correct. Further challenge to AltH emerged with correlations of VQM against each other. When no axiomatic relationshipbetween VQM was apparent, correlations ranged from 0.597 to 0.789. When an axiomatic relationship was ,obvious, correlations ranged from 0.959 to 0.996.

Attached Files