The Influence of Unbalanced Group Sizes on the Choice of Equating Methods Under the Nonequivalent Groups Anchor Test (NEAT) Design: A Monte Carlo Simulation Study

The Influence of Unbalanced Group Sizes on the Choice of Equating Methods Under the Nonequivalent Groups Anchor Test (NEAT) Design: A Monte Carlo Simulation Study

  • Version
  • Download 14
  • File Size 209.35 KB
  • File Count 1
  • Create Date October 28, 2019
  • Last Updated October 28, 2019

The Influence of Unbalanced Group Sizes on the Choice of Equating Methods Under the Nonequivalent Groups Anchor Test (NEAT) Design: A Monte Carlo Simulation Study

ONETARGET Institute for Educational Assessment (OTIEA) is a research-oriented assessment service to schools, as well as directly to individuals in mainland China. Each year, OTIEA conducts assessments to thousands of students from 1-12 on students’ cognitive ability, motivation, self-regulation, and environmental impact using affluent scales. Frequently, students’ scores on these scales are compared across different administrations or grades. It is noted that equating should be implemented to ensure that students’ scores of different administrations and grades are comparable to each other. Although various equating methods have been proposed regarding different equating situations, significant issues in techniques and applications of equating in practice need our continuous attention.
The purpose of this monte carlo simulation study is to investigate the choice of equating methods under the nonequivalent groups anchor test (NEAT) design and unbalanced/unequal group sizes. In the NEAT design, group mean differences and variances were attributed to two variations, i.e., test form differences and examinee group differences (Kolen & Brennan, 2004). In previous empirical and simulation studies, the examinee group differences were studied on population ability differences under the NEAT design (e.g., Brennan, 1990; Dorans & Holland, 2000; Hanson, 1991; Kolen & Brennan, 2004; Moses & Kim, 2007). In addition to the ability differences, this study considers educational realistic scenarios of unbalanced group sizes when comparing groups. One of the considerations is that when test scores are discrete (e.g., numbercorrect scores), some scores could not find equivalent counterparts on equated test forms using observed score equating. This case may be more exacerbating when two groups differ in population ability and in addition, have unbalanced group sizes because unequal sample sizes may result in larger variances differences between groups.
Therefore, the purpose of this study is to evaluate the effects of variation in unbalanced sample sizes between equated groups on equating methods under the NEAT design. The equating methods include equipercentile equating methods and linear equating methods. Following the same procedures in González and Wibergs’s (2017), the effect of unbalanced group sizes on the choice of equating methods will be investigated in a simulation study across 5 simulation conditions (i.e., 5 unbalanced group sizes). Different equating methods (i.e., linear and equipercentile equating methods) will be compared in terms of standard equating error (SE) of equating, relative bias, and root mean square errors (RMSE). It is hypothesized that variation in unbalanced sample sizes between groups will impact on equating methods under the NEAT design. Implications to researchers and practitioners regarding the choice of method for score equating under the NEAT design and unbalanced group sizes will thereby be discussed

Attached Files

FileAction
5837-The Influence of Unbalanced Group Sizes on the Choice of Equating Methods Under the Nonequivalent Groups Anchor Test (NEAT) Design A Monte Carlo Simulation Study.pdfDownload 
Menu
X