The Effect of Item Exposure Control Methods on Measurement Precision and Test Security under Different Measurement Conditions in Computerized Adaptive Testing

Abstract

In this research, it was aimed to investigate the effect of item exposure control methods on measurement precision and test security for dichotomous items within different conditions (different ability distributions, test lengths, sample sizes) in computerized adaptive testing applications. Current research is designed as a Monte Carlo simulation research. Within the scope of the research, short test was designed to have 25 items while long test was designed to have 50 items. In addition to this, small sample was simulated to have 250 observations whereas large sample was simulated to have 1000 observations. Regarding the test-takers’ ability parameters, computerized adaptive testing samples which have normal, right-skewed, left-skewed and uniform distribution for each sample size condition in which ability parameters were within -3 and +3 range were generated. The condition in which item exposure was not controlled for was taken as the reference condition and Sympson Hetter and Fade Away methods were compared with the reference condition. When the variables and conditions within the research were crossed, 48 simulation conditions were established and 4800 data files were generated by doing 100 replications for each condition. Measurement precision and test security indices computed for each condition were compared to one another. The analyses conducted indicate that measurement precision indices corresponding to the different item exposure control conditions did not create substantial differences. It was found out that when Fade Away method was used, item exposure was distributed in a balanced way and higher test security was obtained. As a consequence of this, it was concluded that item pool became more effective and sustainable without revelation (disclosure) of the items. Accordingly, it was discovered that in computerized adaptive testing applications in different conditions, instead of Sympson Hetter method which is frequently used before computerized adaptive testing applications and in which item exposure control parameters are estimated by iterative simulations, the use of Fade Away method which ensures the item exposure control parameters to be estimated simultaneously resulted in higher test security without reducing the measurement precision.