The Influences of the Ability Estimation Methods on the Measurement Accuracy in Multidimensional Computerized Adaptive Testing Author:Po-Hsi Chen
Research Article
The goal of the research was to investigate the influences of ability estimation methods on multidimensional computerized adaptive testing. In stage 1, different quadrature points of the Baysian expected a posteriori (EAP) estimation were manipulated in order to find out the appropriate quadrature point of EAP in multidimensional computerized adaptive testing (MCAT). In stage 2, the maximum likelihood (ML) estimation, the Bayesian maximum a posteriori (MAP) estimation, and the EAP estimation methods were used in two kinds of ability dimensions (two and four dimensions) and two kinds of correlations between dimensions (high correlations and low correlations). The target item numbers of MCAT were 20, 40, 60, and 80. The dependent variables were the average reliability, bias, and the root mean square of error (RMSE) in all ability dimensions. Results in stage 1 indicated that the higher the quadrature point and the ability dimensions, the much higher the estimation time of MCAT. Ten points was appropriate in less than 4 dimensions of MCAT when the estimation time and the reliability of ability estimation were taken into consideration. Results of stage 2 indicated that MAP and EAP methods resulted in higher reliability and lower RMSE than ML method, especially in the conditions of high correlation between abilities, more ability dimensions, and fewer MCAT items. There were advantages and disadvantages in the three estimation methods. The regression bias of MAP, the estimation times of EAP, and the reliability and RMSE of ML were the problems that should be resolved when executing MCAT.