Efficiency of computerized adaptive testing with a cognitively designed item bank - Luo, Hao; Yang, Xiangdong (Frontiers in Psychology, 2024)
Item bank is the key for the application of computerized adaptive testing (CAT). The traditional approach of developing item bank requires content experts to design each item individually, which is time-consuming and costly. The cognitive design system (CDS) approach of item generation can improve this problem. However, the CDS approach has a specific way of calibrating or predicting item difficulty that will affect the measurement efficiency of CAT. A simulation study was conducted to compare the efficiency of CAT under the use of calibration and prediction models. The results show that the predictive model (Linear Logistic Trait Model; LLTM) is worse than the baseline model (Rasch) in terms of Root Mean Square Error (RMSE), but the LLTM only needs a few more items to produce the same RMSE as the baseline model, and that the number of additional items decreases as the explanatory rate increases. It indicates that the loss of measurement efficiency due to prediction item difficulty is acceptable, and the use of prediction item difficulty can reduce or avoid item pretesting to reduce the cost of item calibration.