Identification of parameters for electronic distance examinations - Richter, Robin; Tipold, Andrea; Schaper, Elisabeth (Frontiers in Veterinary Science, 2024)
In the context of this study, log data and response behavior from invigilated in-person electronic timed exams at the University of Veterinary Medicine Hannover, Foundation, Germany were analyzed. The investigation included 216,625 log data from five electronic exams taken by a total of 1,241 veterinary medicine students in 2021 and 2022. The primary focus is on understanding how various factors influence the time needed per exam item, including item format, item difficulty, item discrimination and character count. The aim was to use these results to derive recommendations for designing timed online distance examinations, an examination format that has become increasingly important in recent years.The findings indicate that different item formats require varying amounts of time. For instance, image-based question formats and Kprim necessitated more than 60 seconds per item, whereas one-best-answer multiple-choice questions (MCQ) and individual Key Feature items were effectively completed in less than 60 seconds. This variation underscores the need for careful consideration of the ratio of used item formats when defining a time limit for examinations. Furthermore, there was a positive correlation between character count and response time, suggesting that longer items require more time. A negative correlation could be verified for the parameters "difficulty" and "discrimination index" towards response time, indicating that more challenging items and those that are less able to differentiate between high-and low-performing students take longer to answer.Regarding exam design, the literature mentions that time pressure is a critical factor, since it can negatively impact students' exam performance and some students, such as those with disabilities, are particularly disadvantaged. Therefore, this study emphasizes finding the right time limits to provide sufficient time for answering questions and reducing time pressure. In the context of unsupervised online exams, the findings of this study support previous recommendations that implementation of a stringent time limit might be a useful strategy to reduce cheating.