Assessing medical knowledge: a 3-year comparative study of very short answer vs. multiple choice questions.

Abstract

Purpose

Assessment design significantly influences evaluation of student learning. Multiple choice questions (MCQ) and very short answer questions (VSAQ) are commonly used assessment formats, especially in high-stakes settings like medical education. MCQs are favoured for efficiency, coverage, and reliability but may lack depth in assessing critical thinking. VSAQs require students to generate responses, potentially enhancing depth, but posing challenges in consistency and subjective interpretation.

Methods
Data from parallel MCQ/VSAQ exams over three years was collected. Summary statistics for each exam (marks, time, and discrimination index; DI) and the effect of year and question characteristics were analysed.

Results
VSAQs were associated with lower marks (p<0.001), longer time (p<0.001), and higher DI (p<0.001). Question characteristics (e.g. basic science or clinical stems) significantly affected the mark, time, and DI, changing across years, but not interacting with question format.

Conclusion
While MCQs resulted in higher marks, VSAQs provided higher discrimination of student performance. Response options in MCQs likely enhance recall, however real-world settings also offer contextual cues. Question characteristics affect student performance independently of format, likely due to differences in cohort career progression. Future research should investigate predictive validity and standard setting of VSAQs in a basic science context.