Do You Know How Your Exam Questions Are Performing?
Exam data provides students with important feedback, but exam creators can also use post-exam data to improve test questions and exams as a whole. The word “psychometrics” can be daunting to some educators, but ExamSoft’s item analysis report makes psychometrics easy. We provide an explanation of how your questions are performing and how to make them even stronger to assess your students more accurately.
Item Difficulty Index (p-value): P-value shows what percentage of exam-takers got this question correct. What counts as a good p-value depends on the intention of the question. This stat is your first indication of how your question performed but shouldn’t be the only piece of data you use to determine validity.
Discrimination Index: This index shows the difference in item performance between top and bottom performing students. If lower performing students are getting this question correct more often than your top performing students, there could be a problem with the item. A good index of discrimination can vary, based on other stats from this particular item.
Upper/Lower 27 Percent: Of the students who scored in the top or bottom 27 percent of exam-takers, what percentage of those specific subgroups got the question correct. This can be useful in determining what distractors drew students away from the correct answer.
Point Biserial: Indicates whether getting the question correct correlates positively or negatively with performing well on the exam as a whole. Unlike the discrimination index, point biserial incorporates performance of all students instead of the two specific subgroups.
Kuder-Richardson Formula 20 (KR-20): Indicates the reliability of the assessment, essentially showing whether an assessment was poorly constructed or a valid measure of knowledge. The degree to which an assessment is free from random error, yielding the same result across multiple administrations the same cohort. An ideal KR-20 score is close to one, on a scale of zero to one.
There are lots of guidelines relating to the use of psychometrics. While these rules might generally be correct, you should always remember that your specific circumstances are unique, and to take each rule with a grain of salt. Most importantly, educators must remember that there are many non-statistical factors that can influence the performance of a question, and these factors must be taken into account when determining a question’s validity.