EEC: Using ExamSoft Data to Evaluate Those Pesky “Select All That Apply” Questions

ExamSoft Education Connection Presentation:

Using ExamSoft Data to Evaluate Those Pesky “Select All That Apply” Questions
Dr. Cheryl Frutchey

July 29, 2020 11 a.m.

Literature supports the inclusion of multiple response type questions to assess critical thinking and higher level of cognitive processing in nursing programs. Multiple response type questions are often developed with five or more responses and several correct answers. Following exam completion, exam analysis is conducted to assess the quality of the questions. The point biserial index (PBI) and the p-value are two statistics often presented during the item analysis as the basis for modification or elimination of individual exam items. When evaluating multiple response questions, p-values are often significantly lower than those of multiple choice items. If faculty members were to follow published guidelines, then multiple response questions with a p-value of less than 0.30 would automatically be thrown out. Such extreme modification or removal of multiple response items will reduce the effectiveness of the test and has the potential to weaken a nursing program’s success in achieving curricular goals. Additionally, the continuing removal of multiple response items may influence NCLEX success rates and be detrimental to student and program success. When evaluating such complex exam items, the data suggests that a review of item discrimination ratios (IDR) may support the continued use of a discriminating questions despite low p-values.

Published: January 12, 2022

Updated: January 24, 2022

Related Resources