Background
Holy Name Medical Center School of Nursing (HNMCSON) is a small private nursing school located in northern New Jersey. HNMCSON has two programs housed within the school: a Registered Nursing program, with both a two- and three-year track; and a Licensed Practical Nursing program, which is a one-year course of study. Both programs have used the Scantron platform for testing; however, many issues began to surface from both students and faculty. During the spring 2014 term, HNMCSON began an initiative to research and implement computerized testing. This process required detailed planning and much preparation before the planned introduction of fall 2014.
The Problem
A few concerns had arisen in the last few years related to the lack of electronic testing in the program: familiarity and comfort with electronic questions, question types, and availability of actionable assessment data. Many graduates commented the first time they were exposed to electronic testing was during the NCLEX exam. This presented issue on exam success and comfort with technology in the education and licensure process. Differences between the question format in the NCLEX exam and exams given to students raised concerns on preparedness: the testing platform in use at that time only allowed multiple choice questions; in contrast, the NCLEX uses many different question types.
The final major factor in the decision to move to electronic testing was the lack of actionable student performance information with the current process. HNMCSON required information that could drive student autonomy toward content areas in need of improvement, could assist in decreasing attrition rate, and increase student satisfaction.
The Solution
Once the decision was made to move to an assessment software, a choice had to be made as to which platform to implement. An investigation of three different testing solutions was conducted: the current learning management system (LMS), the testing platform currently in use (a traditional bubble-sheet program), and a server-based software program. HNMCSON faculty were very involved in the decision of which platform to implement. Webinars were scheduled along with training on each platform, so that a determination could be made as to which one met the school’s needs.
Of the three choices, testing within the LMS seemed to be the most logical decision. This would provide the greatest amount of data in the format to which faculty were accustomed and would not come with a significant cost, as it was a part of a system HNMCSON had already purchased and implemented. The bubble-sheet solution did have a high level of discernment; however, the ability to simulate the questions students would find on NCLEX was non-existent. The LMS solution had many of the desired attributes, but it did not have the level of item analysis required.
The server-based testing platform was rapidly ruled out as it did not provide the ability to simulate NCLEX-style questions and this was critical to student success. Despite concerns, the LMS provided the best price along with most of the desired features in a testing platform; HNMCSON decided to move forward with the LMS for testing. Shortly after making this decision, Dr. Barbara Blackwell, Director of the School of Nursing at the HNMCSON attended a conference on technology in Philadelphia, Pennsylvania, where one of the topics of the conference was computer-based testing.
During the course of the conference, many of the schools in attendance recommended was ExamSoft as a computer-based testing solution. The enthusiasm for this product stemmed from the high level of statistical analysis available and the depth of analysis of student achievement. ExamSoft offers data describing exam analysis and validity, the ability to create NCLEX-type questions, and detailed data for analysis of test item reliability. These recommendations then triggered an investigation of the ExamSoft platform.
After speaking with many of the participants at a seminar, the outlay in terms of learning and implementation was well worth the effort. Once Dr. Blackwell returned from the seminar, HNMCSON formed an exploratory committee. After an assessment of the product by faculty, which included several webinars and conversations, ExamSoft was deemed able to provide all of the required features HNMCSON was looking for in an assessment software along with a level of test security not available in the other potential solutions.
After a comparison between the LMS and ExamSoft was made, the committee recommended the full faculty to go with ExamSoft. ExamSoft provided HNMCSON with all of the desired features they required without compromising what would be important to the program. HNMCSON decided to scrap all plans to use the LMS as the preferred testing platform and to implement ExamSoft.
Results
To begin the process HNMCSON created a steering committee that would serve as the stimulus to push through the change and determine policy. This committee consisted of four faculty members, two from the senior faculty and two from freshman faculty. The committee facilitated the decision on what items to track for accreditation purposes and helped to determine policy.
Part of the planning process included determining an approach for item writing, question approval, and the creation of the categories for test blueprints. To ensure that everyone would start with the same knowledge base, the program provided an expert in item writing and exam analysis for a one-day workshop. This training session was extremely interactive and consisted of actual question writing, question critique, and item analysis. This class enhanced the knowledge base for all faculty.
The overall implementation process was long and painstaking, and the effort and learning curve will continue. The outcomes and data acquired for program and student improvement have been well worth the effort.
“ExamSoft provided us with all of the things that we were looking for without compromising what we felt would be important to our program.”
Other Key Takeaways
- The investigation of computer-based testing platforms presented opportunity to affect change within the culture of assessment at the Holy Name Medical Center School of Nursing. Implementation of the ExamSoft platform solved many issues faced by HNMCSON, including the collection of critical assessment data necessary for student success, assurance of learning, programmatic accreditation, and— most important—the preparedness of students for the professional licensure process via the NCLEX.
- The effort of a thoughtful implementation of ExamSoft has provided a significant return to drive program and student success in the preparation of future professional nurses. Not only did faculty grow as teachers during implementation, students now have opportunity for real-time data on their success within the program to better drive their understanding of strengths and deficiencies as the prepare for workforce entry.