With the arrival of a new dean who had used ExamSoft at a previous school and knew how valuable it could be, Baker University School of Nursing adopted ExamSoft. The school had high licensure exam pass rates so some questioned – why change? Sure enough, shortly after adopting ExamSoft, faculty discovered numerous opportunities to improve the quality of exams and instruction. They also realized tremendous improvements in exam security. In the year following implementation, the school’s nursing licensure exam pass rates increased and have consistently remained well above state and national averages since.
Baker University (BU) is private liberal arts college in Kansas. The university is home to four colleges – Nursing, Arts & Science, Education, and Professional & Graduate Studies.
The Baker University School of Nursing is located at Stormont Vail Health, a 586-bed acute care hospital in Topeka, Kansas. Stormont Vail is a Magnet hospital, recognized by the American Nursing Credentialing Center for having the highest standard of care for patients. The 20-person faculty of BU nursing school has been using ExamSoft since 2015.
Decision to Adopt
While many schools pursue new technology solutions to solve a problem, Baker University’s ExamSoft implementation was driven by a desire to strengthen what was already a sound testing program. Some faculty members had reservations. Why make a change when outcomes are good? Afterall, Baker students’ National Council Licensure Exam (NCLEX) pass rates had, for several years, been higher than the Kansas and national averages. Nevertheless, looking to the horizon, the Baker University School of Nursing went live in the fall of 2015.
The faculty’s concerns were quickly assuaged when they began to realize the myriad ways in which the technology was helping them create better, higher-quality tests that would have a positive impact on NCLEX pass rates. Specifically, faculty appreciate:
- A high degree of exam security.
- Robust test banks.
- The ability to efficiently assemble and review exams.
- The ability to categorize questions against a number of parameters including the NCLEX test blueprint.
- Close statistical scrutiny of all exams and test items for the purpose of quality improvement.
It didn’t take long to realize that the extra effort entailed in onboarding ExamSoft and learning to effectively use it was well worth the payout. Time invested on the front end, to learn new technology, paid dividends—many times over—on the back end in terms of efficiency (assembling and grading), security, and valuable data.
Post Implementation: Quick Wins
When faculty began writing assessments in ExamSoft they were able to move from simply writing an exam to categorizing and creating learning outcome focused test blueprints across the curriculum. Through the test construction process, faculty identified the need for more application and analysis questions to better meet the test goals and course outcomes for higher-level students. The Baker curriculum is a leveled one requiring a higher percentage of analysis questions at each subsequent level. ExamSoft provided the ability to track progress and performance.
With the right mix of questions in place, the Baker team shifted focus to the performance of the overall exam as well as the quality of each item—looking at the discrimination index, question difficulty, point biserial, and item response patterns as well as other student performance data. The ability to assess the item performance was helpful for faculty in evaluating students performance and provided focused remediation when closing the learning loop in test review.
“Through ExamSoft we realized we needed to increase application and analysis level questions. Without ExamSoft, I’m not sure we would have realized such specific shortcomings of exams.”
A high priority for Baker, as in most nursing programs, was to insure that students were well prepared to take the NCLEX. While the curriculum exposes graduates to a wide variety of concepts and equips them to understand the subjects covered by the exam, using ExamSoft prepared them for the NCLEX testing environment. To that end, Baker faculty opted to administer all major unit and final exams using ExamSoft with settings that recreate the computer-based licensing exam experience.
Some other settings or features Baker uses to set students up for success include:
- Not allowing backward navigation, which wasn’t initially well received by students, but ultimately, students conceded that it makes the testing experience more like the NCLEX and is good practice.
- Limited use of the highlighting feaure. It’s allowed for the first two semesters, but it is removed in the senior year to better align with NCLEX.
- Exam questions using alternative formats such as Hot Spots and integrating exhibits (attachments) to help prepare student for this type of questioning.
Several students realized, over time, that their peers who graduated from programs that used paper-based tests were really challenged in adapting to the computerized format during the licensure exam. Particularly challenging and anxiety producing was the different question formats and the inability to backward navigate.
“As a faculty, we sit down after every exam and tear the questions apart, diving deep into the data to strengthen our questions and ensuring alignment with the NCLEX exam.”
In our increasingly digital world, there are more and more subtle ways to cheat on tests. Unfortunately, Baker was not immune to this age-old problem. ExamSoft’s ability to provide a secure environment, by randomizing question order and answer options per question for each student and locking students’ computers down so that they couldn’t screen shot, browse the internet, copy/ paste, or access any other applications during testing, made it very attractive to the faculty. An unintended benefit of turning off backward navigation was that students had a harder time recalling and relaying the questions and answers to other students who’d yet to take the exam. The ability to track student activity with the download and upload snapshot, as well as invidual responses, were also useful inputs to help gauge exam security effectiveness…if and who might be sharing test questions.
“Utilizing ExamSoft helps us to be proactive when it comes to question and exam security. Both the secure exam environment and snapshots allow us to monitor exam activity and stay on top of potential issues related to cheating.”
Hurdles to Overcome
Being in a hospital environment, there were a host of network privacy mandates in place to protect patient data, per federal privacy mandates. Adding another layer of security complexity was FERPA, the federal law that protects student records. Within the School of Nursing there were no computer labs of adequate size to accommodate the testing required. For this reason using individual student laptops was the only option for testing.
This created a small hurdle in terms of making sure all students had appropriate devices (PCs or MacBooks) with up-to-date software. There were stumbling blocks with anti-virus software which needs to be turned off while using ExamSoft; not all students knew how to do that. But these minor challenges were quickly resolved with the help of ExamSoft support and test administration ran smoothly.
“ExamSoft provided phenomenal tech support. They really helped us overcome some early hurdles. They were there every step of the way.”
While Baker University School of Nursing had strong NCLEX pass rates before deploying ExamSoft, one goal was to create a stable pattern of high performance. Since implementation in 2015, the average pass rate has been 96% and it continues to improve. For four straight semesters after implementing ExamSoft, 100 percent of students passed the exam. Baker University’s School of Nursing is proud to have high NCLEX passrates that are above both the state and national averages.
Administrators regularly search for quizlets, questions, or other information leaked via the dark web. Before ExamSoft, questions and exams could be found. Now, with the security measures in place from ExamSoft and not being able to navigate backwards, there are no “cheat sheets” or unauthorized exam content to be found on the web.
During a university-wide accreditation visit four years after ExamSoft implementation, accreditors were impressed by how quickly the nursing faculty could access high-quality testing data, not only around student outcomes but also with regard to testing rigor. The faculty are optimistic and looking forward to the next nursing accreditation visit, knowing how easy it will be to pull and analyze data, demonstrating strong results. Data that once took days to pull is easily accessible through ExamSoft.
Lastly, one unintended or unexpected benefit has been a higher degree of accountability from students. Using ExamSoft requires students to be responsible for their devices and what they put on them so they can be ready to test. Upgrades have to be made in a timely manner. Baker requires students to download the test before coming to the test-taking session, or they automatically receive a 3-5 percent reduction in score.
Testing responsibility is placed squarely on the students. As Baker sees it, being a nurse means always being prepared to care for your patient. That responsibility begins in their program with students taking the initiative to understand and comply with the device requirements and be prepared to not only take an exam but have the correct supplies (computer/powercord) to take it.
While Baker was already turning out well-prepared students, with the adoption of ExamSoft, the students’ scores are stronger, their testing experiences richer, and they’re graduating thoroughly equipped for careers in nursing and patient care.
Download the full case study here: