After investing a lot of time and effort to build and maintain a proprietary testing and data administration system, it was time for a change. The University of Nebraska Medical Center’s College of Medicine adopted ExamSoft and has never looked back. ExamSoft provided the collaboration tool faculty needed and a secure test-taking platform students could use on their own devices. The College of Medicine continues to explore new methods of category tagging to track and manage course curricula, learning objectives, and USMLE compliance.
Background
The University of Nebraska Medical Center (UNMC) is Nebraska’s only public academic health sciences center. As such, UNMC works hard to reach all Nebraskans through a wide variety of programs. It proactively recruits and prepares students to serve Nebraska’s underserved communities. UNMC has six colleges that enroll nearly 4,000 students across more than two dozen programs.
One of the oldest colleges at UNMC, the College of Medicine (COM) admitted its inaugural class of 14 students in 1881. After relocating to several sites on and around campus over the years, the COM now delivers instruction to students in two campus buildings, the Sorrell Center, built in 2009, and the Davis Global Center, built in 2019. COM now admits approximately 130 students every year and as of 2017, teaches a curriculum with three phases of student instruction.
UNMC’s College of Medicine (COM) adopted ExamSoft in the Summer 2015. Prior to ExamSoft, the COM used a homegrown system for exam development, administration, and security. The educational office was using multiple software solutions before implementing ExamSoft. They used a Microsoft Access database to store exam questions, associated images, and anecdotal information. They were also using an LMS and a locked down browser to deliver examinations. A statistical software solution helped the team create psychometric analysis every year. However, the multiple pieces, parts, and platforms in place lead to some challenges.
Challenge
While the homegrown system was meeting the college’s needs, it had its shortcomings. Aside from being cumbersome in its multi-platform and multi-step nature, there was constant manual movement of data between tools, increasing the potential for error. It did not allow for collaboration, so it was difficult to route questions for review between the assessment coordinator and faculty instructors. Work had to be done exclusively on-campus or through inefficient remote access.
In the Spring of 2015, the COM determined it would need to close one of two computer labs used for test administration. Soon after, the programmer who had built and managed the homegrown system retired, leaving them without support for the database and statistical calculations.
All signs pointed to adopting a new system. One that:
- Allowed students to take exams on their own devices to offset the loss of a computer lab.
- Offered the security features necessary for a BYOD, large group, amphitheater-style test-taking environment.
- Was cloud-based, allowing collaboration from remote, disparate locations.
- Offered the data necessary to make continuous course and program improvements.
- Enabled faculty to provide meaningful, yet timely feedback to students.
After investing so much time in its proprietary platform, it wasn’t easy to make a change. But if UNMC’s College of Medicine was to make a change, the obvious solution was ExamSoft. The platform was already in use at UNMC’s College of Pharmacy and many peer schools. All were reporting ease of use, significant time savings, better and more secure testing, and rich data to improve instruction.
Adoption and Start Up
When UNMC College of Medicine made the decision to implement ExamSoft in the Summer 2015, they were able to start using the platform for exam administration almost immediately.
Program administrators quickly adapted to the new technology, finding it intuitive and easy to use. They especially liked hands-on aspect of the assessment platform. The course directors liked being able to see the progress on an exam build at any time, working with the assessment coordinator and phase directors in designated “construction zones” for item review and collaboration.
The school dove right into categories, organizing exam items primarily by board exam content, in addition to item author and date — when questions were most recently used. Beyond these broad categories, each block has its own unique category structure for its specific organizational needs. For example, anatomy and embryology divide content further into categories including head and neck, upper limbs and lower limbs, etc. After each exam, students receive detailed performance reports broken down by these categories.
Results
With five years of ExamSoft use under its belt, UNMC’s COM has operationalized ExamSoft and its resulting data, making both an integral part of the program and decision-making processes.
- Demonstrating the quality of instruction, feedback, and overall program effectiveness is easy with ExamSoft. The data makes it clear that the UNMC COM is meeting accreditation objectives and providing meaningful feedback to students to ensure their full grasp of critical concepts
- The web-based faculty portal within ExamSoft is a tremendous benefit to COM. Previous systems were all connected to a server on campus, and remote connection wasn’t always feasible, fast, or effective. With ExamSoft, COM can quickly build exams and immediately begin the review process.
- To lower their environmental footprint and cut back on in-office visits, COM now delivers digital exam reviews and reports to students in groups. This also reduces the faculty time spent in review sessions from 4.5 hours to just 1 hour.
- To help ease the adjustment to remote learning in Spring 2020, COM set a window of time during which students could access secure exams and distributed passwords electronically.
- COM pairs ExamSoft’s complete device control and item randomization features to deliver remote exams on an as-needed basis for students in quarantine.
At the height of the pandemic and remote learning — March through May…we knew some students had kids and spouses at home, so we gave them a larger timeframe to take the exams. I would send out their password at the beginning of the day and the exam was already secured on their computer. — Tamra Nielson, Assessment Coordinator
Next Steps
Tamra Nielson, the assessment coordinator at the UNMC College of Medicine, has helped maintain a steady course with ExamSoft since former curriculum & assessment manager Sarah McBrien accepted a position at the UNMC College of Allied Health Professions. With a firm foundation for delivering high-stakes assessments, the College of Medicine is able to focus on creating a more organized system of categories for faculty and students.
Under Tamra’s direction, the College of Medicine is actively working to tag exam items to course objectives which faculty will then review for accuracy and relevance. Tagging items to course objectives will likely become an ongoing process for the COM in an effort to incorporate the most recent learning outcomes in assessment. Finalized course objectives will also be visible on individualized strength and opportunity reports, allowing students insight into specific areas of their exam performance.
The College of Medicine is also working to streamline item naming conventions in order to quickly identify items by instructor and lecture title. This new naming system will help expedite exam creation by making exam content easy to find among hundreds of faculty members and courses.
UNMC is currently preparing for an upcoming accreditation visit from the Liaison Committee on Medical Education (LCME), for which the College of Medicine will pull assessment data from individual items to demonstrate which learning objectives are represented and how often each is objective is assessed. The College of Medicine’s continual category improvements will contribute to stronger reporting through ExamSoft, revealing curricular alignment with accreditation criteria and helping to prepare students for future licensing examinations.
