ExamSoft for Vet Med Specialty Certification Exams
Dr. Robert Malinowski, a Michigan State University alumnus and faculty member, has coupled his veterinary medicine knowledge with deep experience in educational technology to modernize the administration of veterinary specialty board-certification exams.
Robert’s consultancy, Veterinary Specialty Exams, is applying ExamSoft in a new way to help veterinary medicine specialty colleges:
- Transition to a more efficient, secure, and environmentally friendly electronic platform.
- Improve the quality of tests administered.
- Shorten the time it takes to deliver results from days or weeks to hours.
- Provide feedback to exam takers on areas for continued study.
Background
Dr. Robert Malinowski first became familiar with ExamSoft while teaching at Michigan State University’s College of Veterinary Medicine (CVM). In both the CVM and the College of Human Medicine (CHM), he worked extensively with ExamSoft for instruction and exam administration. He helped transition most of CVM’s courses from paper-based testing to the digital platform over the course of a few years. At the start, there was resistance to change from some faculty. In a short time, as fellow faculty members saw how easy it was to create, deliver, and grade exams through ExamSoft, a digital transformation was under way.
As ExamSoft took, Robert’s colleagues began to see how ExamSoft could go a long way toward improving the administration of specialty exams. After earning a doctorate degree, veterinarians often go on to specialize in one of the 20+ different areas — such as toxicology, ophthalmology, and sports medicine — approved by the American Board of Veterinary Specialties (ABVS) and the European Board of Veterinary Specialists (EBVS).
Problem
Until just a few years ago, most specialty board-certification exams were still being administered via paper through some combination of bubble sheets and long-form written responses, or on homegrown electronic systems with limited functionality. Specialty exam board members, responsible for developing and administering certification exams, are often scattered around the country and only serve in that capacity for a short time, typically one to two years.
Geographic sprawl and turnover made collaborating on paper exams a real challenge. Each year the process would start from the ground up, without an electronic item bank. Printing (and often reprinting) was costly and time consuming. Providing results to exam takers simply took too much time. If the candidate needed to re-take the exam, there was no mechanism in place for giving valuable feedback on areas for focus on their next attempt.
ExamSoft Applied to Specialty Exams
Veterinary Specialty Exams got its start with the American College of Veterinary Sports Medicine and Rehabilitation. An MSU colleague of Robert’s approached him after her first year chairing the exam committee and administering (and grading) the written board-certification exam. It had been a less-than-optimal experience. This colleague knew there had to be a better way and approached Robert for advice.
There were a few challenges to applying ExamSoft in this setting. The first being that many of the board exam candidates didn’t have experience with the platform. Due to the high-stakes nature of the exam, a clear onboarding and training system needed to be put in place to ensure exam takers felt confident and had a high degree of familiarity with the platform before sitting for the actual exam.
To that end, Robert offers test takers several opportunities to become familiar with ExamSoft. Starting with the American College of Veterinary Sports Medicine and Rehabilitation — and now for other specialty exams — exam takers have the opportunity to participate in:
- An online orientation session.
- A practice test at home, well in advance of the board-certification exam.
- A practice run at the site of the exam (usually a hotel or conference center).
With so much advance planning, administrators and test takers can run the traps and make sure everyone has devices that meet the minimum system requirements.
On the administrative side, another challenge for this unique application of ExamSoft was that there are teams of people contributing questions to the exam. While goals are well-defined based on prescribed job tasks, the actual questions are spread out among various sources and in a variety of formats, often Microsoft Word, Excel, or various databases. Bringing order to the questions, eliminating redundancies, and importing items them into ExamSoft requires some effort at the start. But once questions are developed and tagged to key job tasks, the ExamSoft blueprint helps Robert and the board teams get a clear picture of how balanced the questions are — ensuring questions represent exam goals, the higher levels of Bloom’s Taxonomy, and alignment with the job tasks for that specialty.
Results
The first veterinary specialty exam administered for the American College of Veterinary Sports Medicine and Rehabilitation via ExamSoft was in 2012. About 50 candidates sat for that exam; none had any issues with the technology. In fact, the exam takers gave positive feedback on the ExamSoft platform. The exam committee was blown away by how quickly they had results in hand. Grading that had previously taken weeks to complete was now done almost immediately, with grades passed along to governing boards within a few days.
Though exam security hasn’t been the main driver of ExamSoft adoption by the varying specialty groups, they have greatly appreciated ExamSoft’s ability to provide a secure environment by automatically randomizing question order and answer options per question for each exam taker and locking down candidates’ computers so that they can’t take screen shots and share test questions, browse the internet for answers, or access any other applications or files during testing.
For exam takers who don’t pass, the ExamSoft Strengths & Opportunities report has been invaluable. With this information and data, they’re able to focus and study more specifically on problem areas.
The adoption and use of ExamSoft to prepare and administer the American College of Theriogenologists (ACT) Certifying Examination and the support of Dr. Rob Malinowski fortuitously prepared the ACT for the unforeseen need to administer the Certifying Examination remotely in 2020. – Dr. Brian Whitlock, Chair of the American College of Theriogenologists (ACT) exam committee
Future State
Robert has recently implemented additional features — including rubrics, ExamID, and ExamMonitor — with several specialty groups and plans to continue adapting to the changing environment as required. Many specialty groups that had already made the transition from paper tests to computer-based exams are now quickly adjusting to fully remote exams with the help of exam integrity tools, including virtual proctoring. These groups are using performance grading and remote proctoring to stay flexible amid a global crisis.
Use of ExamSoft among veterinary specialty colleges has grown quickly because it’s proven to be accessible, easy-to-learn, and offers a quick return on investment. So much so that interest is mounting to have a pre-board exam test platform in ExamSoft to get residency trainers more involved in the platform and have a clearer line between the training programs and exams. Veterinary Specialty Exams is optimistic about rolling out ExamSoft more broadly in the years to come.
