As a leading medical school entering a new phase of digital learning, the College of Medicine at Alfaisal University* required a holistic assessment platform to cater its needs towards an e-curriculum and bring-your-own-device model. After considering several digital exam solutions, the College of Medicine implemented ExamSoft in 2015. The College of Medicine has since captured valuable insights into exam performance and realized substantial cost savings. In Spring 2020, ExamSoft’s offline assessment platform proved essential to the school’s successful pivot to remote assessment.
Background
Founded in 2008 as one of the first nonprofit, student-centered private universities in the Kingdom of Saudi Arabia, Alfaisal University enrolls more than 3,100 undergraduate and graduate students across its five colleges. International students compose 34% of the student body, and the school offers several international scholarships to help fulfill their mission of achieving a global standard of excellence. Alfaisal University was named the top young institution in Saudi Arabia by Times Higher Education’s 2020 World University Rankings.
The College of Medicine (COM) at Alfaisal offers a Bachelor of Medicine, Bachelor of Surgery (MBBS) program as well as four graduate programs. COM’s MBBS program is offered over a span of six years — students learn basic sciences in years one through three and clinical sciences in years four and five, followed by one year of internship.
Challenge
As the College of Medicine transitioned its MBBS program to a curriculum that supported digital learning and medical informatics, it began the search for an assessment solution compatible with a bring-your-own-device model. COM knew that having a platform to administer offline exams on student-owned devices could reduce the cost of managing computers in their 422-seat exam facility and eliminate exam interruptions resulting from internet connectivity issues.
Reducing paper usage via digital exam management and protecting exam integrity with enhanced security features were among the school’s primary goals for adopting an assessment platform. COM aimed to expand their digitization of records, including performance reports by student and by course, while also diving deeper into item analysis with psychometrics to improve item quality and overall exam efficacy.
Adoption and Start Up
The College of Medicine considered several assessment solutions before adopting ExamSoft in Spring 2016. While many offered the basic services the school required, comprehensive performance reports and offline exam management capabilities were among the main determinants in moving forward with ExamSoft.
Assessment Director Dr. Muhammad Faisal Ikram, under supervision of Dr. Wael Alkattan, Vice Dean Academic Affairs, oversaw the ExamSoft implementation within COM’s centralized Assessment Office. To overcome usual initial resistance to the change, the Assessment Office hosted multiple training sessions and regular workshops to encourage faculty engagement and collaboration.
The first exams administered in the MBBS program were composed entirely of multiple-choice items. COM has since then included short-answer questions to further gauge theoretical knowledge, and it has also added formats for assessing both practical and clinical knowledge (OSPEs and OSCEs).
To discourage academic dishonesty during exam sessions and prevent replication of assessment items, COM has activated ExamSoft’s question and answer randomization feature. COM has also began generating Strength & Opportunity reports throughout the academic year to monitor individual student scoring and reveal any unexplained changes in performance.
With Examplify, ExamSoft’s student application, faculty could send encrypted exams to student-owned devices for later access at the exam facility, without the need for COM to supply computers. Examplify’s offline assessment capabilities have allowed COM faculty to administer secure exams without concern for connectivity issues.
| College of Medicine, Alfaisal University Assessment Solution Requirements | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Results
By the end of Fall 2019, the College of Medicine had successfully administered summative assessments to over 55,000 exam-takers. Having managed the first ExamSoft implementation in the College of Medicine, the Assessment Office at COM now supports other colleges at Alfaisal University in developing and deploying protocols to meet their needs with ExamSoft.
Enhanced security features, including item randomization and complete device control during exam sessions, have helped to preserve assessment integrity. Monitoring student progress with Strength & Opportunity reporting has allowed COM faculty to gain a clearer picture of performance in the MBBS program, identify areas of improvement, and help formulate a future learning plan.
By switching to a bring-your-own-device model for assessment via Examplify, COM has eliminated the cost of supplying and maintaining computers in their exam facility. Managing such a facility would have required an estimated half a million upfront for devices, plus an additional $250,000 annually for depreciation and maintenance.
The Impact of COVID-19
During the lockdown phase of the COVID-19 pandemic in Saudi Arabia, all exams were to be delivered remotely. As the College of Medicine had already adopted a bring-your-own-device model for its MBBS students, the school had the necessary infrastructure in place to pivot to remote assessment.
COM utilized additional in-exam options to minimize the potential for academic dishonesty at home, including randomized questions and answers, disabled backward navigation, and reduced exam duration. Additionally, OSCEs were limited to clinical reasoning questions and all COM faculty were directed to increase scenario-based questions to assess more applied knowledge.
COM administered over 8,600 successful exam encounters in Spring 2020, and the Assessment Office at COM imparted ExamSoft training protocols to over 200 new graduate faculty and students in a matter of weeks.
Next Steps
The Assessment Office plans to continue ExamSoft workshops for all colleges at Alfaisal University. The launch of their special discussion forum for ExamSoft users at Alfaisal, “ExamSoft Coffee Break (ECB)”, has been postponed as a result of the pandemic.
In the 2021-2022 academic year, the College of Medicine is planning to begin using Map, ExamSoft’s curriculum mapping tool. With Map, the school plans to further align its curriculum with programmatic objectives and accreditation standards.
Exams at Alfaisal University have returned to normal, on-campus delivery modes, albeit with strict rules of social distancing. Should another lockdown be required in months to come, COM has the necessary tools in hand to make a successful transition to remote assessment.
*This organization received consideration for participating in this Case Study.
