Mohammed Bin Rashid University of Medicine and Health Sciences (MBRU) was in search of a digital assessment software for its newly established College of Medicine — one that could provide a solid foundation for a variety of assessment types and in-depth reporting where other solutions fell short. The College of Medicine adopted ExamSoft in 2017 to deliver both formative and summative written assessments as well as Objective Structured Clinical Examinations and Multiple Mini Interviews. When the pandemic hit, ExamSoft had all the features the College needed to deliver secure, offline assessments in time for Spring finals and optimizing remote admissions process.
Background
Mohammed Bin Rashid University of Medicine and Health Sciences (MBRU) was established in 2016 as the first medical university established in Dubai Healthcare City (DHCC). DHCC is an economic free zone that serves as the site for such healthcare entities such as Johnson & Johnson, the UK’s Moorfields Eye Hospital, and Mediclinic International of South Africa.
In recent months with the world battling a pandemic, MBRU became home to Dubai’s COVID-19 Command and Control Center responsible for establishing COVID-19 procedures in the UAE. MBRU has played a vital role in researching the strain of the virus in the UAE and dispensing information to help fight the spread.
MBRU’s College of Medicine (COM) currently offers two programs — a Bachelor of Medicine and Bachelor of Surgery (MBBS) and a Master of Science in Biomedical Sciences (MSBS). The MBBS is a six-year undergraduate program that offers students valuable clinical experience from Year 1 through on-campus research and affiliations with local and international healthcare institutions.
Challenge
As IT Manager at MBRU, Jalal Mohamadeya has been responsible for researching and initiating most of the technology solutions currently in use at the university. In 2015, when MBRU was still in its infancy and its College of Medicine had yet to receive its first cohort, Jalal was tasked with finding a digital assessment tool that could keep pace with the growing university.
Eliminating paper-based exams was a major goal in Jalal’s search for a digital assessment platform. He wanted to reduce printing costs and boost faculty efficiency in creating and scoring exams. MBRU also required a solution that offered more advanced reporting tools than their learning management system could provide, including exam blueprinting and psychometrics for analyzing the quality of exams and exam items.
In addition to meeting these criteria, MBRU hoped to find a solution with rubrics-based tools for assessing the clinical competence of future COM students in Objective Structured Clinical Examinations (OSCEs) and Multiple Mini Interviews (MMIs).
Adoption and Start Up
Jalal first learned about ExamSoft in 2015 while attending the EDUCAUSE Annual Conference in Indianapolis. When MBRU’s College of Medicine received its first cohort in 2016, Jalal contacted ExamSoft to discuss implementing the platform at the college. COM adopted ExamSoft in 2017.
To ensure a smooth experience, an ExamSoft implementation specialist visited MBRU to train faculty members and staff to build and manage assessments within the faculty portal and to familiarize them with Examplify, the student-facing exam application.
In Spring 2017, COM piloted its first exam with ExamSoft. The following year, COM began using ExamSoft’s rubrics-based tool, to evaluate students’ performance in clinical scenarios with OSCEs. COM also launched its pilot MMI program in 2018, using ExamSoft to score and rank prospective COM candidates in the admissions process. These MMIs assess the “soft skills” of each candidate, including communication skills, ethical reasoning, empathy, and problem-solving, to ensure that candidates not only meet the rigorous academic requirements of the program, but are competent in non-cognitive areas as well. This initiative was well-received, earning the MBRU MMI Team the “MBRU Excellence Award 2020 – Innovation.”
Excellent | Good | Satisfactory | Unsatisfactory | |
---|---|---|---|---|
Communication Skills | ||||
Problem-solving | ||||
Empathy | ||||
Ethical Reasoning |
Results
As the COM pushed to digitize program curricula, some faculty were resistant to the change. Faculty members, accustomed to face-to-face interactions with students, were worried that student access to exam results would limit their ability (or eliminate the need) to provide meaningful post-exam feedback, which would hinder student remediation. However, faculty soon found that the time saved in building and grading assessments allowed them to devote more time to enhancing their course, thus improving student learning outcomes.
With ExamSoft’s item analysis and psychometrics tools, COM faculty have been able to evaluate overall exam quality and glean statistical insights related to specific items to determine question quality. With these statistics, faculty can observe whether their exams are effectively measuring student knowledge and decide whether individual questions have a place in future assessments.
The College of Medicine successfully delivered its first high-stakes exam in Fall 2017. Students in COM’s first cohort in the MBBS program, graduating in 2022, have been using Examplify extensively since Year 2, and the college has now eliminated all paper-based exams.
COM faculty now have sufficient knowledge for writing their own items, creating and delivering their own assessments, and even implementing new question types. However, Jalal and his colleagues in the (Smart Services department) still offer ExamSoft consultation and training. In fact, Jalal earned the ExamSoft Power User Certificate in May 2019, making him a very knowledgeable resource for MBRU faculty. When faculty want to introduce new assessment modes and methods in their courses, COM’s IT department assists by directing pilot programs for these courses.
The college has utilized ExamSoft’s category tagging and reporting tools to prepare for yearly visits from the UAE Ministry of Education’s Commission for Academic Accreditation (CAA). By using question tagging, ExamSoft’s curriculum mapping tool, COM has created an exam blueprint, which outlines a hierarchy of competencies from course to program, program to college, and college to university. COM is able to quickly generate these reports to demonstrate how they are measuring course objectives and learning outcomes at the course, program, and college level.
With ExamSoft’s statistics, the faculty are able to identify the quality of the exam and questions. They are able also to identify a question is going to be asked again or if they need to change something within that question. They can identify if it’s a good question for our students to be asked again or not.
– Jalal Mohamadeya, MBRU Smart Services Manager
The Impact of COVID-19
In Spring 2020, when COVID-19 cases were beginning to surge in the UAE and worldwide, the Ministry of Education directed all colleges and universities to transition to remote learning and assessment, and advised that institutions adopt proctoring tools to maintain academic integrity.
When the College of Medicine contacted ExamSoft to see what options were available for administering secure, remote assessments, they learned about ExamID and ExamMonitor. The combination of ExamSoft’s auto-identity verification tool and A.I.-powered remote proctoring software was the solution they were looking for. COM quickly implemented ExamID and ExamMonitor for its MBBS program in time to meet deadlines for end-of-semester exams.
With ExamID and ExamMonitor, COM faculty could ensure that the identity of each student was verified prior to accessing exams, and they could review flagged behaviors during exam sessions. With these solutions in place, faculty could easily determine if any of these instances qualified as academic dishonesty.
In the ongoing pandemic, MBRU has adopted the MMI program for the university-wide admissions process. By introducing a Teams conferencing tool to accompany rubrics through ExamSoft, MBRU has optimized the screening process for prospective candidates. Since MBRU can schedule time to meet with prospects at anytime and anywhere in the world, the university is able to interview more candidates and thus enroll a larger, more diverse student body. Though MMI scenarios vary across colleges at MBRU, ExamSoft is flexible enough that it can be effectively used by all to score and rank applicants.
Next Steps
As of November 2020, the College of Medicine continues to deliver remote assessments and has successfully administered its October midterms remotely using ExamID and ExamMonitor. Barring any sudden spikes in cases in the UAE, COM plans to deliver Fall finals on campus.
To ensure a contactless check-in process for these socially distanced exam sessions, ExamID will be mandatory for all on-site exams. Though COM does not foresee the need for ExamMonitor once on-campus classes return, it plans to keep the remote proctoring software on hand should the need for remote assessment return.
Moving forward, the College of Medicine feels confident that ExamSoft’s solutions will offer the flexibility it needs to stay prepared for the unexpected and continue to deliver high-quality assessments.