Seeking a solution to improve institutional revenue without increasing tuition rates, Touro College of Osteopathic Medicine (TouroCOM) adopted ExamSoft to aid in the delivery of embedded assessments for student remediation. Within three years of adoption, TouroCOM implemented ExamSoft in conjunction with a new, dynamic curriculum to improve board passage rates, reduce attrition rates, and reclaim lost revenue.
Background
Touro College of Osteopathic Medicine (TouroCOM) is a medical school situated in the heart of Harlem, NY. The two-fold mission of the school is to increase the number of primary care physicians in underserved communities,
coupled with a desire to improve the diversity of physicians within the medical profession.
Challenge
When TouroCOM first opened, the school’s curriculum committee developed a traditional, static curriculum that was faculty-driven. Faculty wrote the syllabus, created and graded the exams, set the “curve” for these exams, and then posted the results to their LMS. The curriculum committee and administration provided oversight of this process at the end of the year. It was obvious early on in the development of TouroCOM that this faculty-driven process was not adequate. With its student board exam passage rate around 70%, its student dropout rate at approximately 8%, and a limited applicant pool for admissions, it was clear that it was time for a change. Furthermore, student feedback about the curriculum was described as disorganized and inefficient. In response to these issues, TouroCOM developed a series of changes within its existing medical curriculum. Developing a new, dynamic curriculum included measuring student outcomes on objective learning on a daily basis, providing students with the objectives they are expected to learn each day at the beginning of the academic year, and being one of the first medical schools in the United States to deliver high-stakes exams with a computer-based testing platform.
Implementation
TouroCOM developed its new dynamic curriculum using a triangular approach of lectures, exams, and assessment (the LEAH method) with outcomes from student performance at the center, driving all aspects of the curriculum.
In developing this new curriculum, TouroCOM administrators looked at many different educational models. Some of these models considered one part (lectures, exams, or assessment) of the triad, but were limited on the other two. Even fewer models looked at student outcomes as the main output of the system, and administrators were unable to find any model that looked at all three sides of the triangle as equals.
The first step in the implementation of the new approach was to establish objectives for each lecture throughout the entire medical school curriculum for the upcoming year. Before classes began, TouroCOM established learning objectives for courses — as well as for each day the student would be in school for a course — and included these objectives in the syllabus. The school then compiled syllabi for all courses and bound them in a book, which faculty distributed to students on the first day of class. This allowed students to know what objectives would be discussed in class in the months to follow.
The distribution of the syllabi to the students was important to the curriculum, as it kept faculty on track with their curriculum and ensured that core objectives in each course were taught in time with other courses. This longitudinal approach, which has been shown to improve student retention, works best if all classes keep to schedule.
Once TouroCOM set objectives for the entire year, the faculty proceeded to write exams and classroom assessments. The school required faculty to directly link the questions for these exams and assessments back to the objectives detailed in the syllabus. The problems with this part of the implementation were: 1) how to execute it and to track each question along with its objective to allow for detailed assessments, and 2) how to let the student know on what objectives in the curriculum they needed to focus their study efforts. To meet this challenge, TouroCOM administrators selected ExamSoft.
Results
Data-Driven Feedback
ExamSoft allows faculty to enter their questions in an online database and to categorize each question based on an unlimited number of “tags,” including program objective, learning outcome, accreditation standard, and Bloom’s taxonomy.
At TouroCOM, faculty tag each question back to the objective in the syllabus for that course, and the administration tags questions to board exam areas, accreditation standards, and other internal metrics. By doing this, administrators and faculty can see how the students perform with respect to the objectives in each course, how they perform in national board exam areas, and assess how the students perform on the accreditation bodies’ core competencies.
TouroCOM also uses ExamSoft’s reports to gain insight into course and program performance so that faculty might better adjust their lesson plans and syllabi toward students’ needs. Ultimately, this system of assessment allows TouroCOM to evaluate each student individually while giving scale to the institution to see how the curriculum performed globally.
Creating a Dynamic Lecture
The second step in building the new dynamic curriculum was to change the method of lecture delivery. Lectures are now delivered using iTunes, and students watch the lecture prior to coming to class. Classroom time is then devoted to assessment of the student’s knowledge from that lecture using iClicker technology in conjunction with ExamSoft’s question categorization capabilities that integrate into its testing platforms. The professor goes through a series of assessment questions, all of which are “tagged” in ExamSoft, using the following structure:
- The professor prepares 6-10 questions from the iTunes lecture material and readings the students reviewed prior to coming to class. These questions are categorized similarly to exam questions for the course.
- Each question is given in sequence to the students.a. The professor presents the question to the class on a PowerPoint slide.b. The students answer the question using a student response system. The professor does not show the results to students. (1 minute)c. The students work in small groups to determine what the answer is to the question. (2 minutes)d. The professor shows the results using ExamSoft’s “Release Results” feature from the two questions.e. This is where the dynamic part of the lecture begins. If the students get the questions correct, the professor repeats the cycle. If the students get the questions incorrect, the professor engages in a review of the material and works with the students for understanding as to why the answer is correct and the other answers are not. (5 minutes)
This method allows the instructor to focus on the material the students did not understand from the lectures and base this decision on the immediate outcomes from the student’s performance. This is called a “dynamic curriculum.”
An advantage to the dynamic curriculum, which is enabled by ExamSoft’s performance reporting features, is that more students attend class. Lectures are more fun, students are engaged in learning, and the instructor is there to guide students through challenges. This pathway is based on the real-time outcomes of the students in the class. One noticeable change since implementation of this dynamic curriculum is that more students are attending class, likely because they recognize that these new lectures are more valuable to their learning.
The assessments provided in the classroom are created in ExamSoft and thus tagged in the same manner as the high-stakes exam they will take at the end of the semester. With this process, it is possible to perform a daily assessment of students in each class and in each categorized discipline, including competencies related to course objectives, board areas, and accreditation standards. The advantage to performing this daily assessment is that it is possible to identify students who are having difficulty early on in the curriculum — as ExamSoft’s performance reports also determine which students are considered “at-risk” — and to give them the assistance they need to pass the course.
Implementing an Early Warning System
In an effort to reduce the dropout rate and recover revenue for the institution, administrators at TouroCOM implemented an “Early Warning System” (EWS), which involves measuring students’ learning outcomes on a daily basis.
The EWS identifies students that move away from their expected norms (their average grades). This is often caused by an unexpected event in the student’s life — a personal, family, or other problem that is affecting his/her studies and may cause him/her to drop out of school. When the student moves away from his/her expected norm, which is indicated in the performance analysis reports generated in ExamSoft, the administration can call the student in to review his/her academic progress.
Often by calling the student in and asking, “Is there anything wrong?” while reviewing his/her academic performance, it is possible to determine what the issue is and fix it before it becomes a larger problem that could lead the student to drop out of the school. Thereby, TouroCOM’s administration reduces the number of students who drop out. In fact, by using this method, TouroCOM has been able to reduce the dropout rate from 8% to 2%, yielding an improvement of more than $2 million per year in cash flow.
Identifying Student Strengths & Opportunities
By developing this EWS, which ExamSoft’s platforms supplements, and measuring student outcome measures daily, TouroCOM determines the areas in which a student is struggling. Tracking subject areas and board exam areas makes it possible to help direct students’ learning toward weak areas. This is not just for students having difficulty.
Top-performing students also benefit from this assessment because it allows them to see where they are weak in comparison with the students in their class and, on a longitudinal scale, where they can be compared to students in previous years. This system has allowed TouroCOM to improve its board passage rate from 70% to more than 90% in just four years.
Furthermore, TouroCOM is able to matriculate students with a ‘higher risk’ value because administrators and faculty monitor performance daily. Ultimately, this enables TouroCOM to fulfill its mission of promoting student success.
Supporting Enrollment of Underrepresented Students
A third point that the EWS provides the college is an upgraded risk tolerance in student admission. One of the main missions of the college is to get more underrepresented minorities (URM) into medical education. However, these students, on average, often do not perform well on standardized tests, such as the MCAT admission exam. This perceived risk often restricts the URM student from gaining access to medical school, even though the student may be an exceptional one who does not do well on standardized tests.
Because medical school is very competitive (and at TouroCOM there are more than 50 applications for each slot available), and given that if a student drops out they cannot be backfilled, medical colleges are often reluctant to take the risk associated with a student with a low MCAT score. Since the EWS allows TouroCOM to have more comfort with that risk, TouroCOM has been able to accept more URM students as compared with other medical schools, all while maintaining the high level of education required to pass the national board exam.
Conclusion
When developing a dynamic curriculum, it is important to recognize that lectures, exams, and assessment are all equally important in measuring student outcomes. These outcome measures should be reviewed on a daily basis to identify at-risk students, which will yield improved academic performance and ultimately, improved institutional revenue.
To effectively measure these student outcomes, it’s helpful for the institution to have the technology necessary to measure and gain performance insights for each learning outcome, as well as the ability to provide for assessment of objectives from the curriculum, competencies from accreditation bodies, and metrics from national standardized exams.
ExamSoft’s data-driven assessment platform has supplied these criteria for TouroCOM. By performing these assessments, TouroCOM has improved the quality of education, reduced dropout rates, and increased student engagement — ultimately creating a positive and fundamental alteration in the workings of the institution’s economic structure.
