Last year was a challenging one for all of us, with educators, students, and certifying bodies adapting to some necessary technological changes as classes and exams went remote. At ExamSoft, our role was to help ease that transition for our clients while continuing to the provide consistency and information they needed to drive positive learning outcomes. However, with the abrupt and dramatic shift we saw in Spring 2020 and for much of the remaining calendar year, there was bound to be some confusion around the technology being used.
During the Fall specifically, the spotlight was squarely on ExamSoft, as our software was the primary delivery system for thousands of universities and some very high-stakes certification-style exams such as numerous U.S. Bar exams. Because of the unprecedented situation, many of those institutions elected to use our remote proctoring solution, ExamMonitor, in order to deliver a safe and secure examination. Unfortunately, there has been a lot of misinformation and misunderstanding regarding how the ExamMonitor software works and the nomenclature around the process.
I’d like to take a moment to help clear some of this up.
At the heart of a recent question is the word “flag” and what it means when there is a “flag” on the recording of an exam session. To be clear, “flagging” behavioral anomalies isn’t the same as a test-taker cheating. Exam-takers whose exams have “flags” are not being accused of cheating. In fact, the system is designed to empower the exam administrator with all irregularities noted during the exam; it is sensitive to any anomaly, which means that there will be flagged instances that are overridden. To expand further on the difference between flagging and cheating when using ExamMonitor:
Flagging: The ExamMonitor process starts with an automated Artificial Intelligence (A.I.) review of the audio and video files from an exam. The A.I. is looking for anomalies — unusual behaviors — that it then flags for further review. An additional review comes from a professional human proctor who reviews the audio/video recordings and also flags unusual behavior with human context and understanding. The notes of all reviews are captured and presented for final review to the exam administrator (the ExamSoft client).
Cheating: The exam administrator (again, this could be a university faculty member, a certification examiner, or a client integrity committee) then reviews the reports and looks at the anomalies and makes their determinations as to whether there were any instances of cheating,academic dishonesty, or violation of honor code. That decision lies solely with the client’s exam administrator — not ExamSoft or its software. The exam admin may choose to reach out to the exam-taker for explanation or send a request for more information. That process, which occurs outside of ExamSoft software, is determined by the client. Since the exam administrator is the one who sets the rules for the exam, as well as knows the nuances and special circumstances of each exam-taker, they are the only ones suited for this determination. ExamSoft never makes denunciations about integrity breaches. ExamSoft does not “accuse” anyone of cheating, nor does ExamSoft “flag for cheating.” The software and the human reviewers are there to help the exam administrator focus and save time. These resources simply flag anomalies for further review by the exam administrator.
The ExamMonitor product provides information to the client. We empower our clients with data and reporting as well as audio and visual evidence (which can help both the exam-taker and the exam administrator), but ExamSoft does not determine whether or not an exam-taker has cheated.
Again, being flagged does not mean that the client’s exam-takers will be penalized or labeled as cheaters; it simply means further review will be required by the client. This is not markedly different than an in-person proctored exam. During in-person exams, there is often a proctor in the exam room, watching exam-takers as they complete the test. If the proctor sees something concerning, they can provide notes and feedback to the faculty member or administrator responsible for the exam. An in-person proctor may even choose to interrupt an exam-taker if they feel the concern is egregious; however, ExamMonitor is not designed to disrupt the exam-taker experience but instead allows the exam-taker to complete their exam and be reviewed afterwards. Either way, in-person or remotely, the exam administrator (the ExamSoft client) always sets what is or is not acceptable behavior during the exam.
At ExamSoft, we strive to deliver superior software solutions that improve education and learning outcomes. Helping deliver secure exams with reliable assessment and learning data is key to accomplishing that goal.
Vice President, Product