A board writes an assessment into an order. Weeks or months later a result comes back: pass or fail, with a score on each of four sections. Behind that score is a harder question, whether the licensee can reason through the ethics of what happened, and most boards never see how it gets answered.
Episode 3 of Board Insights opens that process with a guest from each end of it. Dr. Greg Bond is a chiropractic specialist with the National Board of Chiropractic Examiners, where he works with the testing and psychometrics teams on exam content, including EBAS scenarios. Dr. Ken Pennell is the EBAS lead grader. He has more than 40 years in chiropractic practice and 30 in regulation and testing. He has scored EBAS essays since 2013 and now leads the graders who score them. Greg Hirschi, VP of Operations at EBAS, hosts.
The conversation follows the assessment in order: where a scenario comes from, why the licensee writes, who grades the essay, and what the score tells a board.
Every Scenario Starts as a Real Violation
A scenario begins with something that happened. Regulatory boards publish their disciplinary actions, and scenarios are modeled on those cases, with identities protected and none of the wording reused. Dr. Bond says that is the first step toward validity, because the scenario mirrors something that happens in practice.
From there the scenario travels what Dr. Bond calls the conveyor belt. A subject matter expert or a regulator submits it, and the testing department rephrases it around what it is meant to test. Then it goes to a test committee of regulators, educators, ethics professors, and testing experts, with the psychometrics department taking part. The committee’s first question is whether the scenario describes something actively happening in the profession. Then it checks the job details, edits where needed, and sends it back to the testing department for a final polish that makes clear who committed the violation and who was affected.
Each scenario also gets its own rubric. It lists everything the graders look for in a complete answer, and every response is scored against it. It is separate from the four-section scoring rubric EBAS publishes for boards and licensees.
After a scenario goes live, the psychometrics department keeps tracking how it performs. A scenario that falls outside the standards is flagged, reviewed with subject matter experts, and fixed. One that still underperforms comes out of rotation.
Why the Licensee Writes
Ethics questions depend on the situation. Dr. Bond’s example is a plumber who dates a client and a medical doctor who dates a patient. The act is the same, but the profession changes the standards and the repercussions, so they are two very different violations. Stealing from an employer sits at the other end. There is no situation, he says, where that is right.
A question that depends on context has to be reasoned through, which is why the assessment is an essay. Four prompts point the licensee toward what is being scored and leave the answer to them, because the goal is to see what they come up with on their own. Dr. Pennell adds that writing an answer out takes critical thinking and shows whether the licensee understands ethics and boundaries.
Dr. Bond often gets asked why licensees don’t write about their own case. Emotion gets in the way, he says: “It’s much easier for us to talk about someone else than it is ourselves.” The scenario matches the licensee’s profession and the category the board ordered, and someone else commits the violation, which gives a cleaner read on how the licensee reasons.
Who Grades, and What Keeps the Score Consistent
Essays are read by people, so a board wants to know the score would hold with a different grader. Dr. Pennell’s answer starts with who the graders are. Every one has regulatory experience and has made decisions on licensees who came before a board. Each is trained and has to prove they can grade to the standard before scoring a live essay. Scores are monitored against psychometric thresholds. Once a year, every grader scores anchor essays whose scores are already known, and a grader whose scores drift between those checks gets coaching. Each essay gets two graders, and when they split on pass or fail, a third grader scores it.
Grading is blind. A grader never sees the licensee’s name, background, circumstances, or demographics. “You just see what they write for the case they’ve been given,” Greg Hirschi says.
The rubric anchors every grade, but years on a board also give a grader a gut read. “We want them to pay attention to their guts,” Dr. Bond says. A gut read that a response is thin, or strong, still has to be backed by evidence from the rubric.
What the Result Tells a Board
The report a board receives is a pass or fail with a score on each of four sections: introductory opinion statements, consequences, public protection, and solutions. Each is scored from 1 to 4, and 12 of 16 is a pass. The section detail shows where the licensee’s reasoning held and where it did not. EBAS makes no recommendation.
In most cases, Dr. Pennell says, the result shows clearly whether the licensee understands ethics and boundaries. That tells the board which question comes next. If the licensee lacks the understanding, the question is what remediation would build it. If the licensee has it and crossed the line anyway, there is no gap to point to, and the question is how to hold them accountable and keep it from happening again.
Dr. Bond calls the result “another tool in their arsenal,” one piece of the board’s overall body of evidence. Regulators already carry a lot of weight in these decisions, he says, and he hopes the result takes some of it off.
Asked what he wishes every regulator knew, Dr. Bond points to the level of detail and complexity behind the assessment. Dr. Pennell wants them to know how much time and effort went into making a test that is fair, reliable, and legally defensible.
Watch the Full Episode
The full conversation has moments a recap can’t carry, like Dr. Pennell on how a police officer’s son ended up policing his own profession, and on the graders who treat the work as giving back. Watch the complete Episode 3.
Join the Conversation
The discussion continues on LinkedIn. Before your board relies on an assessment result, what does it want to know about how that result was made? Follow EBAS on LinkedIn and let us know.
