Some thoughts on the Washington bar exam fiasco

The introduction of the NextGen bar was bumpy, but it's hard to know what it portends

Earlier this year, I previewed the new NextGen bar exam. It offers some promise, but also some pretty significant questions, in my view. I eagerly await to see how scoring of the exam looks and if pass rates looks comparable to previous pass rates in these jurisdictions.

But the story last week was a fiasco in its introduction, particularly in Washington state (which I first learned about from Professor Marsha Griggs (St. Louis University)). Missouri and Maryland experienced relatively small mishaps with Internet connectivity and access to the exam. But Washington’s problems were so significant that the state simply canceled the bar exam for all 647 test-takers. It will offer a new administration in September.

Law deans in Washington have already called for diploma privilege admission, a predictable response that certainly (and rightly) focuses on the student-affected side but, of course, leaves open the consumer protection side. (One helpful measure would be for the state bar to look back at the diploma privilege admissions granted in 2020 and examine whether those cohorts faced higher discipline complaints or related misconduct activity than other cohorts. While interesting, however, I don’t know such research is coming, even if it ought to be.)

I want to acknowledge that details are limited and blame can easily fall to the National Conference of Bar Examiners, but I’ll only note one small separate question. While ten jurisdictions administered the new NextGen bar exam, seven appeared to have no problems and two others only brief ones. The NCBE release blames a “venue-specific infrastructure issue at the Yakima Convention Center.” It remains unclear why this was this relatively remote venue chosen, and there are theories about why this happened in the first place. Regardless of whether law schools or the state bar had a hand in selecting the venue, its inadequacy compared to what took place in other jurisdictions should offer a moment of some introspection, too.

But for the NCBE, its delivery mechanism that failed to get the bar exam in the hands of an entire state’s worth of test-takers cannot be ignored. It’s not a problem that happened with its standard UBE in other jurisdictions. There are certainly going to be bumps in any rollout, but this was a real failure.

Given the near-monopoly the NCBE holds on the bar exam, the inevitable question that arises with any fiasco like this is, could there be an alternative? And that quickly becomes a complicated question.

NCBE offers two major advantages apart from the obvious one of “easy for a jurisdiction to use,” and despite the downsides of things like cost and, well, fiascos. The first is scoring accuracy, and the second is reciprocity.

First, scoring accuracy improves with a large cohort of test-takers and a sound understanding of IRT. It allows a test administrator to evaluate the ability of test-takers and the difficulty of the exam, and to calibrate scoring to ensure a consistent and accurate outcome year over year. I have heard (only anecdotally!) in jurisdictions that have not used the UBE but used a more, shall we say, local scoring system that pass rates can oscillate more erratically in ways that do not seem consistent with test-taker ability. Yes, single jurisdictions can equate scores over times and try to ensure that scoring remains accurate, and it is possible to get there with a lot of work. But the NCBE does this well and does it consistently.

Second, reciprocity. The UBE and now the NextGen offer an identical experience across jurisdictions and a score that can be easily transported. The practice of law increasingly occurs across jurisdictions, and the flexibility for admissions to multiple states, either to move or simply to practice there, is highly desirable, not just for law students but also for law schools. It’s even desirable for state bars—it allows attorneys to enter a jurisdiction more easily to allow greater access to legal services.

Any state that adopts its own test faces challenges, and while it might be able to get over the accuracy hurdle, reciprocity is a real blow. For some jurisdictions, they may simply not care. But it’s been, I think, one of the bigger challenges for jurisdictions thinking about decoupling from the NCBE.

When California tried to pull away for its own bar exam, a fiasco in its own right, it should be noted why it did so—to cut costs. It believed it could save money from moving away from in-person testing centers to a remote administration option.

Nevada has developed its own bar exam in recent years, to little fanfare. (Nevada also requires a small amount of “supervised practice” in limited clinical, pro bono, or government work.) It’s not clear what the outputs of that program have been (e.g., how it’s affected the market for legal services, etc.), but I’m sure those studies are coming.

But in the end, any critique of the NCBE starts to look like critiques of the ABA for accreditation and licensure—what would replace it? A state like Nevada has something, but it is more state-specific, has more requirements than just the bar exam, and is not reciprocal. Any competitor would need to solve the reciprocity problem, I think, to be effective.

It’s possible, of course, for a second organization to pop up akin to how there’s the SAT and the ACT—two college admissions tests doing different things. Some cohort of states might want to think about a second way of testing. If they worked together, that cohort could create a separate consortium of reciprocity, and maybe we could see two sets of reciprocity in the United States. Or they might offer “equivalency tables” to say that a passing score on a new exam is “equivalent” to a NextGen score of some type, much like SAT/ACT equivalency tables.

But really, it’s academic. The bar exam is for a small number of human beings in the United States each year. It would require a significant investment from an organization to decide it wanted to disrupt the market and what that new exam would look like. And not only would it need that effort, it would need buy-in from a lot of stakeholders who have relied on the ease of access of the NCBE. Fiascos like Washington come and go; state bars have short memories, even if the test-takers don’t.

For this reason, I don’t have easy answers for those critical of the NCBE. The tremendous effort to build an alternative requires some first-movers. States like Nevada have done this alone. Whether others move remains to be seen. Many bar exam reform efforts have focused on matters like alternative pathways to licensure, or on some kind of supervised practice requirement before admission to the bar, rather than the bar exam itself. Much energy has been channeled into the NextGen bar exam over developing alternative-alternatives (as I opened this post). But even after a fiasco so great as canceling the entire administration of an exam in a jurisdiction, it would seem unlikely we’ll see much change. But we’ll see how other jurisdictions react in the months ahead. As with the ABA as accreditor, an institution’s position seems very secure until it isn’t.