What we know, and what we don't know, about legal education, accommodations, and extra time to take an exam

A recent Wall Street Journal opinion takes on "cheating" at law schools. Extra time on exams is a matter of concern, but it takes some work to sort out what we know, what we don't, and what to do.

The Wall Street Journal’s opinion page recently featured a piece critical of Pepperdine’s accommodations practices with respect to additional time on law school exams. The piece was entitled “How Cheating Spreads at Law Schools,” and it was certainly intended, by extension, to critique law schools more generally.

By “cheating,” the piece makes the following argument: law school exams are time-sensitive measurements; additional time benefits students on those exams; students manipulate the process to request accommodations to secure extra time; those students achieve higher grades and better careers, a form of cheating.

Of course, the topic of accommodations is always a sensitive one, and I always receive rather remarkable feedback whenever I write about this subject. The area is one I am always learning about, and my comments are often subject to development and nuance over time as I learn more. I ask that you construe any imprecision in language with some grace.

Does extra time on law school exams inappropriately benefit some students entitled to accommodations?

The first question is what additional time on a law school actually does. We can separate accommodations more generally (e.g., large print or braille materials for the visually impaired) from other types of accommodations. And while there are others (e.g., a “break” during an exam, an isolated testing room) that might be beneficial to all test-takers, to some degree, the real focus in these pieces related to accommodations is about extra time.

And we’ll also focus here on those students who are “entitled to accommodations”—that is, indisputably and appropriately diagnosed with a disability that requires an accommodation. (Questions about the abuse of this process will be discussed below.)

For students who have certain kinds of disabilities (e.g., ADHD), additional time on an exam is an accommodation designed to mitigate the effects of that disability. (Medication may also help, but, interestingly, accommodations practices here appear to not account for that.)

Now, an accommodation that brings a student back to par with fellow students is one that “rightly” or “appropriately” benefits students. Think of large-print materials. We would generally agree that providing a visually impaired student with large print materials on an exam would “rightly” benefit that student and bring the student to par with fellow students.

The real concern raised in this opinion piece and related commentary is that some accommodations—and here, we focus on extra time—can “wrongly” or “inappropriately” benefit students. That is, the accommodation does not bring the student to par with other students, but actually provides them with an advantage that fellow students do not share.

So, do we have evidence of this? That is, can we separate “appropriate” from “inappropriate” accommodations, or measure what it looks like with respect to additional time on exams?

We have a few studies, and fewer for legal education.

Some interesting observations to start. One study found for undergraduates, many did not need as much time as was typically given (in this study, 50% extra time)—25% seems to be more appropriate. There is also some suggestion that those with ADHD use additional time less effectively and do not improve—and some studies suggest that extended time as a benefit for those with a disability is inconclusive. There is some evidence it can help, but other studies suggest that is not the case. One study, for instance, suggested that the benefit was more apparent in reading comprehension exams, but not for math problems.

In short, extra time may not particularly benefit students, appropriately or inappropriately. There’s limited evidence to suggestion that it could appropriately benefit students entitled to accommodations.

But it is, in part, telling that universities so quickly and readily offer extra time as an accommodation when the empirical literature suggests it may not actually be the type of accommodation to benefit students entitled to an accommodation. It is also telling that universities often give 50% (or I’ve seen 100%) extra time as an accommodation, when empirical literature suggests shorter time is appropriate.

That said, other studies show some greater concern. There is evidence that tests taken with extra time may predict future outcomes differently than regular tests. For example, analyses by testing agencies have found discrepancies in how well accommodated test scores predict later academic performance. A technical study by the Law School Admission Council (LSAC) examined law students who took the LSAT with extra time and tracked their law school grades. LSAT scores obtained under extended-time conditions were not equivalent to standard scores in their predictive power – in fact, after multiple studies, LSAC concluded that scores from extended-time LSATs were “simply not comparable” to scores from standard administrations.

Data from LSAC shows that accommodated test-takers receive higher scores than non-accommodated test-takers, around four to five points. Most accommodations for test-takers translate into LSAT scores that predict law school success—for instance, a visually-impaired person receiving large-print materials will receive a score that fairly accurately predicts law school success. There is an exception, however, for time-related accommodations, and LSAT scores tend to overpredict law school success when there are time accommodations. Requests for additional time have increased dramatically over the years, from around 6000 granted requests in 2018-2019 to around 15,000 granted requests in 2022-2023.

Additionally, the College Board reported that SAT scores earned with a 50% extended time accommodation had a weaker correlation with first-year college GPA than normal-time scores did. that is, the accommodated SAT scores tended to over-predict students’ college performance.

In competitive timed exams, giving extra time can alter the meaning of the score. A high score with time-and-a-half or double time might not equate to the same level of mastery as a high score under standard time constraints.

Note, for instance, how an accommodation for a visually impaired student might work. We would expect a visually impaired student who has difficulty reading the materials to perform quite poorly on the exam. Given large print materials, that student now should perform comparably to other students. And if we gave large print materials to students who were not deserving of the accommodation, we would expect no change in performance—the accommodation does not do anything.

But it appears that, at least for these types of tests, the accommodations are inappropriately benefiting students—they are performing at a higher rate, not at a comparable rate. And if student who are not entitled to the accommodation get extra time, they might benefit in ways that they might not if they received other accommodations, like large print materials.

The LSAC and College Board studies, however, simply measure all accommodated test-takers who received extra time. This introduces a complexity. What if some of the students who received accommodations were not “entitled” to do so, because the LSAC’s accommodation process is broader than what a narrower scientific definition might identify?

Are students undeserving of additional time manipulating the process?

If there is no benefit to additional time, then it wouldn’t particularly matter if students who were not entitled to an accommodation received one. But it certainly seems that, at least for certain kinds of tests—including the LSAT—additional time may “wrongly” benefit students. And that could be exacerbated if more and more students are requesting it, and if those students do not “deserve” an accommodation.

For many physical disabilities, the disability will be visible to observers. For many mental disabilities, it will not. This puts great weight on a professional diagnosis.

If it is easy to exaggerate symptoms of, say, anxiety, hyperactivity, or distraction of the type of attention deficit, then it can be easier to secure a diagnosis. Please note, I am not saying that students are fabricating stating symptoms. I am saying, it can be easier to exaggerate certain symptoms for a diagnosis. If you have a moderate amount of pain and go to the doctor, and you know the doctor will only diagnose a certain pain reliever if you identify a significant amount of pain, there can be an incentive to exaggerate the symptoms, or to omit mitigating factors, or what not. And when it comes to mental disorders, particularly of the types many 20-somethings might reasonably be able to say they suffer to some degree, then it is easier still.

We know that accommodation requests are climbing rapidly, as the LSAT data shows. It is possible, of course, that we are simply observing a number of students who otherwise properly should be receiving accommodations requesting them. Some might “learn” about the accommodations process as they see other students missing from the exam room—as the WSJ piece suggests—and request accommodations to bring the number up to its true total.

But, others—and the WSJ piece gets at this—may feel that they need to “game” the system because others are doing it, too. They can exaggerate, or selectively report, or omit details regarding, symptoms that could secure extra time as an accommodation. If more students who are not “entitled” to an accommodation are receiving them, then more are disproportionately “wrongly” benefiting from the extra time.

It also appears universities approve accommodations requests at a fairly high rate. Of note:

According to multiple Pepperdine students, more than a third of the school’s law students receive testing accommodations, the most common of which is extended time. They report that the school’s administration confirmed this statistic at a town hall last year and noted that it’s comparable to that of many other law schools. . . .

Law schools don’t disclose their rates of accommodations, but a 2023 Oregon Law Review paper reports data on public law schools obtained through state public-records laws. As of 2021—before the post-Covid rise in disability accommodations—the accommodations rates were 21.3% at the University of California, Hastings (now UC Law San Francisco) and 25.5% at UC Irvine. Private law schools like Pepperdine aren’t subject to public-records laws.

The Law School Admissions Council reports that only 12% of first-year law students nationwide said they had a mental or physical disability in 2023, suggesting that many students who don’t need accommodations are using them to get a leg up. The California Bar Exam’s accommodations rate, by contrast, is around 7%.

This is limited information, because we do not know the rejection rate or the requesting rate, just the total who do receive the accommodation. But it seems that universities are approving them at a higher rate than, say, the bar exam. And if universities are risk averse—that is, they worry about litigation challenges to denial of accommodations—they have incentives to be overly generous in accommodations. There are concentrated benefits and diffuse costs to granting additional time on exams, a classic dilemma.

An Oregon Law Review article also found that schools with higher rates of accommodated students tended to have lower first-time bar pass rates, even after controlling for LSAT scores and other factors. In statistical terms, there was a significant negative correlation between the proportion of a law school’s student body that received exam accommodations in law school and that school’s bar exam performance.

There is debate about interpretation of this study. One possible conclusion is that students who rely on accommodations like extended time in law school may face difficulties on the bar exam if they do not receive equivalent accommodations. And if the bar exam grants accommodations at a lower rate, that seems plausible. But bar examining authorities do grant extra time to many candidates with disabilities, and it is also possible that additional time is still not sufficient in the way that additional time on the LSAT (and perhaps law school exams) may benefit test-takers.

Another conclusion might be that law schools are failing to prepare accommodated students. It could indicate that some accommodated students, despite doing well in law school exams with extra time, struggle with the standardized, fast-paced bar exam. Whether that is a bar problem or a law school problem, or both, is perhaps the subject of debate, and perhaps highly influenced by one’s priors.

Another possibility is that test-taking speed is an affirmative value on the bar exam, and law schools are failing to develop pacing strategies that the bar exam demands. It points toward the need for better support, and perhaps different evaluation methods, for accommodated students transitioning to high-stakes professional testing like the bar exam.

(One side note, and one I am not going to delve into here. Occasionally, questions arise about the relationship between time-pressure examinations in law school, and the fact that much of legal practice includes time sensitive deadlines, or billing by the hour, so that completing tasks in a fixed period of time is directly relevant to the practice of law. This is another fruitful area of future study, and one that perhaps is only touched upon in the bar exam data studies, but one thing I’ll simply flag.)

So if additional time can benefit test-takers in an “inappropriate” way for the LSAT and law school exams, there is an incentive to manipulate the process to gain additional time. If that is happening—and there is a big if here—it may also exacerbate other problems after students graduate, such as higher bar failure rates. (This, of course, would seem to undercut the notion that the accommodations are disproportionately advantaging the best students in the class who secure the best jobs, as these students tend to pass the bar at very high rates. Instead, it is possible that it is masking concerns for students who are closer to the middle of the class. But, that is a different matter, of separating what is true and what is more anecdote about the nature of these accommodations.)

Note, of course, the assumption that additional time on exams helps law students. There aren’t really great studies on this, and most of the assessments are for standardized tests, and elementary and undergraduate education.

Are law schools assessing their accommodations practices? Can they? Are there things individual professors can do?

What can law schools do?

The WSJ piece includes this, the kind of anecdotal, post hoc detail that should be read with great caution:

Pepperdine students say many of their classmates who ranked near the top of the class, made it onto the law review, and secured competitive jobs at major law firms received extended time on tests. The university denied that students with disabilities are disproportionately represented in these groups.

This is certainly the perception. But is it true? And can law schools mitigate the “advantage” that extra time might offer?

Let’s start by thinking about exam formats.

The WSJ piece mentions a “racehorse” exam. This is, frankly, both a common type of exam at law schools and a poor assessment method. A “racehorse” exam is essentially an open-ended fact pattern with potentially dozens of legal issues inside it. It is a “race” to see who can identify the most issues and analyze them in the allotted time. It could measure both breadth and depth.

Worse are “racehorse” exams where professors value “black letter law” statements. If you have 50% additional time to regurgitate the law, you are assuredly going to be at an advantage.

Knowing nothing else about the exam format, then, a, say, three-hour “issue spotter” “racehorse” exam will almost assuredly benefit those who have additional time to take the exam. (Admittedly, it also benefits those who are fast typists!)

Unfortunately, few schools spend any meaningful time assessing the assessment—that is, reviewing whether professors’ exams were appropriate for the course, leaving it to some abstract sense of “academic freedom.” Relatedly, few schools rarely provide meaningful guidance to help faculty reflect on assessments. (This is rather remarkable if more than one-quarter of your student body is requesting accommodations that may inappropriately benefit them, and if students are aware of this and express resentment about it, privately or publicly.)

Suppose this is all true. If it is, it would seem that faculty should try to move away from “racehorse” exams. Here are some thoughts—all tentative, and certainly subject to robust scrutiny!

Word limits. If more time means students can type more, then faculty should strongly consider implementing word limits on exams. While there might be more time to “think” through an answer, spot appropriate issues, and get the law right, there cannot be an opportunity for an “outline dump,” or a “racehorse” of simply putting more words on the page.

More difficult exams. Simply put, if the exam is harder, then more time will typically not redound to someone’s benefit. This is consistent with some of the empirical literature suggesting there is less value for additional time on math exams—it is typically not that people lack time to complete the exam, but that people simply do not know the answer. A more challenging exam, or an exam with more complicated sub-issues, can reduce disparities.

Multiple choice or short answer questions. The LSAT is a multiple-choice test, but one where more time is beneficial. The bar exam includes multiple-choice components (of course, it also has essay components!), and it appears that more time is not as beneficial (appears!). It is possible, then, that specific substantive knowledge on exam questions—more of what law school exams do, which is closer to the bar exam than the LSAT—in the form of multiple choice may not give students with extra time an advantage. The same could be true for short answer questions.

Moving away from a three-hour, in-class exam. If the exam is eight hours, or 24 hours, then it would seem that time-and-a-half is less of an advantage, because the exam format is designed for flexibility for the test-taker. (That said, in the ChatGPT era, schools may understandably be moving away from such take-home formats.) It is possible that paper classes look different, too. And whether oral examinations are feasible is another matter entirely. The point being, law faculty may want to evaluate whether there are alternative formats to assessment that do not place such a premium on time.

Can schools measure this? Without subjecting myself to an institutional review board at the moment, I’ll say that I’ve tried some of these methods to varying degrees and have found some success. I am able to assess a multiple choice component and find that most students, without accommodations, easily complete it within a fixed period of time. Word limits do seem to constrain students who might have the luxury of writing volumes and volumes in the exam in the hopes of “finding” points.

It’s also possible to assess, in the aggregate, whether students who receive time-related accommodations are scoring at higher rates on your exam than others—or if they are comparable. It is possible to evaluate this across classes and see if different modes from different faculty are yielding different results. These are things that can happen on an individual faculty member level or on an institution-wide level. And of course, any results should be read with caution.

Concluding thoughts

It is possible that additional time on law school exams “inappropriately” advantages students. And it is possible that the process for seeking accommodations for additional time is more easily manipulable, both compared to other accommodations and given their availability at some institutions in particular. Assuming both are true (and there are at least some reasons to give that at least some, but perhaps not much, pause, as highlighted earlier), law faculty are not powerless to address the disparities. Setting aside the accommodations process itself (which is perhaps a robust conversation for another post!), faculty have tools at their disposal—or, I should be more modest, faculty potentially have tools at their disposal—to reevaluate their assessments. They should also consider asking administrators to help conduct blind review of grades to see if disparities are arising in their own examination methods. There are potential solutions that can mitigate the risks identified in this WSJ piece, but they require proactive and thoughtful faculty engaging with the present challenges. Communicating these things clearly to students (which requires some thoughtful law school administration strategies) can also help mitigate the complaints that might spread across a discontented student body.