by Ben Bratman
In May, New York became by far the most populous and prominent state to adopt the Uniform Bar Exam, the standardized licensing test for lawyers created and promoted by the National Conference of Bar Examiners. New York is the 16th state to take on this uniform test commonly referred to as the UBE.
With such an influential state on board, the UBE is now all the rage. New York Court of Appeals Chief Judge Jonathan Lippman proclaimed in his announcement of the Court’s decision that it “will reverberate among the other states,” ultimately leading to adoption of the UBE throughout the country. The president of the Florida Bar responded to the decision by predicting that the northeastern states will soon follow New York’s lead and also acknowledging that Florida will give accelerated consideration to the UBE. And UC Irvine Law School Dean Erwin Chemerinsky argued in an L.A. Times op-ed that California should join New York in adopting the UBE.
Before other states react to the reverberations by crowning the UBE as the predominant or sole bar exam in the nation, the entire uniform exam initiative merits closer scrutiny. To be sure, adoption of the UBE throughout the country would make law licenses much more portable for beginning lawyers recently out of law school. That is a worthy goal, but the UBE is a dubious and potentially damaging means for achieving the desirable end of greater interstate license portability, especially because there is a different and more benign way to achieve that end.
A comprehensive consideration of the UBE and all that comes with it reveals five reasons why additional states should step back from the precipice and be very cautious about joining the UBE bandwagon.
- See The Uniform Bar Exam Debate for more background.
1. The UBE perpetuates a flawed bar exam and is fundamentally inconsistent with recent trends in legal education and the legal profession
The UBE is a collection of three testing instruments, each authored and sold to states by the National Conference: The Multistate Bar Examination (MBE), the Multistate Essay Examination (MEE), and the Multistate Performance Test (MPT). The MBE is a six-hour multiple-choice exam with 200 questions testing knowledge of Civil Procedure, Constitutional Law, Contracts, Criminal Law and Procedure, Evidence, Real Property, and Torts. The MEE includes essay questions covering the MBE subjects and five additional ones. The MPT requires applicants to complete assigned written lawyering tasks relying on furnished legal and factual materials.
Right now, the vast majority of states already administer the MBE, all states administer essay questions (about half use the MEE), and about two-thirds of states administer the MPT. Full adoption of the UBE would therefore prop up and perpetuate the current bar exam model—a model that is the subject of vocal and growing criticism from various constituencies and on many fronts.
Prominent among the criticisms is that the exam fails to adequately evaluate many basic lawyering competencies and focuses too heavily on substantive knowledge of law, thereby requiring applicants to engage in extensive memorization solely for the sake of the test. Meanwhile, responding to the demands of the legal marketplace, legal education is increasing its emphasis on experiential learning and skill development. In a 2014 article, Professor Dennis Honabach aptly observed that “the UBE sustains and reinforces an anachronistic over-emphasis on general subject matter knowledge just at the moment when we should all be dynamically shifting focus and resources toward experiential learning and skills training.”
The presence of the MPT redeems the UBE to a limited extent, as the MPT does not test substantive knowledge of law but rather evaluates only lawyering skills. However, the MPT evaluates a narrow range of skills and, as discussed below, receives the lowest scoring weight among the three UBE components.
2. The UBE would represent a regressive change to the current bar exam in several states
By adopting the UBE, a state agrees to administer the MBE, six MEE questions, and two MPT questions. They also agree to weight the individual scores as follows: 50% MBE, 30% MEE, and 20% MPT. The MBE scores are scaled based on level of difficulty, and scores on the written portions of the exam (MEE and MPT) are then scaled to the MBE.
Among the states that have not yet adopted the UBE are several that, giving some credence to criticisms of the bar exam, do not follow the National Conference’s lead. Some do not use all of the three tests, and many others decline to use the number of questions or the 50/30/20 score weighting that the National Conference recommends—the same that it requires of UBE states. Currently, for example, at least eleven states that administer the MBE weight it at less than 50% of the overall score, believing that the ability to write answers is a better reflection of fundamental competencies than the ability to answer multiple-choice questions. In addition, three states (California, Oregon and Georgia) plus the District of Columbia choose to weight the performance test at more than 20%, believing that less than 80% of a bar exam score should depend on questions that require recall of memorized law.
As a result, in the case of numerous states, adoption of the UBE would actually be a step backward in that it would increase the role of multiple-choice questions and/or decrease the role of performance test questions. That is reason enough for those states not to accept the UBE. The circumstances of New York’s decision are illuminating in this respect. New York has been using only one MPT question accounting for a mere 10% of the overall score. In its final report, the Advisory Committee to the Court of Appeals defended its recommendation of the UBE in part by noting the resulting increase in skills testing through a second MPT question. New York’s decision to adopt the UBE is thus a powerful precedent for about a dozen other states, including California, not to adopt the UBE but to reject it.
3. The advantage of having the National Conference set a singular scoring methodology for all bar exams is overstated
To be sure, the National Conference has greater testing expertise and resources than does any individual state. The organization employs psychometric experts and apparently has concluded that the combination of the MBE, MEE and MPT at the scoring weight of 50/30/20 is an appropriate mix to establish maximum testing reliability and validity. (Reliability is the extent to which multiple exams over time produce consistent results; validity is the extent to which the exam tests skills relevant to the profession.)
As a multiple-choice exam, the MBE is indeed an anchor of testing reliability, but reliability in testing, like anything, can be taken to an extreme. The problem is that the MBE does very little to achieve testing validity. For that reason, and given the possible adverse impact of multiple-choice tests on ethnic groups underrepresented in the legal profession, the MBE should be weighted as low as possible to achieve reliability of overall exam scores.
Finding the right balance between reliability and validity is not an exact science, and 50% as the weight for a multiple-choice test with high reliability is not a magic number. Michael T. Kane, Ph.D., testing expert and former Director of Research for the National Conference, wrote in a 2012 article in the National Conference’s own magazine that “any weighting system that assigns at least 40% to the objective component works reasonably well,” as long as the remaining elements are scaled to the MBE. (The vast majority of states using the MBE do in fact scale their written test components to the MBE.) As of this writing, Delaware, Mississippi, North Carolina, Texas and Virginia all weight the MBE at 40% of the overall score, and Pennsylvania at 45%. Merely for the sake of the UBE, these states should not be compelled to increase the extent to which they premise competency determinations on an applicant’s ability to answer multiple-choice questions. Should any state, including the five additional ones that weight the MBE at less than 40%, wish to change its method of scoring to achieve greater testing reliability, it can surely do so on its own volition—separate from any decision about joining the UBE.
Another quirk of the National Conference’s mandated scoring weight for UBE states is its allocation of the remaining 50% between the MPT (20%) and the MEE (30%). Nearly 20 years after the MPT was introduced, and after considerable movement in legal education toward greater and more varied skills training, what is the basis for still giving lesser weight to MPT questions than to essay questions? The MPT might be modestly more expensive to produce, but surely it is the most valid testing instrument on the bar exam, evaluating more lawyering skills than essay questions do while not requiring recall of memorized law.
For many jurisdictions, essay questions are a means for testing knowledge of local law, but if local law can instead be covered through supplemental state CLE courses or tests (a premise of the UBE that actually makes sense), then it is fair to ask whether a bar exam should emphasize essay questions or, dare I say, include them at all. These queries become even more pertinent as law school professors continue to increase their usage of performance test-style questions and other alternatives to traditional essay questions.
4. National implementation of the UBE would decrease the chance of meaningful bar exam reform by taking power away from states and giving it to the National Conference
The National Conference is already a powerful and influential player in bar examinations. Through national implementation of the UBE, it would gain even greater power. As a result, effecting bar exam reforms would be a formidable challenge. Effecting more fundamental reforms to the licensing system—e.g., replacing at least some of the bar exam as we know it with a clinical component akin to that required in medical examinations—might require nothing short of a coup. As Brooklyn Law School Dean Nicholas W. Allard wrote in his recent National Law Journal op-ed, Too Much Power Rests with the National Conference of Bar Examiners, “[t]aking on a powerful organization with a virtual monopoly over the bar-exam system is daunting.” If the UBE becomes the predominant bar exam in the United States, the monopoly will move beyond virtual, and the challenge of reforming the bar exam will move beyond daunting.
Maximizing the National Conference’s influence through national adoption of the UBE is especially problematic because it would all but eliminate a very important player in bar exam innovations: the states themselves. The last significant reform to the content of the bar exam, the addition of the performance test, came about in no small part because of the initiative of states. Indeed, more than a decade before the MPT appeared on bar exams, Colorado and Alaska created and administered their own performance tests as a counterweight to the multiple-choice portion of the exam. California also offered performance tests long before the MPT was born, and to this day the California Performance Test stands apart as a three-hour test, allowing for more in-depth skills evaluation than the MPT’s 90-minute version. Colorado and Alaska have already adopted the UBE. If California and the rest of the states do so, where does the next important innovation in bar exams come from, and how long will it be before we see it?
The Executive Director of the National Conference has endorsed the current construct of the exam and wrote in 2012 that any reforms to the exam will be “more glacial than volcanic.” The legal profession and legal education have undergone and continue to undergo profound change that could hardly be described as glacial. The bar exam needs to change commensurately (albeit cautiously), and meaningful change is far more likely to happen—and to happen sooner—if states continue to play an influential role in advancing exam reforms. The National Conference can and should play a significant role in supporting and working with states, as it did in the evolution of the performance test decades ago, but states adopting the UBE will end up with little influence in that relationship or on any future changes to the exam.
5. There is another means for achieving license portability that has far fewer drawbacks than the UBE
The advantages of the UBE pertaining to license portability have been well touted. Law School graduates can take the UBE in any participating jurisdiction, and after receiving their score, they can promptly seek admission to any UBE state where that score is deemed passing (e.g., minimum of 260 in Alabama and Minnesota; 280 in Idaho). UBE scores are valuable only to recent takers though, as they expire in two to five years, depending on the state. Each UBE state also sets its own character and fitness requirements, and some states require an additional test or course on local law.
In contrast, if a beginning lawyer who has already taken the bar in one state seeks admission in a non-UBE state, in most cases he or she will have to take a bar exam again. For some states, California and Florida included, this remains true no matter how many years the applicant has been practicing law.
However, given the UBE’s drawbacks, it is not the best means for achieving the laudable goal of license portability. In a very important respect, we already have a uniform bar exam. The MBE, as mentioned earlier, is offered in 49 states and is a very reliable test from a psychometric perspective. Why not allow applicants who achieve a passing score on the MBE on any jurisdiction’s bar exam to apply for admission to any other state without the necessity of taking the bar exam again? Minnesota, North Dakota, and the District of Columbia already permit this, and other states need only follow their example.
A state adopting this approach in lieu of the UBE would be giving full faith and credit to whatever the written portion of the exam is in the original jurisdiction instead of forcing applicants to take its written portion. Since most states scale the scores from their written tests to the MBE, the policy is psychometrically sound. In fact, in many cases, the written tests of the two states at issue would be very similar, and the only variable would be awareness of local law, which could be covered through a CLE program.
Instead of putting its energies into national adoption of the UBE, which would decrease the quality of the bar exam in many states and drastically reduce state autonomy and flexibility, the National Conference could simply push for more states to adopt the policy already maintained by Minnesota, North Dakota, and D.C. We already have a uniform bar exam, and the National Conference already has enough power.
About the Author
Ben Bratman is an Associate Professor of Legal Writing at the University of Pittsburgh School of Law, where he also serves as faculty consultant to the Director of Academic Success and Bar Exam Services. Professor Bratman has published several articles on bar exam reform and the intersection between bar exams and legal education. He was also among the first in the country to create and teach a for-credit bar exam preparation course as part of a law school curriculum.