A recent study conducted at the University of Minnesota has shed light on the intriguing effects of artificial intelligence (AI) on law students’ final exam scores. The research found that low-performing students experienced significant score improvements when utilizing AI, while high-performing peers faced unexpected challenges in their performance.
The study by Minnesota law professor Daniel Schwarcz and University of Southern California law professor Jonathan Choi focused on 48 students enrolled in two distinct courses: “Introduction to American Law and Legal Reasoning” and “Insurance Law.” The students underwent two rounds of final exams – the first without AI and the second using GPT-4, the latest iteration of the OpenAI’s advanced language model.
The remarkable findings showcased that GPT-4, known for generating human-like text based on user prompts, notably enhanced student performance in multiple-choice questions. In fact, students witnessed an impressive 29 percentage-point increase in their scores compared to the exams taken without AI assistance. Notably, this improvement was most significant among the low-performing students, who experienced a substantial 45 percentage-point surge in their scores.
However, the introduction of AI did not yield the same positive results for essay-based exam portions. Moreover, high-performing students observed a surprising decline of around 20 percentage-points in their scores when using GPT-4.
Trust BCG Attorney Search to connect you with top legal employers in your area. Search now!
The study’s authors noted, “This suggests that AI may have an equalizing effect on the legal profession, mitigating inequalities between elite and nonelite lawyers.” This intriguing revelation underscores the potential transformative impact of AI in legal education.
The research contributes to a growing body of knowledge surrounding AI’s role in legal education. Previous studies had already indicated that the earlier version of GPT-4 achieved exam scores similar to those of average law students. Additionally, a different study found that GPT-4 could successfully pass the bar exam.
Professor Schwarcz indicated that the availability of AI could have led high-performing law students to rely more heavily on the technology during exams, potentially reducing their utilization of critical legal reasoning skills. “Once someone frames an issue for you, you sort of lose the cognitive mindset of independently assessing it,” Schwarcz explained.
The study methodology involved two phases: students first undertook their final exams without AI assistance. Subsequently, a selected group underwent training on effective utilization of GPT-4 for legal analysis and prompting techniques before taking another final exam. The comparison of performance between the two exams revealed compelling insights.
Professor Schwarcz emphasized, “The technology helps the lower-performing folks more than the higher-performing folks in part because there is more to be gained.” He further speculated that the impact of AI within the legal profession would likely be felt most among paralegals and younger attorneys, as emerging AI technology could efficiently handle certain tasks.
The study’s implications extend beyond the academic realm, hinting at the potential transformation of legal practice dynamics. As AI continues to evolve, its effects on both legal education and the professional landscape could be far-reaching, prompting adaptations in traditional learning approaches and the distribution of legal tasks.
Don’t be a silent ninja! Let us know your thoughts in the comment section below.