The Florida Bar

Florida Bar Journal

Computers Laying Down the Law: Will Judges Become Obsolete?

Trial Lawyers

Anyone beginning their legal career in the last half of the 20th century has seen firsthand the impact of technology on the day-to-day practice of law. First, computers replaced typewriters. In a seemingly short time, computer voice dictation began replacing secretarial shorthand as well as dictaphones. Volumes of books and yards of law office shelf space were replaced by computer workstations and subscriptions to Lexis or Westlaw. E-filing made the law office courier an endangered species. In the next 30 years, self-driving vehicles will lead to the virtual disappearance of car accident litigation, and increasing electronic financial transactions may render the law of negotiable instruments an anachronism.

What other changes might occur in our justice system in light of future technological advancement? While there will always be a need for lawyers, albeit in a somewhat repurposed practice paradigm,1 could computers eventually diminish, or even eliminate, the need for human judges? Although this possibility may have been far-fetched 30 years ago, this question can now be seriously debated because of Watson.

way of background, IBM created a computer program that 17 years ago accomplished what many thought would be impossible — it defeated world chess champion Garry Kasparov in a six-game match.2 Since then, greater computer processor speed and larger electronic data storage capacity, coupled with improved artificial intelligence technology, made natural language database search capability a reality. These advancements led IBM to create a computing system designed to apply natural language processing, information retrieval, knowledge representation, automated reasoning, and machine learning technologies to the field of open domain question answering.3 Beginning in 2007, IBM research began building a computer system that could compete with the world’s best players at the game of Jeopardy! Four years later in 2011, this system known as Watson4 beat two of the world’s best players in a nationally televised two-game Jeopardy! match.5 Instantly, speculation abounded about Watson’s potential uses in other information-intensive fields, such as healthcare, telecommunications, financial services, government, and law. Today, similar language-based information processing has become widely available to the public through Apple’s iPhone application Siri.6

The question of whether computers could be programmed in such a way as to replace some or all of the functions of judges is not new. In 1977, a University of Georgia law review article posited the possibility that, assuming the law is completely determinable from written authorities, an algorithm could be created to readily access all rules, principles, policies, and structures in statutes, constitutions, and judicial precedents, so that even the “hardest” cases can be evaluated and perhaps decided according to existing law.7

In an era in which both plain-meaning textualism and original intent are polestars for many judges in guiding their interpretive analysis of the law, could a computer be even more successful at reaching the right decisions? After all, computers use logic, and, theoretically, do not make outcome-based decisions. Conceivably, we could input all statutes, judicial decisions, and legislative history (with other historical sources) into Watson to serve as its database. Watson’s responses would be free of the extraneous influences that can impact judicial decisionmaking, such as public opinion, or political and personal bias. It would reduce the possibility that relevant, controlling authority is missed, overlooked, or misapplied — which is always a risk when relying on human decisionmaking. Watson could conceivably eliminate judicial activism, and maximize consistency in rendering decisions across all jurisdictional lines. It has also been suggested that this technology could create a societal benefit by minimizing lawsuits and reducing litigation costs for companies and persons if potential litigants had access to Watson to assist them with litigation decisions involving cases pending before any tribunal, including the Supreme Court.8

At the trial court level, I foresee no danger to judges becoming obsolete. Trial judges (and juries) are the fact-finders, whose findings are then used by the appellate courts for determining the correct application of the law. Judges must also make rulings on the admissibility of trial evidence, with implications for the ultimate determination of both facts and issues, including negligence. Fact-finding is also impacted by human evaluations of witness credibility, even to the point of consideration of vocal inflections and facial expressions by witnesses when testifying to help give weight to the responses by assessing reliability of evidence. This “weighing” of the evidence to assess trustworthiness and determine true facts is a particularly human ability. A computer can provide assessments based on information it is given and in conjunction with its database. It could even provide probabilities for the accuracy of any fact based on comparisons for consistency with other inputs.

As D’Amato rightly points out, perhaps the better question is not whether computers can replace judges, but rather should they replace judges. If they did, what would be gained, and what would be lost?

What a computer cannot do is to apply common sense and life-experience judgment to assess the trustworthiness of the evidence presented. The degree of hyper-consistency delivered by computers in their application of the law could result in a one-size-fits-all, or “cookie-cutter” justice system. This would prevent our courts from performing as intended by eliminating the judge’s ability to apply laws flexibly and with reason in specific cases.

D’Amato does not advocate a wholesale delegation to computers for decisionmaking, but does suggest that some areas of law involving low risk and high cost could benefit from enhanced technological involvement and automated resolution:

I would suggest the area of law that covers all determinations in advance of determinations of the merits of substantive controversies. In other words, matters of procedure, jurisdiction, venue, and choice of law would seem to be appropriate subjects for computerized decisionmaking. One need merely look at the specialized reporters containing volumes of cases devoted to jurisdiction, procedure, and conflict of laws to get an idea of the enormous amount of judicial time and clients’ money devoted to such litigation. Surely this vast expenditure does not serve the aggregate interest of the public; courts that are clogged with such cases have less time for other matters, and parties are subjected to enormous legal fees devoted to nothing more than a determination that the case is in the right court or that the right body of law is being applied. If all these cases could be handled expeditiously by computer, who would be hurt in comparison with the enormous sums saved?9

Considering both the advantages and disadvantages of open domain question answering to resolving legal disputes, it seems Watson conceivably poses the greatest danger to the existence of appellate court judges. Even so, this is still an extremely remote possibility. Appellate issues are presented to the courts by written submission, usually through briefs or motions, identifying the specific issues on appeal. These issues are phrased in a manner to allow appellate judges to analyze them according to an established body of law. The relevant underlying facts have been “found” by the lower tribunal; the appellate court considers those established facts according to the applicable law. Or, the court is asked to interpret the meaning of words or phrases in a law to properly apply to a given set of facts. Under either scenario, a computer program like Watson could be programmed to provide answers to such questions submitted to it.

In some ways, the future is already here. Legal scholars, computer science engineers, and commercial companies are building similar databases to identify the significant factors that influence particular legal outcomes using algorithms applied to historical legal data. Such factors can then be used to predict outcomes in posed scenarios. In 2009, using a computer database analysis called “quantitative legal prediction,” a web-based fantasy league was launched for predicting Supreme Court decisions. The league, now overseen by the nonprofit Harlan Institute, known as “FantasySCOTUS,”10 has built up a database of crowd-sourced opinions and analyses of many Supreme Court cases. In 2012, the founder of FantasySCOTUS suggested that future iterations of the website could combine the crowd-sourced data with data from publicly available court filings, then use an algorithm and decision engine to make its predictions.11 During the October 2009 Supreme Court term, over 5,000 FantasySCOTUS members made more than 11,000 predictions for all 81 cases decided. Based on these data, FantasySCOTUS correctly predicted the outcome in more than 50 percent of the cases decided, and the top-ranked predictors forecasted 75 percent of the cases correctly.12

Would a Watson-like system eliminate the potential flaws inherent in human decision making at the appellate level, such as bias? The short answer is no.

Neutral human intervention would still be needed in order to appropriately craft the relevant questions. While this is typically done by the lawyers in their briefs, it is not uncommon for appellate judges to restate or reconfigure the wording of these issues for greater accuracy or comprehension. Computers operate on the GIGO principle,13 and do not make corrections or clarifications to inquiries. Also, consider that while eliminating judicial bias or inconsistency is the intended goal, computer programs are nonetheless written by humans; the biases and subjectivity of a program’s authors would be built in (unconsciously or not) to the algorithms. If the computer is programmed to respond, for example, that the presence of X fact + Y fact = plaintiff wins, this is merely a substitution of one fallible human decisionmaker (perhaps a programmer with little legal training) in place of another (a human judge). Therefore, the goal of achieving unfailing neutrality cannot be achieved — the question of bias is merely shifted onto the program, rather than to the judge.

Also, our courts are routinely presented with issues of first impression. In such cases, no body of law or bank of written knowledge would be available for Watson’s use to discern the correct outcome in those cases. Although legislatures could create new laws to alter Watson’s outcomes, its ability to eventually give meaning to these new laws would be limited. Updated law and interpretive cases would no longer exist. In other words, without human judges, who would write the opinions that would serve as part of the improved database available for Watson’s distillation? Without human judges writing opinions, this database would cease to expand. Using Watson as a decisionmaker would stagnate the law, and “bad precedent” or outdated legal theorems would never be reconsidered, revised, or even overruled. Worse, they would be perpetuated.

We want judges who possess sufficient life experience, common sense, and ethics to be used in making their decisions — something a computer can never do. We also want judges to rely on their professional expertise to decide cases on their merits, while maintaining efficiency, consistency, and coherency in their rulings. We expect them to faithfully adhere to statutory text when rendering their decisions. Decisions are respected and followed, in part, because we respect the individuals who issue them.

This is not to say that computers could not have an important impact on how appellate judges function in the future. It is more likely that a system like Watson could be a valuable tool for the appellate courts, functioning like a law clerk, and conducting basic research upon which judges can construct their opinions thereby assisting them to make their own decisions as effectively and efficiently as possible.14 To that end, it would be a resource no different than perhaps Lexis or Westlaw for providing direction and guidance in deciding outcomes.

Watson could bring the advantages of computer-based analysis to the court’s task of interpreting statutes using the ordinary meaning of language, all without sacrificing the normative discretion, which allows human judges to “get it right” in ways that computers cannot.15

Take a famous example: in Muscarello v. United States, the Supreme Court debated the meaning of the phrase “carries a firearm.” The majority argued that the ordinary meaning of carrying a gun included transporting it in a vehicle. The dissent disagreed, arguing that “carry” required holding a gun on one’s person. The two sides marshaled a vast array of evidence from the public domain to demonstrate that their interpretation was the most ordinary, including dictionaries, news articles, and even the Bible. Watson could have saved the Court’s law clerks a great deal of trouble. The computer would have been able to calculate how frequently the terms “carry” and “vehicle” (or their synonyms) appear together versus “carry” and “person” (or their synonyms). Thus, in at least one sense Watson is better at textualist interpretation than humans—he can not only identify ordinary meanings but can tell us just how ordinary a particular meaning is!16

Additionally, Watson-like technology is already being used by lawyers for help in sifting for information through voluminous documents obtained in discovery, and to help decipher patterns of activity.17 Therefore, its future use in the appellate courts is entirely foreseeable, especially for analyzing large quantities of documents provided to the court in records on appeal.

While a Watson-like program cannot make the type of decisions that we want and expect from judges, nor should it supplant the human element that is so vital to our justice system,18 it can serve to enhance a court’s judicial analysis in several important areas.19 Society should embrace the use of new technology, even when used as an adjunct to dispute resolution, with the recognition that we should always control the tools, rather than allowing the tools to control us. I, for one, have no desire to welcome our new computer overlords.20

1 See Richard Susskind, The End of Lawyers?: Rethinking the Nature of Legal Services (Oxford Univ. Press, 2008) and Tomorrow’s Lawyers: An Introduction to Your Future (Oxford Univ. Press, 2013) for discussions regarding technology as a disrupter in the legal market.

2 The program “Deep Blue,” won two games, lost one, with three draws. William Saletan, Chess Bump: The Triumphant Teamwork of Humans and Computers, Slate (May 11, 2007).

3 IBM, DeepQA Project: FAQ, http://researcher.ibm.com/researcher/view_project.php?id=2099. In information retrieval, an open domain question answering system aims at returning an answer in response to the user’s question. The returned answer is in the form of short texts rather than a list of relevant documents.

4 Watson was named after IBM’s first CEO and well-known business reformator, Thomas J. Watson, Sr.

5 Ken Jennings had the longest unbeaten run at 74 winning appearances, while Brad Rutter had earned the biggest prize pot with a total of $3.25 million. Between them, they had accumulated over $5 million in winnings.

6 Windows released its new natural language system, codenamed Cortana, in 2015.

7 Anthony D’Amato, Can/Should Computers Replace Judges?, 11 Georgia Law Review 1277-1301 (1977).

8 See Blackman, et al., FantasySCOTUS: Crowdsourcing a Prediction Market for the Supreme Court, Northwestern Journal of Technology & Intellectual Property, Vol. 10, 125 (2012) (for a discussion on FantasySCOTUS).

9 Id.

10 LexPredict, Fantasy Scotus, http://www.fantasyscotus.net/.

11 Blackman, et al., FantasySCOTUS: Crowdsourcing a Prediction Market for the Supreme Court, Northwestern Journal of Technology & Intellectual Property, Vol. 10, 125 (2012).

12 Id.

13 “Garbage in, garbage out,” meaning the computer’s output is only as reliable as the data it is given.

14 Jana Potová, Will people be judged by computers in future? (Dec. 12. 2011),
http://www.veda.muni.cz/en/2576-will-people-be-judged-by-computers-in-future#.UaPiG5VfyC8.

15 Betsy Cooper, Judges in Jeopardy!: Could IBM’s Watson Beat Courts at Their Own Game?, Yale Law Journal (Aug. 23, 2011), available at http://www.yalelawjournal.org/forum/judges-in-jeopardy-could-ibms-watson-beat-courts-at-their-own-game.

16 Id.

17 A company called Lex Machina in Menlo Park, CA, currently has a database with information from over 128,000 intellectual property cases that has been categorized, tagged, and coded. The company has used the database to analyze settlement patterns and win rates. The multinational law firm, Seyfarth Shaw LLP, collects data on the amount of time it takes to perform specific tasks to help price its legal services, drawing in part on data from TyMetrix, an e-billing and management vendor. The law firm has also collected information on U.S. Equal Employment Opportunity cases so that when a client has an EEOC charge, the firm can evaluate risk by looking at data on the EEOC investigator and type of claim. See also John Markoff, Armies of Expensive Lawyers, Replaced by Cheaper Software, NY Times, Mar. 4, 2001, available at http://www.nytimes.com/2011/03/05/science/05legal.html?_r=0.

18 For example, the right to trial by jury is a guarantee found in the Sixth and Seventh amendments to the U.S. Constitution.

19 Robert C. Weber, Why “Watson”Matters to Lawyers, National L. J. (Feb. 14, 2011).

20 At the end of the three-day Jeopardy! tournament pitting him and fellow champion Brad Rutter against the IBM supercomputer Watson, Ken Jennings gave the following subtext to his final Jeopardy answer after knowing he had already been beaten: “I, for one, welcome our new computer overlords.” This quip was a play on a memorable line from the 1977 film adaptation of H.G. Wells’ short story, Empire of the Ants. In the film, actress Joan Collins reacts to the insect threat by saying, “I, for one, welcome our new insect overlords.”

Judge Mark W. Klingensmith is on Florida’s Fourth District Court of Appeal.

This column is submitted on behalf of the Trial Lawyers Section, Courtney Kneece Grimm, chair, and Kimberly Ashby, editor.

Trial Lawyers