Generative AI in the modern lawyer’s toolbox
'Google has trained us to put in the least amount of information possible and expect magic. That’s not what we do with AI; it’s a conversation'
Used properly, generative AI can boost a lawyer’s productivity, profitability, and even “cognitive” reserves, says a national expert.
Indiana University Law Professor Josh Kubicki addressed scores of Florida Bar members last month in Orlando for a segment of the Board Technology Committee seminar, “Legal Intelligence: Navigating the Impact of AI on the Legal Practice.”
Lawyers should know that generative AI works much differently than a typical Google search, Kubicki said.
“Google has trained us to put in the least amount of information possible and expect magic,” he said. “That’s not what we do with AI; it’s a conversation.”
The conversation helps users learn the best prompts or how to ask the right questions to get the best results, he said.
Generative AI has the power to digest the facts of a case and produce a flawless brief or legal memo, but Kubicki warned that the results should always be reviewed.
“Is the first page of your Google search result 100% accurate?” Kubicki asked. “Would you lift those results and put them into a legal brief? No! These tools have been known to stretch the truth.”
Generative AI and the traditional Google search tool do share a common trait, Kubicki said.
Ask Google for the best restaurant in town and it will list the restaurant that paid Google the most to advertise, Kubicki said.
“There’s a whole other layer going on in the background that’s giving me my Google results,” he said.
Generative AI tools contain a “system prompt” that users also don’t see, he said.
“That’s where they’re controlling for things like accuracy, bias, and so-called hallucination, although I dislike that term,” he said.
When considering investing in a generative AI tool, lawyers would be wise to ask the creator how the system prompts are designed, he said.
“It’s one way for you as a consumer to lift the hood on these tools and engage in a meaningful conversation around how these models are trained and what their purpose is. When we do understand that we become better users.”
Studies show that generative AI is boosting law firm productivity, but some results can be misleading, Kubicki warned.
“Oh my gosh, AI truly is a game changer, it is,” he said. “But what the headlines don’t capture is that you have to know what to use these tools for.”
Kubicki said Microsoft studied 15 of its in-house lawyers who used the company’s AI tool for “realistic legal tasks.”
In addition to an overall 87% increase in productivity, the lawyers reported completing tasks 32% faster, and with 20% more accuracy.
Kubicki, however, doubts most lawyers who use the tool would see a 20% increase in accuracy.
“That’s where this is a little misleading,” he said. “Microsoft was using its own tool, and they knew how to apply it.”
The technology also poses a challenge for the profession, he said.
Before AI, beginning lawyers could expect to experience 117 “learning interactions” with a senior lawyer per week, including feedback, conversations with a peer, and markups on first drafts, Kubicki said. Adding greater use of generative AI is reducing the number of those interactions at a time when more lawyers are working from home.
“We’re just simply not filling the brains of new lawyers as much as we used to. We’re losing the surface area for them to learn and train on, and generative AI is accelerating that,” he said.
Lawyers who learn the best ways to employ generative AI can see more than a boost in productivity, Kubicki asserted.
Lawyers are “cognitive” athletes who “have the stamina to work with clients, deal with their personal situations, while maintaining a very highly tuned brain.”
That makes them susceptible to the “taxing nature of content switching,” or moving from task to task in rapid succession – dealing with a secretary, a paralegal, an email, a phone call, writing a memo, or responding to opposing counsel.
Generative AI is helping attorneys power through the challenge, Kubicki said.
“I’m not going to say generative AI is the panacea to correct that,” he said. “What I am hearing, through individual conversations, is evidence that, once lawyers start using these tools, our ability to maintain our cognitive reserve is getting stronger.”
