Artificial Intelligence Is Not Ready for the Courtroom

Since June 2023, more than 120 instances of court filings containing fake information attributed to AI have been identified. And the incidence of these cases is accelerating, said the researcher, Damien Charlotin.

“We have not seen any threatened litigation from the lawyers and firms we insure,” said Joseph Barnard, head of FTJ’s ProDefender business unit. “But it’s something lawyers need to be aware of and consider before using AI.”

Barnard said ProDefender’s lawyers professional liability policies would respond to claims arising out of the use of AI.  “There’s no exclusionary language in policies for that – at least not so far,” he said.

 A study by Stanford University found that general AI tools such as ChatGPT provide fictitious information – referred to as “hallucinations” – between 58% and 82% of the time on legal queries.

Specialized tools such as Lexis+ AI and Ask Practical Law AI also produced incorrect information in one out of six queries, while Westlaw’s AI assisted research hallucinated more than a third of the time, according to Stanford’s research.

Stanford found that even new legal tools that use retrieval-augmented generation (RAG) – purported to be the solution to AI errors in legal documentation – make mistakes.

“At this point, lawyers probably ought to think twice before using AI,” Barnard said. “Are they really saving time if they have to double-check every argument, example, fact, and citation the AI generates to assure its accuracy?”

This experience in the legal realm should serve as a warning to professionals in other disciplines that relying on artificial intelligence for work product may lead to negative consequences.

To learn more about professional liability insurance and related insurance protection for law firms, use the link below to visit our ProDefender web site.