AI ETHICS offices and solo practitioners lack not only access to enterprise-level AI systems, but also the resources and expertise to challenge software outputs (including AI-generated outputs) as unreliable in court. 9 This disparity raises urgent questions about competence and fairness, but also requires defenders to proactively engage with the AI tools that are available as a potential equalizer. The proliferation of generative AI among the legal profession, generally, has been controversial and landed many lawyers in trouble with the court. There are countless recent examples of prosecutors and defenders misusing AI in briefing and in the courtroom, but two recent examples highlight the ethical concerns at the heart of this white paper. The New York Times recently published a story about a District Attorney in California who allegedly used AI in an 11-page brief asking the court to deny bail to the accused and keep him in jail, a brief allegedly “rife with errors that bear the hallmarks of generative artificial intelligence” and including “wholesale misrepresentations of the law, as well as quotations that do not actually appear in the cited texts.” 10 And on the other side of the bench, a defender included several misrepresentations of law in a brief before the Illinois Supreme Court, something that was a source of tense questioning during the defender’s oral argument. 11 When one of the justices challenged the defender regarding the inaccurate case propositions, the defender said, “I can only tell you that we rely on the Lexis, the flags and so forth, and my associate was assisting me, so I will have to have a long chat.” 12 Just minutes before, the defender admitted to using X’s chatbot Grok to learn more about certain expert testimony requirements, one of the key issues to be reviewed by the Illinois Supreme Court on appeal. 13 These two examples underscore two serious concerns with overreliance on AI to be discussed in this white paper. First, AI often “hallucinates,” or invents, fabricates, exaggerates, or misstates cases, rulings, and even facts. 14 According to researcher Damien Charlotin, there are at least 500 cases in the United States, identified as of January 6, 2026, where generative AI produced hallucinated content through fabricated, falsely quoted, or misrepresented case law, and a court or tribunal specifically levied warnings or other punishments. 15 Second, in the Smith case mentioned above and in countless other cases, lawyers who have been caught using hallucinated case law often blame the mistake on a lack of supervision of younger or more junior associates. As discussed later in the white paper, both prosecutors and defenders are on notice that AI hallucinates, courts can and will sanction the misuse of AI in briefings and in the courtroom, and blaming a junior associate will not relieve an attorney of their ethical obligations or any punishment for violating them. Beyond the misuse of AI in the courtroom, there are access-to-justice concerns about AI, too. As Professor Drew Simshaw has explained, although legal AI has “the potential to increase the efficiency and cost effectiveness of work done by lawyers” and “help people solve their own legal problems or connect them with licensed legal professionals who can,” overreliance on and legitimization of technology-driven legal services, “will lead to one or more inequitable two-tiered systems.” 16 AI will either be superior, cost more, and only be available to large firms and wealthy clients, or the inverse
10
Parity in Practice: The Defender’s Duty to Ethically Use AI
Made with FlippingBook Online document maker