Authored by Paul Cartwright and Yuveshen Naidoo.
A Johannesburg high court judgment has highlighted a growing risk in the use of generative AI: legal authorities that look convincing but are wrong. In a recent appeal, the court identified multiple incorrect and apparently fictitious case citations in the lower court judgment and stressed the importance of verifying authorities before relying on them. The decision is a reminder that AI can assist with research and drafting, but it cannot replace professional judgment and independent verification.
The dispute itself concerned divorced parents who disagreed about whether their children should continue attending a private school. The full court upheld an order allowing the children to remain at their existing school pending further appeal proceedings, finding that their best interests justified that result.
What attracted wider attention, however, was a separate judgment dealing with a number of citation errors contained in the acting judge’s lower court decision. The appeal court recorded that a series of discrepancies had been identified. These included incorrect citations, passages attributed to cases that did not contain them, and references to authorities that did not appear to support the propositions for which they were used. The court also referred to an apparently non-existent authority.
The court did not make a finding that artificial intelligence had been used. However, the judge observed that some of the errors were consistent with what are commonly known as AI hallucinations, where an AI system confidently produces information that appears authoritative but is inaccurate or fabricated. The matter has been referred to the Legal Practice Council for further consideration.
The significance of the judgment extends beyond the legal profession. Many businesses are increasingly using generative AI to prepare reports, analyse information, summarise documents and draft communications. While these tools can improve efficiency, the judgment illustrates the danger of treating AI outputs as reliable without checking the underlying sources and accuracy. A fabricated citation in a legal document is obvious evidence of the risk, but the same problem can arise in board papers, compliance reports, technical assessments and regulatory submissions.
The lesson is not that AI should be avoided. Rather, it should be used as a tool that assists human decision-making. Where accuracy matters, especially in legal, regulatory and professional environments, every authority, quotation and factual assertion should be verified against the original source and personal judgment should be exercised before it is relied upon. The responsibility for the final product remains with the person who signs it off or uses it.
F.J.L v T.G.O (2025/220239) [2026] ZAGPJHC 875 (31 July 2026)