Keep ChatGPT out of court, warns Pretoria law expert after AI blunder

AI is transforming the legal sector, but a University of Pretoria law lecturer has warned that its unchecked use could threaten ethical standards, fairness and access to justice.

Rather than replacing human judgement, AI should complement it.
Rather than replacing human judgement, AI should complement it. (Shutterstock.png)

Artificial intelligence (AI) is transforming the legal sector, but a University of Pretoria (UP) law lecturer has warned that its unchecked use could threaten ethical standards, fairness and access to justice.

The caution comes after the Mavundla v MEC: Department of Co-Operative Government and Traditional Affairs KwaZulu-Natal and Others case, where AI was used to generate legal citations, some of which turned out to be false.

“The adoption of AI in legal settings introduces a range of ethical concerns,” said Lucinda Kok, a lecturer in the department of private law at UP.

“AI-powered legal tools process vast amounts of data, often including sensitive personal information. Without robust security measures and strict compliance with data protection regulations, there is a heightened risk of breaches that could compromise the legal rights of individuals.”

Kok emphasised that AI tools trained on historical legal data may reinforce existing prejudices in the system.

“If left unchecked, these biases could perpetuate systemic inequities rather than promote fairness,” she said. “Ensuring AI impartiality requires diverse and representative training data, ongoing audits and transparent AI development practices.”

Kok also pointed out the importance of explainable and transparent AI systems, saying: “AI tools should be designed with clear, explainable mechanisms that allow legal practitioners and the public to understand how decisions are reached.

“This fosters trust in AI applications and ensures accountability in legal proceedings.”

Another major concern is the digital divide. While AI tools can enhance access to justice, people in disadvantaged or rural areas risk being left behind due to limited digital literacy and infrastructure.

“To bridge this digital divide, policymakers and legal institutions must prioritise inclusivity and digital equity,” said Kok. 

“This includes investing in accessible legal technology, expanding digital literacy initiatives, and developing AI-driven tools that accommodate users with varying levels of technological proficiency.”

Despite these risks, Kok acknowledged AI's potential to improve the legal process from automating admin tasks to streamlining case analysis and providing cost-effective legal assistance.

However, she still maintained that AI must not replace human judgment. “Legal reasoning requires critical thinking, ethical deliberation, and contextual understanding elements that AI cannot fully replicate,” she said.

“Ultimately, the success of AI in law depends on a balanced approach that prioritises innovation while preserving fundamental legal and constitutional principles.”

TimesLIVE



Would you like to comment on this article?
Sign up (it's quick and free) or sign in now.

Comment icon

Related Articles