Artificial intelligence in law: valuable tool or risky replacement?

Artificial intelligence (AI) has rapidly established itself within the legal sector and is frequently presented as a transformative technology. It offers the promise of greater efficiency, improved accessibility and reduced costs. It can produce drafts within seconds, condense lengthy legal materials and, at first sight, appears to make legal information more readily available to those who do not have professional representation.

However, behind the polished language and confident responses lies a more complex reality. AI can produce content that resembles legal advice and often appears convincing. What it cannot do is exercise professional judgment. The courts are increasingly seeing the consequences of parties relying on AI-generated material without appreciating that distinction.

The attraction of “instant law”

It is easy to understand why AI-generated legal assistance has become so appealing to businesses and private individuals alike. Faced with escalating legal costs and increasingly complicated disputes, many people are tempted to seek answers from a chatbot rather than a solicitor.

This trend is particularly noticeable among litigants in person, who may use AI platforms to draft court documents, research legal authorities or formulate legal arguments. On one hand, this can be seen as expanding access to legal knowledge. On the other, it creates a significant risk: placing trust in a system that does not understand legal principles but instead predicts what a legal answer might look like.

As the Law Society has recognised, generative AI presents considerable opportunities, but the risks associated with its use have not yet been fully explored or understood.

When AI gets it wrong: from error to sanction

The principal concern with AI is not merely that it can make mistakes. Rather, it is that those mistakes are often presented in a manner that appears entirely credible and authoritative.

Courts across various jurisdictions have already encountered cases involving AI-generated legal material that was wholly fictitious, including fabricated case names, citations and judicial reasoning. In Harber v HMRC [2023], a UK tribunal considered a case in which a self-represented litigant relied upon authorities which, despite appearing genuine, simply did not exist.

Likewise, in a civil matter before the courts in Manchester, a litigant-in-person relied upon four authorities generated by ChatGPT. Upon examination, one of the cited cases had been entirely invented, while the remaining authorities bore no resemblance to the propositions they were said to support.

These examples are far from isolated. Across the world, courts have identified numerous cases involving AI-generated inaccuracies, including the submission of fabricated citations and authorities.

The repercussions can be serious. In the United States, legal professionals have faced fines and sanctions for relying upon AI-generated content without carrying out appropriate verification, with courts criticising submissions containing significant citation errors. More recently, a federal court took the unusual step of pausing proceedings, removing legal representatives from the record and imposing financial penalties after inaccurate AI-generated material was filed on behalf of both parties.

The judiciary in England and Wales has also made its position clear. The High Court has emphasised that parties remain responsible for the accuracy of any material placed before the court, regardless of whether it originated from an AI platform. In serious cases, reliance upon false authorities may amount to contempt of court and could potentially give rise to criminal consequences.

The hidden dangers for litigants in person

The risks associated with AI are particularly pronounced for litigants in person.

Unlike qualified legal professionals, litigants in person are not trained to scrutinise legal sources, evaluate evidential weight or ensure procedural compliance. AI often appears to bridge that gap by providing answers that are well-structured, persuasive and apparently authoritative. The problem is that those answers may be incomplete, inaccurate or entirely fabricated.

There are a number of potential consequences of this.

• Misleading the court, even unintentionally. Reliance upon incorrect authorities or flawed legal arguments can damage a party’s credibility from the outset.
• Procedural errors. AI may overlook important nuances within the Civil Procedure Rules, leading to defective pleadings, missed deadlines or procedural non-compliance.
• Adverse costs consequences. Even where a party is unrepresented, reliance on inaccurate material can increase costs, delay proceedings and result in sanctions.
• Damage to an otherwise strong claim or defence. Litigation outcomes often depend not only on the underlying merits of a case but also on how those merits are presented and argued.
• Creation of potentially disclosable material. Information entered into AI platforms does not necessarily benefit from privilege in the same way as communications between solicitor and client. Depending on the circumstances, AI chat histories may be disclosable, and documents uploaded to AI systems may lose protections which would otherwise have applied.

Research indicates that a significant proportion of AI-related errors identified in court filings originate from self-represented parties. Although courts may show some understanding towards litigants in person, that sympathy does not prevent the practical consequences of time being wasted, credibility being undermined and opportunities being lost.

Regulation, responsibility and the limits of technology

The regulatory position remains straightforward: AI is a tool, not a replacement for professional judgment.

Guidance issued by both the Law Society and The General Council of the Bar (Bar Council) makes clear that legal professionals must review and verify any AI-generated content, maintain appropriate oversight and exercise independent judgment throughout. Blind reliance on AI carries the risk of falling below the standards expected of competent practitioners.

More broadly, the UK’s approach to AI regulation is built upon principles of safety, accountability and transparency. Those principles are particularly important within a legal setting.

Fundamentally, the issue is simple; AI has no ability to determine whether the information it generates is true or false. It cannot distinguish between binding legal authority and persuasive commentary. It does not understand context, nuance or judicial discretion. What it produces is persuasive language, rather than genuine legal reasoning.

A recent example encountered by our team illustrates the point. An AI-generated analysis relating to a property ownership dispute incorrectly advised that a party had no viable claim because the relevant agreement had not been recorded in writing. In reaching that conclusion, the AI entirely failed to consider the possible existence of a common intention constructive trust, which may arise even where an agreement concerning ownership was made orally.

The example demonstrates a significant limitation of AI: answers can sound entirely convincing while being fundamentally incorrect.

Why legal representation still matters

The growing body of cases concerning AI misuse highlights a recurring theme. The issue is rarely the technology itself. Problems tend to arise when AI-generated output is accepted without proper scrutiny or oversight.

A solicitor’s role extends well beyond producing documents or identifying legal authorities. Effective legal advice requires a practitioner to assess the merits of a claim, evaluate evidence, identify procedural risks, develop litigation strategy and apply legal principles to a client’s specific circumstances.

Crucially, lawyers are trained to recognise nuance. Legal disputes seldom turn on broad principles alone; outcomes frequently depend upon facts, context and strategic considerations that cannot easily be reduced to a simple prompt.

By comparison, AI can recreate the appearance of legal advice without providing the expertise, judgment and accountability that underpin it.

For those involved in litigation, the greatest risk is often not an obviously wrong answer. It is the false confidence created by an answer that appears entirely plausible until it is examined more closely by the court.

Proceed with caution

There can be little doubt that AI will continue to influence the future development of legal services. Its advantages are genuine and its potential is significant. Equally, its limitations should not be overlooked.

For litigants in person, the temptation to treat AI as a substitute for professional advice is understandable. In many cases, however, doing so may prove to be a false economy, resulting in greater expense, delay and difficulty in the long term.

Ultimately, AI is best viewed as a helpful tool rather than a replacement for expert advice. Informed legal judgment remains at the heart of effective legal representation and, at least for the foreseeable future, that remains something artificial intelligence cannot provide.

The Birketts view

At Birketts, we recognise the opportunities that innovation and emerging technologies can bring to the legal profession. The responsible use of AI can improve efficiency and enhance client service. However, technology must operate alongside rigorous legal analysis and professional oversight.

We believe that technology should support legal practice rather than replace legal judgment. Our specialists work closely with clients to:

• navigate disputes strategically and effectively
• ensure legal arguments and authorities are properly verified
• avoid the costly consequences that can arise from AI-generated inaccuracies
• achieve practical solutions informed by both legal expertise and commercial understanding.

As technology continues to evolve, the importance of high-quality legal advice becomes even more valuable.