Lawyers are already using ChatGPT for research, drafting and summarising. The tool is fast, and it also invents case law.
This post covers what ChatGPT can do in legal research, what it has already cost lawyers who did not check its work, and the confidentiality and accountability rules that now apply to Australian practitioners.
The Role of ChatGPT in Legal Research
ChatGPT answers legal research questions in plain English and holds a back-and-forth exchange, so you can narrow a question across several turns instead of rebuilding a search query each time. That is faster than reading through case law and precedent by hand.
The speed comes with one specific failure. ChatGPT produces citations that look correct and do not exist, including case names, reporter references and quoted passages. Check every citation against the primary source before you rely on it.
Its answers are also only as good as its training data. The Supreme Court of New South Wales makes the same point in Practice Note SC Gen 23, warning that a Gen AI program’s underlying data may be incomplete, out of date or not relevant to New South Wales or Australia.
Ethical Considerations for Legal Professionals
Using ChatGPT in practice raises ethical questions that start with the fabrication problem. Fabricated authority in a filed document is a professional failure, not a software bug.
In Mata v Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023), two New York lawyers filed a brief citing and quoting six judicial decisions that ChatGPT had invented. On 22 June 2023 Judge P. Kevin Castel fined the two lawyers and their firm US$5,000 and ordered them to write to their own client and to every judge falsely named as the author of a fabricated opinion.
The case set the terms of the debate that followed: whether the time AI saves is worth the errors it introduces, and who carries the cost when a client’s matter is the one that goes wrong.
AI-generated advice is also impersonal. Clients still expect advice from a person who is accountable for it.
The conversation around the ethical use of AI in law extends beyond the accuracy of its outputs. It encompasses issues of accountability: who bears responsibility when AI tools provide faulty advice or information?
Australian courts have started answering that question in writing. Practice Note SC Gen 23 commenced on 3 February 2025 and applies to all Supreme Court of New South Wales proceedings. Gen AI must not be used to generate the content of affidavits, witness statements or character references. Those documents must now carry a disclosure that it was not.
AI’s Potential to Transform Legal Practice
Those limits do not remove the upside. ChatGPT can take a first pass at legal research and document summarisation, which frees lawyers for the parts of a matter that need judgement. Practice Note SC Gen 23 permits Gen AI for chronologies, indexes, witness lists, briefs and the summarising or review of documents and transcripts. Lower time on that work means lower cost to the client.
AI also reads large volumes of material quickly, which surfaces patterns and precedents a manual review can miss. That gives you more to work with when you build an argument.
Getting there asks the profession to change how it works. Lawyers have to rethink how they run research and case management, and decide which steps a person must still perform. That change takes work, and it is where the time savings come from.
Navigating Confidentiality and Compliance Challenges
Confidentiality and data security are the other half of the problem. Australian practices answer to the Privacy Act 1988 (Cth) and the Australian Privacy Principles, and to the GDPR where they act for clients in the European Union. Before you put client material into an AI tool, confirm what the provider does with it.
Practice Note SC Gen 23 draws a hard line here. Material under a non-publication or suppression order, material covered by the Harman undertaking and material produced on subpoena must not be entered into a Gen AI program unless the practitioner is satisfied it stays inside the platform, is used only for that proceeding, and is not used to train the model.
The same test is a good default for any client material. Check whether the provider trains on your inputs, where the data is stored and who can reach it.
Cloud-based tools also raise data sovereignty. Client material processed offshore sits under another country’s law, so check where a provider stores and processes your data before you send anything sensitive to it.
Looking Ahead: The Future of AI in Law
ChatGPT can make legal work faster and cheaper, and it can put fabricated authority in front of a judge. Both are true at once, and the rules now in force treat them that way.
Where Gen AI has been used to prepare written submissions, Practice Note SC Gen 23 requires the author to verify in the body of the submissions that every citation and legislative reference exists, is accurate and is relevant. That verification must not be done using a Gen AI tool alone.
That is the working rule for everything else too. Use ChatGPT for first drafts, chronologies and summaries of material you can check, and verify every citation against the primary source yourself.