With the rage that it has quickly become, what can we learn about ChatGPT and the legal profession? CoinDCX in-house counsel Shalini Saxena takes a look down the rabbithole
Artificial intelligence tools cannot yet replace a human’s ability to create connections with clients. Law businesses still need people to empathise, engage, personalise and individualise service for each client.
But ChatGPT systems and its successors may be capable of handling some of the behind-scenes work for law firms, like drafting individual clauses for contracts.
At its heart, ChatGPT is a chatbot; a conversational tool that performs routine tasks efficiently. For earlier examples, think of Apple’s Siri or Amazon’s Alexa virtual assistants. ChatGPT is a step further, notable for its artificial learning capabilities and ability to have human-like conversations.
ChatGPT launched in November 2022, and by February 2023, India had “Lexi”, the country’s first AI powered by ChatGPT, which has been integrated with analytical tools offering business solutions. India’s Ministry of Electronics and Information Technology is, apparently, planning to integrate ChatGPT with WhatsApp to help Indian farmers understand and learn about several government schemes.
ChatGPT recently passed law exams in four courses at universities in the US, according to professors at these schools. After completing 95 multiple choice questions and 12 essay questions, the chatbot performed on average at the level of a C+ student, achieving a low but passing grade in all four courses.
At the Wharton School of the University of Pennsylvania, ChatGPT fared even better, earning a B to B- grade on a five-question experiment. In a paper detailing the performance, a Wharton business professor said ChatGPT “did an amazing job” at answering basic operations management and process-analysis questions, but struggled with more advanced prompts and made “surprising mistakes” with basic math. “These mistakes can be massive in magnitude,” he wrote.
On the flip side to these results, India has joined other countries in banning ChatGPT in universities.
So, there are concerns alongside the excitement, and we all need to start thinking how to leverage this new technology while discouraging misuse.
Legal profession and ChatGPT
To quote Professor Kristian Hammond of the Northwestern University (US) and director of the Center for Advancing Safety of Machine Intelligence who spoke from a panel hosted by the university on 10 January 2023: “You have to understand the length and breadth of the technology and where it collapses, and make sure the task is not one that demands something beyond its limits. ChatGPT might be good at taking a test. But, because of the nature of the underlying mechanism, it may never be capable of genuine reasoning, being imaginative, or thinking beyond the moment.”
Hammond predicts a new “prompt engineer” role will emerge in the short term to improve the system and refine for specificity. “Users who understand enough about a domain will engineer the appropriate prompts to guide the system in the right direction, and the prompts will become the learning driver for the next generation,” he says.
And it all makes sense. In what we can predict of the near future, AI tools cannot make lawyers obsolete, but they will be helpmates that we will all have to learn to build into our working style. This extends beyond the legal fraternity; any function will need to question how to add value in a world where the simpler tasks will be taken away by machines.
ChatGPT-level AI tools are a much-needed solution in multiple areas. For instance, they can:
- Instantly search vast amounts of data and provide relevant results, saving time on tedious tasks like researching case law, finding legal documents, or verifying facts;
- Prepare early-stage documents;
- Automate tasks like document retrieval, summarisation and analysis; and
- Generate blog posts and other content quickly creation.
Let’s look at some useful opportunities in the legal industry, where some drawbacks are also obvious.
Clause creation. The ChatGPT tool is limited in the amount of text it can generate. The longer the prompt, the less reliable the results. It would seem that ChatGPT is not well suited to drafting an entire contract. However, it is perfect for drafting individual clauses in accordance with specific instructions.
This capability will allow legal teams to create error-free clauses in minutes, provided a human lawyer properly indicates the requirements beforehand and checks the end result.
Document search. Data currency is a basic necessity. However, ChatGPT is updated only up to 2021. This means that, as far as ChatGPT is concerned, everything that happened in 2022 never happened. And we are now in 2023.
Let’s talk about how much data is created in the world every day. The current estimate stands at 1.145 trillion MB per day.
If we talk data available in courts in India alone, a good place to start is the National Judicial Data Grid (NJDG). This is an online database of orders, judgments and case details of 18,735 district and subordinate courts and high courts. Litigants can access case status information on more than 22 billion cases and 20 billion orders and judgments. To track cases related to land disputes, land records data of 26 states have also been linked with NJDG.
This is just the tip of the iceberg. Every day, data is being generated across the globe. How will ChatGPT stay relevant and perform the way the creators intend?
Compliance to data law and some other softer matters. It is not surprising that ChatGPT itself seems to be unable to answer this question conclusively and, in fact, recommends that a legal professional be consulted on specific concerns about the GDPR compliance of ChatGPT.
In a recent move, PwC introduced a chatbot service for its lawyers. This bot is meant to help speed up work on due diligence and regulatory compliance, but will ultimately move to broader legal advisory and legal consulting services.
Other professional legal services firms also seem to be moving towards using AI to expedite tasks. For example, Allen & Overy recently announced a chatbot to help lawyers draft contracts and client memos.
These are real cases and many more seem likely to follow. The results will help us both to understand and better leverage AI tools.
Will AI tools make lawyers redundant? This author doubts it.
Today there is ChatGPT; tomorrow something else. We live in an extremely dynamic world where every day there is a new skill set, a new disruptor. This teaches us not to be complacent with our existing skill sets but to be constantly focusing on adapting to these evolutions and these new technologies.
Technological tools can be a friend rather than an enemy for any profession. Humans will be irreplaceable and hence professionals will always exist. AI cannot replace the human ability, for example, to create connections with the client. The bottom line:
- What we can do as legal professionals is be attentive to new technologies and know how this can help in the execution of our tasks.
- What we cannot do is reject technology on the basis of believing that a machine would be unable to match up to the experience derived from our careers.
The ChatGPT rage
Before we jump to conclusions let’s understand what we are dealing with
ChatGPT is a natural language processing tool driven by (AI) technology that allows a person to have human-like conversations and much more with a chatbot. The language model can answer questions and assist you with tasks such as composing emails, essays and code.
Potential immediate uses include:
- Customer service, like answering common questions and providing information;
- Being a virtual assistant, helping users to manage their schedules;
- Chatbot creation – ChatGPT can create chatbots for websites, messaging apps and other platforms; and
- Content generation for websites, social media posts and other written material.
It is clear there is no single source of truth for ChatGPT. It gets its information from a large variety of sources that include Wikipedia, books, news articles and scientific journals that are fed to it. As per news articles, this dataset is only up until 2021, meaning that it lacks information on subsequent events.
While initially ChatGPT did not have internet access to glean information from the web, it is now possible to connect ChatGPT to the internet and access more up-to-date information. Basically, it can search and answer. Answers will be based on ChatGPT’s instilled training.
While ChatGPT’s responses seem human-like, it does not have human experience and human common sense.
Neither does it possess true emotional intelligence, despite responses that appear empathetic. It cannot detect subtle emotional cues or respond appropriately to complex emotional situations. It may struggle to grasp the subtle nuances of human communication, which in turn may lead to providing a response that is inappropriate or irrelevant.
ChatGPT is specifically programed not to provide harmful responses. It will avoid answering all such questions. The model performs best when it’s given a single task or objective to focus on. If you ask ChatGPT to perform multiple tasks at once, it will struggle to prioritise them, which will lead to a decrease in effectiveness and accuracy.
There is a cost. Someday we will need to pay for use. Any specific use cases will need fine-tuning the model, which can be time-consuming and resource-intensive. Being complex to operate efficiently, it may require access to specialised hardware and software systems.
Shalini Saxena is the head of legal at crypto firm CoinDCX.