Filling in the blanks Pt. II - The difference between a cloze passage and a legal argument (cont.)
Your honour, I'm blanking out...
My friends, I will be the first to confess that writing today’s piece was a little bit of a challenge. Now that I have completed the week’s work, I can now turn my mind fully to the parcel that awaits me at home: The Legend of Zelda - Tears of the Kingdom. Pre-ordered months ago, and now finally at my doorstep. It still feels so very far away. But I digress.
At the end of last week’s piece, I ended by concluding that some forms of legal argumentation involve some form of blank-filling. I then asked whether there are forms of legal argumentation which do not involve any blank-filling. Since I asked the question, I have come around to the view that all forms of legal argumentation involve blank-filling.
This is linked to my view that the latest developments in large language models give enough cause for worry1. A big part of that is because I have been thinking about complexity. In other words, what it means to have something which whole is greater than the sum of its parts. It goes to the very core of how we think about the blanks to be filled.
We will start first with the intuition that many, if not all, white-collar tasks - and the legal domain is no exception - is to some extent blank-filling.
Which begs the next question: what are these blanks? Are they simply elements to fill in a framework? Can the frameworks themselves be blanks? How big are these blanks?
Let’s turn to courtroom argumentation. At the risk of over-simplification, many more routine arguments will turn on whether the framework set out by statue (legislation passed by Parliament) or case law (law developed incrementally on a case-by-case basis by judges, as novel issues appear before them) has been fulfilled. What are the elements? These are the elements; here are your blanks. I want you to argue whether or not they have been filled. From this angle, one is simply filling in the blanks.
From another angle, how you argue whether these blanks have been filled is also another set of blanks to be filled. So one is really talking about blanks to be filled at differing levels of abstraction, or different levels of building blocks so to speak2.
A more visual example of this is probably best explained using Lego! I recently completed LEGO’s The Starry Night set with my wife (and some assistance from my friend who works at LEGO). When we were putting together the individual bricks, it was difficuly to imagine where the building was headed. The bricks were also packaged in several numbered packs. Each led to the completion of certain larger elements. For example, the breeze in the middle of a night sky was actually made up of several such elements.
I think a lot of legal argumentation is similar.
Let’s consider appeal judgments. Very briefly, appeal judgments can do several things. These include:
Setting out frameworks of blanks to be filled e.g. going back to our earlier example, what goes into a harm/culpability framework?
Setting out what constitutes those blanks e.g. what is harm?
Filling in the blanks at the penumbra of statute and case law
Rectifying instances where it is clear how blanks should be filled in, but where blanks have been filled in correctly (in lower courts for example). These could be cases where lower courts have incorrectly applied previous appeal court judgments, or cases where appeal courts have never ruled
Finally, and most controversially, where blanks have not been filled in incorrectly, but the most fundamental argument is that times have changed. Whether the courts are the right forum to explore such questions have been, and always will be a controversial question. At this point, it is only apposite to point out that arguments, moral or political, about how things should be, is blank-filling at some level of abstraction as well.
At this point, one might of course argue that characterising everything as some sort of blank-filling at some level of abstraction is too quick, too reductive, too simple to capture what is going on. This might be true. But the performance of the latest language models shows that they are learning to fill in blanks at increasingly higher levels of abstraction. A classic test, as told by the godfather of AI, Geoffrey Hinton, is whether a large language model can explain what humour is. GPT4 was successful at explaining his particular joke. If you have met me before, you know I sometimes struggle to explain why something is so funny. Quite apart from that, humour is actually a very sophisticated concept, that requires a huge amount of rich and complicated contextual information. But large language models have picked up on this, and far better than many human children and youths, who have not accumulated enough contextual information.
Looking further afield, large language models have been trained on plenty of examples of how people fill in blanks at differing levels of abstraction. The results speak for themselves.
Where does this leave young associates, or new entrants to the legal sector? I think there is a strong argument that law school training emphasises critical thinking and analysis skills by asking many of the same questions that appellate judgements may ask and answer. When confronted with the reality of legal practice, which 90% of the time does not involve such thinking processes, a young associate may experience no shortage of cognitive dissonance.
Practice is at the same time more elementary and routine blank-filling, and at the same time, more demanding in requiring new ways of blank-filling, and of new blanks - but using the same first principles. Without quite the same demands on writing and lengthier discourse as appellate submissions and judgments involve.
What does this mean for younger associates, and for the legal profession at large?
At a very general level, I think it means that legal professionals must view legal careers less as set milestones to be met, with regular blanks in experience to be filled.
Looking at problems and asking again how problems should be solved anew, or more effectively, tends to reveal new blanks. These blanks might have been lurking the entire time, awaiting comprehension. Or they might simply be new demands. We need better ways of identifying them. Because there are problems facing the profession at large as well.
And in what I promise will be the last meta-turn, I think this will enable the profession to better solve our clients’ problems, and better serve society.
We should err more on the side of invoking the precautionary principle, than on the side of charging ahead with progress (even as I use DALL-E for the picture below).
In many discussions about how AI will never be able to replace lawyers, much ink has been spilled about the ingenuity, novelty, creativity, and nuance that is required to fashion arguments that are incisive enough to advocate a client’s case. The answer usually is that AI is nowhere near there, and will not be there for decades to come. The recent wave of large language models changes that timeline. My discussion of blank-filling is one way of explaining why I think that timeline has changed, and why LLMs may be more capable of these traits than we had thought just 6 months ago.