The tempest of naming legal AI after TV characters
Or why the most recent firewind shouldn't raise an eyebrow
Hi all,
Interrupting the programme of posts I’d scheduled to post a quick one about a recent firewind you might have seen brewing on Reddit, but more probably on LinkedIn. This is for both lawyers, and everyone else.
While I have my reservations about a startup sharing the same name as a very good-looking but highly unethical lawyer1, the firewind has my sympathies. Starting a company is difficult. Running one even more so. The court of public opinion is often not a kind one, and seldom a fair one. While there is a certain wisdom to crowds, it’s sometimes difficult to tell when a group is instead a mob.
The best thing we can do is to move on, and to keep plugging away at the real problems. The enterprise play has its role, but there is also everyone else.
To navigate the AI storm, you need to be clear about the difference between tools and services. Bear with me as I work through this: the questions matter more than quick answers.
There are only two major themes to be observed about the whole episode.
I say this as someone building tools for legal. I don’t really care for thought leadership at this particular point in time. It’s more important to me that the legal sector keeps apace with what is going on. In the best case, we at Elefant have a product that actually adds value2, and that we are able to market effectively. In the worst case, we will have personal toolkits that serve us and enhance the personal edges that we have, while bulwarking our weak spots.
First, AI literacy is of paramount importance. I would never have guessed that learning epistemology3 before college would one day prove this useful. But it’s allowed me to avoid asking embarrassing questions, but also allowed me to maintain a beginner’s mindset, to learn as much as possible, as quickly as possible.
Second, only with AI literacy does the distinction between a tool and service become clear. When we say that we are benchmarking AI, or evaluating it4, to borrow an analogy from woodworking, are we assessing the tool, the user of the tool, or the company of a user of tools, the specific collection of processes and practices of the company, or the whole industry of complementary services and tools (tool-maker, tool-user, complementary tool-maker, etc.)? We should assess what we pay for, but there’s a risk that we use the wrong benchmarks.
Why does this matter?
Let me put it another way using an analogy that I am very acquainted with, but that you would also understand: even if I managed to procure a chisel made by the best chisel-maker in the world, would I, an amateur, be able to make use of it? You may as well hand a sniper rifle to a blind person.
There are many questions implicit here:
How do I judge the skill of the tool-maker?
How do I judge the skill of the tool-user?
Should I judge it in tandem with the quality of other tools?
Should I judge it in tandem with the user’s skill with other tools?
Should the final product of usage of all these tools affect my judgment of these tools and/or the tool user?
Should I judge services that are complementary to the product as being essential the final judgment e.g. provision of credit, ideal payment terms, the opportunity to schmooze and be schmoozed, delivery, etc.
Should I judge it the same way I judge lawyers?
Should clients now judge me the same way I judge LLMs being applied to legal?
These are not questions that admit easy answers. The only thing that I know for sure is that time will tell.
A lot of evaluations, in my opinion, are targeted at a collection of services, that markets itself as a great tool. Where is the actual intelligent tool within this byzantine machine?
There are also other questions, like who the great machine services.
With Elefant, we’re focusing on the long-term, and I think there are serious questions to be asked about who we are pricing in, and more importantly, out. Who we are ultimately going to be serving, and at what cost and price. Legal AI cannot just exist for enterprise-level customers, because the law applies to everyone. It cannot be otherwise if the law is meant to prevent people from harming each other, but also to help them flourish.
You might have noticed a slight transition there. I focussed on the law, rather than focusing on preventing AI from harming, and encouraging its capacity to aid human flourishing. That’s because I think, at base, AI is a tool. Well, at least until we reach general artificial intelligence (GAI). I treat my tools well whether or not they’re GAI. I think of AI like any other tool in my life. I love cooking, and my kitchen knives, pans, and other accoutrements are all bought with a view to lasting decades.
They are tools that work well when they are treated well, and when they are used well. I intend the same for my use of AI, of our use of AI.
I hope you follow along in our journey.
When I was preparing for university, I made my first suit in Hong Kong, at Cheung Hing Tailor. It’s another story how I came to know of them, but they made some of my favourite suits. Probably a fair amount of nostalgia involved. When making that first suit, I showed them a picture of Harvey Specter looking fantastic in a suit. I had a build closer to Harvey than to Mike, so I told myself. The tailor, in very colourful cantonese, said “Sure, I’ll make you a C-E-O!” I later realised Suits was more like a 101 in how to get disbarred - fast.
This suit story captures exactly what’s happening with legal AI - there’s a chance many are trying to look like Harvey Specter without understanding what actually makes a good lawyer.
We believe in do your homework, and do it well. We prefer to one day let the work speak for itself.
Theory of Knowledge - which explores such questions as how we know what we know, what we know about what we know, and other related questions - was a compulsory subject on the International Baccalaureate. It addresses why lawyers started off thinking GPT was automagic. And it was a pre-university subject: not even worthy of tertiary, or professional accreditation. In Singapore’s system, it was even seen as a level up from the General Paper, which examined critical thinking of current affairs. It’s weird that we have pre-university students grapple with questions that legal professionals rarely turn their minds to.
I think these are different things. Creating a benchmark is like creating a bar exam. Evaluations are more frequent, and less monolithic, but absolutely essential to guaranteeing day to day performance. But should you benchmark the tool, the tool-user, or the company of tool users, which will invariably provide a bevy of of related services?