How to Evaluate Legal AI Tools

Despite what scare-tactic news has to say about robots, aka artificial intelligence (AI), stealing all of our human jobs, we know that AI-powered legal tools will not replace legal professionals (read more about why here.) The truth is that these solutions bring efficiency, freeing up your time for more complex and strategic work. Now that sounds… niiiice. 

The key to the kingdom is separating the real and valuable AI tools from the over-hyped ones. In this blog post, we’ll outline easy techniques for evaluating an AI solution for those of us without any formal education in AI. Let’s get started. 

What is AI?

Definitions of AI can get very technical, but a digestible and accurate definition is software that learns and improves over time without those improvements being explicitly programmed by humans. When software programs itself, real AI "magic" happens. 

AI magic means you can develop software to do tasks you may not specifically program under normal circumstances. For example, you can train AI to recognize common clauses in contracts. Then once it's trained, imagine you can ask the AI to do that kind of analysis at the speed and scale of modern software. That’s the type of benefit legal teams can’t afford to miss out on. 

How Do You Know AI Is Real?

You can tell if an AI solution is legit by the task it’s trying to perform and the data on which it was trained. AI-powered software is good at three general types of tasks.

• AI can classify data using a complex set of criteria. AI sorts through a lot of information and determines which items qualify under particular categories. 
• AI can automate tasks that stem from classification. For example, Gmail can recognize when someone emails you an invitation and can automatically turn that email into an appointment on your Google Calendar. 
• AI can predict outcomes based on past data. AI is fair at prediction but isn't necessarily better than humans who try to forecast the future without a crystal ball – AI just does it much faster.

To perform these tasks accurately, AI needs good, well-annotated training data. To evaluate whether an AI solution can do what it promises, ask yourself whether the AI claims to do more than the tasks above. Then ask yourself if the AI developer had access to good data to train the AI in whatever task it claims to do well. If they can't deliver or can't account for where they got their training data, odds are, something is off. 

Beware: If their answer to the training data question is "we'll just use your in-house data to train the AI," be very skeptical. Vendors have no idea how good your data is, so their promises are unlikely to be true. For a legitimate explanation, read this blog post about how we develop our AI here at LinkSquares.  

AI for Legal 

Legal teams are positioned to benefit from AI because the legal profession has spent hundreds of years documenting case law and sample contracts. This means that legitimate legal AI developers have access to a gold mine of training data. 

Plus, common legal tasks are great for AI-powered solutions because they involve repetitive work that AI can learn how to do. And once it learns, the robots can do that work more efficiently. AI serves as a welcome relief to the boring work that eats away at your time. The classification work that legal teams spend too much time on can be automated by AI, giving you hours (or even days) back in the week for more strategic work. 

Artificial intelligence isn't magic, but it’ll make a magical impact on your legal team – as long as you identify a solution that delivers on its promises. If your legal team is ready to harness the power of AI, contact LinkSquares today.