Can AI Draft Your Contracts? What the Benchmarking Really Says

Written by

mromney

in

,

Recent benchmarking data from legalbenchmarks.ai has turned heads recently. According to their study, top AI tools now produce reliable first drafts of standard contracts at rates that meet or exceed the performance of experienced human lawyers working without AI assistance. The best AI tools in the study scored 73.3% on a first-draft reliability metric, compared to 56.7% for a benchmark group of in-house commercial lawyers with an average of ten years of experience. The tasks were primarily junior-to-mid-level in complexity, and “reliability” measured whether the draft met minimum standards for compliance with instructions, factual accuracy, and legal adequacy. Even more surprising, the study claimed that specialized legal AI tools raised risk warnings in 83% of outputs about compliance of those outputs with applicable law, whereas human lawyers raised zero warnings for the same tasks.

At first glance, those numbers are shocking. They seem to directly contradict the current consensus about the use of AI in specialized tasks, including legal drafting. But do they mean that AI can now replace your lawyer? The short answer–no, that could be a costly mistake. But they do mean that the value proposition of legal services is shifting — and if you’re making decisions about how to allocate legal spend at a technology company, understanding that shift will help you buy legal services more effectively.

What’s Actually Changing

The drafting layer — the mechanical production of a first-pass document — has crossed a speed and baseline-quality threshold for standardized agreements like NDAs, template SaaS subscriptions, and routine vendor contracts. AI has reached the point that it can handle the first drafts of these documents in seconds with reasonable accuracy, a task that used to take a junior attorney up to an hour using templates, clause libraries, and examples from prior deals.

But it is critical to keep in mind what hasn’t changed. Even if you used the best performing AI tool in the study, 1 in 4 of your contracts would have critical deficiencies. Human-in-the-loop is still the best way to minimize risk exposure in your contracting.

It is also critical to remember that when you retain outside counsel, you’re not just buying text. You’re buying malpractice coverage that backstops the work product. You’re buying market-standard consistency — the assurance that your customer agreement looks like what sophisticated counterparties expect, not like something generated from a training corpus with no awareness of current market norms. You’re buying accountability: someone whose professional license is on the line and whose name is on the engagement letter. And you’re buying judgment — the ability to tell you which redlines matter and which don’t, based on having seen hundreds of similar deals.

How This Should Affect Your Legal Spend

The practical implication is not that legal services should cost less across the board. It’s that the composition of what you’re paying for should be shifting, and you should expect transparency about that shift.

For routine, high-volume work — standard NDAs, template-driven agreements, recurring amendments — ask your firms how AI is integrated into their drafting workflows. If the first draft takes seconds instead of hours, the engagement model should reflect that. Flat fees, capped arrangements, or subscription structures often make more sense than hourly billing for work where the drafting itself is no longer the bottleneck.

But also be realistic about the cost structure. Ethics rules require lawyers to verify AI outputs against primary sources. Even the best retrieval-augmented AI systems still produce hallucination rates between 17% and 33% in legal research contexts, and professional responsibility obligations are non-delegable. That means firms are now paying for enterprise AI tooling and for the associates who verify, refine, and take responsibility for the output. Whether that nets out to savings for the client depends on workflow maturity — for well-tuned playbooks on routine agreements, the savings are probably real. For anything more complex, it’s genuinely unclear.

The right question isn’t “why isn’t this cheaper?” It’s “what am I paying for, and does the workflow match?”

Where to Focus Your Attention

Invest in playbooks with negotiation logic. The real efficiency unlock isn’t faster drafting — it’s faster decision-making on counterparty redlines. A well-built playbook defines your fallback positions, acceptable variations, and escalation triggers so that routine deviations can be processed against predetermined risk tolerances rather than requiring a judgment call on every markup. AI flags the deviations; the playbook tells your team what to do about them. Reserve outside counsel for the points that genuinely require strategic judgment.

Rethink your in-house / outside counsel split around complexity, not volume. AI-augmented processes can handle standardized contract workflows with minimal outside counsel involvement — provided someone experienced built and maintains the playbook. Outside counsel earns their fee on mid-complexity negotiations where cross-document consistency, counterparty leverage, and commercial context drive the outcome, and on high-stakes transactions where the precision rate of AI tools drops meaningfully.

Ask about AI governance, not just AI capability. What tools does the firm use, and are they enterprise-grade, closed systems that protect your confidential information? What are the verification protocols for AI-generated output? How are prompts and outputs retained, and what’s the discovery exposure? Privilege may not attach to AI interactions the way it does to lawyer-client communications, meaning the workflow itself can create risk if not properly managed.

The Bottom Line

The technology has crossed a meaningful threshold for routine contract work, and that should change how you structure and price certain legal engagements. But the value of experienced counsel — accountability, market awareness, strategic judgment, and the ability to see around corners in a negotiation — hasn’t diminished. If anything, it’s becoming easier to distinguish from the commodity drafting work that AI now handles well.

The executives who navigate this transition best will be the ones who stop paying premium rates for work AI does reliably, start investing in the playbooks and processes that make AI-assisted workflows actually work, and continue to value the human judgment that no model has come close to replicating.

For further reading:

More posts