Short answer. Parse the contract, extract it against a 20-clause taxonomy, require a quote, page and character span for every finding, validate the quotes, then route high-risk clauses to a lawyer. Measure recall on 50 gold contracts. This is not legal advice; counsel review is required.
What to extract: the 20-clause taxonomy
The taxonomy is the product. A generic "find risky clauses" prompt cannot be evaluated. A fixed list gives comparable results and a consumable schema. Each clause needs a playbook position; version the taxonomy.
| # | Clause | Extraction question | Risk flag |
|---|---|---|---|
| 1 | Parties and capacity | Who is bound, and in what capacity? | Entity mismatch, missing affiliate |
| 2 | Definitions and precedence | Which document wins on conflict? | Order-of-precedence gap |
| 3 | Scope and deliverables | What is included and excluded? | Open-ended scope |
| 4 | Acceptance | How is acceptance deemed or refused? | Silent acceptance |
| 5 | Fees and taxes | What is payable, when, and excluding which taxes? | Uncapped pass-through |
| 6 | Payment terms | Net terms, late interest and suspension rights? | Long terms, penalty interest |
| 7 | Term and renewal | Initial term and auto-renewal notice? | Evergreen renewal |
| 8 | Termination for convenience | Who can exit, on what notice, with what fee? | Asymmetric exit |
| 9 | Termination for cause | Cure period and material breach definition? | Short cure, broad breach |
| 10 | Confidentiality | What is confidential, for how long, with which exceptions? | Perpetual or one-way |
| 11 | Data protection | Roles, DPA, subprocessors, breach notice, transfer mechanism? | Missing DPA or notice |
| 12 | Intellectual property | Who owns deliverables, background IP and feedback? | Broad assignment |
| 13 | Warranties | What is warranted, and what is disclaimed? | Unlimited warranty |
| 14 | Indemnities | Who defends whom, for what, and with what control? | Uncapped IP indemnity |
| 15 | Limitation of liability | Cap, exclusions and carve-outs? | No cap or broad carve-outs |
| 16 | Insurance | Coverage types, limits and evidence? | Limit below contract value |
| 17 | Force majeure | Which events, notice and termination right? | Too broad or too narrow |
| 18 | Assignment | Consent required, change-of-control trigger? | Free assignment |
| 19 | Governing law and disputes | Law, venue, arbitration seat and language? | Unfamiliar forum |
| 20 | Boilerplate | Notices, entire agreement, amendments, severability, counterparts? | Missing notice address |
Output schema with evidence spans
Ask for one object per contract, not prose. Each clause should say whether it was found, summarise it, assign a risk level and cite the exact text. A finding without a quote is not reviewable and should be treated as missing.
{
"contract_id": "MSA-2026-0042",
"taxonomy_version": "2026-11-01",
"governing_law": "England and Wales",
"clauses": [
{
"type": "limitation_of_liability",
"present": true,
"summary": "Liability capped at 12 months of fees, with carve-outs for IP and confidentiality.",
"risk": "medium",
"playbook_position": "acceptable_with_review",
"evidence": {
"page": 7,
"start_char": 18422,
"end_char": 18710,
"quote": "In no event shall either party's aggregate liability exceed the fees paid in the twelve (12) months preceding the claim."
},
"confidence": 0.86
}
],
"missing_clauses": ["insurance"],
"human_review_required": true
}
Validate three things: the quote appears in the parsed text after whitespace normalisation, the offsets fall inside the document, and every high-risk clause has a quote. Reject paraphrased quotes and retry with a narrower page slice.
Pipeline and code
Use a parser that preserves page and section boundaries. Losing page numbers makes evidence unverifiable. Send a bounded slice per request, not the whole 80-page agreement, and merge results by clause type.
import json
from openai import OpenAI
client = OpenAI(api_key="PLUGSKY_KEY", base_url="https://api.plugsky.com/v1")
SYSTEM = """You extract contract clauses. Return JSON only.
For each clause in the taxonomy, set present, summary, risk (low|medium|high),
and evidence {page, start_char, end_char, quote}. If not found, present=false
and evidence=null. Never invent a quote."""
resp = client.chat.completions.create(
model="plugsky-chat",
response_format={"type": "json_object"},
messages=[
{"role": "system", "content": SYSTEM},
{"role": "user", "content": json.dumps({
"taxonomy": TAXONOMY,
"page": page_number,
"text": page_text,
})},
],
)
result = json.loads(resp.choices[0].message.content)
assert all(c["evidence"] is None or c["evidence"]["quote"] in page_text
for c in result["clauses"])
Redline workflow with human review
For each playbook deviation, retrieve the approved fallback and ask for a minimal edit. A lawyer reviews the diff and exports tracked changes.
- Extract: clause, risk, evidence and playbook position.
- Propose: one fallback per deviation, with a rationale tied to the playbook.
- Validate: the proposal must keep defined terms, cross-references and numbers intact.
- Review: a qualified lawyer approves, edits or rejects; no auto-send.
- Log: model, prompt version, taxonomy version, reviewer and timestamp.
The 50-contract eval
Build a gold set of 50 executed contracts across five types: NDA, MSA, SaaS, professional services and software licence. Ten per type exposes taxonomy gaps. Two reviewers label each independently, then adjudicate disagreements with counsel.
Measure clause presence F1, evidence exact match, high-risk precision and recall, missing-clause recall and review time. Set thresholds first; 95% evidence match and 90% high-risk recall are reasonable bars, but risk tolerance decides.
Report by contract type. A model strong on NDAs and weak on liability caps can look fine in aggregate and fail where it matters. Re-run after every change.
Privilege, confidentiality and residency
Contracts contain commercial terms, personal data and sometimes privileged advice. Sending them to a third-party model can be a disclosure, and privilege rules vary by jurisdiction. A vendor's confidentiality clause does not decide whether privilege survives.
Before processing privileged material, counsel should approve the vendor, data flows, retention and access. Prefer no training on customer data, a defined retention window and an audit log. For sensitive matters, use a private deployment in the client's jurisdiction.
Cost per 100 contracts
Assume 12,000 input and 1,500 output tokens per contract, two calls each: about 1.2 million input and 150,000 output tokens for 100 contracts. Plugsky Hobby is $8 per month with 2 million tokens per day, so the batch fits one day.
Longer contracts need page splits and more calls. Starter is $20 with 5 million tokens per day; Builder $60 and Scale $120 have no monthly cap, subject to fair use. Reviewer time usually costs more than tokens.
FAQ
Can an LLM review contracts reliably?
It can extract clauses, flag deviations and draft fallback language, but it should not give legal advice or send a redline without a lawyer. Reliability depends on the taxonomy and verifiable evidence spans.
How many clauses should the taxonomy have?
Twenty is a practical starting point for commercial contracts: parties, scope, fees, term, termination, confidentiality, data protection, IP, warranties, indemnities, liability, insurance, force majeure, assignment, governing law and boilerplate. Add clauses only when your playbook has a position for them.
What is an evidence span?
An evidence span is the exact page, character range and quote supporting a finding. The quote must appear in the source. If the model cannot produce one, mark the clause as not found rather than guessed.
Does using an API waive privilege?
It can, depending on jurisdiction and the terms of the engagement. Privilege is fact-specific and is not guaranteed by a vendor contract. Counsel should approve the workflow, the vendor, the retention settings and who can access the outputs before privileged material is processed.
What does it cost to review 100 contracts?
At roughly 12,000 input and 1,500 output tokens per contract, 100 contracts use about 1.2 million input and 150,000 output tokens. That fits inside Plugsky Hobby at $8 per month if the daily request and token caps are respected; heavier volumes move to Starter at $20 or Builder at $60.
Reviewed 20 November 2026. Contract workflow design is editorial. Sources checked 20 November 2026: confidentiality and privilege rules against ABA Model Rule 1.6; structured output patterns against the OpenAI structured outputs guide; risk management against the NIST AI RMF. This page is not legal advice; counsel review is required before production use.