Private AI in law

A practitioner's guide to running legal AI inside your own perimeter — written for the partner who has to approve it, the COLP who has to defend it, and the IT lead who has to run it. No vendor sponsorship, no product placement, and a clear line between what we know and what we think.

SECTION 01The state of play, September 2026

Three things happened in 2026 that changed this decision for UK firms, and they happened close together.

First, the vendors moved. In August 2026 Thomson Reuters announced a proprietary legal model rather than continuing to build purely on third-party foundation models — a signal that the largest players in legal AI now treat model ownership as part of the confidentiality answer.¹ Whether or not you buy that product, the strategic point is unavoidable: if ownership of the model matters to the biggest vendor in the market, it should at least be a question in your own build-or-buy decision.

Second, the regulator spoke about supervision rather than technology. The SRA issued a warning notice on the misuse of AI in legal practice on 17 August 2026,² and the themes reported around it are consistent with what we find in audits: outputs that were trusted without verification, supervision that was assumed rather than evidenced, and files that do not show what anyone checked. None of that is fixed by a better model. Some of it is fixed by architecture — because a system that logs what it retrieved and what a human accepted produces the record as a by-product.

Third, "AI" became a client question. Enterprise clients, public bodies and insurers now include AI governance in diligence: what tools process their data, where it rests, who reviewed the output, and whether anyone independent has tested any of it. For firms with those clients, the AI register has joined the AML review as a document you are expected to produce.

SECTION 02The confidentiality problem, stated properly

Most conversations about AI confidentiality are really conversations about contracts. A vendor's data-processing agreement, enterprise tier or zero-retention option limits what the supplier may do with your content. That is valuable. It is not the same as your professional obligation, and the difference matters in three specific ways.

Transfer is not the same as use

Client content leaves your systems the moment it is sent to a third-party model for processing. Whether the recipient may train on it, retain it, or read it is a separate question. Your duty of confidentiality is about the disclosure, not only about what happens afterwards — and a client who discovers their matter documents were processed on infrastructure they did not agree to is unimpressed by a clause that permitted it.

Privilege is a narrower door than confidentiality

Privileged material is a subset of confidential material, and the consequences of mishandling it are correspondingly worse. Most practical AI risk in firms concerns privilege rather than confidentiality: privileged drafts, advice notes and litigation strategy pasted into tools that were never assessed for that category of material. Any private deployment should be designed to keep privileged material inside the perimeter by default, not by user discipline.

Your insurer asks architectural questions

When a firm tells us its AI pilot stalled, the reason is usually a risk conversation that could not be closed: where does the data rest, who can access it, what is logged, who at the firm is accountable, and what happens when it goes wrong. Contracts help with two of those five. Architecture answers all five, which is why the private route often exists for governance reasons before it exists for performance reasons.

SECTION 03Three deployment archetypes

Almost every option in the market is a variation on one of three patterns. The distinctions that matter are who controls the infrastructure, where retrieval happens, and who carries the maintenance burden.

Note what is not on the list: a fourth option in which everything is solved by a good policy. We see firms attempt it every month. It works until someone is tired.

SECTION 04What private actually costs

Numbers below are indicative UK ranges for 2026, covering infrastructure and licences only — not our fees, not your staff time. Treat them as a shape, not a quote, and insist on a written model for your own firm before committing.

SECTION 05Choosing a model

You are choosing between families of open-weight models rather than a single product, and the differences that matter in legal work are not the ones leaderboards measure. Four practical tests:

• Instruction-following under long context. Legal tasks arrive as a question plus several thousand tokens of precedent or correspondence. A model that degrades at 30,000 tokens is useless for matter-level work however clever it is at trivia.
• Citation behaviour. Does it quote the retrieved text faithfully, or paraphrase it into something that sounds authoritative and is wrong? Test this with retrieved passages that contain a deliberate trap.
• Refusal discipline. A model that invents a definitive answer where the sources are silent is a supervision problem. You want "the retrieved documents do not answer this" to be a common output.
• Licence and provenance. What does the licence permit commercially, what are the use restrictions, and can you keep running the weights if the publisher changes direction? Read the licence, not the blog post.

Model size is a negotiation with your hardware. Two useful rules of thumb: a mid-size open-weight model with good retrieval outperforms an enormous one without it, for document-grounded legal work; and quantisation (running weights at lower precision) buys you capacity at a small cost in quality that is usually worth paying in a firm of your size. Both claims should be tested on your tasks before you take our word for anything.

SECTION 06Retrieval: where projects fail

A private model with no access to your documents is an articulate stranger. Value comes from retrieval — and retrieval is where confidentiality gets broken by well-meaning engineering, because the fastest way to make a demo work is to index everything.

Permissions must be inherited, not reimplemented

Your document management system already knows who may see what: matter walls, ethical walls, restricted clients, departed-fee-earner access. Any retrieval layer must inherit those rules at query time rather than copying documents into a parallel store with its own access model. If a vendor cannot explain how permissions are enforced at query time, that is your answer.

Scope deliberately

Start with a defined corpus — a practice group's precedents, a set of published know-how, one matter type — rather than the whole DMS. Narrow scope makes evaluation possible and failure survivable. Widen it only after the evaluation set passes.

Retrieve passages, not vibes

Every answer used in client work should be traceable to the documents it came from, with the passages shown. This is not a nicety; it is the mechanism by which a supervisor can check the work in two minutes instead of re-doing it, and it is what makes the output defensible when a file note says an AI-assisted draft was reviewed.

SECTION 07Evaluation on your own matters

Vendor benchmarks are marketing. The only evaluation that means anything is one built from tasks your fee-earners recognise, scored by people who know the right answer. Ours looks like this, and it takes about three weeks to assemble properly.

1. Assemble 50–150 tasks across the work you intend to support: summarise a bundle, extract terms from an agreement, draft a first-pass letter, find the clause that governs a scenario, answer a question from the file.
2. Record the correct answer for each, from an experienced fee-earner, before the model sees anything.
3. Include traps. At least ten tasks where the retrieved material does not contain the answer, to see whether the model says so or invents one. This is the single most informative test we run.
4. Score three ways: correct, correct-with-caveats, or wrong — and separately record whether the output showed its sources well enough to check.
5. Re-run after every change to model, prompt or retrieval. A regression set is what turns a pilot into a managed capability.

Two things usually happen. First, retrieval quality dominates model choice, which surprises people who expected the opposite. Second, the failure rate on citation-heavy tasks is higher than anyone assumed — which is exactly the information the SRA's warning notice implies firms should have before they deploy.

SECTION 08Governance that survives questions

A private deployment does not exempt you from the paperwork; it changes what the paperwork can say. Four documents do the heavy lifting.

On standards: ISO/IEC 42001 is the AI management-system standard that maps most naturally onto what a law firm already does for AML — policy, risk assessment, controls, evidence, review cycle.³ Firms pursuing certification should note that the certifying body must be independent of the consultant who prepared them,⁴ which is why we prepare and never certify. On EU exposure: if you act for EU clients or process data subject to the EU AI Act, obligations for certain higher-risk uses phase in through 2026–2027 and the surrounding guidance has itself been in flux;⁵ take specific advice on your fact pattern rather than a general summary like this one.

SECTION 09Training fee-earners differently

A private model does not behave like the consumer tool your staff use on their phones. It is usually more literal, more dependent on the retrieved context, and — this is the important part — sometimes confidently wrong in a different accent. Three training adjustments we make for private deployments:

• Teach the retrieval boundary. "It can only see what is in scope" is the single most useful thing a user can know, because it explains both the failures and the fix (point it at the right documents).
• Teach verification against sources, not vibes. Users check outputs they can trace; they wave through outputs that merely sound right. If the tool shows sources, teach people to click them — every time, including when busy.
• Teach the file note. What to record when AI assisted: the tool, the task, what was checked, who checked it. One sentence. Firms that train this sentence have fewer findings in every audit we run.

SECTION 10The 90-day rollout plan

The point of the plan is week 13. A deployment that cannot be reviewed honestly at week 13 becomes a fixture nobody owns, which is how firms end up with an AI capability that no one can describe to an insurer two years later.

SECTION 11Failure modes we keep seeing

1. Buying a model instead of building a capability. Weights are the cheap part; retrieval, evaluation and logging are the work.
2. Indexing everything to make the demo impressive. The fastest route to a confidentiality incident in the category.
3. No evaluation set. Then every opinion about quality is anecdote, and nobody can tell whether an update made things worse.
4. Service-account access. The tool sees more than the user should, and no one notices until an audit.
5. Training delivered once, to a room, in launch week. Behaviour reverts within a fortnight; the second session matters more than the first.
6. Governance deferred to "phase two". Phase two does not arrive, and the DPIA is written retrospectively in the week a client asks.
7. Logging switched off for performance. Then the evidential trail — the reason for going private — is missing.
8. No internal owner. The deployment is a project until someone is named in a job description.
9. Cost model with no maintenance line. Year two arrives with an update nobody budgeted for.
10. Scope creep without re-evaluation. New corpus, new practice area, same scores from six months ago.

SECTION 12Total cost, worked

Two illustrative models. Both exclude our fees and assume you already run a document management system with a usable API. Treat them as arithmetic to argue with, not a quotation.

The comparison that decides most cases is not private-versus-public on cost. It is private-versus-doing-nothing on the work you currently cannot automate. If the confidential half of your practice is excluded from AI because of where the data would go, that exclusion is the cost you are already paying.

SECTION 13Glossary

SECTION 14Sources and further reading

1. Thomson Reuters announced a proprietary legal model in August 2026 — reported by SiliconANGLE, “Thomson Reuters launches proprietary AI model for legal work”. See also Thomson Reuters' Future of Professionals — Legal and their 2026 practitioner survey commentary.
2. Solicitors Regulation Authority warning notice on the misuse of AI in legal practice, 17 August 2026 — see the SRA's own guidance at sra.org.uk and the summary at Herbert Smith Freehills Kramer. We have not reproduced its text; read the notice itself.
3. ISO/IEC 42001:2023 — information technology, artificial intelligence, management system. Published by ISO/IEC; see iso.org.
4. ISO/IEC 17021-1 — requirements for bodies providing audit and certification of management systems, including independence from consultancy.
5. EU AI Act implementation has been subject to amendment and guidance during 2026; for the current position see this overview of the AI Omnibus and take advice specific to your firm.
6. ICO guidance on AI and data protection: ico.org.uk. Money Laundering Regulations 2017 (as amended): legislation.gov.uk.

How to cite this guide: Probative Co, Private AI in law: the 2026 deployment guide, September 2026 edition. Share it internally; attribution appreciated, permission not required. This is general information, not legal advice. Probative Co is not a law firm and does not provide reserved legal services. Where this guide touches your own regulatory position, take advice on your facts.