A field guide is one thing. Running code is another. This week we built the public→hybrid→private deployment path as two live systems on our own infrastructure — partly because we use them, mostly because "trust me, private AI works" is not an argument. Here's both, with numbers.
A router that refuses to leak
The first system is a model router: text comes in, gets classified for sensitivity, and is answered by either a frontier API or a model on hardware we control — never the wrong one.
The classification layer is deliberately unglamorous: deterministic pattern matching for IBANs, phone numbers, card numbers, ID-shaped strings. An LLM assists only on ambiguous text, and it is only allowed to escalate to sensitive — never to downgrade. If the rules find hard PII, no model gets asked for a second opinion, because you can't ask a cloud model whether text is too sensitive to send to a cloud model.
Three behaviors, all verified against the live system:
- Public text routes to Claude and comes back answered.
- Sensitive text routes to a 7B model running on our own machine, reached over an encrypted private network. Round trip: about a second once warm.
- Sensitive text with no private endpoint configured is refused — in about 1.03 seconds, with zero external calls made. Early on, before our private endpoint was live, this was the behavior in production. We'd argue it's the most important feature of the three: the failure mode of a privacy system should be silence, not leakage.
Every request leaves an audit record — route taken, reason, matched pattern types, and a hash of the input. Not the input itself. An audit trail that stores the sensitive text it was protecting is a second copy of the problem.
The claim I care most about: for PII requests, the public-API node never executed. That's not inferred from the response — we pulled the execution graphs from the database and checked which nodes ran. Structural proof beats promises.
RAG without the vector-database tax
The second system is a retrieval-augmented assistant over our own site content — same canvas, two modes.
The contrarian design choice: no vector database. Our corpus is 25 pages, 32 chunks. Embeddings live in the workflow's own storage; retrieval is a cosine-similarity loop that runs in 1–3 milliseconds. For corpora under a few thousand chunks, vector infrastructure is a cost center cosplaying as architecture. You can add it when the corpus earns it — that's the whole deployment-path thesis in miniature.
The behaviors, again live-verified:
- Grounded questions come back with citations to the actual pages, in 3–8 seconds.
- Out-of-corpus questions get an honest "I don't know from the corpus" — in 55 milliseconds, at zero cost, because refusing early means never invoking a model to hallucinate confidently.
- We baited it with a question about a price that isn't in the corpus. It declined to invent one. That test should be routine for any RAG deployment; most demos would fail it.
- Private mode answers from an 8B model with locally computed embeddings, on our hardware. Nothing leaves the network. Same workflow, one field changed.
Public mode costs about $0.0065 per question. Private mode's marginal cost is electricity.
What the exercise taught us
Model choice is a per-workload decision, not an ideology. Our first private model refused a perfectly benign business summarization on "safety" grounds — we swapped it for a different 7B model that behaves, one configuration line, no re-architecture. That's the practical meaning of hybrid: the freedom to move any single workload without moving your worldview.
If you're weighing where your own workloads belong, the workload placement tool is the two-minute version of this analysis, and the audit is the real one. Both are run by people who have done this to themselves first.