Quorum: Cost per Correct Decision

A five-phase build of an eval-governed control plane for visual AI decisions — routing easy calls to a cheap detector, ambiguous ones to a vision-language model, and the genuinely uncertain to a human queue, with every decision priced and recorded. Built with an AI agent pair, documented in Confluence ADRs throughout, and completed for £0 of external spend against a £10 cap.

The premise

I built an AI system on the worst detector I could find. Deliberately.

The vertical is recycling-stream contamination: a materials recovery facility runs thousands of items past a camera every hour, and a single battery in a paper bale is a fire risk. The obvious pitch is "train a better model." I did the opposite — I took a free, hosted, visibly weak object detector and spent all five build phases on the system around it: a confidence gate, an escalation queue, an eval harness, a drift monitor, and a cost ledger.

The bet: for most operational AI, the model is not the product. The product is knowing when to trust the model, what each decision costs, and who should decide instead. By the end, the system had measured its own components failing — and routed around them. That's the whole story, and every number below comes out of the system's own records.

The shape of the system

Every image becomes a decision row that walks a state machine. A four-rule gate reads the detector's output and picks one of three lanes:

Three design rules did most of the work:

1. Thresholds are data, never code. Every gate parameter lives in a versioned database row; every decision pins the version that made it. Tuning writes a new row. When routing behaviour changes, the audit trail says exactly when and why.
2. The escalation model never gets to guess. The VLM answers each question twice, independently. Agreement decides; disagreement goes to a person, and no machine verdict is recorded at all.
3. Silent drops are forbidden. Anything that fails permanently lands in a dead-letter state and gets counted. Accuracy is computed over attempted decisions, not convenient ones.

The eval that called my bluff

Phase 3 built a 184-item golden set from the TACO litter dataset and replayed it through the full pipeline. The headline numbers were bad — accuracy 0.68, F1 0.58, routing recall 0.22 — and the interesting part was why. The detector's mAP@0.5 came out at effectively zero, which smelled like a bug in my metric code. It wasn't:

The detector finds objects almost perfectly and names them almost randomly. 423 of 514 predicted boxes sat on a real object; 3 carried the right label out of 59 classes. That single measurement explains everything downstream: a gate keyed on class labels under-escalates because the labels are noise, and 44 real contaminants sailed through marked clean — including a battery pile confidently labelled "Other plastic, 0.96".

A composite accuracy score would have hidden this completely. Splitting localization from classification — and splitting routing correctness from verdict correctness — is what turned a bad number into a diagnosis.

Watching the world shift

Phase 4 added the monitoring layer: CLIP embeddings for every replayed image, per-class centroids from a "training era" window, and a drift alarm on the mean distance of recent decisions from those centroids. Then I deliberately broke the world — sixty familiar TACO images, followed by forty from a different dataset shot under different conditions, salted with photos of the one class held out of everything: batteries.

The alarm fired cleanly — baseline windows sat at 0.134, shifted windows jumped to ~0.51 against a 0.35 threshold. And on one battery seed, the zero-shot fallback did exactly what it was built for: detector blind, embedding far from every known class, CLIP text-match says "battery" — a class the system had never been configured to detect, flagged with no retraining.

The honest miss: three other battery photos never triggered the fallback, because the detector confidently misidentified them, and the trigger rule requires the detector to be unsure. Confident-but-wrong is the one gap the current rule can't see — every replayed decision now records its embedding distance so that "confident but far from anything known" is one dashboard filter away.

The dial that turned out to be flat

Phase 5 built the operator dashboard: a threshold dial, economics inputs (£ per rejected bale, £ per reviewer-hour, £ per thousand VLM calls), and a cost-quality frontier simulated by re-gating the entire golden set — from cache, so sweeping 30 candidate configurations costs zero external calls. Here is that sweep, on real data:

Thirty threshold configurations, and they all land in a smudge two points wide. The dial was flat — and that flatness was the most valuable output of the whole project. The frontier simulation prices escalation using the measured quality of the escalation path, and the local 7B vision model I was escalating to had scored 0.44 on adjudications — worse than a coin flip. Escalating more items to a coin flip buys nothing, at any threshold. The instrument didn't just fail to find a better setting; it proved where the next unit of investment had to go — a better adjudicator, not better thresholds. That's the go/no-go evidence for the paid-model upgrade, produced by the harness instead of by opinion.

Failure paths, exercised for real

The parts of the system I'm most attached to are the ones that fired without permission:

• A decision dead-lettered live during the first full adjudication run. Root cause: the local model server rejects large images — a 1.5 MB photo alone consumed more context than the default window. The decision retried twice with backoff, landed in the dead-letter state, stayed countable, and the fix (a bigger context window, recorded as a cache-key parameter so old outputs can't silently replay) is one line of config history.
• The judge got audited before being believed. An LLM scores the adjudicator's written rationales — but its scores are labelled uncalibrated in every report until they agree with human scoring at κ ≥ 0.6 on a blind sample. A judge you haven't calibrated is a vibes generator.
• The adjudicator contradicted itself in writing. One rationale described cigarette butts in detail and then concluded "clean." It's stored verbatim in the ledger — that sentence is now a test case for the next rubric revision.

What I'd tell another PM

1. Buy measurement before capability. The £10 model budget went unspent because the eval harness kept proving the bottleneck wasn't where I assumed. Every pound you spend before you can measure is a guess.
2. Split your metrics until they confess. Accuracy hid everything. Localization vs classification, routing vs verdict, measured vs modelled — each split converted a grade into a cause.
3. Config is data with a history. Versioned thresholds cost one join and bought a complete answer to "why did the system decide that, that day?" — which is also roughly what incoming AI regulation asks for.
4. Design the failure paths first. Dead letters, retry budgets, stale-claim reaping, and no-guessing rules were all built before they were needed. All of them fired.
5. An honest bad number beats a flattering composite. The flat frontier and the 0.44 adjudicator are the most persuasive artifacts this project produced. Nothing sells a measurement system like watching it catch your own assumptions.

Stack: Python, FastAPI, SQLAlchemy/SQLite (WAL), LangGraph, Roboflow-hosted TACO detector, Qwen2.5-VL via Ollama, CLIP ViT-B/32, Llama 3.1 as judge, Streamlit. 271 tests. Total external spend: £0.00 of a £10 cap. The explainer video above was rendered programmatically — Pillow frames, ffmpeg, and a local neural-TTS voiceover, with scene timing derived from the narration audio.