Structured-output verification patterns for production LLM pipelines

Every provider now ships schema-constrained decoding, and every provider’s documentation quietly overpromises what it buys you. Constrained decoding guarantees the output parses. It does not guarantee the output is true, grounded, or internally consistent. The gap between those guarantees is where production incidents live.

This piece documents the verification stack we run on a document-extraction pipeline processing roughly 40K documents daily against frontier-model APIs, with the failure rates each layer catches. Numbers are from June–July 2026 production traffic; your distribution will differ — measure it.

The failure taxonomy schema validation can’t see

Schema-valid failures cluster into four families:

1. Type-valid hallucination. The model returns "invoice_total": 4820.00 — a perfectly typed float that appears nowhere in the document. Constrained decoding increases the risk at the margin: when the model is uncertain, the grammar forbids it from expressing uncertainty outside the schema.
2. Unit and format drift. Dates parsed day-first from US documents, totals in cents where the schema assumes dollars, percentages as 0.15 vs 15. Type-correct, silently wrong by 100×.
3. Cross-field contradiction. Line items that don’t sum to the stated total; an end date before a start date; a currency: EUR beside a $-prefixed source span.
4. Enum coercion. Forced to choose from ["invoice", "receipt", "credit_note"], the model files a purchase order as an invoice rather than failing. Closed enums without an escape value convert “I don’t know” into confident misclassification.

In our pipeline, 2–4% of schema-valid extractions exhibit at least one of these per day, varying with document mix.

Layer 1: schema — but design for refusal

The schema layer is table stakes; the design detail that matters is giving the model somewhere to put uncertainty:

{
  "doc_type": { "enum": ["invoice", "receipt", "credit_note", "other"] },
  "confidence": { "enum": ["high", "medium", "low"] },
  "fields_not_found": { "type": "array", "items": { "type": "string" } }
}

Adding "other" plus a fields_not_found array cut our enum-coercion rate by more than half. The model was never unable to say “I don’t know” — the schema had just made it grammatically impossible.

Nullable-with-reason beats required-with-guess for every field a document might legitimately lack.

Layer 2: semantic invariants

Every schema ships with unstated physics. Write them down as executable checks that run on every extraction:

def invariants(x: Extraction) -> list[Violation]:
    v = []
    if x.line_items and abs(sum(i.amount for i in x.line_items) - x.total) > 0.01:
        v.append(Violation("line_items_sum", severity="block"))
    if x.due_date and x.issue_date and x.due_date < x.issue_date:
        v.append(Violation("date_order", severity="block"))
    if x.total and not (0 < x.total < 10_000_000):
        v.append(Violation("total_range", severity="review"))
    return v

Invariants are cheap, deterministic, and catch the contradiction family almost completely. Ours run in three severities: block (re-extract with violation appended to the prompt), review (human queue), log (monitoring only). The re-extract-with-feedback loop resolves roughly 70% of block violations on the first retry; the remainder queue for review.

The prompt-side counterpart matters as much: state the invariants in the extraction prompt. Models violate constraints they were never told about at several times the rate of stated ones.

Layer 3: provenance grounding

Type-valid hallucination survives layers 1 and 2 when the invented value is plausible. The countermeasure is requiring the model to cite its work — every extracted value paired with a source span:

{ "total": 4820.00, "total_span": "TOTAL DUE: $4,820.00" }

Verification then checks the span actually occurs in the source (fuzzy match tolerating OCR noise and whitespace) and the value is derivable from it (number extraction on the span, comparison against the field). Span-not-found or value-mismatch routes to review.

Grounding catches the failures that matter most — invented financial values — at a cost: span fields roughly double output tokens for dense schemas. We ground only the six fields whose wrongness is expensive, not all forty. Failure rate at this layer: ~0.5% of extractions that passed layers 1–2, which is exactly the population you could not have found otherwise.

Layer 4: statistical monitoring

Per-document verification misses distribution-level drift: a serving-stack update that shifts date formats, a new document template that quietly halves field-found rates. Distribution monitors watch daily aggregates per field — null rate, mean, P95, enum mix — against trailing 28-day bands, alerting on excursions.

This is the layer that catches silent model updates. Same API identifier, new serving revision, moved behavior: we have caught three such shifts in a year of operation, each visible in field-level distributions days before any per-document check fired. If your pipeline pins vendor aliases rather than dated snapshots, this layer is not optional.

Layer 5: selective human escalation

Escalation is a budget allocation problem: review capacity is fixed, so route it where per-item expected loss is highest. Our routing score is a weighted sum of layer signals — invariant review flags, grounding mismatches, model-reported low confidence, and monetary value of the document. The result: ~1.5% of daily volume queues for human review, and that 1.5% contains an estimated 80%+ of surviving errors (estimated via periodic random-sample audits — which you must also run, or your escalation model grades its own homework).

The stack, summarized

Layer	Catches	Cost	Our observed catch rate
Schema + refusal design	Malformed output, forced guessing	~0	eliminates class
Semantic invariants	Contradictions, range violations	CPU-trivial	2–4% of valid outputs flagged
Provenance grounding	Plausible hallucination	~2× output tokens on grounded fields	~0.5% post-invariant
Distribution monitoring	Silent drift, template shifts	infra-light	3 serving shifts / year
Selective escalation	Residual tail	fixed human budget	~80% of survivors in 1.5% of volume

None of these layers is novel; the system property comes from running all five. Schema validation alone is a seatbelt bolted to a car with no brakes — it makes the crash tidier, not less likely.

What we’d build differently today

Two revisions from a year of operation. First, we would adopt grounded spans from day one rather than retrofitting them after the first hallucinated-total incident; retrofit cost exceeded first-build cost by an embarrassing multiple. Second, we would version extraction prompts and schemas in the same artifact with the same review gates as code — the worst drift we shipped came from a “harmless” prompt wording change that moved enum distributions 9%, discovered by layer 4 eleven days later. Everything upstream of the model call is code; treat it with code’s discipline.