Updated June 2026

If your code does json.loads() on model output, you have a contract, and contracts need enforcement. The options, from weakest to strongest.

Level 0: ask nicely

“Respond with JSON in this format” works most of the time, and “most of the time” is the problem. The failure modes are predictable: a friendly preamble before the brace (“Here’s the JSON you requested:”), markdown code fences around the payload, a trailing explanation after it, single quotes, a comment, an extra field. At 99% reliability, a pipeline handling ten thousand items breaks a hundred times. Fine for interactive use where a human retries; not fine for code.

Level 1: ask precisely

Cheap upgrades that close most of the gap: show an exact example of the desired output (one concrete example beats three paragraphs of schema prose), state the negative space explicitly (“respond with only the JSON object, no code fences, no text before or after”), and keep the schema flat and dumb. Deeply nested optional structures invite improvisation; a flat object with obvious fields doesn’t.

Level 2: enforce it

Modern APIs have structured output modes that constrain generation to a schema you supply: the output is guaranteed to parse and to match the shape. Where this exists (it does for Claude and most major providers), use it for anything programmatic and stop hand-rolling regex extractors for fence-wrapped JSON. Two caveats: schema support has limits (recursive schemas and numeric range constraints typically aren’t enforced, so validate business rules yourself), and a guaranteed shape is not a guaranteed truth. The schema ensures "confidence": 0.95 parses; whether 0.95 means anything is on you.

The often-better alternative: tool calls

If you’re using tool calling anyway, defining a tool whose parameters are your schema gets you well-structured arguments without a separate output mode, and it composes naturally with agent loops. A classifier can literally be one tool named record_classification with an enum parameter. This was the standard trick before native structured outputs, and it’s still often the cleanest design.

Design the schema for the model, not just the parser

  • Enums over free text wherever the values are knowable. "category": "billing" from a fixed list beats hoping the model spells it consistently.
  • Let it say “none of the above.” A schema with no escape hatch forces a wrong answer on inputs that don’t fit. Add an other value or a nullable field, or the model will shove square pegs into your round enum.
  • Order fields for reasoning. Models generate in order, so putting "reasoning" before "verdict" lets the conclusion benefit from the thinking. Verdict-first means the reasoning is a post-hoc justification.
  • Validate at the boundary anyway. Parse, check business rules, and on failure retry once with the validation error in the prompt. A model shown its specific mistake usually fixes it on the second attempt.

Rule of thumb: humans reading the output, ask nicely; code reading the output, enforce; agent in the loop, make it a tool.