Skip to main content

6.3 Output handling

Producing structured output​

MechanismShapeUse
output_config.formatA JSON schema constraining the responseThe response itself is the data
Tool with strict: trueA JSON schema constraining tool inputThe model is calling into your system; you want validated arguments
messages.parse()SDK helper that validates the response against your schemaThe default ergonomic path
Prose plus a parser—Last resort; every other option is more reliable
const schema = {
type: "object",
properties: {
invoice_number: { type: "string", description: "As printed, e.g. INV-4471" },
total: { type: ["number", "null"], description: "Grand total; null if not present" },
currency: { type: "string", enum: ["EUR", "USD", "GBP"] },
confidence: { type: "string", enum: ["high", "medium", "low"] },
unresolved_fields: { type: "array", items: { type: "string" } },
},
required: ["invoice_number", "total", "currency", "confidence", "unresolved_fields"],
additionalProperties: false,
} as const;

Design notes visible in that schema: absence is representable (null), the closed set is an enum, uncertainty is a field rather than a guess, and the consumer is told which fields were unresolved instead of receiving invented ones.

Two constraints worth remembering: output_config.format is incompatible with citations (400), and strict: true requires additionalProperties: false plus required.

Validating the response​

Structured output constrains the shape. It does not make the content true.

LayerCheck
Transportstop_reason — max_tokens means truncated, so do not parse it as complete; refusal means there is no answer to parse
SyntaxParses as JSON at all
SchemaValidates against the schema, including types and enums
SemanticsLine items sum to the total; the date is in range; the ID exists in your system; the referenced order belongs to this customer
Business rulesWithin policy limits, permitted for this user, not a duplicate
ProvenanceThe quoted text actually appears in the source document

Only after the semantic and business layers may the value reach a database, a payment, or another API.

The validation-retry loop​

call → parse → validate
ok → use it
invalid → retry ONCE, feeding the specific validation error back
invalid → fall back (partial result, flag for review) or escalate

Feed the specific error back ("currency must be one of EUR, USD, GBP; you returned 'euros'"), not "invalid output". Bound the retries — an unbounded validate-retry loop is a cost incident. And log every failure: a rising validation-failure rate is the earliest signal of prompt drift, schema drift, or a model change.

Defensive parsing​

RuleWhy
Narrow on block type before reading fieldscontent is a discriminated union; content[0].text is not safe
Concatenate all text blocksCitations and thinking split the response into several blocks
JSON.parse / json.loads on tool_use.inputEscaping varies by model; string matching is a latent bug
Never eval model output, never interpolate it into SQL, a shell command, or HTMLIt is untrusted input to those systems
Treat missing optional fields as missing, not as defaultsA silent default is a silent wrong answer
Bound sizes and array lengths before iteratingA 10,000-element array will be produced eventually
Version the parser with the schemaOld stored output must remain readable

Skepticism toward confident output​

The model's tone carries no information about its accuracy, and self-reported confidence is poorly calibrated — an agent that is wrong about a hard case is usually confidently wrong about it.

Do not use as a gateUse instead
"I am 95% confident"Verification against a source of truth, or a deterministic check
Fluent, well-formatted proseSchema validation plus semantic checks
The absence of hedgingExplicit criteria for when the model must say "not found"
Sentiment or urgency in the textRules keyed on facts you can observe

Design for it structurally:

  • Make "not found" / null / unknown a first-class, easy answer. If the schema has nowhere to put uncertainty, the model will invent certainty.
  • Verify with tools rather than trusting recall — look the record up instead of asking the model what it remembers.
  • Require provenance for claims: source, page, quote, date. A claim with no source is not usable downstream.
  • Route by fact, not by feeling — thresholds on validated data, human review for the classes you have decided need it.
  • Cross-check high-stakes results with a deterministic computation (sums, totals, reference lookups) rather than a second opinion from the same model.

Quiz​

  1. Question 1

    Structured outputs guarantee the response matches your JSON schema, and your schema already constrains field types, lengths and enum values. Which remaining check needs state the model never had access to?

  2. Question 2

    A response fails schema validation. What is the correct recovery?