2.3 Claude API mechanics
Everything goes through POST /v1/messages. Tools, thinking, caching, and output constraints are features of that one endpoint, not separate APIs. The supporting endpoints are Batches, Files, Token Counting, and Models.
The request
- TypeScript
- Python
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic(); // resolves ANTHROPIC_API_KEY, or an auth token, or a CLI profile
const response = await client.messages.create({
model: "claude-opus-5",
max_tokens: 16000,
system: "You are an invoice extraction service.", // top-level, not a message
thinking: { type: "adaptive" },
output_config: { effort: "high" },
messages: [{ role: "user", content: "Extract the totals from this invoice." }],
});
for (const block of response.content) {
if (block.type === "text") console.log(block.text);
}
import anthropic
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-opus-5",
max_tokens=16000,
system="You are an invoice extraction service.", # top-level, not a message
thinking={"type": "adaptive"},
output_config={"effort": "high"},
messages=[{"role": "user", "content": "Extract the totals from this invoice."}],
)
for block in response.content:
if block.type == "text":
print(block.text)
| Field | Notes |
|---|---|
model | Exact ID string, no date suffix. Pin it. |
max_tokens | Required. An enforced ceiling on output the model is unaware of. Hitting it truncates. |
system | Top-level parameter, or a list of blocks (needed for cache_control). Not a messages entry. |
messages | Alternating-ish turns; first must be user. Consecutive same-role messages are merged. |
tools | Definitions with name, description, input_schema; optional strict: true. |
thinking | {"type": "adaptive"} on current models. |
output_config | effort, format (structured outputs), task_budget. |
stream | Or use the SDK .stream() helper. |
The API is stateless. There is no server-side conversation for the Messages API — you resend the full history every turn, which is why history hygiene and caching are cost decisions, not cosmetics.
response.content is a list of content blocks, a discriminated union: text, thinking, tool_use, plus server-tool result blocks. Narrow on type before reading fields; never index content[0] blindly.
Stop reasons
The loop contract. Branch on it; never parse prose for completion.
stop_reason | Meaning | Correct handling |
|---|---|---|
end_turn | Finished naturally | Done — return the text |
tool_use | One or more tool_use blocks emitted | Execute all, append all tool_result blocks in one user message, iterate |
max_tokens | Output cap hit mid-generation | Not completion. Raise max_tokens, stream, or shorten the requested output |
stop_sequence | A custom stop sequence matched | Treat per your protocol |
pause_turn | A server-tool turn paused | Send the response back to resume |
refusal | A safety classifier declined | Read stop_details (type, category, explanation) and handle; stop_details is null for every other stop reason |
Streaming
Streaming is server-sent events over the same endpoint. Use it whenever input is long, output is long, or max_tokens is large — it prevents request timeouts, and it is required by the SDKs for very large max_tokens values.
- TypeScript
- Python
const stream = client.messages.stream({
model: "claude-opus-5",
max_tokens: 64000,
messages: [{ role: "user", content: prompt }],
});
stream.on("text", (delta) => process.stdout.write(delta));
const final = await stream.finalMessage(); // full Message, usage included
with client.messages.stream(
model="claude-opus-5",
max_tokens=64000,
messages=[{"role": "user", "content": prompt}],
) as stream:
for delta in stream.text_stream:
print(delta, end="", flush=True)
final = stream.get_final_message() # full Message, usage included
Do not hand-roll a promise around .on() events when you only need the result — the SDK's finalMessage() / get_final_message() exists for that. Event order matters if you consume raw events: message_start, then per-block content_block_start / content_block_delta / content_block_stop, then message_delta (carrying stop_reason and final usage), then message_stop.
For tool inputs, eager_input_streaming: true on a tool definition streams partial JSON as it is produced — a plain tool-definition field, not a beta feature and not on client.beta.
Multimodal input
| Input | Content block | Constraints |
|---|---|---|
| Image, URL | {"type": "image", "source": {"type": "url", "url": "..."}} | — |
| Image, base64 | {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "..."}} | No newlines in the base64 string |
| PDF, base64 | {"type": "document", "source": {"type": "base64", "media_type": "application/pdf", "data": "..."}} | 32 MB request, 600 pages (100 on 200K-context models); place the document block before the text block |
| Uploaded file | {"type": "document", "source": {"type": "file", "file_id": "..."}} | Upload once with client.files.upload(...), reference many times; the block type must match the file's MIME type |
Citations — set citations: {"enabled": true} on each document block (all or none). The response splits into multiple text blocks; cited ones carry a citations array with cited_text, document_index, document_title, and a location: char_location for text, page_location (1-indexed) for PDF, content_block_location for custom content. Citations are incompatible with output_config.format — pairing them returns a 400.
Thinking
| Model family | Config | Omitting thinking |
|---|---|---|
| Claude Opus 5 | {"type": "adaptive"}; {"type": "disabled"} only at effort high or below | Runs adaptive — thinking is on by default |
| Claude Opus 4.8 / 4.7, Sonnet 5 | {"type": "adaptive"} is the only on-mode | Runs without thinking on 4.8/4.7 |
| Claude Fable 5 | Omit it — thinking is always on; explicit disabled or a budget returns 400 | Adaptive |
| Pre-4.6 models | {"type": "enabled", "budget_tokens": N} | No thinking |
display: "summarized" returns a readable summary of the reasoning; the default on current models is "omitted" (thinking blocks arrive with empty text). Display changes visibility only — thinking is billed either way. Echo thinking blocks back unchanged when continuing on the same model.
budget_tokens on Claude Opus 5 / 4.8 / 4.7 / Sonnet 5 / Fable 5 · temperature, top_p, top_k on those same models · an assistant prefill as the last message on any 4.6-or-later model · the deprecated output_format (use output_config.format). Items are written with these as distractors.
Prompt caching
A prefix match. Render order is tools → system → messages; any byte change anywhere in the prefix invalidates everything after it.
- TypeScript
- Python
const response = await client.messages.create({
model: "claude-opus-5",
max_tokens: 16000,
system: [
{ type: "text", text: FROZEN_INSTRUCTIONS },
{ type: "text", text: POLICY_CORPUS, cache_control: { type: "ephemeral" } },
],
messages: [{ role: "user", content: userQuestion }], // volatile content, after the breakpoint
});
console.log(response.usage.cache_creation_input_tokens); // written (~1.25x cost)
console.log(response.usage.cache_read_input_tokens); // served from cache (~0.1x cost)
response = client.messages.create(
model="claude-opus-5",
max_tokens=16000,
system=[
{"type": "text", "text": FROZEN_INSTRUCTIONS},
{"type": "text", "text": POLICY_CORPUS, "cache_control": {"type": "ephemeral"}},
],
messages=[{"role": "user", "content": user_question}], # volatile content, after the breakpoint
)
print(response.usage.cache_creation_input_tokens) # written (~1.25x cost)
print(response.usage.cache_read_input_tokens) # served from cache (~0.1x cost)
Maximum 4 breakpoints per request; default TTL 5 minutes, ttl: "1h" available. Top-level cache_control on the request auto-caches the last cacheable block. Placement strategy and the silent-invalidator list are on 5.4 Cost & token management.
Tools
{
"name": "lookup_invoice",
"description": "Fetch one invoice by its INV-prefixed identifier. Use for invoice-level queries; use lookup_purchase_order for PO-level queries.",
"input_schema": {
"type": "object",
"properties": { "invoice_id": { "type": "string", "description": "e.g. INV-4471" } },
"required": ["invoice_id"],
"additionalProperties": false
},
"strict": true
}
| Mechanic | Detail |
|---|---|
strict: true | Top-level field on the tool definition, not on tool_choice. Requires additionalProperties: false plus required. Guarantees input validates |
tool_choice | auto (default), any (must call some tool), {"type": "tool", "name": "..."} (must call that one), none |
| Parallel calls | On by default — one assistant message may hold several tool_use blocks. Return all results in one user message |
| Errors | tool_result with is_error: true. Never drop a block |
| Schema-less Anthropic tools | bash_20250124, text_editor_20250728, memory_20250818 — declare type and name, no input_schema |
| Server tools | Run on Anthropic's infrastructure: web_search_20260209, web_fetch_20260209, code_execution_20260521, tool search. Results arrive as content blocks in the same response |
| MCP connector | mcp_servers: [{type: "url", url, name}] and tools: [{type: "mcp_toolset", mcp_server_name: "<same name>"}] with beta mcp-client-2025-11-20. One half alone is a validation error |
Server-tool failures return HTTP 200 with an error object inside the result block — they do not raise. Branch on the block's content shape.
Structured outputs
Constrain the response format with output_config.format, and/or the tool input with strict: true. The messages.parse() helper validates the response against your schema for you. output_format is deprecated. Full patterns on 6.3 Output handling.
Batches
- TypeScript
- Python
const batch = await client.messages.batches.create({
requests: docs.map((doc) => ({
custom_id: doc.id, // your key — results come back in ANY order
params: {
model: "claude-opus-5",
max_tokens: 4096,
messages: [{ role: "user", content: doc.text }],
},
})),
});
// Poll until ended, then stream results.
let status = await client.messages.batches.retrieve(batch.id);
while (status.processing_status !== "ended") {
status = await client.messages.batches.retrieve(batch.id);
}
for await (const entry of await client.messages.batches.results(batch.id)) {
if (entry.result.type === "succeeded") store(entry.custom_id, entry.result.message);
}
from anthropic.types.messages.batch_create_params import Request
from anthropic.types import MessageCreateParamsNonStreaming
batch = client.messages.batches.create(
requests=[
Request(
custom_id=doc.id, # your key — results come back in ANY order
params=MessageCreateParamsNonStreaming(
model="claude-opus-5",
max_tokens=4096,
messages=[{"role": "user", "content": doc.text}],
),
)
for doc in docs
]
)
while client.messages.batches.retrieve(batch.id).processing_status != "ended":
pass
for entry in client.messages.batches.results(batch.id):
if entry.result.type == "succeeded":
store(entry.custom_id, entry.result.message)
| Batch fact | Detail |
|---|---|
| Cost | 50% of standard rates |
| Window | Processed asynchronously, within 24 hours |
| Result order | Any order — key by custom_id, never by position |
| Result types | succeeded, errored, canceled, expired — handle all four |
| Not available with | Streaming semantics per request, and the fallbacks parameter |
Realtime versus batch
| Choose realtime when | Choose batch when |
|---|---|
| A human is waiting on the response | Nobody sees the output until later |
| The request is part of an interactive loop or agent turn | The workload is a bulk pass over a known input set |
| Volume is low enough that per-request cost is not the driver | Volume is high and cost is the primary constraint |
| You need tools executed in your process turn by turn | Each item is independent and self-contained |
"10,000 documents overnight, non-urgent, cost is the primary concern" → Message Batches API. Parallel synchronous calls do not reduce per-token cost, lowering max_tokens does not address the tradeoff, and dropping to the smallest model sacrifices quality for a saving batch already gives you.
Data access patterns
| Need | Mechanism |
|---|---|
| Same large file across many requests | Files API — upload once, reference file_id |
| Same large text prefix across many requests | Prompt caching |
| Verifiable quotes back to a source | Citations on document blocks |
| Cost estimate before sending | client.messages.count_tokens(...) — never a third-party tokenizer |
| Live model capabilities | client.models.list() / client.models.retrieve(id) — fields include max_input_tokens, max_tokens, capabilities (there is no context_window field) |
| Long conversation that will outgrow the window | Compaction or context editing (1.3) |
| Operator instruction mid-conversation | Append {"role": "system", ...} to messages — preserves the cached prefix; model-gated, and it cannot be messages[0] |
Third-party vendors
Use the platform's dedicated client class, not the first-party client with a base_url override.
| Platform | Client | Model ID form |
|---|---|---|
| Amazon Bedrock | AnthropicBedrockMantle / AnthropicBedrockMantle({awsRegion}) | anthropic.-prefixed, e.g. anthropic.claude-opus-5 |
| Google Vertex AI | AnthropicVertex(project_id, region) | Bare ID for current models; dated snapshots use @, e.g. claude-opus-4-5@20251101 |
| Microsoft Foundry | AnthropicFoundry(...) | First-party IDs; billed through the Microsoft Marketplace at standard API rates |
Auth differs: Bedrock uses AWS credentials, Vertex uses GCP application default credentials (no Anthropic key), Foundry uses its own credential type. Feature availability differs per platform — several capabilities (fast mode, Managed Agents, some server tools, refusal fallbacks) are first-party only, and web fetch is unavailable on Vertex. Check availability before promising a feature on a partner platform.
Errors, retries, and client config
| Concern | Default / rule |
|---|---|
| Retryable | 408, 409, 429, 5xx, connection errors — and 529 (overloaded) |
| Not retryable | 400 (bad request), 401, 403, 404, 413 — fix the request |
| SDK retries | max_retries defaults to 2, with backoff |
| Timeout | 10 minutes default; units differ — seconds in Python/Ruby, milliseconds in TypeScript |
| Wall clock | Up to timeout × (max_retries + 1), because timeouts are retried |
| Error handling shape | A most-specific-first chain (NotFoundError → RateLimitError → APIStatusError → APIConnectionError), never one broad catch and never string-matching messages |
Quiz
- Question 1
A developer must process 10,000 documents overnight to produce a non-urgent analytics report. Cost is the primary concern, and results are not needed until the following morning. Which approach best fits the requirement?
- Question 2
A batch job writes each result to the database using that result’s index in the results stream. Occasionally a document is stored under another document’s ID. What is wrong?
- Question 3
You need Claude to answer questions about a 300-page PDF, quote the exact source text with page numbers for every claim, and return the whole thing as a strict JSON object. What breaks?
- Question 4
Which statement about the Messages API is correct?
Links
- https://platform.claude.com/docs/en/api/messages
- https://platform.claude.com/docs/en/build-with-claude/streaming
- https://platform.claude.com/docs/en/build-with-claude/vision
- https://platform.claude.com/docs/en/build-with-claude/pdf-support
- https://platform.claude.com/docs/en/build-with-claude/citations
- https://platform.claude.com/docs/en/build-with-claude/prompt-caching
- https://platform.claude.com/docs/en/build-with-claude/batch-processing
- https://platform.claude.com/docs/en/build-with-claude/extended-thinking
- https://platform.claude.com/docs/en/build-with-claude/token-counting
- https://platform.claude.com/docs/en/api/errors
- https://platform.claude.com/docs/en/api/claude-on-amazon-bedrock
- https://platform.claude.com/docs/en/api/claude-on-vertex-ai