4.5 Batch processing strategies
Message Batches API properties
Full reference: batch processing.
| Property | Value |
|---|---|
| Cost | 50% savings vs synchronous |
| Processing window | Up to 24 hours |
| Latency SLA | None guaranteed |
| Multi-turn tool calling within a request | Not supported — cannot execute tools mid-request and return results |
| Correlation | custom_id on each request/response pair |
| Result ordering | Arbitrary — key by custom_id, never by position |
Appropriateness
| Suitable | Not suitable |
|---|---|
| Overnight technical-debt reports | Blocking pre-merge checks |
| Weekly audits | Anything a developer waits on |
| Nightly test generation | Interactive agent turns |
"Switch both the blocking pre-merge check and the overnight debt report to batch for 50% savings." The answer is batch for the overnight report only. "Batches are often faster than 24 h" is not acceptable for a blocking workflow, and a timeout-fallback-to-realtime design adds complexity when matching each API to its use case is simpler. Batch result ordering is a non-issue — that is what custom_id is for.
SLA arithmetic
With a 24-hour worst-case batch window and a 30-hour end-to-end SLA, you have 6 hours of slack — so submit at least every 4 hours to keep a comfortable margin. Expect an item that asks you to compute submission cadence from a stated SLA.
- TypeScript
- Python
const batch = await client.messages.batches.create({
requests: documents.map((doc) => ({
custom_id: doc.id, // the correlation key
params: {
model: "claude-opus-5",
max_tokens: 16000,
tools: [extractInvoice],
tool_choice: { type: "tool", name: "extract_invoice" },
messages: [{ role: "user", content: doc.content }],
},
})),
});
// Poll, then stream results and key by custom_id.
for await (const result of await client.messages.batches.results(batch.id)) {
if (result.result.type === "succeeded") {
store(result.custom_id, result.result.message);
} else {
failed.push(result.custom_id); // resubmit only these
}
}
from anthropic.types.message_create_params import MessageCreateParamsNonStreaming
from anthropic.types.messages.batch_create_params import Request
batch = client.messages.batches.create(
requests=[
Request(
custom_id=doc.id, # the correlation key
params=MessageCreateParamsNonStreaming(
model="claude-opus-5",
max_tokens=16000,
tools=[extract_invoice],
tool_choice={"type": "tool", "name": "extract_invoice"},
messages=[{"role": "user", "content": doc.content}],
),
)
for doc in documents
]
)
# Poll, then stream results and key by custom_id.
for result in client.messages.batches.results(batch.id):
if result.result.type == "succeeded":
store(result.custom_id, result.result.message)
else:
failed.append(result.custom_id) # resubmit only these
Failure handling
Resubmit only the failed documents, identified by custom_id, with appropriate modifications — for example chunking documents that exceeded context limits.
Refine before you scale
Run prompt refinement on a sample set before batch-processing large volumes. First-pass success rate is the lever that keeps resubmission costs down.
Quiz
- Question 1
Real-time Claude calls power two workflows: a blocking pre-merge check developers wait on, and a technical debt report generated overnight for the next morning. Your manager proposes switching both to the Message Batches API for its 50% cost savings. How should you evaluate this?
- Question 2
A contract requires every submitted document to be processed within 30 hours of arrival, and documents arrive continuously through the day. The Message Batches API completes a batch within 24 hours with no guaranteed latency inside that window. How often should batches be submitted?
- Question 3
Three CI workloads are proposed for the Message Batches API: a pre-merge security scan that developers wait on before merging, a weekly dependency audit, and a nightly technical debt report. Which should move to batch?
- Question 4
A 5,000-document batch returns with 78 failures. All 78 are documents whose content exceeded the context limit for the request. What is the correct recovery?