Batch processing strategies

What you’ll be able to do

Match each workflow to the synchronous API or the Message Batches API based on whether anyone is blocked waiting
Calculate a batch submission frequency that meets an end-to-end SLA given the 24-hour processing window
Correlate results with custom_id and resubmit only the failed requests, modified where needed
Refine prompts on a sample set before batching a large volume

What you need to know

The trade the Batches API offers

The Message Batches API takes a set of requests, processes them asynchronously, and lets you collect the results later by polling for completion. In exchange for giving up immediacy, you get:

50% cost savings compared with sending the same requests synchronously
A processing window of up to 24 hours
No guaranteed latency SLA: a batch may finish in minutes or take most of the window, and you can't plan on either

That last point decides most batch questions. "Batches often finish quickly" doesn't make a batch safe for work where someone is waiting, because nothing guarantees it.

Latency tolerance decides the API

Ask one question about each workflow: is anyone, or anything, blocked until the result arrives?

Non-blocking, latency-tolerant: overnight reports, weekly audits, nightly test generation, re-extracting a document archive. Results are consumed hours later. The batch API fits, and the cost saving is real.
Blocking: a pre-merge check a developer waits on, a support reply, an interactive tool. Use the synchronous API. A cheaper result that arrives tomorrow is worthless for a merge someone wants to make now.

A single team often has both. The right answer is usually to split them: batch for the nightly technical-debt report, synchronous for the pre-merge check. Moving everything to one API, or adding a fallback timer to force batches to behave like synchronous calls, just adds complexity.

No multi-turn tool calling inside a batch request

Each batch request is processed on its own, without your code in the loop. The batch API doesn't support multi-turn tool calling within a single request: if the model asks for a tool, your code can't run it mid-request and return the result for the model to continue. Workloads that need an agentic loop (call a tool, read the result, decide the next step) belong on the synchronous API. Workloads where each item is a self-contained, single-pass job, such as "extract these fields from this document" or "summarize this file," are the natural fit for batching.

custom_id ties results to requests

Every request in a batch carries a custom_id you choose, and each result comes back with the same custom_id. Use it, not the position in the results, to match each result to its input. Choose IDs that map directly to your own records (a document ID, a file path plus revision) so that joining results back to source data, and finding the failures, is a lookup.

correlate by custom_id, resubmit only the failures

requests = [{"custom_id": f"doc-{d.id}", "params": build_params(d)} for d in documents] # ... submit, poll until processing ends, then collect results ...

failed = [r.custom_id for r in results if not succeeded(r)] retry = [] for cid in failed: doc = docs_by_custom_id[cid] if exceeded_context(cid): retry += [chunk_request(doc, i) for i in range(n_chunks(doc))] else: retry.append(original_request(cid)) submit_batch(retry) # only the failures, modified where needed

Handling failures: resubmit only what failed, and fix why

When some requests in a batch fail, resubmit only those, identified by custom_id. Rerunning the whole batch pays again for work that already succeeded. And modify them where the failure calls for it: a document that exceeded the context limit will fail again unchanged, so split it into chunks, each with its own custom_id that traces back to the parent document.

Calculating submission frequency against an SLA

If documents arrive continuously and you promise results within some end-to-end SLA, the batch's 24-hour window is only part of the delay. A document also waits for the next submission. The worst-case latency is roughly:

worst case = time waiting for the next submission + up to 24 hours of processing

With a 30-hour SLA, submitting every 4 hours gives a worst case of about 4 + 24 = 28 hours, inside the SLA with a margin for collecting results and resubmitting failures. Submitting once a day would give 24 + 24 = 48 hours and break it. Work backwards from the SLA: subtract 24 hours of processing and a safety margin, and what remains is the longest gap you can leave between submissions.

Refine on a sample before you batch the volume

A prompt flaw in a 10,000-document batch fails 10,000 times, and you find out only after the batch finishes. Run the prompt on a representative sample first (different document types, edge cases, the longest inputs), fix what goes wrong, and then submit the full volume. A higher first-pass success rate means fewer resubmission rounds, and each round can add up to another 24 hours.

Key concept

Use the Message Batches API (50% cheaper, up to 24 hours, no latency guarantee) for work nobody is waiting on, and the synchronous API for blocking checks. Each batch request is single-shot with no multi-turn tool calling. Correlate by custom_id, resubmit only failures (chunking oversized ones), size the submission interval from the SLA, and test the prompt on a sample first.

When a scenario proposes batching to save money, check one thing first: is a person or a pipeline blocked on the result? If so, batch is the wrong tool regardless of the savings.

Practice scenario

ScenarioA contracts team must deliver extracted clause data within 30 hours of a contract being uploaded. Uploads arrive throughout the day. The current design submits one batch at midnight. Last week's batch also had 40 failures out of 3,000, all very long contracts that exceeded the context limit, and the team reran the entire batch to recover them.

Work it through, then open this

A contract uploaded just after midnight waits nearly 24 hours for submission, then up to 24 hours of processing, which is about 48 hours worst case and well past the SLA. Submit on a regular interval instead. Every 4 hours gives about 28 hours worst case, leaving margin. For the failures, resubmit only the 40 failed custom_ids, not all 3,000, and chunk those contracts first, since they failed on context limits and would fail again unchanged.

Build exercise — Design a batch pipeline with failure handling

Intermediate · 35 min

What you’ll learn

Deciding batch versus synchronous per workflow
Using custom_id to correlate results and target resubmissions
Sizing the submission interval to an SLA

List the model-backed workflows in a system and mark each as blocking or non-blocking, and as single-pass or needing multi-turn tool calls.
- Why: Only non-blocking, single-pass work belongs in a batch.
- You should see: A clear split, typically pre-merge and interactive work on the synchronous API and reports, audits, and nightly generation on batch.
Run the batch prompt on a 20-document sample covering your document types and longest inputs, and fix the failures before submitting the full set with a custom_id per document.
- Why: Fixing a prompt on 20 documents is far cheaper than discovering the flaw across thousands and resubmitting.
- You should see: A first-pass success rate on the full batch close to the sample’s.
After the batch completes, collect failures by custom_id, chunk any that exceeded context limits, and resubmit only those. Then compute your worst-case latency for your submission interval against your SLA.
- Why: Targeted resubmission avoids paying twice; the latency calculation tells you whether the schedule meets the promise.
- You should see: A resubmission batch containing only the failures, and a worst-case figure (interval + 24 hours) below your SLA with some margin.

Exam traps

Moving a blocking pre-merge check to the batch API to save cost

Batches can take up to 24 hours with no latency SLA. Blocking checks need the synchronous API; batch suits overnight and weekly work.

Assuming batches usually finish fast enough to rely on for latency-sensitive work

“Often fast” isn’t a guarantee. With no SLA, latency-sensitive work can’t depend on it.

Designing a batch request that needs multi-turn tool calling

The batch API can’t run tools mid-request and return results. Agentic loops belong on the synchronous API.

Matching batch results to inputs by position instead of by custom_id

custom_id exists to correlate each request with its response. Use IDs that map to your own records.

Resubmitting the entire batch when only some requests failed

Identify the failures by custom_id and resubmit only those. Rerunning everything pays again for work that already succeeded.

Resubmitting oversized documents unchanged after they failed on context limits

They’ll fail the same way. Modify them first, for example by chunking, with each chunk traceable to its parent.

Batch-processing thousands of documents with a prompt never tested on a sample

Refine on a representative sample first to raise first-pass success and avoid costly resubmission rounds.

Sources

Claude Developer Platform Docs
Anthropic, Claude Certified Architect – Foundations Exam Guide v1.0 (July 2026)