Claude API mechanics

What you’ll be able to do

• Explain why statelessness drives cost, overflow, and “it forgot” bugs
• Branch on every stop_reason before touching response content
• Match streaming, batch, caching, vision, and thinking to the right requirement
• Place a cache checkpoint so it actually engages, and verify that it did

What you need to know

The API is stateless — and your bill proves it

Every call starts from zero. Claude holds nothing between requests. Your application is what remembers the conversation, and it does that by resending the relevant turns every single time.

Three production problems trace directly back to this one fact:

• Cost creep — a 40-turn conversation resends roughly 40 turns of input on turn 41, not just the new message. Cost grows with conversation length even when each user message is short.
• Context overflow — long sessions eventually hit the window ceiling. Pruning or summarizing before that happens is your application's job, not the model's.
• The "it forgot" bug report — it didn't forget. Something upstream truncated the history or dropped the system prompt, so it was never told.

Read stop_reason before you read content

The most common failure in code that works in a demo and breaks in production is parsing response.content[0].text unconditionally.

Each branch exists because a distinct production incident lives there. Treating a max_tokens truncation as a finished answer ships half a sentence to a user. Treating a tool_use response as text throws an index error or returns nonsense. A refusal parsed as content silently loses the fact that the model declined.

Matching capability to requirement

The exam phrases these as business sentences and expects you to name the mechanism:

• Streaming — "users should see it appear as it's written." Fixes perceived latency. Total generation time is unchanged.
• Vision — "patients photograph their insurance cards." Images and scanned documents as input.
• Extended thinking — "get the hard cases right." Buys accuracy on genuinely difficult reasoning, costs tokens and latency.
• Prompt caching — "the same long prefix on every call." Bills the stable portion once.
• Message Batches API — "process last night's uploads by 9am." Large, non-urgent, nobody watching.
• Token counting — the count_tokens endpoint returns input_tokens for a request without generating anything, and is free to call. It is how you measure before optimising instead of guessing.

Real-time vs Batch: find the one deciding fact

This is the single most repeated pattern on the whole exam, and every version of it hides the same deciding fact: is anyone waiting right now?

A hospital's patient-portal chat needs real-time — someone is watching a cursor blink, cost is secondary. That same hospital's overnight reconciliation of 50,000 claims has nobody watching anything, which is exactly what Batch is priced for.

The concrete terms, worth knowing rather than paraphrasing as "a discount":

• 50% off standard rates, on both input and output tokens. Applied automatically — there is no flag to set.
• 24-hour ceiling, not an estimate. Most batches finish far sooner; nothing lets you ask for faster.
• Up to 100,000 requests or 256 MB per batch, whichever you hit first.
• Results kept 29 days, and they come back out of order — you match them to inputs by custom_id.
• Requests that expire past 24 hours are not billed, but you also get no result for them.
• No streaming inside a batch. You get the final message per request, never deltas.

Wrong answers reliably try to fix a batch-shaped problem with a real-time lever: "use a smaller model," "run the requests in parallel." Both shave a little cost or time without addressing why the workload was expensive — it never needed to be synchronous at all.

Prompt caching: placement is everything

Stable content first, checkpoint immediately after it, volatile content last. That's the entire rule.

The failure is silent and specific: put anything that changes between requests — a timestamp, a request ID, a user ID — before the checkpoint, and the cache misses on every single call. No error is raised. The flag is set, the code looks right, and the bill never drops.

So verify rather than assume. Check usage.cache_read_input_tokens on the second identical request. A nonzero value means it engaged; zero means your checkpoint is in the wrong place. Its sibling usage.cache_creation_input_tokens tells you what you paid to write the cache.

The economics are worth carrying in your head, because they explain why placement matters so much:

• Cache read: 0.10× base input rate — a 90% discount on the cached prefix.
• Cache write: 1.25× base input for the default 5-minute TTL, or 2.0× for the 1-hour TTL.
• Reading a cached block resets its TTL, so a high-frequency endpoint rarely pays the write cost after warm-up.

That write premium is why a misplaced checkpoint is worse than no caching at all: you pay 1.25× to write a cache that never gets read, on every single call.

One combination the exam can reach for: caching stacks with Batch, and both discounts apply. But the default 5-minute TTL expires partway through anything but a tiny batch — use the 1-hour TTL so the prefix survives the run.

Key concept

The API is stateless, so your app owns history and pays for it on every turn. Branch on stop_reason before reading content. And a cache checkpoint only works if everything before it is genuinely stable.

When a scenario describes a cost that won't come down despite caching being "enabled," the answer is nearly always checkpoint placement, not cache size or model tier.

Practice scenario

Build exercise — Catch your own caching and stop_reason bugs

Intermediate · 30 min

What you’ll learn

• Proving whether a cache actually engaged
• Why a misplaced checkpoint costs more than no caching
• Handling every stop_reason branch

1. Take a request with a long stable prefix. Check whether anything volatile — a timestamp, a request ID, a user ID — sits before the cache checkpoint.

   • Why: This is the most common caching mistake and it fails silently with no error.
   • You should see: Either a correctly placed checkpoint, or a volatile field quietly breaking it on every call.

2. Send the same request twice and read usage.cache_read_input_tokens and usage.cache_creation_input_tokens on the second response.

   • Why: Setting cache_control is a request, not proof. These two fields are the evidence.
   • You should see: A nonzero read count on call two. If reads stay at zero while creation keeps firing, you are paying the 1.25x write premium every call for nothing.

3. Force a truncated response by setting max_tokens very low, then check what your parsing code does with it.

   • Why: Most demo code treats a max_tokens stop as a finished answer and ships half a sentence.
   • You should see: Either a clean branch on stop_reason, or a bug you just found in your own code.

Exam traps

Parsing content[0].text without checking stop_reason

A max_tokens truncation looks like a complete answer to code that never asks. This is the highest-frequency production bug in the domain.

Putting a timestamp or request ID before the cache checkpoint

It invalidates the cache on every call, silently, with no error raised. The flag is set and the bill never moves.

Assuming caching engaged because cache_control was set

Setting the field is a request, not proof. usage.cache_read_input_tokens on the response is the evidence.

Fixing a batch-shaped workload with a smaller model

It trades away accuracy to solve a problem caused by unnecessary synchronicity. Batch addresses the actual cause.

Believing Batch is faster

It is explicitly not. Batch trades latency for cost — choosing it for a user who is waiting is the inverse of the right call.

Expecting batch results in submission order

They come back out of order. Matching by custom_id is required, not optional.

Leaving the 5-minute cache TTL on a long batch

The prefix expires partway through the run and the cache stops paying. Use the 1-hour TTL when caching inside a batch.

Sources

• Claude Developer Platform Docs