Quick reference

Every trap from all 32 lessons — 193 of them — grouped by domain. This is revision material, not a substitute for the lessons: the traps make sense once you've read the reasoning behind them. Print-friendly with Ctrl/Cmd + P.

Integration

19% · 12 questions · 8 lessons

3.1 Adding audit logging to a dangerous tool the role never needs, instead of removing it
3.1 Relying on a system-prompt instruction to stop the agent from calling a tool it still has
3.1 Treating a confirmation prompt as equivalent to not granting the capability
3.1 Connecting a whole MCP server because one of its tools is needed
3.1 Choosing a larger model to make an over-privileged agent follow the rules
3.1 Scoping the tool list tightly but leaving the credential behind it broad
3.2 Assuming the user is authorized because they authenticated to the chat front end
3.2 Backing a multi-user agent with one shared service credential that can see every user's data
3.2 Enforcing data-access rules with system-prompt instructions to the model
3.2 Passing tokens or API keys through the model's context or as tool parameters
3.2 Filtering retrieved documents by permission after they have already been placed in context
3.2 Shipping a newly networked MCP server with no auth because the local version never needed it
3.3 Choosing the most capable model and deepest reasoning for every request by default
3.3 Downgrading the model for speed without measuring accuracy on the hard cases
3.3 Treating perceived latency and total completion time as the same requirement
3.3 Running sequential tool calls or retrieval steps that could run in parallel
3.3 Serving offline, non-urgent workloads through the synchronous interactive path
3.3 Justifying a configuration with benchmark scores instead of your own task evaluations
3.4 Logging each model call in isolation with no correlation across tools and services
3.4 Monitoring only HTTP status codes when the costly failures are semantic
3.4 Tracking total token spend with no attribution to tenant, feature or step
3.4 Storing full prompts and tool payloads indefinitely, including personal data and secrets
3.4 Uniformly sampling traces so rare failures and expensive outliers are discarded
3.4 Treating a successful final answer as proof the intermediate tool calls behaved
3.5 Splitting every document into fixed-size chunks regardless of its structure
3.5 Stripping headings and surrounding context so chunks can't be understood on their own
3.5 Indexing without metadata for source, version, date, tenant and access control
3.5 Mixing embeddings from different models or chunking versions in one index
3.5 Updating documents in place with no way to remove stale or deleted chunks
3.5 Blaming the model for wrong answers before inspecting the retrieved chunks
3.6 Embedding structured tables and answering aggregate questions with vector similarity
3.6 Using pure semantic search for identifiers, codes and exact names
3.6 Retrieving more chunks to fix a precision problem
3.6 Forcing a single retrieval step on multi-hop questions that need several lookups
3.6 Choosing a retrieval strategy without a labelled query set to compare options
3.7 Wrapping every internal function in its own MCP server regardless of reuse
3.7 Delegating to another agent when a deterministic API call would do
3.7 Hard-coding the same integration separately in many clients instead of sharing it once
3.7 Running a shared, multi-user MCP server over stdio
3.7 Assuming an integration standard handles authorization and trust for you
3.7 Ignoring interface versioning and failure contracts at the integration boundary
3.8 Loading every tool definition, document and instruction into every request by default
3.8 Assuming a bigger context window removes the need for context selection
3.8 Writing vague capability descriptions that the model can't use to decide what to load
3.8 Using progressive discovery for small, always-needed context that would cache well
3.8 Forgetting that discovery adds round-trips and can miss content the model never thinks to load

Solution Design & Architecture

17% · 11 questions · 4 lessons

1.1 Starting from "where can we use Claude?" instead of from a business outcome and its baseline
1.1 Using an LLM for logic that is deterministic and auditable in code
1.1 Measuring success with model-centric proxies (tokens, accuracy on a demo set) instead of the business metric
1.1 Claiming every value pillar at once, so no trade-off is ever decided
1.1 Ignoring the cost of human review and error correction when computing ROI
1.1 Defaulting to full autonomy when the error cost calls for a human decision point
1.2 Designing only the model call and treating input handling and output delivery as someone else's problem
1.2 Choosing an agent because it is the most capable pattern, not because the path is unpredictable
1.2 Forcing an open-ended task into a rigid workflow that cannot adapt to what it discovers
1.2 Collecting user feedback that never reaches an evaluation set or a change process
1.2 Sending model output straight to downstream systems without schema or business-rule validation
1.2 Relying on the system prompt alone to enforce a hard business constraint
1.3 Splitting into many agents because it mirrors the org chart, not the task
1.3 Giving every subagent the full tool set "for flexibility"
1.3 Letting subagents talk to each other directly so no single component sees the whole flow
1.3 Passing free-text summaries between agents with no status, sources or error information
1.3 Treating a subagent failure as an empty result and continuing silently
1.3 Ignoring the token and latency multiplication that parallel agents create
1.4 Decomposing so finely that context is lost at every handoff
1.4 Running one giant prompt over everything and accepting attention dilution
1.4 Splitting the work but never designing the integration step that catches cross-piece issues
1.4 Fixing the plan up front for an investigation whose steps depend on what is found
1.4 Leaving deterministic sub-steps (math, lookups, format checks) to the model
1.4 Defining subtasks without acceptance criteria, so failures surface only in the final output

Evaluation, Testing & Optimization

16% · 10 questions · 5 lessons

4.1 Reporting a single blended "quality score" that hides which dimension regressed
4.1 Tracking average latency instead of tail percentiles and time to first token
4.1 Measuring cost per API call instead of cost per successfully completed task
4.1 Counting only harmful outputs as safety failures and ignoring over-refusal
4.1 Treating security as a one-off penetration test instead of a regression metric
4.1 Defining success criteria after the system is built, so the metric fits whatever it already does
4.2 Building the dataset only from happy-path examples the team wrote by hand
4.2 Using an LLM judge without ever checking its agreement with human graders
4.2 Asking humans to grade everything, so the suite is too slow and expensive to run on each change
4.2 Using a model judge for criteria that code could verify exactly
4.2 Tuning prompts against the same examples used to decide whether to ship
4.2 Running each test case once and treating a single pass as proof of reliability
4.2 Letting the dataset go stale while production inputs drift
4.3 Changing the prompt, model and retrieval settings in one release and crediting the result to one of them
4.3 Skipping offline evaluation and testing every idea directly on production users
4.3 Stopping the test as soon as the dashboard shows a lead
4.3 Randomizing per request in a multi-turn product, so a user sees both variants in one conversation
4.3 Declaring the winner on the primary metric without checking guardrails and segments
4.3 Comparing variants while a floating model alias changes underneath both
4.4 Upgrading to a larger model before checking whether the right context reached the model
4.4 Adding "do not make things up" to the prompt as the fix for every hallucination
4.4 Debugging from a paraphrase of the request instead of the exact logged prompt and context
4.4 Ignoring the timing: not correlating onset with deploys, index refreshes or model changes
4.4 Treating output cut off by max_tokens as a model reasoning failure
4.4 Assuming a failure is model nondeterminism without rerunning it to see if it reproduces
4.5 Cutting cost by switching to the smallest model without re-running the eval suite
4.5 Optimizing input tokens when output length is what drives latency and cost
4.5 Sending latency-tolerant bulk work through the real-time API
4.5 Monitoring only infrastructure metrics (uptime, error rate) and assuming quality holds
4.5 Logging full prompts and outputs with personal data and no retention or access controls
4.5 Alerting on averages, so a drift in one segment or the tail never triggers

Governance, Safety & Risk Management

14% · 9 questions · 5 lessons

5.1 Enforcing a hard business or security rule only through system-prompt instructions
5.1 Relying on an input classifier alone to stop prompt injection
5.1 Giving the agent a broad service credential and trusting the prompt to limit what it does with it
5.1 Keeping an unneeded high-risk tool and adding logging or a confirmation step instead of removing it
5.1 Treating documents, web pages and tool results as trusted instructions because they came from your own pipeline
5.1 Validating the model's output only for format, not for what it would cause downstream
5.2 Assuming a fluent, confident answer is a correct one
5.2 Blaming the model first when the symptom points to retrieval, tools or context assembly
5.2 Treating the model's self-reported confidence as a reliable error detector
5.2 Rating risks by likelihood alone and ignoring reversibility and blast radius
5.2 Assuming a pre-launch evaluation keeps describing production behavior after data, prompts or models change
5.2 Expecting a larger or newer model to remove a failure mode that comes from system design
5.3 Requiring human approval on every action, so reviewers start rubber-stamping
5.3 Implementing the approval step as a prompt instruction the model is asked to respect
5.3 Showing reviewers only the model's conclusion, without the source evidence behind it
5.3 Stopping review entirely for categories that were automated because they looked accurate
5.3 Measuring review by throughput rather than by whether reviewers catch planted or known errors
5.3 Letting reviewer corrections disappear instead of feeding them back into evaluations and prompts
5.4 Assuming a system is compliant because the model provider is, or because one component is covered
5.4 Covering the model endpoint but sending regulated data to logging, analytics, tracing or third-party tool services outside the agreement
5.4 Honoring a GDPR erasure request in the primary database but not in vector indexes, transcripts, caches or eval datasets
5.4 Sending whole records to the model when the task needs a few fields
5.4 Fully automating a decision with legal or similarly significant effects on individuals without the safeguards GDPR expects
5.4 Treating a FedRAMP authorization as covering every service, region and feature a vendor offers
5.4 Logging full prompts and responses indefinitely by default for debugging
5.5 Concluding a system is fair because overall accuracy is high
5.5 Removing a protected attribute and assuming bias is gone while proxies remain
5.5 Treating the model's generated reasoning as a faithful, auditable explanation of how it decided
5.5 Adding a "be unbiased" instruction to the prompt as the main fairness control
5.5 Picking a fairness metric by default instead of deciding it with stakeholders for the use case
5.5 Deploying an AI-assisted decision without disclosure, a way to contest it, or a named accountable owner

Stakeholder Communication & Lifecycle Management

14% · 9 questions · 4 lessons

6.1 Starting discovery by choosing a model, framework or architecture pattern
6.1 Accepting "as accurate as possible" as a requirement
6.1 Gathering requirements only from the executive sponsor
6.1 Leaving data access, residency and legal review until after the design is approved
6.1 Validating feasibility on synthetic or hand-picked examples instead of real data
6.1 Setting a success target without measuring today's baseline
6.1 Assuming every part of the problem needs an LLM
6.2 Presenting a single recommended design with no alternatives or downsides
6.2 Describing model quality with benchmark scores instead of results on the stakeholder's task
6.2 Promising that the system will not hallucinate or make mistakes
6.2 Giving every audience the same technical deck
6.2 Leaving decisions recorded only in meeting notes and chat threads
6.2 Escalating a disagreement without framing the options and who owns the decision
6.3 Committing to an availability or latency SLA stricter than your upstream dependencies support
6.3 Defining a quality SLA without saying how quality will be measured
6.3 Collecting thumbs-up and thumbs-down feedback with no triage or owner
6.3 Fixing each reported bad answer by patching the prompt without adding it to the eval set
6.3 Treating a stakeholder's anecdote as a trend without checking the metrics
6.3 Letting scope expand through feedback without revisiting targets and cost
6.4 Handing off a system with architecture diagrams but no runbooks or eval suite
6.4 Documenting prompts and model choices outside version control
6.4 Treating go-live as the end of the project
6.4 Shipping prompt or model changes after launch without a regression evaluation
6.4 Leaving known limitations out of the documentation to avoid alarming the new owners
6.4 Waiting for a model deprecation notice before planning the migration

Claude Models, Prompting & Context Engineering

13% · 8 questions · 4 lessons

2.1 Choosing the most capable model for every step because it is the safest default
2.1 Upgrading the model to fix failures that come from missing or wrong context
2.1 Choosing the cheapest model for a high-stakes step because it scored well on average
2.1 Selecting a model from public benchmarks instead of an evaluation on your own task
2.1 Pointing production at a moving model alias with no re-evaluation when it changes
2.1 Routing by a model's self-reported confidence instead of an observable signal
2.2 Enforcing a hard business or compliance rule with prompt wording alone
2.2 Fixing ignored instructions by adding capital letters and repetition
2.2 Interpolating user or retrieved text into the instructions without delimiters
2.2 Editing production prompts in place with no version, owner or regression run
2.2 Writing one monolithic system prompt that tries to cover every tenant and use case
2.2 Trusting instructions found inside retrieved documents or tool output
2.3 Adding chain-of-thought to every call regardless of whether the task needs reasoning
2.3 Using many near-identical few-shot examples that teach surface features instead of judgment
2.3 Reaching for few-shot before the zero-shot prompt states its criteria explicitly
2.3 Parsing the final answer out of free-form reasoning text
2.3 Treating the model's written reasoning as a faithful audit trail of how it decided
2.3 Judging a technique change on a handful of hand-picked examples
2.4 Filling the context window because the model can accept it
2.4 Putting timestamps, user IDs or other per-request values at the top of the prompt
2.4 Summarizing long conversations in a way that drops amounts, dates and commitments
2.4 Assuming caching is working without checking cache read tokens in usage data
2.4 Loading every playbook into the system prompt instead of loading it when relevant
2.4 Ignoring output tokens and verbose tool results when reducing token cost

Developer Productivity & Operational Enablement

7% · 4 questions · 2 lessons

7.1 Writing a security rule into CLAUDE.md and treating it as enforced
7.1 Putting team conventions in a personal user-level file so they never reach teammates
7.1 Committing API keys or tokens inside a shared MCP configuration
7.1 Granting broad allow rules to stop permission prompts instead of narrowing them to safe commands
7.1 Relying on each developer's settings for a policy that must hold organization-wide
7.1 Copying the same configuration into every repository by hand so it drifts
7.2 Measuring AI productivity by lines of code generated or number of prompts
7.2 Letting an assistant change code with no test or check to verify the result
7.2 Giving an incident-triage agent write access to production "to save time"
7.2 Accepting a confident root-cause explanation that is not tied to log or trace evidence
7.2 Running headless automation with broad permissions because no one is watching the prompts
7.2 Applying AI to the step that is not the bottleneck in the delivery process