4.2 Few-shot prompting
Few-shot examples are the most effective technique for consistently formatted, actionable output when detailed instructions alone produce inconsistent results.
| What few-shot examples do | Concretely |
|---|---|
| Demonstrate ambiguous-case handling | Which tool to pick for an ambiguous request; a branch-level coverage gap |
| Enable generalization to novel patterns | The model applies the judgment, not just the listed cases |
| Reduce hallucination in extraction | Informal measurements, varied document structures |
| Lock the output format | location, issue, severity, suggested fix |
How many, and what shape
Use 2–4 targeted examples for ambiguous scenarios, and show the reasoning for why one action was chosen over a plausible alternative:
### Example 1
Request: "My package says delivered but it's not here."
Chosen: lookup_order (need carrier status and delivery scan before anything else)
Not chosen: process_refund — refund eligibility depends on the carrier scan, which
we do not have yet. Acting first would risk refunding a delivered-but-misplaced item.
### Example 2
Request: "Can you match Competitor X's price?"
Chosen: escalate_to_human
Not chosen: autonomous resolution — the price-adjustment policy covers our own site
only and is silent on competitor matching. A policy gap is an escalation trigger.
Applications tested by the exam
- False-positive reduction: examples distinguishing acceptable code patterns from genuine issues, which reduces false positives while enabling generalization (unlike a hard-coded allowlist).
- Varied document structures: inline citations vs bibliographies; methodology sections vs details embedded in prose.
- Empty/null extraction of required fields: add examples showing correct extraction from documents whose format differs from the training examples.
Quiz
- Question 1
Your CI test-generation step produces good tests for straightforward functions, but on branch-heavy code it is inconsistent: sometimes a case per branch, sometimes only the happy path, sometimes one test with a dozen assertions. The prompt already describes the expected coverage in detail and lists the fixtures available. What is the most effective improvement?
- Question 2
Extraction accuracy on research papers is 94% for papers carrying a bibliography section and 61% for papers using inline parenthetical citations, where the citations array frequently comes back empty even though the citations are plainly in the text. The schema, the instructions and the model are the same for both. What is the most effective fix?
- Question 3
Extraction of quantity from purchase orders is inconsistent when a line reads "2 cases (24 units)": sometimes 2, sometimes 24, occasionally 224. The schema says quantity is an integer in the order’s stated unit of measure, and the instruction says so too. What is the most effective addition?
- Question 4
You add few-shot examples covering the eight document layouts you have seen, and accuracy on those eight rises from 71% to 96%. A ninth layout appears the following month and scores 68%. A colleague proposes adding one example per layout as each new one appears. What is the better approach?
Links
- https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices
- https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview
- https://en.wikipedia.org/wiki/Prompt_engineering
- https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)