Use-case Prompt Templates and Evals Code

Ready-made prompt templates for use cases that come up again and again in real projects, plus code for evals and embeddings. The Console tools are here too.

26.1 Customer support bot

<role>You are the customer support assistant for "AcmeTV Care".</role>

<knowledge_base>
{{FAQ_AND_POLICIES}}
</knowledge_base>

<rules>
- Answer only from the knowledge_base. If there is no answer, say:
  "I can't confirm this, I'm connecting you to a human agent."
- Refunds, account deletion, payment issues → always hand off to a human.
- Tone: polite, short (3-4 lines), in the customer's language (Hindi/English).
- Never ask for a password / OTP.
</rules>

26.2 Ticket routing / classification

Classify the support ticket below into ONE category.

<categories>
BILLING - payment, recharge, invoice
TECHNICAL - signal, box, app errors
ACCOUNT - login, profile, KYC
OTHER - anything else
</categories>

<ticket>{{TICKET_TEXT}}</ticket>

Return JSON only: {"category": "...", "confidence": 0-1, "reason": "one line"}

In production, use Haiku + structured outputs (JSON schema) + low effort for this: cheap, fast and guaranteed parseable.

26.3 Data extraction and content moderation

<!-- Extraction -->
Extract these fields from this resume. Use null for anything you can't find; don't guess.
Fields: name, email, phone, total_experience_years, skills[], last_company
<resume>{{RESUME}}</resume>

<!-- Moderation -->
Check the user message against our community guidelines.
<guidelines>{{GUIDELINES}}</guidelines>
<message>{{USER_MESSAGE}}</message>
First the reasoning in <analysis>, then <verdict>ALLOW / REVIEW / BLOCK</verdict>.

26.4 LLM-as-judge eval (code)

import json, anthropic
client = anthropic.Anthropic()

test_cases = [   # real user questions + expected key facts
    {"q": "My recharge failed but the money was deducted", "must_have": ["48 hours", "refund"]},
    {"q": "The set-top box shows 'No Signal'", "must_have": ["dish alignment", "cable"]},
]

def judge(question, answer, must_have):
    prompt = f"""Grade the answer.
<question>{question}</question>
<answer>{answer}</answer>
<must_cover>{must_have}</must_cover>
Give a 1-5 score (5 = all points correct and polite). JSON only: {{"score": n, "why": "..."}}"""
    r = client.messages.create(model="claude-opus-5-5", max_tokens=300,
                               output_config={"effort": "low"},
                               messages=[{"role": "user", "content": prompt}])
    return json.loads(r.content[-1].text)

scores = []
for t in test_cases:
    ans = support_bot(t["q"])                 # your app
    scores.append(judge(t["q"], ans, t["must_have"])["score"])
print("Average:", sum(scores) / len(scores))

• Give the judge model a rubric; don't just ask "is it good?".
• Validate the judge on some human-graded examples (the judge can be wrong too).
• Run the same eval after every prompt/model change; track cost and latency along with the score.

26.5 Embeddings + mini RAG (Voyage AI)

# pip install voyageai numpy anthropic
import voyageai, numpy as np, anthropic
vo, claude = voyageai.Client(), anthropic.Anthropic()     # VOYAGE_API_KEY, ANTHROPIC_API_KEY

docs = ["If a recharge fails, the refund arrives within 48 hours.",
        "For a No Signal error, check the dish alignment and the cable."]
doc_emb = np.array(vo.embed(docs, model="voyage-3.5", input_type="document").embeddings)

def ask(q, k=1):
    q_emb = np.array(vo.embed([q], model="voyage-3.5", input_type="query").embeddings[0])
    top = np.argsort(doc_emb @ q_emb)[::-1][:k]          # cosine (normalized vectors)
    context = "\n".join(f"<doc>{docs[i]}</doc>" for i in top)
    r = claude.messages.create(model="claude-sonnet-5-5", max_tokens=500, messages=[{
        "role": "user", "content": f"{context}\nAnswer only from the docs above: {q}"}])
    return r.content[-1].text

print(ask("Money was deducted but the recharge didn't go through"))

26.6 Claude Console (platform.claude.com) tools