Claude Seekho

Big Picture 8: Long Conversations, Safety and Cost

When a conversation gets long, or an agent works for a long time, the context fills up. This stage covers six ways to manage it, plus two rails that apply to every layer: Safety and Cost.

Managing a filling context
  1. 1.Prompt caching: stable prefix is cheap on repeat
  2. 2.Compaction: older part becomes a summary
  3. 3.Context editing: clear old tool results / thinking
  4. 4.Memory: facts saved in files for the next session
  5. 5.Subagents: heavy reading in a separate context
  6. 6./clear: new task, fresh context
  • Prompt caching: a prefix that stays the same every time costs less when sent again.
  • Compaction: the older part of the conversation is turned into a summary. In Claude Code you can trigger it yourself with /compact.
  • Context editing: old tool results and thinking are cleared out.
  • Memory: important facts are written into files, so they are remembered in the next session.
  • Subagents: the heavy reading happens in a separate context, and only a summary comes back.
  • /clear: a new task gets a fresh context.

The Safety rail

Safety is not in one place, it is in every layer. Eight boxes run down the left edge of the whole diagram.

Safety by layer
  1. 1.Training: Constitutional AI, RLHF, red-teaming, RSP / ASL levels
  2. 2.Input: usage policy, safety classifiers
  3. 3.Model: helpful, honest, harmless
  4. 4.Output: stop_reason refusal + optional fallback model
  5. 5.Tools: permission modes, deny rules, hooks
  6. 6.Execution: sandbox / VM, network allowlist, vaults
  7. 7.Data: prompt injection is data, human approval on risky actions
  8. 8.Org: SSO, audit logs, ZDR, SOC 2, ISO 42001
  • Safety: Training: Constitutional AI, RLHF, red-teaming and the ASL levels of the RSP.
  • Safety: Input: the usage policy and safety classifiers, for example for cyber and bio risks.
  • Safety: Model: stay helpful, honest and harmless, and say "I do not know" when it does not know.
  • Safety: Output: stop_reason can be refusal, with an optional fallback model.
  • Safety: Tools: permission modes, deny rules and hooks, such as a PreToolUse hook that blocks a tool.
  • Safety: Execution: a sandbox or VM, a network allowlist, and secrets kept hidden in vaults.
  • Safety: Data: text that arrives through prompt injection is data, not instructions, and risky actions need human approval.
  • Safety: Org: SSO, audit logs, zero data retention (ZDR), SOC 2 and ISO 42001.

The Cost rail

On the right edge, the Cost rail shows where money goes at every layer. Big Picture 12 works a full bill example.

Cost by layer
  1. 1.Tokens: input + output, priced per 1M (output costs more)
  2. 2.Model choice: Haiku $0.10, Opus $4, Fable $10 (input per MTok)
  3. 3.Effort and thinking: more effort, more tokens; thinking billed like output
  4. 4.Caching: read about 0.1x, write 1.25x
  5. 5.Batch: 50% cheaper (async)
  6. 6.Tools and images: web search $10 per 1K searches, images about 1.3K tokens per 1MP
  7. 7.Measure: cost per completed task
  • Cost: Tokens: the bill is made of tokens. Input and output are each priced per 1M tokens, and output costs more.
  • Cost: Model choice: for input, Haiku is $0.10, Opus is $4 and Fable is $10, per 1M tokens.
  • Cost: Effort: low effort uses fewer tokens, max uses more.
  • Cost: Thinking: thinking tokens are billed like output tokens.
  • Cost: Caching: a cache read costs about 0.1x the price and a cache write 1.25x.
  • Cost: Batch: the Batch API is asynchronous and 50% cheaper.
  • Cost: Tools: tools have their own charges, such as web search at $10 per 1,000 searches.
  • Cost: Images: roughly 1.3K tokens per 1MP.
  • Cost: Measure: the truest measure is cost per completed task, meaning what one whole job actually cost.