Case study

Lemmath: $2,100+/mo → ~$240/mo

Lemmath is an AI math tutoring agent. Before we touched it, it was spending over $2,100/month on API costs — barely breaking even, with profit reduced to breadcrumbs. After optimization, it spends around $240/month while still tutoring with very few mistakes. Same task volume, same token usage — the only thing that changed was what it cost to run.

The biggest change: prompt caching

Lemmath's system prompt — the tutoring instructions, worked examples, and formatting rules — was sent as a plain string on every single call. That meant the model re-read and re-priced the entire thing at full input cost, every time a student asked a question.

The fix was to wrap the system prompt in a content block with a cache breakpoint. Same text, same behavior — but now Anthropic caches it server-side and only charges full price on the first call. Every call after that reads from cache at roughly a tenth of the cost.

const response = await client.messages.create({
  model: MODEL,
  max_tokens: 4096,
  system: SYSTEM,        // plain string
  tools: TOOLS,
  messages,
})

const CACHED_SYSTEM: Anthropic.TextBlockParam[] = [
  { type: 'text', text: SYSTEM, cache_control: { type: 'ephemeral' } },
]

const response = await client.messages.create({
  model: MODEL,
  max_tokens: 4096,
  system: CACHED_SYSTEM, // same text, now cacheable
  tools: TOOLS,
  messages,
})

Second change: model routing

Lemmath was running every tutoring session on Opus. Opus is a great model, but it's far more model than a structured math-tutoring task needs — most of the conversation is explaining steps, checking work, and following a fixed pedagogical format, not open-ended reasoning. We moved the workload to Haiku, which handled the same tutoring quality with very few mistakes at a fraction of the per-token cost. Between the model switch and prompt caching, that's what took the bill from $2,100+/month down to around $240/month.