How we cut token costs by 32% without touching a prompt

Token spend is the biggest line item for most AI teams. Over the last quarter we shipped three changes that reduced the average cost per run by 32%, with no change to output quality.

1. Route by difficulty

Not every step needs the largest model. A tiny classifier decides whether a step is simple or complex and routes it to the right model.

2. Cache by meaning, not by text

Semantic caching matches requests that mean the same thing, even when the wording differs. For support agents, the hit rate is often above 40%.

3. Trim the context

• Summarize long tool outputs before passing them on
• Drop conversation turns that no longer matter
• Keep system prompts short and reuse them across agents

All three optimizations are on by default for every Nova workspace. You can see the savings on your usage page.