Back to blog
Product5 min read
Introducing Agent Evals: test your agents like you test your code
Evals bring regression testing to AI agents. Write a test once, run it on every deploy, and catch quality drops before your users do.
Ava ThompsonCo-founder & CEO
Shipping an agent is easy. Knowing it still works after you change a prompt, swap a model or add a tool is much harder. Today we're launching Agent Evals to make that part boring.
Why evals matter
Agents fail in quiet ways. A response gets a little less accurate, a tool call gets skipped, latency creeps up. Without tests, you only find out when a customer complains.
- Define test cases from real production traces in one click
- Score outputs with rules, reference answers or an LLM judge
- Block deploys automatically when scores drop
Your first eval in five lines
import { evals } from "@nova/sdk";
evals.define("triage-accuracy", {
dataset: "support-tickets",
scorer: "exact-match",
});“Evals caught a regression in our triage agent before it ever reached production. It paid for itself on day one.”
Agent Evals is available today on the Pro and Enterprise plans. Open any agent and click Evals to get started.