Sage: measuring AI you can't fully trust

How do you know your AI agents are behaving in production? Sage's answer is a platform we built: every service team measures its own agents, safely, on shared rails.

The problem

Sage ships agentic AI across many product teams. Each team knows what "good" means for its own agents; nobody wanted a central committee marking everyone's homework, or teams running unreviewed scripts with production access.

What we built

A secure evaluation orchestrator. Teams publish their own evaluation scripts; the platform fetches and runs them inside tightly scoped IAM roles, isolated from each other and from the crown jewels. Scheduled test runs, plus live evaluation triggered straight from production traces by a one-line Python decorator.

The plumbing that makes it stick

Results land as Parquet and surface through Athena; traces flow to self-hosted Langfuse; the whole estate is CDK-defined and deployed by pipeline. Autonomy for the teams, guarantees for the platform.