At Zapier I build the evaluation infrastructure for agentic automation: simulated worlds that agents act in, end-to-end scenarios grounded in real outcomes, and the tooling that turns those measurements into release decisions.
I've been circling the same question for a while. At Genentech I built simulation engines so a portfolio of three hundred clinical trials could be planned before reality arrived. At Taltrics, which I co-founded, we shipped multi-agent systems to enterprises and post-trained them to stop hallucinating tool calls. The instrument keeps changing; the question — how do you trust a system before it meets the world? — hasn't. Along the way I published on trustworthy AI in healthcare.
Agents move decisions out of control flow and into the model. That part of the specification can only be recovered by watching the system act, which makes the eval the place where it gets written.
A model's job is to be honest about what it doesn't know, in a form a decision can use. Three years of simulating clinical trials taught me what that means.