Eval-Driven Development

An agent can satisfy every contract its builders wrote and still fail the task. The API accepts its arguments. The tools return valid responses. The workflow compiles. The result is wrong.

Testing has always helped engineers find requirements they did not know they had. Integration tests, exploratory testing, and acceptance-driven development all do this. Agents make the problem acute for a specific reason: the model now supplies decisions that engineers used to encode in control flow. It chooses tools, interprets failures, carries identities between steps, and decides whether it has enough information to act. Valid interfaces constrain those decisions without specifying them.

That is the part of the specification nobody can write in advance, because it lives in behavior rather than in code. So I no longer treat agent evals as tests applied after implementation. An eval is where an observed failure becomes a durable contract: the conditions under which the system must succeed, the effects it may produce, and the evidence that separates success from a plausible attempt.

Evals discover the contract

A test asks whether an implementation satisfies a known property. An eval asks which properties matter once a model, its tools, and a realistic task interact. The difference is not that evals use probabilistic assertions or run a model several times. The difference is that the behavior under evaluation reveals requirements that no component owner had reason to write down.

I saw this while evaluating an agent that assembled a multi-step sales workflow. A search for one CRM record worked five times out of five. The agent selected the action, supplied valid inputs, and received a record. A conventional integration test could have stopped there.

Inside a realistic sequence, nine of ten attempts failed. The workflow had to find a lead, look up the associated account, and draft an email from information spread across both records. The agent confused similarly named fields, carried large responses between steps, and sometimes introduced an unnecessary AI action to recover from its own confusion.

The search contract was satisfied. It was also insufficient. Returning the requested records did not establish that the agent could preserve their identities and use their fields correctly across a workflow. The useful specification included field provenance, context size, cross-step identity, and the agent’s ability to ask for more without receiving everything at once.

The fix that satisfied it happened to be progressive disclosure: large results came back as a compact manifest, and the agent asked for particular fields when it needed them. The scenario then passed eight of ten attempts, the field confusion disappeared, and runtime fell from roughly 210 seconds to 75. Those samples did not establish a rule about progressive disclosure. They gave me a concrete failure, an intervention that addressed it, and a scenario to keep when the model or implementation changed. The lasting requirement was correct use of the records. Progressive disclosure was one candidate for meeting it.

This pattern recurs because agent behavior emerges from combinations. Each component can satisfy its local contract while the model fails to compose their outputs under realistic context, ambiguity, and sequence length. The failure is not in any interface. It is in the decisions the model makes between them.

The usual objection is that this creates an infinite obligation: if the contract is discovered by observation, one could observe forever and never ship. That is not the claim. Most possible tasks have little information value. A scenario earns its place when it challenges a consequential assumption, distinguishes plausible implementations, or preserves a boundary that has already failed. The first scenario at such a boundary usually pays for itself by rejecting a claim that several locally correct implementations had allowed everyone to assume.

Nelson Elhage describes a test suite as a classifier that predicts whether a change is acceptable to ship.¹ An agent eval also classifies claims about the product. The first task is not to raise the score. It is to formulate a scenario capable of proving the claim false.

The world can make the spec false

If the eval serves as the specification, it has to describe the right world and judge the right consequences. It can fail in three places. The fixture or simulator can present the wrong environment. The grader can mistake activity for success. The harness can contaminate the evidence by allowing attempts to affect each other or the machinery observing them.

These are specification failures, not ordinary test maintenance. A broken unit-test fixture usually produces an obvious false result near the code under test. A broken agent world produces a plausible trajectory, a valid workflow, and a reassuring score. The agent appears to solve the task because the difficult part of reality never entered the run.

My first attempt to evaluate large CRM responses failed this way. I created a fixture with dozens of records and hundreds of fields, then ran a workflow intended to stress the agent’s context handling. The scenario passed.

The trace showed that the simulated action had returned {}.

The grader checked whether later workflow steps referred to the search output. It did not check whether that output contained the fixture’s records, or whether the final email used the correct fields. The simulator omitted the difficult input, and the grader rewarded the shape of the workflow. Two errors agreed with each other and produced a green result.

I replaced the generic response with deterministic mappings from action inputs to world state. The real fixture data reached the agent, and the grader inspected which records the final workflow used. Only then did the field-confusion failure appear. The eval became harder to pass because it finally represented the claim it was supposed to test.

The distinction is not “mocked bad, live good.” Live services contain changing data, rate limits, shared accounts, and effects that are difficult to reset, and they make repeated attempts less comparable. I prefer real product and integration code running against controlled worlds whose state is explicit and reset for each attempt. The simulator owns external records. Production code still owns discovery, validation, and execution.

A controlled world can still lie. It may infer a missing parameter that production rejects. It may expose generic fields where an authenticated account exposes resource-specific ones. It may accept an entity shape that the real API does not. More fixture data does not fix any of this. Fidelity comes from preserving the constraints relevant to the claim.

The third failure is the harness contaminating its own evidence. I found it while running five attempts of a workflow-repair scenario in parallel. The attempts shared one execution sandbox and one worker log; they looked concurrent from the runner’s perspective, but execution was serialized underneath, and evidence from one workflow could be attributed to another. What looked like variation in model behavior was partly a concurrency bug. Attempt-specific sandboxes, with logs, runs, and external effects bound to the attempt that produced them, turned five parallel runs into five separate evidence chains.

Agents do not return a value to their environment; they act inside it. Once evaluated code can mutate state, trigger retries, choose routes, or influence shared processes, the harness becomes part of the causal system being measured. So before I trust an eval, I want to see a deliberately broken candidate fail and a different valid solution pass, and I want evidence that the current attempt caused the observed effect. A grader that cannot distinguish a product failure from a fixture or harness failure does not define a usable specification.²

Outcomes cross ownership boundaries

An outcome-based specification crosses service and repository boundaries because users experience the composed system, not its internal ownership map. This is why end-to-end evals find failures that component tests cannot. Each component test checks one owner’s interpretation of a contract; the eval checks whether those interpretations compose into the promised outcome.

This is not an argument for replacing component tests. Unit and integration tests locate failures cheaply once the contract is known. The eval identifies contracts that no single component owns, then gives each owner a concrete boundary to enforce.

I encountered this while evaluating an agent that built a durable workflow containing an AI-powered action. The action advertised two structured inputs: a collection of tools and a collection of knowledge sources. The application declared schemas for both. An upstream API reduced them to opaque JSON fields before a downstream agent saw them.

Every component remained internally consistent. The application knew the schemas. The API returned valid fields. The downstream client accepted JSON. The agent still could not configure the action generically because the structure had disappeared between those layers. One client could have memorized the missing keys in its prompt; that would have made the scenario pass while preserving the platform defect. Instead, the fix propagated the application-declared JSON Schema through the API so any client could inspect the same contract.³ The same scenario surfaced a smaller disagreement of the same kind: an authentication field one schema marked optional and the execution service required.

A second scenario showed how several small failures combine into an unacceptable external effect. An agent built a workflow that searched Trello before creating a card. It invented search actions that did not exist, and the workflow SDK accepted them. During execution, the searches failed. The AI-powered step interpreted those failures as “no matching card” and proceeded with the write. The card was created successfully. A later output-validation error caused the durable runtime to repeat the step, producing five duplicate cards.

There were several places to intervene: action discovery, reference validation, interpretation of prerequisite failures, and retry behavior after a successful write. Blaming the model for inventing an action described only the first visible mistake. The outcome-based specification described the failure: a failed prerequisite search must not authorize a dependent write, and a failure observed after a successful external effect must not repeat that effect.⁴

This pattern recurs because distributed agent systems contain semantic gaps between layers. Types can prove that a response is valid JSON. They cannot prove that one service’s “no match” means the same thing as another service’s “search failed,” or that retrying a function is safe after the function has already changed the outside world.

The model receives the blame because its trajectory makes the failure visible. Sometimes the model did guess. The more useful question is where the system first had enough information to prevent the wrong outcome. In the Trello case, several layers did. The eval turned one failed run into separate contracts for discovery, validation, failure semantics, and retries, and none of those contracts dictates a tool sequence. They constrain the outcome across whatever sequence the implementation chooses.

The suite is the design surface

Once the eval becomes the specification, the suite becomes the place where product behavior gets decided, and the implementation becomes one candidate for satisfying it. This reverses the usual order. Teams make a product decision in a design document, encode it in a prompt or tool, and later add an eval to check the implementation. For agent behavior, the disputed cases appear only after a scenario forces the system to choose. The suite does not record the decision. It creates the conditions under which the decision can be made precisely.

Workflow recovery made this clear to me. Suppose an agent encounters an expired connection and finds another connection that appears to belong to the same person. Switching may complete the task. It may also change permissions, billing, data residency, or the account whose records receive a write. “Repair the workflow” does not say whether that substitution is authorized.

The same issue appears when a destination field disappears or an application introduces a new mandatory field. The agent may be capable of choosing a plausible replacement. Capability does not establish authority. The product decision concerns what the agent may change on the user’s behalf, and ordinary product language leaves it open.

We used recovery scenarios to make those boundaries explicit. A successful repair had to address the original failure, replay the same input, preserve unrelated workflow behavior, and stop for approval when the repair expanded authority. The scenario did not decide the policy for us. It presented a concrete choice that broad language had left unresolved, and once the choice was made, it gave us an executable way to preserve it.

That distinction matters. An eval suite cannot replace product judgment, and a passing score cannot justify whatever behavior the grader happens to reward. People still decide which outcomes are acceptable. What the suite changes is where and when that judgment gets exercised: on a specific trajectory, before the behavior hardens into code, rather than in the abstract.

Outcome-based specs also make implementation disposable. The CRM scenario did not require progressive disclosure by name. It required the agent to use the correct fields without drowning in irrelevant data. Progressive disclosure happened to satisfy that requirement better than returning the full payload. A later model may handle larger contexts, request fields differently, or remove the need for the mechanism. The scenario can remain.

This matters in light of Rich Sutton’s Bitter Lesson.⁵ Hand-coding one expected reasoning path makes an eval hostile to general methods that improve with more computation. Defining the world, the permitted effects, and the success condition does something different. It allows a stronger system to discover a new path while holding the product contract fixed.

The thesis would be wrong if the decisions a model makes inside a system could be enumerated before the system runs. In that world, conventional tests could encode every relevant precondition, outcome, and failure boundary, and evals would add sampling and monitoring without serving as specifications. I have not seen that world. I have seen locally correct components compose into wrong identities, missing schemas, confused fields, unsafe retries, and unauthorized recovery choices, and in each case the missing requirement became clear only after the agent produced a plausible but unacceptable trajectory.⁶

There are domains where the argument weakens. Open-ended research and long-horizon memory may produce outcomes too delayed or contested for a controlled scenario to settle. My experience comes from systems that eventually create records, send messages, publish workflows, select accounts, or ask for approval. Their effects give the specification something concrete to constrain.⁷ Even there, an eval is evidence within the conditions it exercises. Eight successful attempts out of ten is evidence of progress; it is also evidence that the task remains unreliable, and a controlled world’s assumptions still have to be checked against production.

The durable asset is not the prompt, the model, or the workflow representation. It is the executable account of what the system may do, what it must accomplish, and which evidence proves the difference.

Footnotes

1. Nelson Elhage, “Test suites as classifiers”, 2020. ↩

2. Repeated runs help distinguish large behavioral changes from isolated model variance, but five or ten attempts do not estimate a precise production success rate. ↩

3. Carrying the schema through the platform also serves conventional clients. The fix removed the need for every consumer to maintain a private copy of the input structure. ↩

4. Idempotency keys can prevent some duplicate writes. They do not remove the runtime’s need to distinguish execution failure from a failure observed after execution. ↩

5. Sutton argues for general methods that improve through search, learning, and increased computation rather than systems filled with human theories of intelligence. A controlled world supplies consequences without dictating the policy. ↩

6. I have changed identifying details from internal incidents. The causal structure and measurements remain the same. ↩

7. Even concrete effects require identity. Sending the right message to the wrong workspace is structurally valid and operationally wrong, so account and world identity belong in the evidence chain. ↩

@misc{spezzatti2026evaldrivendevelopment,
  author = {Spezzatti, Andy},
  title = {Eval-Driven Development},
  year = {2026},
  howpublished = {\url{https://andy.spezzatti.com/writing/eval-driven-development}}
}