> ## Documentation Index
> Fetch the complete documentation index at: https://docs.protege.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Write an eval contract

> Define what a correct answer is for one task. Nothing can be routed or trained until this exists.

An eval contract is a fixed definition of correct for one task, with scored
examples behind it. It is the gate every route change passes through.

```mermaid theme={null}
flowchart LR
  A["Pick one task"] --> B["Write the rule"] --> C["Collect examples"] --> D["Score the incumbent"] --> E["Baseline"]
```

**Time:** a few hours, mostly spent arguing about edge cases. No GPU cost.

<Steps>
  <Step title="Pick the task, not the workflow">
    One task from your cost split, usually the most expensive one. If you cannot
    write the rule below in one sentence, the task is still too broad. Go back to
    [Split a workflow](/tutorials/split-a-workflow).
  </Step>

  <Step title="Write the rule">
    One sentence, in the language of the domain, that a non-engineer who does this
    job would agree with.

    | Weak                         | Strong                                                                                       |
    | ---------------------------- | -------------------------------------------------------------------------------------------- |
    | "The extraction is accurate" | "The total matches the invoice's stated total to the cent"                                   |
    | "The reply is good"          | "The reply answers the question asked and states no policy we do not have"                   |
    | "The intent is right"        | "The intent is one of the eight defined labels, and matches what an agent would have picked" |

    The strong versions are checkable by someone who is not on the ML team. That
    is the bar.
  </Step>

  <Step title="Collect examples, including the hard ones">
    Pull real inputs from the task's traces. Aim for coverage of the shapes you
    actually see, not a round number.

    <Warning>
      A set of easy examples produces a contract everything passes, which gates
      nothing. Deliberately include the cases that caused a retry, an escalation
      or a complaint. Those are the ones a cheaper model will fail on.
    </Warning>
  </Step>

  <Step title="Score the incumbent">
    Run your current model against the set and record the score with its spread
    and its cost. This is the number every candidate has to beat, and without it
    "cheaper" means nothing.
  </Step>
</Steps>

## What the contract buys you

Once it exists, routing can move the workload, because there is something to
gate the move on. Until it exists, it stays on the model you name.

It is also yours. The contract and its examples leave with you if you leave, in
the same way the weights do.

<Card title="Next: route a task down" icon="route" href="/tutorials/route-a-task-down">
  Use the contract to move the task to a cheaper model.
</Card>