Skip to main content
With a contract in place, this is mostly waiting and one decision. Prerequisite: an eval contract on the task.
1

Switch the task to auto

Nothing changes yet. With no approved route the task still serves from your default model, so this deploy is safe on its own.
2

Wait for a candidate

Cheaper models are replayed against the contract out of band. Your production traffic is not used as the experiment, and no candidate touches a live request before you accept it.You get the candidate’s score against the contract and its measured cost delta, both on the same examples.
3

Read the proposal, not just the price

Two questions, in this order:
  1. Does it clear the contract, including the hard examples?
  2. What does it cost?
A candidate that clears by a hair on a set with no hard cases is not a candidate. If the margin looks too comfortable, the contract is probably too easy, and that is worth fixing before accepting anything.
4

Approve, then watch the route field

After approval, new calls for that workload use the new route. Log the response’s model so a quality change can be traced to a route change rather than guessed at.

​
If it goes wrong

Roll back. It is a route change, not a redeploy of your application, and the previous route is versioned. Pinning the old model in your request body also works as an immediate stop.

​
If nothing ever clears

That is a real answer, and a useful one: no existing cheaper model can do this task. It is the point at which a model trained on the task’s own traces and failures becomes the cheaper option rather than the expensive one.

How failures decide what to train

Why the calls that went wrong are the training signal.