AI Gateway adds confidence-based decision fallbacks: when the primary model's answer misses your threshold, a backup model takes over

Decision requests through AI Gateway gain an escalation path: if the primary model's answer falls inside a user-set confidence condition, the request is handed to a fallback model to decide again. Previously fallbacks fired on model names alone, catching outright errors; now a fallback can also be triggered by the confidence of the answer itself. Confidence conditions cover Choice and Score questions, while Boolean questions escalate on a probability range; requests without the conditional object keep their existing behavior.

It is enabled by adding one conditional object to providerOptions.gateway.models that names which model takes over under which condition. The conditional object uses when to describe the trigger, such as escalating a given question when its confidence falls below 0.6; leaving out question makes the condition check every question of the matching type. In the example the intent question is typed as Choice and the condition is written as confidenceBelow 0.6, after which the fallback model supplies that question's choice answer.

The demonstrated scope is fallback configuration for AI Gateway decision requests: confidence conditions apply to Choice and Score questions, Boolean questions escalate on a probability range, conditions can be combined, and requests without the conditional object behave as before.

A triggered fallback runs a second decision, so both stages are billed; the material gives no guidance on choosing a confidence threshold, and the 0.6 in the example is only a configuration form.