Skip to main content
Every answer carries a confidence between 0 and 1 and a confidence_source.
  • calibrated — the primary engine (Jev) computes confidence from the shape of the probability distribution: all the mass on one option gives 1.0; an even spread gives a low value.
  • self_reported — the LLM fallback states its own confidence; Branch Pilot subtracts a malus (default 0.10) because language models tend to be overconfident.
For classify decisions, confidence is the smallest margin between a label’s probability and its threshold: a label hovering around its threshold makes the whole decision less certain.

​
Below the threshold

Each decision has a confidence threshold (default 0.70). When the answer’s confidence is lower, the uncertainty policy applies:
The response has status: "uncertain". For route decisions, choice is "__uncertain__"; probabilities are still returned so you can inspect them. Give this case its own branch in your workflow: ask the user, apply a default, or escalate.

​
Feedback

Any run can be marked good or bad in the history, with a note. Feedback does not retrain the engine; it feeds the statistics per decision and per version (rate of bad feedback, average confidence) so you can spot drift and rewrite criteria before it becomes a problem. Any run can also be turned into a test case in one click.