Teaching an AI system to say "I don't know"
How Evident grounds every decision in retrieved evidence, uses a bounded agentic loop, and refuses to recommend when the evidence is too thin.
Most LLM failures in production aren't dramatic. They're quiet. The model returns a fluent, confident answer that happens to be wrong, and nothing in the output tells you it was guessing. For a system whose job is to decide who is worth contacting, that quiet confidence is the entire problem. Evident is built around the inverse instinct: ground every decision in retrieved evidence, expose the reasoning, and make refusal a first-class outcome.
Three outcomes, not two
A naive version of this system has two outcomes per contact: recommended or not. Evident has three, recommended, not_recommended, and insufficient_evidence. That third outcome is the important one. It's an explicit refusal: the system is saying the public evidence it could find is too thin to justify a confident decision either way. Surfacing that honestly is more useful than forcing a binary call the evidence can't support.
The refusal has to live in one place
The interesting engineering problem isn't deciding to refuse, it's making sure the refusal can't be quietly overridden. Evident evaluates contacts down several paths: the primary LLM evaluation, a heuristic fallback when the model is unavailable, and a second pass for uncertain candidates. If each path had its own notion of "is this strong enough," an over-confident model on the second pass could upgrade a thin contact to recommended and undo the refusal.
The fix is a single refuse-when-weak gate applied uniformly across all three paths. One definition of sufficiency, enforced everywhere. An over-confident model can't talk its way past it.
Bounded, not open-ended
When a contact is uncertain, Evident does something agentic: it retrieves more evidence and re-evaluates. But "agentic" is where a lot of LLM systems become unpredictable and expensive. So the loop is deliberately bounded, at most one adaptive retrieval pass and one re-evaluation, then the system finalizes. That cap keeps the reasoning explainable and the cost predictable. You can always answer the question "why did it decide this, and how much did it spend getting there?"
Cost as a correctness property
Predictable spend isn't a nice-to-have bolted on at the end; it's part of the design. A deterministic pre-filter removes weak candidates before any model budget is spent. Per-run caps bound evaluations, drafts, retries, and outbound fetches. Timeouts and backoff keep a slow dependency from turning into a runaway bill. Treating cost as a property the system enforces, not a number you hope stays low, is what makes the whole thing safe to actually run.
Why it matters
The pattern generalizes well beyond outreach: ground decisions in retrieved evidence, expose the reasoning, enforce a single sufficiency gate across every code path, bound the agentic work, and make cost a first-class constraint. A system that can say "I don't know", and mean it, is more trustworthy than one that always has an answer.