There is a moment in building any safety tool where you have to decide what it says when it does not know.
Mine came while I was working on a static analyser — a tool that reads Apex without running it and works out how far a given agent action can actually reach into an org. I had a case in front of me that the analysis genuinely could not resolve. The tool had two output states at that point: finding, or clean. Nothing in the code path fit either one, so whatever I did next was going to be a lie of some kind.
The easy lie was to call it clean. Nobody would ever have complained. Clean results do not generate questions.
That is exactly why I did not do it.
In a safety tool, the dangerous output is not a false alarm. It is a silent pass — because nobody investigates a green light.
A false alarm costs an hour; a silent pass costs the incident
Let me put the two mistakes side by side, because the asymmetry is the whole argument and it is easy to feel it backwards.
A false alarm is when the tool flags something that turns out to be fine. Somebody spends an hour looking, finds nothing, mutters about the tool, and moves on. The cost is an hour and a little irritation.
A silent pass is when the tool says clean about something that is not. Nobody spends an hour, because nothing asked them to. The cost arrives later, wearing a different name — a data exposure, an audit finding, a customer’s record on the wrong screen — and by then nobody connects it back to a green tick from weeks ago.
Those two mistakes are not on the same scale, and a tool that treats them as symmetrical is quietly optimising for the comfortable one. School taught me the same lesson in a gentler setting: a pupil who tells you they do not understand has handed you something you can work with. A pupil who nods and says nothing has handed you a problem you will meet again, in an exam.
Three answers, not two
So the analyser has a third state, and it says something specific:
- Finding — I have determined this reaches somewhere it should not.
- Clean — I have determined this does not.
- Undetermined — I could not determine it. This is my answer, not my silence.
The important word is determined. A finding and a clean result are both conclusions, and both carry the tool’s authority. Undetermined carries none, and that is the point: it hands the question back to a human, with the question still intact.
What it must never be is a fourth, unwritten state — where the tool could not tell, and reported clean, and you had no way to know which of the two you were reading.
The blind spot I chose to write down
One class of finding is permanently undetermined by design: polymorphic lookups.
If you are new to the term, a lookup field normally points at one object — a Contact, an Account. A polymorphic lookup can point at several different objects, and which one it points at is decided by the data at runtime, not by the schema. The classic example is a task’s related record, which might be an Account today and an Opportunity tomorrow.
A static analyser reads code without running it. So when a path leads into a polymorphic lookup, the honest statement is that the destination depends on data the analyser is not allowed to see. Not “safe”. Not “unsafe”. Undetermined.
I could have picked the likely case and reported on that. Instead the tool reports it as undetermined and the documentation names it as a permanent limitation — written down, in the open, where a user reads it before they trust a result rather than after they are surprised by one.
A documented blind spot is a boundary. An undocumented one is a trap, and the person it catches is usually the person who trusted you most.
Degrading honestly when a dependency is missing
The same principle shows up in a less obvious place: what a tool does when part of it is unavailable.
The analyser has a preferred backend for its analysis. When that dependency is absent, it falls back to a weaker one — and it says so. It does not quietly produce thinner results with the same confident presentation, and it does not present a partial analysis as though it were the full one.
This is worth building deliberately, because the failure mode is so easy to write by accident. A try/catch, a fallback path, no message — and now your tool has two very different levels of rigour that look identical from the outside. That is a silent pass wearing a different coat.
If your tool can run in more than one mode, the output must say which mode produced it.
Two deliberate gaps in an evidence pack
The strongest version of this idea is in a different tool of mine: an EU AI Act evidence pack — a bundle of artefacts you can hand to somebody who needs to see how an AI system was checked before it went anywhere near users.
It ships with two deliberate gaps, and they are locked in place by tests. If someone were to quietly fill them in, the test suite fails.
That sounds perverse until you have been on the receiving end of one of these packs. A report where every single item is green does not read as reassuring. It reads as marketing — because the reader knows, from experience, that nothing real is complete. The green-everywhere report tells them nothing about the system and quite a lot about the author.
Two honest gaps do something a hundred green ticks cannot: they establish that the author was willing to write down what they had not covered. That willingness is the only reason to believe the rest of the document. And locking the gaps with tests means the honesty is structural — not a habit that erodes the first week someone is in a hurry before a demo.
What this looks like in your own org
You do not need a static analyser to apply any of this. Look at the checks you already have — a validation rule, a deployment gate, a dashboard tile, a health-check report — and ask each one a single question:
When this cannot tell, what does it say?
If the answer is “it says the same thing as when everything is fine”, you have a silent pass in your org, and it is currently indistinguishable from good news.
Your next step
Pick one green indicator you rely on and give it a third state this week. It can be crude — a status of UNKNOWN, a report line reading “not evaluated”, a note saying which mode produced the result. Then find the one thing your check genuinely cannot see, and write it down where a user will read it before they trust the output.
An unknown you can see is a task. A pass you cannot verify is a surprise waiting for a worse day.