You've put your finger on exactly the right nerve — and it's also where Chitos parts ways with static analyzers.

Your critique targets static reachability proofs: the "safe" verdict inheriting the edges the call-graph never saw. Chitos's confirmed verdicts don't come from there. Phase 3 fires real payloads and observes real responses, so a confirmation is an executed round-trip, not an inferred reachable path — it's what the target actually did, not what our model claimed it would do. For positives, that sidesteps the call-graph-blindness problem.

Where your point lands fully is on negatives. That's precisely why Chitos never emits "proven-safe." Unconfirmed is reported as not demonstrated, never closed. Absence of a proof is not a safety claim — and we work hard not to blur that line in the UI.

On auditability, I completely agree. Today each finding already carries its attack vector, the payloads attempted, the response delta, and the verifier's reasoning. The next step is making that "what I tried and what I trusted" trail a first-class citizen for negatives too — because an un-auditable green light is, as you say, just a prettier suspect list. Thank you for the framing.

it dovetails with our invariant: Chitos never emits 'proven-safe' (absence is not a safety claim). That handles the "don't trust a green light" half; the coverage denominator you're pointing at is the other half.

How we mean to express the denominator: we model the attack surface as an enumerable space — reachable entry points × parameters/sinks × vuln classes. Then every 'not demonstrated' can carry coverage = exercised nodes / modeled nodes, per phase and per class. e.g. "exercised N% of the modeled SQLi surface; the unreached region has this shape."

Candidly: today Chitos emits the numerator (what fired) and the invariant (no proven-safe). The modeled-surface denominator (coverage %) is what we're building toward. And we'd flag the deeper limit — it's coverage against the surface we modeled, not the true surface; unknown-unknowns still escape, so the denominator is itself a function of the model, and we'd label it as such.

Genuinely keen on your take on modeling the surface — that's what decides whether the denominator means anything.

You've pinned it exactly. Surprise rate turns the denominator — the modeled surface — from an assertion into a measurement: it makes the coverage number falsifiable.

Honest answer to your question: today Chitos records what Phase 3 fired (the numerator), but it does not yet formally diff "nodes actually reached" against "nodes the static model predicted reachable" and emit that as a surprise rate. That's precisely the gap you've named, and it's the right next step.

Here's how we intend to fold it in: every time a Phase 3 payload lands on a node the static model marked unreachable, we log it as a surprise, feed it back into the enumerator (the surface model gets falsified by its own execution), and report it per phase and per vuln class alongside each "not demonstrated." A low surprise rate is then what earns the coverage number, rather than us asserting it.

One extension: surprise rate is most meaningful stratified by vuln class (1% on the SQLi surface and 1% on the XSS surface don't mean the same thing), and to separate "accurate model" from "blunt prober" it has to be read together with a payload-diversity measure — a prober incapable of surprising will trivially report a low rate. Thanks for this; it's the kind of push that keeps the tool honest.