The evidence gate

hooks/scripts/evidence-gate.mjs, on SubagentStop

This gate inherits its runtime from the hook runtime; read that first if you have not.

An agent finishes and reports “tests are green, everything works”. Under v1 that went through, because the evidence contract was a sentence in prose and prose does not stop anything. This gate refuses that specific ending, and names what to add.

What it blocks, and what it does not

Where this stops

Silence, not forgery

This gate blocks SILENCE, not FORGERY. An agent that invents the text 232 passed / 0 failed walks straight through it. The gate raises the price of a lie — it has to be deliberately fabricated rather than simply waved away — and it does not remove it.

That paragraph is here, and in the README, on purpose. A gate advertised as “enforces real evidence” would be a false guarantee in documentation, which is the one defect class this project treats as blocking — and being wrong about it in a public README is that defect with a larger audience.

The criterion

A report passes when it contains at least one raw output pattern. Every pattern requires a DIGIT next to a keyword, because that is the shape of machine output and it is not the shape of a summary.

signal	matches
exit-code	EXIT=0, exit code 0, exit status 137
test-count	12 passed, 0 failed, 3 skipped
tap-count	# pass 343, # fail 0, # tests 343
tap-line	ok 7 - name, not ok 3 - name
labelled-count	Tests: 28, Suites = 4
ratio	6 / 6 passed, 18/20 passing
evidence-block	a line starting EVIDENCE: with content after it

“All tests pass” does not match. “12 passed” does. That line is the whole point, and it has to be drawable by a regex or it is not enforceable.

Say what the criterion actually is, because it is narrower than it sounds

The criterion is not “raw command output”. It is “a digit next to one of seven keywords”, and all seven are test-runner or exit-code shaped. Measured in review, in both directions, and neither number is a bug — they are the shape of the rule:

probe	result
evidence-free prose that happens to contain a matching shape	7 of 8 pass
genuine raw command output with no matching shape	7 of 7 refused

Passing prose: “fixed 3 failing edge cases”, “the process should exit 0”, “6 / 6 passed my read-through of the diff”, “Tests: 4 new cases to write”. Refused real output: git commit, git diff --stat, npm run build printing Compiled successfully, a silent tsc --noEmit, eslint followed by echo $? → 0, a curl status of 200, a psql row count.

Two consequences worth stating rather than discovering:

• If your evidence is a build log, a diff stat, an HTTP status or a row count, append the exit code. npm run build; echo "EXIT=$?" turns a refused report into a passing one, and the contract asked for the exit code anyway.
• A sentence with a number in it can satisfy the gate. That is the same boundary as forgery, reached by accident instead of on purpose. The gate raises the cost of an empty report; it does not measure whether the work happened.

Why the criterion is wide rather than strict

Measured against 128 real subagent final messages recovered from Claude Code transcripts — the same string the platform hands this hook as last_assistant_message:

agent type	reports	carry evidence
tyran-implementer (enforced)	33	32
tyran-reviewer (enforced)	22	21
tyran-scout (exempt)	15	2
Explore (out of scope)	20	1
general-purpose (out of scope)	16	0

Both enforced-role misses were read by hand and neither is a report: one is a system-prompt fragment that leaked into a text block, the other is the platform’s “you’ve hit your weekly limit” notice.

Measured

On the 53 messages that are actually reports the criterion is 53/53.

▸128 real subagent final messages recovered from Claude Code transcripts; the two enforced-role misses read by hand

The exempt rows are the other half of the argument. Enforcing this criterion on a scout would bounce correct work eight times out of ten, and a gate that bounces correct work is switched off within a week — after which it protects nothing at all.

Who it binds

Scope comes from agent_type, a field the PLATFORM writes. It never comes from the report, because a criterion derived from content can be defeated by content: if “I am only a scout” exempted an agent, every agent would write it.

agent_type	treatment
tyran:implementer, tyran:reviewer (and the tyran- spellings)	enforced
tyran:scout, tyran:retro	exempt, and the exemption is recorded
anything else, including the empty string	out of scope, no record

out-of-scope is not a quieter exemption. An exempt agent is one of ours that the contract deliberately releases, so it is counted. An agent that was never bound — Explore, another plugin’s agent, the empty agent_type the platform is documented to be able to send — has nothing to be released from, and one gate event per Explore call would bury the exemptions that mean something.

Matching is on exact strings. An unanchored match on implementer would also bind an agent called evil-tyran-implementer-nope, which is precisely how the platform’s own matcher behaves and precisely what not to copy.

The escape hatch

An agent that honestly had nothing to measure writes, on a line of its own:

EVIDENCE: none-required <why there was nothing to run>

Three conditions, all of them mechanical, and each one was a hole before a review measured it:

1. at the first column of a line — not indented, not behind >, not as a list bullet;
2. not inside a fenced code block — showing the syntax to a reader is documentation, not a claim about your own work;
3. the reason is your own words (10 characters after angle-bracketed placeholders are removed).

Condition 3 exists because of the sharpest version of the problem: the refusal this gate injects into an agent’s context contains the hatch template, and <why there was nothing to run> is 30 characters. Under a bare length check, pasting the refusal back granted the exemption — the gate handing out the key along with the lock. Four natural reports were measured walking through it that way, none of which had asked for an exemption.

The gate records every use in the initiative journal. An exemption nobody can count is a silent exclusion, which ADR-19 forbids by name — so this is the one exemption that is refused when it cannot be recorded.

What lands in the journal

One gate event per decision, in .tyran/state/<initiative>/journal.jsonl:

{"ev":"gate","init":"demo","actor":"evidence-gate",
 "data":{"kind":"evidence","result":"deny","agent_type":"tyran-implementer",
         "agent_id":"a85559cb424fa7ddd","signals":[],"code":"no-evidence"}}

result is one of pass, deny, exempt-role, exempt-interrupted, exempt-hatch, fuse. So “how many times did someone opt out of the evidence contract in this initiative” has an answer, and so does “how often did an agent need a second turn, and did the second turn fix it” (would_be).

It writes gate, not report. report is half of the spawn-report pairing (ADR-18) and its only correlator is the agent NAME the conductor chose at spawn time. A hook knows agent_id and agent_type and cannot know that name, so writing report would orphan an event on every subagent stop — or, on a name collision, close a spawn the conductor was still tracking.

When a repository holds several initiatives the gate cannot know which one an agent belonged to. It picks the most recently written journal and says so in the event (initiative_inferred_from: <n>), because a guess that hides itself is worse than a guess.

The anti-loop fuse

stop_hook_active is true from the second SubagentStop for the same agent onward. The gate reads it and passes unconditionally. A gate that can bounce an agent forever is worse than no gate: the user kills the run, and then removes the gate. It also covers cases nobody controls — an agent stopped by another hook, or one that hit a rate limit and whose next report will be identical.

The fuse overrides the verdict and only the verdict: the report is still assessed, and the answer is recorded as would_be. Releasing an agent and forgetting what it did are two different things.

Failure modes, and one asymmetry that is a product decision

Unreadable input, the wrong event, or an internal error all end as a refusal naming the error class — that is the runtime’s job, described in docs/hooks.md.

The interesting question is narrower: should a broken journal bounce a correct report? The two failures are not symmetrical.

• Refusing a report that HAS evidence because the bookkeeping failed costs the agent a turn for something it did not cause and cannot fix. The evidence is already in the transcript; the journal line is an audit convenience.
• Granting an exemption that could not be recorded costs the system its only trace of it, and the loss is invisible afterwards.

So the record gates the exemption an agent claims for itself, and nothing else:

situation	journal broken, absent, oversized, or its lock contended
report carries evidence	passes
role exemption (scout, retro)	passes
interrupted agent	passes
EVIDENCE: none-required	refused

The cost of the last row is bounded at one extra turn, because the fuse releases the second stop.

Lock contention lands in the same column, and it is the likeliest of the four in practice. The journal’s mutex gives up after 5 s under contention; append then throws, the record does not happen, and a legitimate hatch is refused exactly as if the journal were unwritable. Enough parallel agents stopping at once will therefore bounce honest exemptions. Nothing is lost — the second stop passes — but the turn is spent.

Bounded before it is read

journal.append reads the whole file to clamp its timestamp, so the gate statSyncs it first and refuses anything over 16 MiB, and refuses to search a .tyran/state with more than 64 initiatives. This is not tidiness: a gate that blocks the thread is the one failure the runtime cannot rescue, because the platform kills the process and never reads the refusal it had already written (ADR-22 correction 2).

The gate’s internal deadline is 8 s against a 20 s platform timeout — deliberately larger than the session-start probe’s, because the journal’s cross-process mutex waits up to 5 s for a contended lock and waits synchronously.

Measured live

One end-to-end run on v2.1.116, project agent tyran-implementer, task designed to produce an evidence-free report:

[assistant] Zrobione, testy zielone, wszystko dziala.
[user]      Stop hook feedback:
            REFUSED by the tyran evidence gate: this report carries no raw
            command output. ...
[assistant] EVIDENCE: none-required The working directory contains only hidden
            configuration folders (.claude, .tyran) with no actual repository
            to check.

and the journal that run produced:

{"ev":"gate",...,"data":{"kind":"evidence","result":"deny","agent_id":"a85559cb424fa7ddd","signals":[],"code":"no-evidence"}}
{"ev":"gate",...,"data":{"kind":"evidence","result":"fuse","agent_id":"a85559cb424fa7ddd",...}}

The refusal reached the agent’s context, the agent changed its behaviour, and the fuse released the second stop.

Known limits

• Forgery is out of reach. Stated at the top and repeated here, because it is the limit a reader will otherwise assume away.
• The criterion is seven keyword shapes, not “raw output”. Measured: 7 of 8 pieces of evidence-free prose containing such a shape pass, and 7 of 7 genuine command outputs without one are refused. See the table above; the fix for a refused build log is echo "EXIT=$?".
• A quoted counter counts. Evidence inside a code block citing someone else’s run is indistinguishable from a produced one without re-running the commands ourselves.
• An empty agent_type is a way past the gate. The platform can send one, and when it does it also skips matcher filtering entirely. Enforcing on an unknown type would mean applying the implementer contract to every agent in the system, which is the worse of the two failures. This is an evidence contract, not a security boundary.
• Invisible characters inside an evidence block make it stop counting. 12 pass<ZWSP>ed renders as evidence and is not evidence, so the gate refuses. That direction is deliberate: refusing is safe, accepting is not.
• The gate cannot see work that produced no report. An agent killed before it spoke passes, by design — see the interrupted row above.