Self-improvement — how Tyran learns your repo

agents/retro.md · skills/retro/SKILL.md · hooks/scripts/retro-gate.mjs

This is Tyran’s centerpiece: you bring the harness, it does the improving. The more initiatives you run, the better it fits your repo and your style.

The loop

After every closed initiative, the tyran-retro agent reads the initiative’s journal, decision ledger, and agent reports — the densest source of signal — plus the git history of the work itself. Then it may improve Tyran (never your product code):

• distill a repo rule (“full test suite, not just touched files — a regression slipped in sideways once”),
• write a repo-specific skill for recurring work,
• tune an agent prompt variant,
• or delete/weaken one of its own earlier rules — the loop self-corrects downward, too.

What compounding looks like

Not a log of a real install — the shape the loop is designed to produce:

Initiative 1   Infers your stack, your validation commands, your deployment
               style from how the repo is actually worked. Asks once, about
               what it genuinely could not establish.
Initiative 5   Knows your shared-file hot zones, your flaky tests, your
               review taste. Its retro has already DELETED two of its own
               rules that weren't earning their keep.
Initiative 20  Has written repo-specific skills for your recurring work and
               merged two agents that overlapped. You mostly approve gates.

The anti-bloat filter (default answer: change nothing)

Every candidate change must pass all of:

1. Will it recur — in another initiative, another domain? One-off circumstances don’t qualify.
2. Was the cost real and measured — a lost agent cycle, a rework wave? Hypothetical discomfort doesn’t qualify.
3. Is it already covered by an existing rule? Then the problem is enforcement, not documentation — don’t add words to a rule that was ignored; build a mechanism or do nothing.
4. Can it be a one-sentence edit instead of a new section? Then only that is allowed.

Plus the extraction curator (adapted from the best pattern we found in the wild): is it non-googleable? specific to THIS repo? earned through real debugging? Generic programming knowledge is never extracted.

Changing nothing is a correct outcome

A retro that changes nothing is a correct, common outcome. Rejected candidates are logged — they protect future retros from re-litigating.

Hard limits on a single retrospective

Stated in the agent’s own contract (agents/retro.md), because a filter with no ceiling is a preference rather than a limit:

• At most three changes to the conductor skill per retro. More than that is bloat with certainty rather than risk; the surplus is recorded as candidates for next time and not implemented.
• Deletion is preferred to addition, in that order: (a) delete or merge what duplicates, (b) fix an existing sentence, (c) only then add. Several retros in a row with no deletion means the filter is too soft.
• Do not breed agents. A new agent only when the work has a genuinely different mode — different tools, different model tier, different lifecycle — never merely a different topic. A roster nobody can hold in their head is its own cognitive tax.

Guardrails (designed to be enforced, not promised)

Class	Examples	Who decides
AUTO	knowledge facts, rule tweaks, .tyran/config.yaml, agent overrides (since 0.1.44), new repo-specific skills (must pass an activation test)	retro commits autonomously; ledger entry; git revert rolls back
GATED	new/changed hooks, CLAUDE.md, budgets, deleting safety rules	retro proposes, you approve
KERNEL	the enforcement hooks, the rollback mechanism, this classification itself	humans only, by hand

The classification is a file in your repo (.tyran/policies/autonomy.yaml) and is enforced by a PreToolUse hook on write paths — the boundary does not depend on the model behaving.

.tyran/config.yaml is AUTO and it holds the deployment class, which is the one place that table trades safety for usability on purpose. It was GATED until a real install showed what GATED costs there: setup had inferred pnpm test, which in that repo is bare vitest — watch mode, never exits — and the agent that discovered every future agent would hang could not repair the file that said so. If that trade is wrong for your repo, set the rule back to GATED (see policy gate, which also measured the same escalation happening under GATED wherever Write is allow-listed).

Every AUTO entry carries confidence and usage/helpfulness counters; entries that stop earning their keep get degraded or retired by later retros. Two learning loops run at different speeds: fast and local (your repo), slow and curated (pull requests to the core — reviewed by humans).

The counters are fed by a closed loop rather than by promises. Every handoff carries a knowledge brief — scripts/knowledge.mjs brief selects the entries whose applies_to globs intersect the story’s predicted files, into a budget — and the agent’s final report owes a verdict on the entry ids it received. The retrospective folds those verdicts into the counters at close (the procedure lives in the retro skill, its one home). An entry nobody reports as helpful is a retirement candidate on evidence, not on taste.

The budget is pressure, and somebody has to look at the total. The brief names what it omitted on every run, but nothing added those omissions up — and measured on a real install the store had reached the state where brief '**' returned 1 of 31 entries: 104,178 codepoints of hard-won detail, read by nobody, while a per-entry oversize warning fired five times and read as a small untidiness each time.

node scripts/knowledge.mjs audit --dir .tyran/knowledge

It answers the aggregate question — how many entries could reach ONE brief, how many could not, and which are so wide they cannot appear even alone, where no budget a caller passes will ever reach them. doctor --state carries the same number as knowledge-store-unreachable.

It is a MEASUREMENT and never an edit. Which of two overlapping entries is the true one is a judgement, and a script that guessed would delete exactly the detail the store exists to hold. That has not changed: nothing scores text similarity, and no tool picks a winner.

What the store gained is a way to record the judgement without paying for it. A merge is an append: write the entry that states the fact once, give it a supersedes: naming every id it replaces, and touch nothing else. knowledge.mjs brief drops any entry named by another entry’s supersedes, so the replaced ones stop reaching handoffs immediately — while keeping their bytes, their provenance and the counters they earned over months, because no file holding them was edited.

That shape is the whole safety argument. A bad merge is not a lost history to be prevented by review; it is one file to delete, after which every entry it retired comes back whole. The audit reports the reclaimable total — superseded entries cost nothing in any brief, so deleting them is a git rm whenever you feel like it, and never a tool.

The one mechanical failure is a supersedes naming an id that is not in the store: the retirement silently does not happen, so the old entry keeps competing and the new one adds to the total. audit names those, and doctor --state counts live and superseded separately so its number still reconciles against the files on disk.

Retirement without a replacement is the other half, and it now runs on evidence rather than taste. audit lists the entries whose counters have written them off — reported wrong more often than helpful, or delivered three times or more and never once reported helpful. An entry delivered once is never a candidate: that is absence of evidence, not evidence. And if no entry in the store carries a non-zero counter at all, the audit says so plainly — the fold at retro close is not happening, which is a more useful finding than any list it could print instead.

This page promised a consolidation step that did not exist until 0.1.35 corrected it, and knowledge.mjs audit printed the same promise in its own output. It may describe one now because selectEntries genuinely suppresses a superseded id — a guard test asserts that by running a brief, not by grepping for a function name.

Where a lesson lands: four stores, four questions

A fourth store is only defensible if it answers a question none of the others answers. MISTAKES.md — plain prose at your repository root — does, and the question is recurrence.

Store	The one question it answers	Lifetime	Writer	How an agent receives it
journal finding	what did I discover during THIS run, and what proves it	one initiative	any agent, mid-run	nothing — it dies with the initiative, correctly
MISTAKES.md	what has gone WRONG here, how often, and what would have prevented it	the repository’s	the retrospective, at close	nothing — there is no read channel, on purpose
.tyran/knowledge/*.yaml	what must an agent about to touch THESE paths know	until the counters retire it	the retrospective	knowledge.mjs brief, pasted verbatim into every handoff
CLAUDE.md, inside the tyran:rules fence	the law that binds every session here, whatever it touches	until someone deletes the line	mistakes.mjs promote --law	it is in context on every turn

Read the axis, not the topics. “Bugs here, conventions there” is unfalsifiable and produces three spellings of one rule inside a month. The axis is shape and delivery: knowledge is forward-looking, path-scoped, machine-delivered and budgeted; MISTAKES.md is backward-looking, narrative, repo-global and counted; CLAUDE.md is law — no scope, no counters, no budget, paid on every turn.

MISTAKES.md is where evidence accumulates, the knowledge store is where it gets delivered, and CLAUDE.md is where it gets enforced.

Setup seeds the file create-only, and deleting it is the whole opt-out — no config knob for a file you can remove. Nothing reads it at session start: an unscoped injection would spend context on every session to deliver what the knowledge brief already delivers scoped.

The graduation pipeline — 3 to knowledge, 5 to law

a correction or a breakage, mid-run
   │  the agent appends a `finding` event (claim + proof, + the command)
   ▼
a `finding` in the journal                   [one initiative; dies at close]
   │  the retro at close: did something actually BREAK, or is this a discovery?
   │  mistakes.mjs add --signature <s> --proof F-12
   ▼
a MISTAKES.md entry, status `open`           [repo lifetime, yours to edit]
   │  mistakes.mjs repeats --threshold 3     (three OPEN entries, one signature)
   ▼
.tyran/knowledge/<topic>.yaml                [delivered in every matching handoff]
   │  mistakes.mjs promote --signature <s> --status knowledge:K-12
   │  ...and it happened twice more anyway — 5 open-or-promoted
   ▼
CLAUDE.md, inside the tyran:rules fence      [law, on every turn of every session]
   mistakes.mjs promote --signature <s> --law --rule '...'

The signature is the judgement; the count is mechanical. The model decides whether two failures are the same failure — a near-synonym resets the count to one and the lesson never graduates — and the script counts what it is given. Promotion never deletes an entry; it rewrites one status token, because the entry is the evidence that earned the rule.

Why 3 and 5. Three is “not a coincidence, and cheap to act on”: the knowledge store is scoped, budgeted, and retired by its own counters, so a wrong promotion costs one brief line. Five is the expensive one, because law is paid on every turn — and the gap between them is the diagnostic. A signature that reaches five has already been delivered in every matching handoff and broke anyway, which is the strongest argument for a rule there is.

Tyran writes the fence itself. At five occurrences the retrospective edits your CLAUDE.md between two markers, and never a byte outside them:

<!-- tyran:rules start -->
- Link the dependency directory into every new worktree before the handoff. (`worktree-missing-deps`, 5 occurrences — MISTAKES.md entries 2026-05-02, 2026-06-11, 2026-07-03, 2026-07-28, 2026-08-09)
<!-- tyran:rules end -->

One line: the rule, its signature, and the dated entries that earned it — the evidence is not decoration, it is how you disagree well. There is no approval gate. You meet the change the way you meet every other autonomous act — a decision event in the journal, a row on the board, a diff — and you say no by deleting the line, which does not come back (those entries are already at status law). --dry-run prints the line and writes nothing. An absent fence is appended once, with its heading, at the end of the file; a malformed one is refused by name and left alone.

The shipped policy classes CLAUDE.md as GATED, and that bans the hand, not the mechanism: a subagent’s free-hand Write is denied, while the script that demands five recorded occurrences passes. The boundary is not “agents may not change the law” — it is “the law changes only through the mechanism that requires evidence.”

State its floor with it, because a guarantee read as absolute is worse than a narrower one read correctly: the class binds the Write/Edit hand, and only hooks/** and .tyran/policies/** are refused to shell commands as well. A repo-root file is not in those globs, so a subagent’s Bash redirect into CLAUDE.md is not refused — measured, and stated where the class is set (policy-gate). What the mechanism guarantees is that the law arriving through it carries its evidence; what catches the other route is the diff, the same way you meet every other autonomous act.

Seeding has always been create-only, so a repo adopted before this rule shipped keeps its own policy until it copies the line from templates/policies/autonomy.yaml.

Authored, not a projection — and what that costs

MISTAKES.md is prose you edit. It is deliberately not a generated projection of the journal:

• a projection reports a hand edit as drift and destroys it on the next render — and a human correcting a wrong root cause is the most valuable edit this file will ever receive;
• journals are per-initiative, so archiving an old initiative would silently shrink a projected history, taking the recurrence count with it;
• most finding events are discoveries, not mistakes, and projecting them would fill the file with non-mistakes.

The cost, stated rather than discovered: nothing checks MISTAKES.md against the journal byte for byte, so it can drift — a fabricated entry, or a real breakage nobody wrote down. Every entry cites its initiative and the journal event id that proves it, so drift is auditable by a human and not enforced by a gate. That is a weaker guarantee than the projections carry, and no test will catch an entry nobody lived through.

doctor --state surfaces the ledger without nagging: mistakes-repeat-unpromoted (info) when a signature has earned promotion, mistakes-unreadable (warning), and claude-md-fence-missing (info) when entries claim law but no fence exists to hold the rule. An absent file produces no finding at all.

The one part that is not built: the update delta-review

Not built yet

Everything above runs today. What does not exist is the delta-review that runs after /plugin update: the step that compares a new core version against what your repo has already learned and proposes the reconciliation (“the core absorbed a rule you learned locally — delete the local duplicate”). Until it ships, a core update leaves your .tyran/ data and your local skills untouched — which is safe, and is also why a rule can end up stated twice after an update. Reconciling it is a manual read of the changelog against .tyran/knowledge/.