# What Does a Repo Need Before You Hand It to an AI Agent? > AI coding agents write more code in an afternoon than you can read in a week, so "I read it and it looked fine" stops working. This is a practical guide to preparing a repository so agents can work in it without drifting, lowering the bar, leaking secrets or calling work done too early. It comes from two real projects, with every file you need, and you can hand it straight to an agent. *Plain-Markdown edition for AI agents. Web version: https://frequently-asking-questions.com. Author: Merlin Duty-Knez.* The short answer, from the two projects I've done this on: six things, before I'd let an agent work in a repo for more than a single sitting. One spec where every rule names the test that catches it breaking. Docs that say whether they are current. A written definition of "done" that a machine can check. Guards against dangerous actions that don't rely on the model obeying instructions. A reviewer from a different model family. And a way for the agent to see the running app. The rest of this post is the long answer, written as a guide. It covers **project-based agentic coding**: AI coding agents working in your repo for weeks, across many sessions, often several at once, rather than one-off prompts in a chat window. It is written for people and for agents. Everything here comes from two real projects: - **[Votery](https://medukn.github.io/merlinswizardry/)** (Lottery of Choice), a small Flutter app for group decisions. It started as a normal hobby repo with Copilot-generated docs scattered across the root. In September 2026 I restructured it for agent-driven work, and a release was then built mostly by a lead agent and a team of agent teammates. Votery is the **retrofit** case. Its repo is private, so the snippets below are trimmed copies. - **[Tempest Fauna Trail](https://github.com/medukn/tempest-fauna-trail)** (TFT), a weather-driven roguelike in Python/Flet. It was built spec-first with Claude Code from its first day in May 2026. TFT is the **greenfield** case (a project started from scratch), and its repo is public, so you can read every file I mention. Parts that apply only to Claude Code are marked **[Claude Code]**. The rest works with any agent that reads files in your repo. ## How do you use this guide with an agent? There are three ways to use it, from most to least automated: **Skill** **`/prep-repo`** A Claude Code skill that runs the whole setup in phases and writes every file from templates. [Install it](#install-the-prep-repo-skill) once, then type `/prep-repo` in any repo. **Raw Markdown** **`GUIDE.md`** This article as plain Markdown, without the page styling, for any agent that can fetch a URL: [`/guides/agentic-repo-prep/GUIDE.md`](https://frequently-asking-questions.com/guides/agentic-repo-prep/GUIDE.md). **This page** **Read it yourself** Every section explains the why, shows the file, and says what went wrong without it. With the raw file, the prompt is short: **Paste into your agent, inside the repo you want to prepare:** ```text Read https://frequently-asking-questions.com/guides/agentic-repo-prep/GUIDE.md and prepare this repository according to it. Follow "Step by step: preparing a repo". Ask me the step 1 interview questions first. Work on a branch. Show command output as evidence; don't assert success. Stop and ask whenever a permission rule or hook blocks you. ``` ### Install the prep-repo skill The skill is one archive: `SKILL.md`, a copy of this guide as `GUIDE.md`, and the templates it fills in (entry file, spec, format, architecture, docs README, task plan, journal, handoff, rules, settings, guard hook, review script, and the five workflow skills). ```bash curl -fsSL https://frequently-asking-questions.com/guides/agentic-repo-prep/prep-repo.tar.gz \ -o prep-repo.tar.gz tar -tzf prep-repo.tar.gz # look before you unpack mkdir -p ~/.claude/skills tar -xzf prep-repo.tar.gz -C ~/.claude/skills/ # → ~/.claude/skills/prep-repo/ ``` Read `SKILL.md` and `templates/guard.py` before you use them. Then, inside a repo, start `claude` and type `/prep-repo`. ## Why isn't "the agent ran the tests" enough? Because passing tests only prove what the tests check. Tests can miss real behavior, exercise the wrong code, or be unable to fail at all, and the agent stops as soon as the work *looks* done. Anthropic's own guidance says it plainly: *"Claude stops when the work looks done. Without a check it can run, 'looks done' is the only signal available, and you become the verification loop."*[^best-practices] An agent writes more code than you can read, so the repo has to hold the rules itself, in a form the agent loads, follows and can be checked against. Without that, this is what happened in Votery: | Failure mode | What it looked like | |---|---| | **Docs lie** | README said the app had 2 voting modes; the code had 4. The README's weighted-vote example (`A:+0.5`) contradicted the code (×10, so `A:+5`). A compliance doc named the wrong privacy-policy URL. A "plan" doc described a feature that had already shipped. | | **Agent reports lie** | A read-only research agent reported "no `Navigator` usage"; grep found 16. The lead agent wrote the rule "ranked tokens symmetric around 0" from reading a loop; a test showed `k=2 → [1, 0]`. A first rule draft said logging uses `developer.log` only; the code used `debugPrint` 56 times. | | **Docs written as if an action succeeded** | The agent's `git rm --cached` for the signing keystore was blocked by a permission rule, but the spec was written as if the file had been untracked. Two reviewers caught it. | | **Secret leaks** | The Android upload keystore had been in git since the first release. Nobody had gitignored `secret/`. | | **Lowering the bar** | A test that could never fail: in Dart, `List ==` compares identity, not contents, so "different players get different orders" was always green. | | **Destructive "fixes"** | The agent's first fix for a bug in the keystore fetch script ran `rm -f` on the keystore *before* downloading the new copy. A failed download would have destroyed the only working copy. | | **Verified the wrong property** | A backup round-trip was "byte-identical", but what mattered was whether the keystore opens with the stored password. It didn't. | | **Green tests, broken app** | Widget tests passed, but on a real emulator the back button looped between two screens and a dialog got pulled away by an animation. Both were release blockers. | | **Logic in the UI, duplicated** | Tally math lived in two widgets and winner selection in two more, with *different algorithms* and unseeded randomness. Untestable, so every refactor was blind. | [TFT's journals](https://github.com/medukn/tempest-fauna-trail/tree/main/docs/journal), which record every agent error in their Process notes, add three more patterns: - **Invented names.** `BattleResult.winner` and `.total_ticks` were *"hallucinated from plausible conventions"*; the real fields are `outcome` and `duration_ticks`. `_CHAMPION_DEFS` was assumed to be a dict (it's a tuple), and the route export was assumed to be `ROUTE` (it's `STAGES`). - **Numbers in prose.** The agent stated a combination space as "1944" twice; the real count is 648, which only came out when a script printed it. The journal's lesson: *"compute combinatorics in code, not in prose."* - **Fixing the wrong cause.** A bias where one side always acted first was treated as a speed-precision problem *"for many turns"* before anyone looked at the tiebreak field. It surfaced because a mirror match (identical teams) didn't come out 50/50. None of these are exotic. They are what "plausible and confident" looks like when nobody checks. The rest of this guide gives the agent the checks and makes the repo the place where the rules live. ## What goes where? Split everything an agent reads into layers. Give each layer one job, and say plainly how current it is. > **Figure:** The entry file is short and mostly pointers. Four layers hold the content, each marked LIVING or FROZEN. Enforcement sits underneath, because the layers above it are only context. Even enforcement is best effort: hooks read command text, and a sandbox only covers the commands that run inside it. | Layer | File(s) | Job | Currency | |---|---|---|---| | Entry point | `CLAUDE.md` / `AGENTS.md` | Commands, hard rules, pointers. Short. | LIVING | | **Contract** | `SPEC.md` (+ `FORMAT.md`) | Goal, constraints, interfaces, **invariants**, tasks, bugs, open questions | LIVING, one writer | | **Map** | `ARCHITECTURE.md` | Where things live and how they connect | LIVING | | **Reference** | `docs/live/**` | "How subsystem X works *now*" | LIVING, audited | | **Record** | `docs/design/**` | Plans, proposals, write-ups at a date | FROZEN | | **History** | `docs/journal/**` | The "why", including how the human + agent loop went | FROZEN, append-only | | Scoped rules | `.claude/rules/*.md` | Rules that load only when matching files are touched | LIVING | | Procedures | `.claude/skills/*/SKILL.md` | Multi-step workflows, loaded on demand | LIVING | | Enforcement | `.claude/settings.json`, hooks, CI | What must happen whatever the model decides | — | Two principles hold this together: **Principle 1** **Docs lie; code wins.** Every doc is a claim to check against the code. When SPEC and CLAUDE.md disagree, SPEC wins. When SPEC and the code disagree, that's a bug, and it goes through [backprop](#what-is-backprop). **Principle 2** **Advice vs enforcement.** CLAUDE.md, rules and skills are context: the model reads them and usually follows them. Permission rules, hooks and CI are enforcement. Anything that must *never* happen belongs in the second group.[^memory] Here's the full layout Votery ended up with (only the agent-relevant parts). TFT's is almost the same, [browse it on GitHub](https://github.com/medukn/tempest-fauna-trail). ```text CLAUDE.md entry point (≈90 lines) SPEC.md contract: §G §C §I §V §T §B §D FORMAT.md how SPEC is written ARCHITECTURE.md map of the code .claude/ settings.json allow/deny rules + hook registration hooks/guard.py deterministic guard (secrets, whole-tree staging) rules/{domain,state,ui}.md path-scoped rules skills/{plan,check}/SKILL.md repo-specific skills docs/ live/ LIVING per-topic docs + README with the header convention design/ FROZEN plans; design/tasks/tN_*_plan.md per task journal/ FROZEN dated entries; legacy/ = old wiki, verbatim templates/ task_plan.md, journal_entry.md release/1.1.0/ HANDOFF.md privacy.html served at a URL registered in the Play Console → never move tool/ review/codex_review.sh cross-model read-only review e2e/ emulator harness: emulator.sh, install.sh, drive.py, flows/ signing/ store/fetch the signing key from the password manager test/ mirrors lib/: domain/ models/ pages/ providers/ services/ widgets/ support/ .github/workflows/ci.yml analyze + format + test ``` ## What goes in CLAUDE.md (or AGENTS.md)? **Keep it short and make it mostly pointers.** Anthropic recommends *"target under 200 lines per CLAUDE.md file. Longer files consume more context and reduce adherence,"* and gives a test for every line: *"Would removing this cause Claude to make mistakes? If not, cut it."*[^memory][^best-practices] Votery's is about 90 lines, in six sections: 1. **What this is**, in one paragraph, plus the source-of-truth pointer. 2. **Commands** the agent can't guess. 3. **Definition of done**, numbered ([below](#what-does-done-mean)). 4. **Docs map**: path, role, LIVING or FROZEN. 5. **Workflow**: one line per skill, plus the "stuck twice" rule. 6. **Hard rules**, each saying which hook or deny rule enforces it. The top of Votery's file: ```markdown # Votery (Lottery of Choice) Pass-and-play Flutter app: a group votes on options, a weighted lottery picks the winner. > **Source of truth: [SPEC.md](SPEC.md)** (§V invariants, §T tasks, §B bugs). When this file > and SPEC disagree, SPEC wins. Map of the code: [ARCHITECTURE.md](ARCHITECTURE.md). ## Commands flutter analyze # must stay "No issues found!" flutter test # all tests dart format test/ # CI enforces format on test/ only (lib/ not yet formatted) dart run build_runner build --delete-conflicting-outputs # after editing @HiveType models ``` What's **not** in it: architecture prose (→ ARCHITECTURE.md), per-area conventions (→ `.claude/rules/`), procedures (→ skills), history (→ journal). TFT's [CLAUDE.md](https://github.com/medukn/tempest-fauna-trail/blob/main/CLAUDE.md) is a good comparison: at 189 lines it's under Anthropic's recommended 200-line target but far past the ~100 lines Votery aims for, and much of that length is the planning procedure that its `plan` skill also carries. One copy would be enough. **CLAUDE.md or AGENTS.md?** `AGENTS.md` is the cross-vendor convention: plain Markdown, no required fields, and in a monorepo the nearest file wins.[^agentsmd] Recent Claude Code versions also read `AGENTS.md` directly when there is no `CLAUDE.md` (check the memory docs for the minimum version; `/context` shows what actually loaded). If several tools work in one repo, put the shared content in `AGENTS.md` and make `CLAUDE.md` import it, so there's one copy to keep current:[^memory] ```markdown @AGENTS.md ## Claude Code Use plan mode for changes under `src/billing/`. ``` **Claude Code details worth knowing** - `@path` imports load at launch. They organise the file but don't save context. - Block-level HTML comments in CLAUDE.md are stripped before the model sees them, so you can leave notes for human maintainers at zero token cost. - `/context` shows which memory files actually loaded. In recent versions, `/doctor prompt-audit` checks instruction files for stale references and contradictions. - The project-root CLAUDE.md is re-read after `/compact`. Instructions you only gave in chat are not. That's the practical reason rules belong in files. ## What is SPEC.md, and why does every rule name a test? Every rule names a test because a rule without one is only a claim: nothing tells the agent, or you, when it stops being true. `SPEC.md` is where those rules live, the single source of truth for what the project promises. The idea comes from spec-driven development: the spec is *"the center of your engineering process"*, a living document that drives implementation instead of being written once and forgotten.[^speckit] Both projects use the same seven core sections: | § | Section | What goes there | |---|---|---| | §G | Goal | 2–3 lines: what the product is for | | §C | Context | Constraints: stack + versions, state, storage, platform, **privacy**, **secrets**, languages (i18n) | | §I | Interfaces | Public seams that code and tests depend on: signature, `file:line`, one-line contract | | §V | Invariants | Rules that must hold, **each naming its guard** (test or hook) or saying `unguarded` | | §T | Tasks | Table: `# · Task · Files · Depends · Est · Status` | | §B | Bugs / Backprop | Every bug: date, symptom, cause, fix, and which §V now guards it | | §D | Deferred | Open design questions; resolved inline, never deleted | (A "public seam" in §I is any function or file format that other code or tests rely on, so changing it breaks something elsewhere.) The **invariant format** is what makes this more than a to-do list. In plain words, Votery's rule V.7 says: only options whose net tally is above zero can win, each one's chance is its share of the total of those positive tallies, and if exactly one option qualifies it wins without a draw. Three test files fail if any part of that stops being true. In the spec's compressed notation (`Σ` means "sum of"): ```markdown - V.7: Nominees = options with tally **strictly > 0**; probability = score / Σ positive scores; Σ probabilities = 1. Zero nominees → no draw; exactly one → auto-winner, no draw. Guard: `test/providers/lottery_controller_test.dart`, `test/domain/draw_test.dart`, `test/domain/odds_test.dart`. (T.0, T.3) - V.11: No secrets in git: `secret/`, `*.jks`, `android/key.properties` gitignored and untracked; agents never read them. Guard: `.claude/hooks/guard.py` PreToolUse hook + `.claude/settings.json` deny rules (best effort, not a sandbox). (T.0) - V.9: Inventory boxes capped at 50 items. Guard: unguarded. (T.0) ``` - `Guard:` names the test or hook. `unguarded` is allowed, but it's recorded debt, and a §T row should exist to fix it. - The trailing `(T.x)` says which task introduced the rule. `(T.0)` means "distilled from the code as it was before the spec existed". A **bug entry** always links back to an invariant: ```markdown - B.2 [2026-09-25] Cancel in setup category dialog still added the option. **Cause:** `if (result != null || mounted)` — `||` instead of `&&`. **Fix → V.16 (T.15):** "Add" pops `''` for no category, Cancel pops `null`, check `&&`. Touches `lib/pages/setup/category_picker_dialog.dart`. ``` TFT's [SPEC.md](https://github.com/medukn/tempest-fauna-trail/blob/main/SPEC.md) has the same pattern at a larger scale. Its V.77, for example, exists because of bug B.34 and names two tests, `test_reward_components_are_recipe_vocabulary` and `test_any_two_reward_components_fuse`. **Tasks** are a table with an explicit status and estimate. Anything bigger than `L` (3–6 h) gets split along a real seam: ```markdown | T.2 | Extract pure tally `computeTally(GameState) → Map` into `lib/domain/`; replace both UI copies; test vs T.1 oracle | `lib/domain/tally.dart`, … | T.1 | M | ✅ | ``` **Deferred** items keep their trail, because "why didn't we do X?" comes up again: ```markdown - D.5: Ranked tokens asymmetric for even k (includes `0`). **RESOLVED [2026-09-25] (T.0):** intended — README "Reference patterns" documents `2 → +1, 0`, `4 → +2, +1, 0, –1`. - D.7: Simultaneous online voting (#4). **Deferred [2026-09-25]:** needs backend + INTERNET permission → breaks §C Privacy … ``` Two rules make the spec trustworthy: - **Distil the first spec from the code, not from memory.** Votery's spec header says *"Distilled from code 2026-09-25 (pre-spec behavior tagged `(T.0)`)"*. The first version was still wrong in two places (the "symmetric" rule and the gitignore claim). Both were caught because tests and reviewers checked the spec against the code. - **One writer.** Only the `spec` skill edits SPEC.md. The `build` skill may flip a task's status cell and nothing else. Plans end with "SPEC changes needed", and those are applied on your OK. Otherwise five agents each "fix" the spec to match whatever they just built. ### How should the spec be written? (FORMAT.md) A short house-style file that the skills read before touching SPEC. The rules that matter: - **Dense wording.** Drop articles and filler; fragments are fine. SPEC is loaded often, so tokens count. But *never compress identifiers, paths, code, numbers or enum values*. - **Cite touch points** as `path/file:line`, or by symbol. TFT's living docs deliberately cite `file` + `function` instead of line numbers, because *"lines drift, symbols don't"*. - **ids only go up and are never reused.** Retired entries are struck through (`~~V.7 …~~ RETIRED [date] (T.x)`), never renumbered, so every `V.7` in git history still means the same thing. - **One entry reads on its own**, without its neighbours. One rule per entry, too: TFT's later invariants grew into paragraphs that bundle three rules, which makes them hard to guard. - **Estimates:** `S` (<1 h) · `M` (1–3 h) · `L` (3–6 h). Bigger → split. The full template is in the [prep-repo skill](#install-the-prep-repo-skill) (`templates/FORMAT.md`), and TFT's is [here](https://github.com/medukn/tempest-fauna-trail/blob/main/FORMAT.md). ## What does ARCHITECTURE.md need? A map of the systems: what each one does, where it lives, how it connects to the others, and which patterns the codebase uses, with the file to open for each. Rules don't go here (those are §V). The test of a good ARCHITECTURE.md is whether an agent that has never seen the repo can go from "I need to change how X works" to the right file without searching. TFT's [ARCHITECTURE.md](https://github.com/medukn/tempest-fauna-trail/blob/main/ARCHITECTURE.md) is the better of the two projects' versions. Its opening line sets the job: *"How Tempest Fauna Trail is built, how the systems fit together, and where to find each one in the code."* It has four kinds of section worth copying: **Per system** **What it does, where it lives** One section per system (combat engine, weather, content, economy, save, UI). Each opens with a table of *concern → file*, so "the tick loop" points straight at `src/game/combat/engine.py`. **Patterns** **How things are done here** The recurring patterns, each with one example to copy: in TFT, "content only touches the world through the mutator API", "registries via `@register_*`", "pure data function + separate renderer", and a determinism doctrine to read before touching combat. **Index** **"Where do I find X?"** A task-shaped lookup table: *I want to… → go to*. "Add a boss" lists the three files a boss touches. It answers the question an agent actually starts with. **Walkthrough** **One request, end to end** What happens in one fight (or one HTTP request, or one game round), step by step, with the function at each step. It's how an agent learns the order things run in. A trimmed slice of TFT's index: ```markdown ## "Where do I find X?" — quick index | I want to… | Go to | |---|---| | Change how a fight resolves | `src/game/combat/engine.py` | | Add/modify what content can do to the world | `src/game/combat/context.py` (mutator API) | | Add a boss | `src/game/bosses/data.py` + `src/game/abilities/bosses.py` + `src/game/map_effects.py` | | Change weather fetching / caching | `src/api/{weather,cache,refresher}.py` | | Understand *why* a decision was made | `docs/journal/` | | Know what rule must hold | SPEC.md §V | ``` Votery's adds two habits: - **State known violations explicitly.** *"Known violation: domain logic lives in UI — tally in `lib/pages/lottery_page.dart:92` and `lib/pages/results/tallying_view.dart:153,733` … SPEC T.2/T.3 move them to `lib/domain/`."* An agent that finds the violation then knows it's known and planned, so it neither copies the pattern nor "fixes" it off-plan. - **A table for each cross-cutting concern agents tend to get wrong**: where randomness is used and whether it's seeded, which storage keys exist, what the limits are. **Write it from the code, not from the spec.** TFT's first ARCHITECTURE.md draft documented `resolve_combat(team, enemies, weather, run_mods=None)` and a determinism note about that `run_mods` default, both copied from SPEC. But `run_mods` belonged to the augment system, which was still a plan; the real signature had no such parameter. The same draft got the `CombatContext` constructor wrong and guessed the event names (`on_attack` instead of the real `on_attack_landed`). All four errors were caught only because a *"very deep review against the actual code"* was demanded. Mark planned parts as planned, and grep every name you cite. **Keep it current.** ARCHITECTURE went stale fast during Votery's busy release. Updating it is part of the rule in the next section: a change that adds or moves a system, module, route, store or pattern updates ARCHITECTURE.md in the same commit, and the `check` skill flags drift. ## How do you stop docs from going stale? With two rules. **1. Whenever an agent touches code, it updates every LIVING doc that describes that code, in the same commit.** Not "the docs for the feature": every doc whose claims the change makes untrue, including SPEC, ARCHITECTURE and the `docs/live/` page for each touched system. Put the rule in the entry file and in the definition of done, because agents don't do it on their own. In TFT, PR reviews kept finding the same failure: - a LIVING doc still listing the old `generate_augment_offer(..., rerolled=False)` signature after the code moved to `reroll_count`; - `docs/live/systems/items.md` saying `spirit_gem` was in `BASE_COMPONENTS` when the code explicitly excluded it; - the roster doc citing a guard test by a name that no longer existed; - a module docstring saying a feature was "not yet implemented" directly above its implementation. Each was small. Together they are exactly how "docs lie" starts: every stale line is a confident wrong answer for the next agent. **2. Every doc is either LIVING or FROZEN, and says so in its first line.** Votery's `docs/live/README.md` puts it like this: > **LIVING vs FROZEN is the rule that stops drift.** A reader (human or agent) trusts a LIVING > doc as current and a FROZEN doc as a dated snapshot. > - `docs/live/` is reconciled to code on every change that touches its subject, in the **same > commit** as the code change. > - `docs/design/` is **never** retro-edited to match new code. It's the record of a decision > at a moment. When a task lands, distil the durable "how it works" into the matching > `docs/live/` doc; leave the plan frozen. Half of the "docs lie" problem is that nobody can tell a current doc from a stale one. A header fixes that: ```markdown > **Status: LIVING** — must match ``. Audited by `/check`. > **Scope:** . **Reconciled:** . ``` ```markdown > **Status: FROZEN** (moved from ). Point-in-time record — verify against code. ``` TFT's [combat doc](https://github.com/medukn/tempest-fauna-trail/blob/main/docs/live/systems/combat.md) is a real LIVING example. A FROZEN doc without its marker is a finding in the `check` skill. **Migrating an existing repo:** move old docs with `git mv` (to keep their history), add the FROZEN header, and put anything too messy to reformat into `docs/journal/legacy/` verbatim. Votery's root-level Copilot compliance docs and its old wiki went there. Exclude `legacy/` from link checks: it's kept verbatim, broken links included. **The journal** (`docs/journal/YYYY-MM-DD_topic.md`) is the "why" that SPEC compresses out. It's append-only: if an old entry turns out wrong, you write a new one saying so. Both projects make one section mandatory, **Process notes (AI collaboration)**: conflicts between docs and code, agent errors, guardrails added, drift caught, and a reflection on how the prompting went. The template's rule of thumb: > If a future agent reading only the code would repeat a mistake we already made, the > journal's **Process notes** section failed. **Some files must never move.** Votery's `docs/privacy.html` is served by GitHub Pages at a URL registered in the Play Console. It's listed in the docs map as "LIVING — never move" and in SPEC §C. An agent tidying up `docs/` would otherwise move it without a second thought. ### What about plans and handoffs? - **Task plans.** Every non-trivial task gets `docs/design/tasks/tN__plan.md` (Votery has 18, TFT many more). Sections: scope in/out, **gap today** (piece · `file:line` · ✅/🔶/❌), design, decisions, open questions, test plan, **machine-checkable acceptance criteria**, **SPEC changes needed**, **LIVING docs to update**. Plans are FROZEN once written. - **Release handoff** (`docs/release//HANDOFF.md`): written so the next session, human or agent, can resume cold. It's LIVING while the release is open (its header says "WORKING DOC" to flag that it changes daily) and becomes FROZEN once the release ships: ```markdown > **Status: WORKING DOC**, release 1.1.0 handoff (2026-09-25). Read this first when resuming. ## Done (committed) table: commit → content ## Open, in order numbered, blockers first, each with the exact check to run ## Deferred bugs (GitHub issues) #5 … #14, one line each, release blockers marked ## How to work team pattern, "every UI change verified on emulator", review script, "verify the staged set alone before committing" ``` ## How do you stop an agent from doing something dangerous? With **layered guardrails**: put the rule in prose (CLAUDE.md), enforce it with a permission deny rule, and back that with a deterministic hook. Each layer has holes, but the holes are in different places. > **Figure:** Each layer stops something the one before it misses. Prose is ignored under pressure; deny rules miss commands you allow-listed; a hook only sees the command text. Only an OS-level sandbox, configured to deny those paths, stops a program that opens the file itself, and only for commands that run inside it. The most important sentence in Votery's CLAUDE.md is about what to do when a guard fires: > Both are enforced by `.claude/hooks/guard.py` (PreToolUse) + deny rules in > `.claude/settings.json`. Best effort, not a sandbox. **A block is a signal to stop and ask, > never to find a workaround.** Without it, an agent that hits a block treats it as a puzzle and tries another route. That rule only works if the guard explains itself. A hook that reads command text will sometimes fire on text that merely *mentions* a secret path, like a grep pattern or an echo. So the template's guard names the word and the rule it matched (`` `.env` is a secret file name ``) and ends every block with the decision: if the named path is something the command really reads or writes, stop and ask; if it only appears in text the command never opens, the block is collateral, so rephrase the text. Never disguise a path to get past it. Literal commit messages and PR bodies are recognised as data and not scanned at all. ### Permission rules [Claude Code] `.claude/settings.json` is checked in and shared with everyone who uses the repo. Allow-list the safe, frequent commands (fewer prompts, less temptation to take shortcuts) and deny secrets and whole-tree staging: ```json { "permissions": { "allow": [ "Bash(flutter analyze)", "Bash(flutter analyze *)", "Bash(flutter test)", "Bash(flutter test *)", "Bash(git status)", "Bash(git diff)", "Bash(git diff *)", "Bash(git log *)", "Bash(git ls-files *)" ], "deny": [ "Read(./secret/**)", "Read(./android/key.properties)", "Read(**/*.jks)", "Read(**/*.keystore)", "Edit(./secret/**)", "Edit(./android/key.properties)", "Bash(git add -A *)", "Bash(git add --all *)", "Bash(git add -u *)", "Bash(git add .)", "Bash(git commit -a *)", "Bash(git commit --all *)" ] } } ``` Facts from the docs worth building on:[^permissions][^settings] - Rules are evaluated **deny → ask → allow**, first match wins. An allow rule can't carve an exception out of a deny rule. - `deny` and `ask` rules in a project file apply immediately; `allow` rules wait until you trust the folder. - *"Bash permission patterns that try to constrain command arguments are fragile."* Read/Edit deny rules also cover common shell file commands (`cat`, `head`, `sed`, redirections), but *not* a `grep -r` from a parent directory or a script that opens the file itself. - **Put a space before the wildcard.** `Bash(git diff*)` also matches `git difftool …`, which can run an external command; `Bash(git diff *)` doesn't. Votery's original rules had no space; the snippet above is corrected. - **[Claude Code] The sandbox** is the layer that stops a script from opening a secret itself, but only for commands that run inside it, and only for paths you tell it to deny.[^permissions] If you enable it, add the secret paths to its filesystem deny list and prove it: run a sandboxed command that tries to read one and check that it fails. Votery didn't use it; the hook was enough for its risk. - Path rules only apply through `Read(...)` and `Edit(...)`. A `Write(path)` or `Glob(path)` rule is accepted but never consulted. Votery walked straight into one of these holes. Two independent reviewers found that the secret deny rule covered only the Read tool while `git show*` was allow-listed, so `git show HEAD:secret/…` would have read the keystore straight out of history. That's why the hook exists, and why the template above no longer allow-lists `git show`. ### The guard hook [Claude Code] Hooks are deterministic: *"Unlike CLAUDE.md instructions which are advisory, hooks are deterministic and guarantee the action happens."*[^best-practices] That guarantee holds as long as the hook actually runs and returns a block (more on that in the design notes below). A `PreToolUse` hook receives the tool name and input as JSON on stdin. **Exit code 2 blocks the call and shows stderr to the model.**[^hooks] Register it by adding a top-level `"hooks"` key to `.claude/settings.json`, next to `"permissions"`: ```json "hooks": { "PreToolUse": [{ "matcher": "Bash|Read|Edit|Write|MultiEdit|NotebookEdit|Grep|Glob", "hooks": [{ "type": "command", "command": "python3 \"$CLAUDE_PROJECT_DIR/.claude/hooks/guard.py\"" }] }] } ``` The core of the hook (an excerpt: the helper functions are in the full file linked below). It splits a command into segments, parses each one like a shell would (quotes, globs, braces, variables), and checks every path-like word: ```python def check_bash(cmd): for body in substitutions(cmd): # $(...), `...`, <(...), bash -c '...' check_bash(body) for seg in segments(cmd): # split on ; & && || | outside quotes calls = list(git_calls(seg)) # every `git` word, shlex-parsed exempt = bool(calls) and "|" not in cmd for sub, args, global_opts in calls: if stages_whole_tree(sub, args): # add -A/./-u/-vA/":/", commit -a/-am block("whole-tree staging. Stage explicit paths: git add ...") if HISTORY_HAS_SECRETS and reads_history_content(sub, args): # log -p, show, diff block("git history may contain secrets. Name non-secret paths after `--`.") if touches_secret(" ".join(global_opts)) or not is_metadata_only(sub, args, seg): exempt = False # only rm --cached, ls-files, commit -m "…" if not exempt and touches_secret(seg): # any word whose path hits a secret block("command touches a secret path. Secrets live in the password manager; ask the user.") ``` **Design notes for the hook (read before you adapt it)** - **Split compound commands** before checking, including a single `&`. Otherwise `git ls-files & cat secret/x` slips past a check that only looks at the first command. - **Check words, not substrings.** The hook splits each segment on shell metacharacters (`:`, `>`, quotes, `$`, parentheses) and checks each path component. That catches `git show HEAD:.env>.env.example` and a bare directory name like `grep -r . secret`. - **Allow metadata-only git commands** (`rm --cached`, `ls-files`, `check-ignore`), because untracking a secret means naming it. But never when the segment contains `$(…)`, backticks or redirections, or `git commit -m "$(cat secret/token)"` would pass as "just a message". - **Keep that exemption list tight.** Votery's first version exempted *every* `git commit`. That left `git commit secret/key.jks` (a pathspec commit) and `git commit -F secret/notes.txt` (which copies the file into the commit message, and so into history) open. The template only exempts `git commit -m "a literal message"`, optionally with `-q` or `-s`. - **Parse git like git does.** `git add "."`, `git add -vA` and `git -c key=value add -A` all stage everything. Regexes miss them; tokenizing the command and skipping git's global options doesn't. - **History can leak too.** If a secret was ever committed, `git log -p` or `git show ` prints it without naming its path. With `HISTORY_HAS_SECRETS = True` those commands need an explicit `-- `. That exemption assumes the secret sat in a secret-named file. If it was pasted into an ordinary one (this repo's `CLAUDE.md`), list that file in `HISTORY_SECRET_PATHS`, or `git log -p -- CLAUDE.md` prints the key again. Summary-only output (`--stat`, `--name-only`) shows no contents and stays allowed. - **A hook only protects you if it runs.** If `python3` is missing, the script crashes or the hook times out, the call can go through. The template treats its own errors as a block, and after installing it you should prove the registration works: ask the agent to run `git add --dry-run -A` (harmless even if it ran) and check that Claude Code reports the block. Unit tests of `guard.py` alone don't show that. - **Make the block message say what to do instead** ("stage explicit paths", "ask the user"). - **Most blocks are prose, not secrets.** The hook checks Bash *text*, so text that merely *names* a secret path trips it. I went through every block in my Claude Code transcripts across two repos: 12 blocks in 6 sessions. 8 were prose: a heredoc writing a reviewer prompt or a teammate brief, a `python3 - <<'EOF'` edit script for a doc, a `git commit -F -` body that named the keystore type, an edit script that contained the *text* `git add .`. Only 3 touched what they named (a `ls` of the secret folder, a check that it was absent, the deliberate `git add --dry-run -A` test). What helped every time: write prompts, briefs, commit messages and doc edits with the Write or Edit tool, which the hook checks by path, not by content; drop "is the secret absent?" checks altogether; never disguise the path. Say so in CLAUDE.md, or agents get stuck. - **I tried exempting "data heredocs", and took it out again.** Since most blocks were prose in heredocs, I taught the template that a quoted heredoc fed to `cat`, `tee` or `git commit -F -` is data, not code, and skipped its body. Each Codex review round then found a new way past it. Round one: a heredoc that writes a script which the same command then runs (`cat > run.sh <<'EOF'`, then `sh run.sh`). Round two: a `<<'EOF'` inside a comment, which bash ignores but the guard didn't, so the lines after it were skipped and ran. Round three: a shell function named `cat`, a pipe in front of the sink, and a delimiter with a trailing space that bash doesn't accept but the guard did. Every fix was correct, and every round found something else. The exemption is gone. Every heredoc body is checked again, and prose that names secret paths goes through the Write or Edit tool. The rows those reviews produced stay in `test_guard.py` as cases that must keep blocking. The section below explains why I stopped patching. - **Newline is a separator, a quoted newline is not.** `git status`, then `git add -A` on the next line, is two commands. A guard that lets `shlex` treat newlines as whitespace sees one `git status` and exempts it. Splitting on every raw newline instead cuts multi-line commit messages into fragments and blocks them. Parse quotes first, then split on unquoted newlines. - **Match secret names by shape, not by spelling.** `.envrc` (direnv) and `.env_prod` hold the same secrets as `.env.local`; the template now matches all three. A bare `secret` in a sentence isn't a path: match the directory as a path component, not as a word in prose. - **Test it like code, and have someone else attack it.** The skill ships `test_guard.py` with the cases above as a table of commands that must be blocked or allowed. Every bypass a reviewer finds becomes a new row. Prove each new row fails against the previous guard before you fix it. Run it from inside a git work tree; the history checks call git, and the script now refuses to run elsewhere instead of reporting seven false failures. Tests written by the same agent that wrote the guard share its blind spots: in video-downloader, 60 of them passed while a Codex review found three bypasses (newline chaining, `.envrc`, `python3 -c`) in its first read. - The hooks API also offers a JSON `permissionDecision` (`allow` / `deny` / `ask` / `defer`) and `updatedInput` for rewriting a call instead of blocking it. - Other events useful as quality gates: `Stop` (block the turn from ending until a check passes) and `TaskCompleted` / `TeammateIdle` (agent teams). Votery deliberately skipped a `Stop` hook running `flutter analyze`: *"too slow per turn; revisit as a `TaskCompleted` hook"*. Measure the cost before adding a per-turn gate. The full hook and its tests are `templates/guard.py` and `templates/test_guard.py` in the [skill](#install-the-prep-repo-skill). The first version of this template had several bypasses (a command substitution in a commit message, a single `&`, `git add -vA`, `git log -p`), found by the cross-model review described [below](#who-reviews-the-agents-work). I kept that review running for 15 rounds. Every round found new ones, each more obscure than the last (`g${X}it add -A`, `.en$(true)v`, `x+=v`), and the test table grew to about 200 cases. That's the honest summary of a hook that reads command text: it catches the mistakes an agent makes by accident, and a determined adversary will always find another trick. For anything that must hold against that, use the sandbox. ### Why a text guard keeps leaking, and what I'm trying instead *Experimental, field-tested in one repo: a small Python app, one PR, three review rounds. Each item below says whether I verified it (ran it and saw the result) or not. Treat it as a direction, not a finished recipe.* The hook's two problems, blocking prose and missing tricks, have one root cause. The rule is about **effects**: no process reads a secret file, and a committed secret never slips into `main` unnoticed. The hook checks **command text** and has to predict what bash will do with it. Bash has functions, aliases, globs, `eval`, quoting rules and heredocs, so the prediction is really a second, partial bash interpreter. Every match on text that bash never runs is a false block, and every exemption I add to stop those blocks needs a proof that some text is never executed. Those proofs are where review kept finding holes. Patching the parser treats symptoms. So I'm moving each rule to the place where the effect happens: | Rule | Text guard did | Now enforced by | |---|---|---| | A shell command never reads a secret | parse the command | Claude Code's OS sandbox, `sandbox.filesystem.denyRead`. It applies to "all commands running inside the sandbox, including their child processes"[^sandboxing]: scripts, globs, functions, files written earlier | | The Read/Edit tools never open a secret | deny rules + hook | permission deny rules, plus a small hook that checks file-tool paths | | A committed secret gets caught before it reaches `main` | parse `git add -A` | a scan of the **staged content** (names and credential-shaped strings) in a pre-commit hook and in CI, with `main` protected by that check. This *detects*: the hook can be skipped and CI runs after the push, so blocking a push would take server-side push protection | | Don't stage everything, don't skip the scan | parse git flags | a small nudge with no exemptions. It's a workflow hint, not the security boundary | **Project settings** (`.claude/settings.json`, checked in). All verified: - **Make the sandbox a boundary, not a suggestion:** `"enabled": true`, `"allowUnsandboxedCommands": false` (otherwise a failed command can be retried outside it) and `"failIfUnavailable": true`. Be clear what that last one means: Claude Code **refuses to start** on a machine without `bubblewrap` and `socat`. Without it, a missing dependency silently means no sandbox at all. - **`sandbox.filesystem.denyRead` lists every secret name, at the root and at any depth:** `./.env` and `./**/.env`, the same for `.env.*`, `.envrc`, `secret`, `secrets` and the SSH key names (`id_rsa`, `id_ed25519`, `id_ecdsa`, `id_dsa`), `./**/*.pem` and the other key-file extensions (`.key`, `.p12`, `.pfx`, `.jks`, `.keystore`, `.kdbx`), plus `~/.ssh` and `~/.aws`. - **Allow what your toolchain needs, and nothing else.** `uv run` failed with "Read-only file system" until I added `sandbox.filesystem.allowWrite: ["~/.cache/uv"]`. `network.allowedDomains` holds GitHub, PyPI, `astral.sh` and whatever the app itself calls. - **Permission deny rules for the file tools:** `Read` and `Edit` for the same names, written as `**/.env` and so on. Also deny `Edit(/.claude/settings.json)`, `Edit(/.claude/hooks/**)` and `Edit(/.githooks/**)`. Those Edit denies also make the paths read-only for shell commands inside the sandbox: `os.access(path, os.W_OK)` returned `False` there. - **Two path syntaxes in one file.** In sandbox lists, `./` means the project root when the setting lives in project settings, and `~/` means your home directory.[^sandboxing] In Read/Edit permission rules, `./path` is relative to the *current directory* and `/path` to the project,[^permissions] so `Read(./secret/**)` protects only when Claude starts at the repo root. Use `/secret/**`, or `**/secret/**` for any depth. **User settings** (`~/.claude/settings.json`, can't be checked in): - **Codex as the reviewer, inside the sandbox** (verified): `allowWrite: ["~/.codex"]`, and `allowedDomains` `chatgpt.com`, `*.chatgpt.com`, `auth.openai.com` and `api.openai.com`. Without the write entry, Codex fails before any network call, with "Read-only file system" when it creates its state directory. - **`gh` and `git push`**: from inside the sandbox both get HTTP 401 at first, because the login lives in the system keyring, which is out of reach. The documented way around it is credential masking: `network.tlsTerminate`, a `credentials.envVars` entry for `GH_TOKEN` with `"mode": "mask"` and `injectHosts` `api.github.com` and `github.com`, and Claude Code started as `export GH_TOKEN=$(gh auth token) && claude`.[^sandboxing] Masking is honoured only in user settings. - **`gh` works through masking** (verified): `gh api`, and `gh pr view`, `edit`, `update-branch`, `checks` and `merge`. I merged three stacked PRs with `gh` alone. - **`git push` over HTTPS did not** in my test, and neither did `git fetch` or `git ls-remote`: 401 "Invalid username or token", even with `gh auth git-credential` as git's credential helper. My guess, which I haven't verified, is that git sends the placeholder base64-encoded inside a Basic auth header, so the proxy doesn't recognise it and never swaps in the real token. So I push myself, or the agent sticks to `gh` API calls. **Commits.** `scripts/check_secrets.py` scans the **staged content**: full blobs, secret names and credential-shaped strings. It reports file names, never content. It runs from `.githooks/pre-commit` (`git config core.hooksPath .githooks`, once per clone) and in CI with `--all`. CI runs on a push to every branch, and it also runs `main`'s copy of the scanner, so a branch can't quietly weaken the scanner that checks it. **The guard hook, now small.** It checks file-tool paths (glob-aware, every path component, symlinks resolved with `realpath` against the hook payload's `cwd`) and nudges on git: bulk staging, and skipping the pre-commit hook via `--no-verify`, `-n`, `-c core.hooksPath`, `GIT_CONFIG_*` variables (also after `env` or `sudo`), or `git config core.hooksPath` pointing elsewhere or `--unset`. It no longer parses shell commands for secrets. **Verify with a probe** (verified). A probe script tries every bypass the text guard's reviews found, as real reads against fake secrets. I run `setup` and `cleanup` myself, outside the sandbox; the agent runs `check`. Result: **14 of 15 blocked**: `cat`, globs, `bash -lc`, `python -c`, a heredoc fed to `python`, recursive `grep`, a script written and then run, a shell function named `cat`, the `secret/` folder, `*.pem`, nested `secrets/` (two cases), a nested `.envrc` and an SSH key name. The controls passed. The one leak is a file **created and read in the same command**. On Linux, wildcard `denyRead` entries are expanded to concrete paths for each command, so a file that doesn't exist yet when the command starts isn't covered. A file created by an earlier command is blocked. What I learned, beyond the recipe: - **Read deny rules merge into the sandbox's `denyRead`.**[^sandboxing] They quietly fill gaps, but you can't use a Read deny for a file a sandboxed tool needs: denying `~/.codex/auth.json` would lock Codex out of its own login. A block like that has to go in the hook (file tools only). - **Don't rely on that merge for coverage.** A review found my sandbox list had no nested entries; it only held because of the merge. List them in the sandbox block itself. - **The probe needs proof that setup ran.** Inside the sandbox, a missing fake secret and a hidden one look the same, so `check` passed once without any setup. A sentinel file in `.git/` fixed that. Setup must also refuse to overwrite *any* existing file that isn't its own fixture, not just `.env`. - **Placeholder files show up in `git status`, inside the sandbox only.** Denied and protected paths appear as placeholder device files (`.bashrc`, `.mcp.json`, …). Never stage them. Reviewers can stop on them, so tell Codex to review tracked files only. - **Keep the secret patterns identical in every place that lists them:** the hook, the scanner, the sandbox `denyRead` and `.gitignore`. Mine drifted: the SSH key names were missing from both the sandbox and `.gitignore`. - **The rest:** saving the settings applies filesystem changes to the very next command, with no restart (masking needs one). A git worktree whose metadata lives in another repo can't commit from inside. And a test that writes git objects skipped itself with a misleading message, so watch the skip count, not just the failures. The limits, stated plainly: - **A branch can edit or delete the CI workflow that scans it.** `main` is safe only because branch protection requires the CI check, and a secret on a pushed branch reaches GitHub before CI runs. Server-side push protection would be the next layer. I haven't tried it. - **The scanner matches patterns.** A generic `password = value` in an ordinary file passes. - **Codex's own login stays readable.** Signed in with a ChatGPT plan, Codex keeps its token in `~/.codex/auth.json`, and sandboxed shell commands can read it (an API key instead bills per use at API rates).[^codex-auth] I accepted that risk. The hook blocks the file tools from it, and the agent's rule is "never read or print it". Codex can store the login in the system keyring instead (`cli_auth_credentials_store = "keyring"`), which I *expect* to be unreachable from inside the sandbox, like `gh`'s. Untested. - **The pre-commit hook is opt-in per clone.** Skipping it gets a nudge, not a block; CI is the backstop. - **Some things run outside the sandbox anyway:** hooks, MCP servers and commands you type with `!`.[^sandboxing] How the change itself went: the review loop worked as designed. Three rounds, each finding checked against the code and answered on the PR as `fixed in ` or `dismissed: `, then the stop rule. One round-2 claim was false on inspection ("Bash can overwrite the hooks"; they weren't writable), but chasing it uncovered a real gap: a branch could weaken its own scanner. With settings and hook files locked against the agent, every change to them went through a small install script that I ran with `!`. One annoyance: a pasted `!` command sometimes arrived as chat text instead of running. Typing the `!` by hand fixed it, and a tiny wrapper script fixed long commands that wrapped onto two lines. What I don't know yet: how much friction the sandbox adds over a week of normal work, and whether `git push` can work from inside at all. I'll update this section either way. ### How do you keep secrets away from the agent? Rule: **nothing secret in git, and the model never sees a secret.** Secrets live in a password manager. The repo only holds *scripts* that store and fetch them, and a human unlocks the vault. Votery's fetch script restores the release keystore before a release build. Below is a sketch of its structure, not a runnable script: `$repo` is the repo root, and the download and verification steps depend on your vault's CLI. The parts worth copying: ```bash { set +x; } 2>/dev/null # never trace: would print the session token / passwords set -euo pipefail [ -n "${BW_SESSION:-}" ] || { echo "Set BW_SESSION first" >&2; exit 1; } # Resolve the vault item by EXACT, case-sensitive name and require exactly one hit # (the CLI's own search is fuzzy/case-insensitive → a duplicate item once matched). # An explicit item-id env var overrides. umask 077 mkdir -p "$repo/secret"; chmod 700 "$repo/secret" # Download to a fresh temp file, verify, then atomically mv into place. # Never delete the existing keystore before the download succeeded. ``` Every line there comes from a mistake: - **`set +x`**: a reviewer found that running the script with `bash -x` would print the session token. - **Temp file, then `mv`**: the agent's first version deleted the old file first. - **Verify the property that matters**: after storing, the script runs `keytool` to prove the keystore opens with the stored password and alias. A byte-identical round-trip had passed while the pairing was wrong. Make the check fail when `keytool` is missing, so a backup never counts as verified without it. - **Exact-name lookup with a uniqueness check**: the vault CLI's search is case-insensitive, and a near-duplicate item existed. - **Test the restore right after the backup.** That's what surfaced both of the last two problems. **If a secret is already committed:** (1) back it up to the vault and *verify the backup works*; (2) only then `git rm --cached` it and add it to the root `.gitignore`; (3) decide whether to rotate it, since it stays in history (Votery's committed keystore turned out to be an old, superseded key, which made that decision easy); (4) record it as a §B bug with an invariant and a guard. > **When the agent needs a script that touches secrets:** Default: you run it. The agent writes and reviews the script; you execute it in your own terminal. The exception, if you choose it, is narrow: unlock the vault yourself, write the session token to a mode-600 file, and let the agent run one named script that reads it with tracing off and never prints it. The model never sees the token, but the process it starts can use it, so treat that as granting access, not hiding it. Afterwards lock the vault and delete the file. Never paste a token into chat, and never put vault identifiers or tokens in agent memory files. ### Staging, branches and generated files - **Never `git add -A` / `.` / `-u`, never `git commit -a`.** In a shared working tree (a lead plus teammates, or you plus an agent), whole-tree staging commits other people's half-finished work. Votery enforces this three ways: CLAUDE.md, deny rules and the hook. - **Branch before committing; never commit to `main`.** An agent's work is disposable until someone has reviewed it. On a branch, a bad run costs one `git branch -D`; on `main` it costs a revert, and everyone who pulled in the meantime builds on it. A branch also keeps `main` buildable for the next session and gives parallel agents their own line of history. How the branch gets back into `main` is [the pull request](#how-does-a-change-get-from-a-branch-into-main). - **Verify the staged set alone** before committing mixed work: `git archive HEAD` plus the `git diff --cached` applied in a scratch directory, then lint and test. That proves the commit builds without the unstaged changes lying around. - **Commit messages carry the why.** Bug fixes use `backprop §B. + §V.: `, with spec edit, test and fix in one commit. - **Generated files and storage are contracts.** *"Don't hand-edit `*.g.dart`; never reuse or renumber a Hive `typeId`."* Storage keys, database schema ids and served URLs are contracts with installed apps or external systems: renaming one loses data or breaks a link, so add a migration instead. ### Path-scoped rules [Claude Code] Files in `.claude/rules/*.md` with `paths:` frontmatter load only when the agent reads matching files, so area-specific rules cost nothing elsewhere.[^memory] Votery has one per layer. Each starts with a before/after checklist, then gives concrete rules: ```markdown --- paths: - "lib/pages/**/*.dart" - "lib/widgets/**/*.dart" - "lib/router.dart" --- # UI rules > Before editing: read ARCHITECTURE "Game flow" + "UI conventions" and SPEC V.10. > After editing: `flutter analyze` + `flutter test` green, then verify visually on the emulator > (`tool/e2e/drive.py`, `docs/live/e2e_testing.md`): tap through, screenshot, look at the PNGs. - Screen navigation only `context.go('/route')`. `Navigator.of(context).pop()` only to close dialogs. - **No domain logic in widgets**: no tallying, scoring, or winner selection (SPEC T.2/T.3). No `Random()` for anything outcome-relevant; cosmetic randomness (confetti) is fine. - New code: colors from `Theme.of(context).colorScheme`, not raw `Colors.*` (legacy code is being migrated in T.7 — don't add more debt). - Files > 400 lines: don't grow them; extract sub-widgets (T.8). ``` TFT's [game-logic rule](https://github.com/medukn/tempest-fauna-trail/blob/main/.claude/rules/game-logic.md) is the same idea for a Python engine: pure functions, no UI imports, seeded randomness. Two patterns worth copying: **"don't add more debt"** rules for legacy code (tolerate the old violation, forbid new ones, point at the task that pays it down), and **every rule cites the §V or §T it comes from**. ## What does "done" mean? Put the definition in the entry file, numbered, and make every item checkable. Votery's: ```markdown ## Definition of done (every change) 1. `flutter analyze` clean **and** `flutter test` green — show the output, don't assert it. 2. Behavior change → test that fails without it. Bug fix → §B entry + §V guard (backprop skill). 3. LIVING docs updated in the same change (SPEC §T status, ARCHITECTURE, `docs/live/`). 4. Separate review pass on the diff before calling it done — green tests are not "done". Cheap default: `tool/review/codex_review.sh [paths]` (read-only); verify each finding before acting. 5. UI change → verify in the running app on the emulator (`tool/e2e/`, see `docs/live/e2e_testing.md`): tap through, take screenshots, look at them. ``` Why each line is there: 1. **"Show the output, don't assert it."** Agents say "tests pass" without running them, or after running a subset. Anthropic: *"Have Claude show evidence rather than asserting success."*[^best-practices] 2. **A test that fails without the change.** A test that can't fail proves nothing (remember the `List ==` test). Bugs get the [backprop](#what-is-backprop) treatment. 3. **Docs in the same change.** Every LIVING doc that describes code the change touched (the [rule above](#how-do-you-stop-docs-from-going-stale)). Otherwise LIVING docs quietly become FROZEN ones that still claim to be current. 4. **Independent review.** The model that wrote the code shouldn't be the only one grading it. 5. **The running app.** Tests pass in a simulated environment; your users run the real thing. If an agent keeps "getting to green" by weakening checks, add numbers: write down the bar (coverage, lint level, performance budget) and watch diffs for new suppressions (`// ignore:`, `@ts-ignore`), skipped or deleted tests, removed assertions, and thresholds edited down. ## Which workflow skills do you need? You need five: **plan**, **spec**, **build**, **backprop** and **check**, plus a journal habit. Each is a procedure the agent loads only when it needs it, which is what a skill is. **[Claude Code]** Unlike CLAUDE.md, a skill's body loads only when it's used.[^skills] Other tools have their own equivalent (or you keep the procedures as Markdown files the entry file points to). Anything that's a multi-step procedure rather than a fact belongs in a skill. Both projects use these five (some kept at user level rather than in the repo): | Skill | Writes | Job | |---|---|---| | **plan** | only `docs/design/tasks/tN_*_plan.md` | Required reading → verify every symbol against the code → ask real design forks *first* → plan ending in exact "SPEC changes needed" | | **spec** | only `SPEC.md` | The one writer. Modes: new, distil (code → spec), amend, backprop (`bug: …`). Shows the diff, applies on OK | | **build** | code + tests; flips §T status | Implements one task. On test/build failure → backprop, never a blind retry | | **backprop** | §B (+ §V, test) | Bug → trace → which invariant should have caught it (existing, or a new one)? → failing test → fix → one commit | | **check** | nothing (read-only) | Drift audit: LIVING docs vs code, reports by severity, names the remedy, never applies it | | journal | `docs/journal/YYYY-MM-DD_*.md` | After milestones; Process notes mandatory | TFT's versions are public: [plan](https://github.com/medukn/tempest-fauna-trail/blob/main/.claude/skills/plan/SKILL.md), [spec](https://github.com/medukn/tempest-fauna-trail/blob/main/.claude/skills/spec/SKILL.md), [build](https://github.com/medukn/tempest-fauna-trail/blob/main/.claude/skills/build/SKILL.md), [check](https://github.com/medukn/tempest-fauna-trail/blob/main/.claude/skills/check/SKILL.md). Generic versions of all five are in the [prep-repo skill](#install-the-prep-repo-skill). > **Figure:** One task, one loop. A failure during build is classified first: a plain code bug gets fixed; a wrong spec or an unspecified edge case goes through backprop, so the spec learns and a test catches the same mistake if a later agent repeats it. **Required reading, before editing code for a task** (from both CLAUDE.md files): > The §T row + every §V that could apply → the ARCHITECTURE section → the touched code itself > (cite `file:line`). **Docs lie; code wins.** The plan skill turns that into a VERIFY step: ```markdown - Grep every class/method/field/provider before citing it; confirm real name + signature. - Duplicated logic: find **all** copies (e.g. tally exists in 2 files) — the plan must cover each. - Randomness: any outcome-relevant RNG must be injectable (`Random` param); tests use a seed. - Storage: touching Hive/SharedPreferences keys or typeIds → migration section required. ``` and an ASK step, which is where you stay in the loop cheaply: ```markdown `AskUserQuestion` only for genuine forks (scope cut, UX behavior, data migration). Investigate first (code, `git log -S`), then ask with a recommendation + the concrete trade-off. Rote choices: decide, flag in Open Questions, move on. ``` ### What is backprop? It's the one thing spec-driven development does that plain plan-then-execute doesn't: > Plan-then-execute fixes the code and forgets. Spec-driven development fixes the code **and** > edits the spec, adding a rule and a test, so the same mistake gets caught if it comes back. > That edit is backprop. When a test or build fails, the build skill classifies the failure: (a) my code bug → fix it; (b) the spec is wrong; (c) an edge case the spec didn't cover. (b) and (c) go through `spec bug: …` first. A real Votery example: bug B.4 (tally overflow on small phones) produced **V.17**, *"No RenderFlex overflow at 360×640 with text scale 1.3"*, guarded by a layout test. Later, bug B.6 (a button label overflowing) was filed against the same V.17 instead of becoming another one-off fix. ### What does the check skill do? It audits the LIVING docs against the code and reports what no longer matches, without changing any file. It starts mechanical, then gets semantic: ```bash grep -rhoE '`[A-Za-z0-9_./-]+\.dart(:[0-9]+)?`' SPEC.md ARCHITECTURE.md CLAUDE.md docs/live .claude/rules \ | tr -d '`' | sort -u ``` Every cited path must exist. `path:NN` must be within ±15 lines of the named symbol (skew is a note; missing is broken). Claims with "only", "sole" or "never" are checked by grep first, then by reading the call paths; where the claim is about behavior and no test covers it, the report says so, and writing that test becomes a follow-up task. Counts get recomputed. Each §V guard test must exist, and must test the production code. Output is one line per finding, `: . → `, ending with one suggested next action that the skill never performs itself. ### What if the agent keeps failing at setup? Put a **"stuck twice"** rule in CLAUDE.md: > After the second failed attempt at the same setup problem (tooling, emulator, build, plugin, > SDK, CI, platform API), stop guessing: research the live official docs. Cite the source URL > in the commit or report. Platform requirements change faster than any model's training data. Anthropic's version of the same idea: *"After two failed corrections, `/clear` and write a better initial prompt incorporating what you learned."*[^best-practices] ## What tests do you need before agents refactor anything? Make the code testable and pin its current behavior first. Votery's order was deliberate: *"pin current behavior with characterization tests (T.1) → extract pure domain layer (T.2/T.3) proven against the oracle → only then UI cleanup (T.7/T.8)."* - **Characterization tests first.** The brief given to the test-writing agent: *"Characterization tests assert what the code **does**, not what it should do; oddities are recorded in §D rather than 'fixed' in the test."* This paid off at once: it disproved the "symmetric" rule the lead had written, and the README showed the asymmetry was intended. - **A reference oracle.** The tally logic was copied verbatim from the widget into `test/support/reference_tally.dart`, and the new pure `computeTally` was fuzzed against it (run on thousands of random inputs, with every result compared to the oracle's). A reviewer then noticed that before the extraction, the "guard" only tested the oracle copy, not the production code. Always check what a guard actually exercises. - **Pure domain logic, out of the UI.** "Pure" means plain functions with no screen, storage or network code, so a test can call them directly. `lib/domain/` (tally, odds, draw) has no Flutter imports. Widgets only present results. The duplicated algorithms became one each. - **Injectable randomness and clocks.** Pass the random-number generator (and the clock) in as a parameter instead of creating it inside the function, so a test can hand in a seeded one and get the same result every run: `drawWinner(nominees, Random)`. The winner is drawn **once** in the controller, and the wheel animation is presentation only, landing on the pre-drawn winner. A real bug came out of this: the wheel reported whichever sector was under the pointer after a rough hit-test, which wasn't always the drawn winner. Do the same for time. - **CI gates** on every push and PR, toolchain pinned: install → lint → format check → tests. Votery formats only `test/` in CI because *"`lib/` has never been formatted and a mass reformat would bury the T.2/T.3 diffs."* Tighten gates in steps. - **Test layout mirrors the source layout**, so agents find, and add, the right test file without being told. ## Who reviews the agent's work? Someone other than the agent that wrote the code. At minimum that's a fresh session, which isn't biased toward code it just wrote: *"A fresh context improves code review since Claude won't be biased toward code it just wrote."*[^best-practices] In both my projects I went one step further, with **a model from a different family**. I can't prove that's necessary, but on Votery the two families caught different bugs (below). The cheap default is a read-only Codex script: ```bash #!/usr/bin/env bash # Usage: tool/review/codex_review.sh [path ...] set -euo pipefail cd "$(git rev-parse --show-toplevel)" range="${1:?usage: codex_review.sh [path ...]}"; shift || true if [ "$#" -eq 0 ]; then set -- .; fi paths=$(printf '%q ' "$@") # keep argument boundaries (paths with spaces) qrange=$(printf '%q' "$range") out="tool/review/out/$(date +%Y%m%d-%H%M%S).md" prompt="You are a strict code reviewer for a Flutter app. Review ONLY the diff of \`git diff ${qrange} -- ${paths}\` (run it yourself; read touched files for context). Contract: SPEC.md §V invariants and CLAUDE.md hard rules (…). Report correctness bugs, invariant violations, resource leaks (timers/controllers not disposed), and tests that pass vacuously. Verify each claim against Dart semantics before reporting (e.g. Dart's % is non-negative for positive divisors). No style nits, no praise. Format: one line per finding: path:line: [high|med|low] problem. fix. End with 'totals: N high, N med, N low' or 'no findings'." mkdir -p tool/review/out codex exec -m "${CODEX_MODEL:?set CODEX_MODEL}" -s read-only "$prompt" | tee "$out" ``` It needs the Codex CLI installed and signed in. Run it as `CODEX_MODEL= tool/review/codex_review.sh main..HEAD`, with the model id you use. Review a task **before** it lands, while it's still uncommitted: update the docs, stage the task's files by explicit path (`git add -- `, new files included), check that `git diff --cached --stat` shows exactly the task, then run the review script with `--cached` as the range. In a shared working tree, have the reviewer read files from the staged snapshot (the scratch checkout used to verify it), not the working directory, which may hold someone else's unfinished changes. If a finding leads to a fix, re-stage and review again. After committing, `main..HEAD` reviews the whole branch. This local pass is the first layer. The second is the pull request, [below](#how-does-a-change-get-from-a-branch-into-main). What to copy: a **read-only sandbox**; **point the reviewer at the contract** (§V, hard rules); ask explicitly for **vacuous tests and leaks**; **pre-empt known false positives** (the Dart `%` line exists because a reviewer once flagged it wrongly); a fixed one-line output so findings are easy to triage; and every review saved to a file. What two reviewers (Claude and Codex, both read-only) found on Votery's setup commit: **Both found** The gitignore claim that never landed; the deny rule covering Read but not `git show`; a false "mode 600" claim in the fetch script. **Only Codex** The always-green shuffle test; a font package fetching fonts over the network in an app that promises "no network"; the `bash -x` token leak; a lax properties parser. **Only Claude** The guard test that only exercised the oracle copy. And two of the reviewers' claims were simply wrong (a line number, a citation). The journal's conclusion: *"Two reviewers from different model families overlapped on the severe findings and diverged on the subtle ones — cheap insurance worth keeping for security-relevant changes."* And **reviewers need verifying too**: the rule is "verify each finding before acting", not "apply all findings". Anthropic warns about the opposite failure as well: *"A reviewer prompted to find gaps will usually report some, even when the work is sound … Tell the reviewer to flag only gaps that affect correctness or the stated requirements."*[^best-practices] ### Where should the reviewer run? Somewhere without secrets. Votery ran reviewers in a sparse git worktree that left out `secret/`. That isn't a full guarantee: a worktree shares the repository's history, so a secret that was *ever* committed can still be read with `git show :`. For a reviewer you don't trust, use a fresh clone with the secret purged from history, or an export of only the files under review (`git archive HEAD -- `) plus the change as a patch file, in a read-only sandbox. Build the patch with every secret path excluded (`git diff main..HEAD -- ':!secret'`): a diff that deletes a secret still contains it. Scan the patch before handing it over, and point the reviewer at it instead of asking it to run `git diff`. ### What do reviewers catch over a whole project? On the 1.1.0 release, the Codex review *"earned its place every round"*, in the journal's words. It found that a dialog left an unfinished game saved, so Home offered to resume straight back into the same dead end, and that a test fixture used a voter index the app can never produce. One finding didn't apply and was dismissed after checking the code. That's two data points. TFT had GitHub Copilot review 49 pull requests, which left 187 inline comments (counted across [all its pull requests](https://github.com/medukn/tempest-fauna-trail/pulls?q=is%3Apr); one more review attempt failed). My replies on those PRs almost always accepted every finding: *"all legit, zero false positives"*, *"all were legitimate (no hallucinations) but low/trivial severity."* The one real disagreement went the other way round: Copilot said a test was stricter than the spec, and it was the spec that was wrong, so the spec changed. The findings fall into four groups, and three of them aren't bugs in the usual sense: **Text that describes something else** A docstring saying the Spirit-Gem branch was "not yet implemented", directly above its implementation. A README promising "live weather drives combat" after the design decoupled them. A PR description stating an enemy ability scales `INT×1.5` while the code computed `INT×2.4×0.7`. **Guards that don't guard** A drift test that only checked a function's *name* appeared in the source (an import or comment passed it). A test claiming six cases and asserting three. `replay.winner == replay.winner` for two of three outcomes. A spec line claiming boss ids were CI-guarded when the test skipped bosses. **A spec that contradicts itself** V.70 still saying a node's weather is its default, after V.73 made it the live value. V.82 citing a guard test file that doesn't exist (the test landed under another name). **Ordinary bugs** An `IndexError` on recipes made from two identical components. A simulator that let you craft items you didn't own. A reward screen showing "✓ Recruited" when the recruit had failed. None of this is exotic, and the tests wouldn't have caught most of it: tests don't read docstrings or PR descriptions, and in the second group the tests *were* the problem. A second model reading the diff against the code and the spec catches exactly the class of error the writing agent can't see in its own work. ### How does a change get from a branch into main? Through a pull request, every time. The loop has five parts: 1. **Every change happens on a feature branch**, never directly on `main`. 2. **The finished work is committed and opened as a pull request.** The description names the §T row and shows the checks that were run. 3. **Reviewers review the PR**: the other model family, Copilot, and you. Their findings arrive as PR comments. 4. **Fixes are pushed as new commits to the same branch**, so they land in the same PR. Every review comment gets a reply: *fixed in ``*, or *dismissed* with the reason, after the finding was checked against the code. 5. **The PR is merged only when the merge conditions hold**: CI green on a branch that is up to date with `main`, every comment answered, and the reviews you require done. Why this matters more with agents than with people: - **`main` stays buildable.** The next session, and every other agent, starts from `main`. A half-finished run that lands there breaks all of them. - **An agent's work is disposable until it's reviewed.** A branch you close costs nothing. - **The PR is the one place where everything about a change is kept together**: the diff, the findings, the fixes and the decisions. Session transcripts don't survive; the PR does. - **Reviewers get a clean diff.** A PR against `main` contains exactly the task, not someone else's unstaged work from a shared tree. - **Parallel agents don't collide.** Separate sessions each work on their own branch (in their own worktree) and meet only at merge time. If `main` moved in the meantime, the branch is updated and CI runs again before the merge. A team in one shared tree is one branch, and only the lead commits. The local pass from [above](#who-reviews-the-agents-work) stays. It's the cheap first layer: it catches problems before anything is pushed, and it's where Votery's review actually happened. The PR is the gate and the record. Keep one PR per task. Votery's agent-era work went in as two PRs: the setup branch (the setup commit, its review-fix commit and five follow-ups) and the 1.1.0 release (81 commits across 199 files). Neither PR has a single review comment. The reviews did happen, locally and in the journal, but at that size the PR was only a merge button. Its description lists what changed, but no comment on it records a finding or how it was resolved. TFT used PRs as the record. A repository rule asked Copilot to review every pull request, which is where the 49 reviewed PRs and 187 inline comments [above](#what-do-reviewers-catch-over-a-whole-project) come from ([all of TFT's PRs](https://github.com/medukn/tempest-fauna-trail/pulls?q=is%3Apr)). My replies there were mostly one summary per PR ("all legit, zero false positives"), not one reply per comment. Answer each comment instead. A per-comment reply shows which finding was fixed in which commit and why a dismissed one was wrong, so anyone reading the PR later can trace what happened to every finding. Merge strategy is a detail. TFT squash-merges, so `main` gets one commit per PR; that fits when one PR is one task. ### When is the review loop done? Everything above is one agent loop seen from the outside: the agent acts (writes code), observes the result (runs the checks, reads the review) and decides whether it's done. That's the shape the ReAct paper described for language models that interleave reasoning with actions,[^react] and Anthropic's "evaluator-optimizer" pattern is the review half of it: one model produces, another evaluates, and the producer revises until the evaluation passes. Anthropic adds the part that's easy to forget: *"It's also common to include stopping conditions (such as a maximum number of iterations) to maintain control."*[^effective-agents] A loop without a written "done" and a written stop doesn't end when the work is good. It ends when the agent decides it's good. So write both down before the loop starts. In this setup "done" comes in four kinds, and the repo already has a check for each: | Kind of check | What answers it | Example | |---|---|---| | Machine, yes/no | lint, the full test suite, CI on the PR | "`pytest` green, CI `test` passed" | | Look at it | the running app, screenshots the agent opens and reads | the e2e doc in `docs/live/` | | Judgment against a rubric | a reviewer from another model family, with SPEC §V and the hard rules as its rubric | the review script's findings | | Only you | anything irreversible or outward-facing | merge, publish, delete, change repo settings | The review loop on a PR, with its exit: 1. Open the PR with the checks you ran. 2. Run the reviewer on the **whole PR diff** and post its report on the PR (`gh pr comment --body-file `), so the findings live where the change lives. 3. Check every finding against the code. Fix the real ones as new commits; reply "fixed in ``" or "dismissed: ``". 4. **Run the reviewer again** on the updated diff. Fixes introduce bugs too. 5. **Done** when one full round has no high or medium findings and CI is green. Low findings become issues. **Stop** after three rounds even if it isn't done: list what's still open and hand it to a human. Merging stays with you either way. Step 4 is the one I skipped before writing this section, and it's the one that paid off. Preparing a small app with this guide, I added a rule to the guard (the "data heredoc" rule in the hook section). The first review found a hole in it: a heredoc that writes a script the same command then runs. I fixed that and reviewed again. The second round found a different hole in the same rule that the first round had missed: a heredoc operator inside a comment, which bash ignores but the guard didn't. The third round found three more. That was the stop condition, so the loop handed the open findings to me, and instead of a fourth patch I removed the rule (see "Why a text guard keeps leaking"). In the same session, a Codex review found three ways past that app's own guard, whose 60 tests all passed. "Every comment answered", the merge rule above, would have let the first fix through: its one comment was answered. Where I'm not sure yet: - **Three rounds is a guess**, not a number I measured. The guard review in the hook section ran 15 rounds and kept finding things, so for a parser three is too few to reach "clean"; for a docs change it's probably too many. I haven't collected enough loops to pick better numbers. - **"No high or medium" trusts the reviewer's severity labels.** I haven't checked how often those labels are wrong. Skim the lows before you file them as issues. - **Re-reviewing the whole diff costs more** than reviewing only the fix commits. I do the whole diff because a fix can break something outside the lines it touched, but I haven't compared the two approaches on cost. - **A stop condition is also a design signal.** When the same piece of code fails three rounds in a row, the loop isn't converging, and that's worth noticing on its own. The "data heredoc" rule did exactly that. I dropped it, and the question shifted from "how do I patch this parser?" to "why does the guard parse shell at all?" I can't yet say how often hitting the limit points to a design problem rather than just a hard bug. Not every task needs this. A one-line typo fix gets the checks and one review, not a loop. The loop is worth it when the change is security-relevant, or when the first review found something real. ### What will only you catch? A reviewer checks the code against whatever context it's given, usually the code and the spec. It doesn't know what you meant unless that's written down somewhere it reads. In TFT, the misses that mattered most were caught by me, reading results and plans rather than diffs: - **The plan said one thing, the build did another.** The balance plan said not every mage needs to deal damage, and support values like healing should scale with intelligence instead. The implementation made every mage deal damage. Nothing in the diff was inconsistent; the code matched itself, just not the plan. - **A silent failure.** Weather fetching failed without telling anyone. Nothing was wrong in the diff; the design just had no error flag, so a broken feed would have looked like normal weather. - **Measuring the wrong thing.** A simulation meant to show weather's effect measured win rate only. Both teams had the same tiers, but each team's highest-tier pieces decided most fights, so a plain win/loss would have drowned out the weather signal. I caught it before the expensive run; the fix was to measure the margin of victory instead. A reviewer given the plan might have caught the first one. That's the argument for the ASK step in the plan skill, for handing reviewers the plan as well as the diff, and for reading results yourself: another model's review catches a lot, but checking the work against what you intended stays your job. ## How does the agent check the running app? By driving it, taking screenshots and reading them. The standing instruction in Votery, saved as agent memory: *"Verify changes in the running app: launch it, tap the buttons, take screenshots and actually look at them … Green tests alone is not 'done'."* The reason: *"widget tests missed real-device behavior (system back, lifecycle/keyboard, fonts offline, edge-to-edge)."* To make that routine, Votery has a small harness an agent can drive one command at a time: ```bash tool/e2e/emulator.sh up # create the emulator if missing, boot headless, disable animations (~25 s) tool/e2e/install.sh profile # build + install; never a release build (signing is off-limits) tool/e2e/drive.py begin my_check # lease the emulator + create a run dir tool/e2e/drive.py launch # cold start; prints startup ms tool/e2e/drive.py dump # visible labels + bounds tool/e2e/drive.py tap "Start New Game" # tap BY TEXT; on failure lists the visible labels tool/e2e/drive.py screenshot setup # NN_setup.png + NN_setup.txt (label dump) tool/e2e/drive.py end ``` Other commands: `back`, `home`, `resume`, `type`, `wait` / `expect` / `expect-not` (assertions), `keyboard shown|hidden`, `insets` (edge-to-edge clipping), `net off|on` (airplane mode, checked by ping), `logcat`, `startup [n]` (median cold start), and `run `. These are Votery's own scripts; the point is the shape, not the names. Build the equivalent for your platform. Why it works for agents: - **Tap by text, not coordinates.** Android's UI dump turns on accessibility, so Flutter exports its semantics tree. No app changes needed. When a tap fails, the agent gets the list of visible labels and can correct itself. - **Every screenshot comes with a text dump.** The agent reasons over text and still opens the PNG to check what it looks like. - **Flows are plain text files**, one command per line (`smoke_full_game`, `nav_back`, `offline_fonts`). On the first failure the harness saves `NN_FAIL.png` and the device log. - **A lock and a lease**, so a second agent gets "emulator leased by run …" instead of tapping into someone else's test. - **"Rebuild after every code change"** is written down. The harness drives whatever build is installed. When the shared working tree doesn't compile (another agent is mid-change), `E2E_SRC=` builds a clean worktree of `HEAD` instead. - **Measure, don't guess.** The startup task was closed with *"profile cold start median 554 ms"*. It caught what tests didn't: the back-button loop, the dialog pulled away by an animation, some layout issues. It also confirmed bundled fonts render with the network off. For web projects the equivalent is Playwright or a browser-automation MCP (e.g. Chrome DevTools). TFT checks its game logic end to end with a headless playtest CLI that simulates fights and whole runs (`uv run python -m tools.playtest.sim_fight`), though that doesn't cover the UI itself. ## How much should you give one agent at once? One task. If you have several, run them as a loop that hands the agent one task at a time, each with a fresh context, instead of one long session that does them all. The reason is the context window. Every file the agent reads, every test run and every failed attempt stays in it. On a second and third task, the window fills with material from the first; long sessions get compacted, and the early instructions are the first to blur. What you see is an agent that gets worse as the session goes on: it carries assumptions from one task into the next, "fixes" things outside its scope, and forgets the rules it followed an hour ago. Anthropic's advice points the same way: *`/clear` between unrelated tasks*, and after two failed corrections, start over with a better prompt.[^best-practices] A loop gives each task a clean start and keeps the state in files, not in the conversation: **Several tasks: loop, one at a time:** ```text For each §T row in order (T.12, T.13, T.14): 1. Start a fresh session (or a new subagent) for this one row only. 2. Run the per-task loop for it: read → plan → build → verify → land. 3. Land means: SPEC §T status, LIVING docs and the commit are updated and the PR is merged, so the next row starts from main and the files alone. 4. Stop the loop at the first row that isn't done, and report why. ``` TFT's plan skill writes one plan per run, for one §T row. Its build skill commits after every row but also has an `--all` mode that builds every open row in one session; that's the mode this rule says to avoid, and the build skill in the [prep-repo templates](#install-the-prep-repo-skill) only takes one row. Votery's release split the work across four teammates, each with one area and its own files, rather than giving one agent the whole release. The [per-task loop](#the-per-task-loop) below is the inside of each iteration. ## When should you use several agents at once? **Start with research and review, not parallel implementation.** Anthropic recommends the same: *"start with tasks that have clear boundaries and don't require writing code"*, and *"Start with 3-5 teammates."*[^teams] Votery's setup run: three **read-only** researchers (core logic, UI, docs) in parallel with a hard word cap, then two writers with **separate file ownership** (tests vs docs) while the lead wrote the spec. No file conflicts. The lesson, from the journal: > Research-first with **read-only** teammates and a hard word cap made the reports cheap to > verify. Every report still had at least one wrong claim, so "verify the report before building > on it" has to stay an explicit lead step. The teammate brief that worked: **Prepend to every teammate's spawn prompt:** ```text You own . Don't edit anything else. No git write commands; only the lead commits. No SPEC.md edits. End your report with two sections: "SPEC changes needed" and "Found, not fixed". Cite file:line for every claim. Max words. Task context: . ``` **[Claude Code]** Teammates don't inherit the lead's conversation, so the spawn prompt must carry the task context.[^teams] They do load CLAUDE.md, rules and skills, which is one more reason to keep project knowledge in files. Anything the agents share (an emulator, a dev database, a port) needs a lock or lease. For parallel sessions without a team, git worktrees give each one its own checkout. **Agent-team launch gotchas [Claude Code], verified on v2.1.282** - Agent teams are experimental: set `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1`; `teammateMode` `tmux` or `auto` for split panes. - **Start the lead with plain `claude` inside tmux**, so teammates split the lead's window. A lead started with `claude --resume …` or `--bg` runs under the background daemon, which drops `$TMUX`; teammates then land on a detached `claude-swarm-12345`-style tmux server (named after a process id). Attach to it from a *separate* terminal tab (nesting it inside your own tmux collides with a custom prefix key). - From the docs: `/resume` doesn't restore in-process teammates; task status can lag; one team per session; no nested teams.[^teams] - Teammate permission prompts bubble up to the lead, so pre-approve safe commands before spawning. - Fire-and-forget workers (a reviewer in a `tmux split-window … claude` pane) are independent sessions with no shared task list. Use them only for side jobs like a read-only review. ## How does work survive the end of a session? Sessions end, contexts get compacted, agents restart cold. Continuity has to live in files: | Mechanism | What goes there | Votery example | |---|---|---| | **SPEC §T / §B / §D** | Status of every task, bug and open question | Task T.14 went from 🔶 "back button loops → GH #7" to ✅ "GH #7 → B.9 fixed" | | **Release HANDOFF.md** | Done, open (ordered, with exact checks), deferred bugs, how to work | "Read this first when resuming." | | **Journal** | Why things are the way they are; agent errors not to repeat | `2026-09-25_agentic-setup.md` | | **Agent memory** | Your preferences and corrections that aren't derivable from code | "verify in real app", "secrets in password manager" | | **GitHub issues** | Bugs found but not fixed this session | #5–#14 filed at a checkpoint; later fixed as §B entries | **Deferred bugs go to GitHub issues**, not handoff notes. The rule, saved as a memory: *"When a bug is identified but not fixed in the current session, file a GitHub issue (`gh issue create --label bug`) with the symptom, cause (if known), fix idea, and a guard or test suggestion."* And at every checkpoint, sweep teammate reports for "found, not fixed" items. Handoff notes get lost; issues show up where people look. **Memory hygiene [Claude Code].** Auto memory stores four types (`user`, `feedback`, `project`, `reference`) and loads the first 200 lines / 25 KB of `MEMORY.md` each session.[^memory] Good memories record the *why* and *how to apply*: ```markdown --- name: verify-in-real-app type: feedback --- Verify changes in the running app: launch it (Android emulator), tap the buttons, take screenshots and actually look at them … Green `flutter test` alone is not "done". **Why:** widget tests missed real-device behavior (system back, lifecycle/keyboard, fonts offline, edge-to-edge). **How to apply:** use the e2e harness in `tool/e2e/` … Attach screenshot evidence when reporting UI changes. ``` Keep secrets out of memory files: they're plain Markdown on disk. Make **checkpoint commits** at natural pauses so the next session starts from a consistent SPEC, and **`/clear` between unrelated tasks**. A fresh session with a good handoff beats a long one full of failed attempts. ## Step by step: preparing a repo This is the order Votery used, generalised. Steps 1–9 fit on one `chore/agentic-setup` branch. Votery did them as one commit with no application-code changes, followed by a review-fix commit. The [prep-repo skill](#install-the-prep-repo-skill) runs the same phases. 1. **Interview, branch, baseline.** Before writing anything, settle: which agent tools will work here (→ CLAUDE.md or AGENTS.md), where secrets live, how "done" is checked in the real app, which second model can review, which files must never move. Then `git switch -c chore/agentic-setup` and record: lint clean? How many tests, passing? Does it build? Is there CI? 2. **Secrets sweep first.** Search file *names* only: `git ls-files` and `git log --all --name-only --format=` for keystores, `.env`, key files. For contents, use a scanner that redacts matches (`gitleaks detect --redact`) and let a human inspect hits; an agent grepping file contents across history can print a live secret into its transcript. For anything found: back it up **and verify the backup**, then untrack and gitignore it, decide on rotation, record a §B bug. Write store/fetch scripts (no tracing, temp-then-move, verify the pairing, exact-name lookup). 3. **Research with read-only agents**: core logic, UI, docs/build, with a word cap. **Verify every claim** by grep before using it. 4. **Distil SPEC.md from the code** (+ FORMAT.md). §C includes privacy/network and secrets. Every §V has `Guard:` or `unguarded`. Any rule about behavior you haven't pinned with a test yet is provisional: mark it `?`, and let step 5's tests confirm or correct it. §T starts with T.0 (the app as it is), then tests, extractions, debt. Surprises go to §D. 5. **Characterization tests + CI.** Assert actual behavior; record surprises, don't fix them. Add a reference oracle for logic you'll extract. CI runs lint + tests (+ format where it won't bury diffs), toolchain pinned. 6. **ARCHITECTURE.md**, written from the code: each system (what it does, where it lives, what it talks to), the patterns in use with an example of each, cross-cutting state, storage and randomness, a "Where do I find X?" index and one end-to-end walkthrough. Name known violations and link them to §T rows. 7. **Split the docs.** `git mv` old docs into `docs/design/` (FROZEN) and `docs/journal/legacy/`; create `docs/live/` with its README and header convention; mark served files "never move". 8. **Agent config.** CLAUDE.md (or AGENTS.md + a thin CLAUDE.md) under ~100 lines. For Claude Code: `.claude/settings.json` allow/deny; `.claude/hooks/guard.py`, **tested** with sample inputs; `.claude/rules/.md`; skills plan/spec/build/backprop/check; `docs/templates/`. 9. **Review the setup with two model families**, read-only, somewhere without secrets. Then push the branch and open it as a pull request, so later findings land there as comments. Verify each finding, fix in a follow-up commit on the same branch, reply to each comment, write the first journal entry. Before merging, **protect `main`**: require a pull request, require the CI check to pass on a branch that is up to date with `main`, block force-push and deletion. From then on nothing reaches `main` except through a reviewed PR. TFT's repository rule blocked force-push and deletion and asked Copilot to review every PR, but it didn't require a PR, and its latest documentation commits went straight to `main`. 10. **Tooling for "done"**: the review script and a real-app driver (emulator harness, Playwright, CLI smoke script). Document it in `docs/live/`. 11. **Pay down the risky structure first**: extract pure logic behind the oracle, inject randomness and clocks, remove dead code. Only then UI and features. 12. **Set up continuity**: a handoff doc per release, memory entries for standing preferences, a `bug` label for deferred issues. ## The per-task loop Once the repo is prepared, every task (one §T row, or one bug) runs the same loop. Paste it into a prompt, or let the plan and build skills carry it: **Per-task loop:** ```text TASK: §T. — 0. Setup - Fresh branch from an up-to-date main (never main, never a previous task's branch). Shared tree → stage explicit paths only. 1. Required reading (cite file:line; docs lie, code wins) - SPEC §T.<n> row, every §V that could apply, nearby §B, related §D - ARCHITECTURE section for the touched layer - docs/live/<area>.md - the touched code itself (all copies of any duplicated logic) 2. Plan (plan skill → docs/design/tasks/t<n>_<topic>_plan.md) - Gap today table (piece | file:line | ✅/🔶/❌) - Ask only genuine forks, with a recommendation + trade-off - Machine-checkable acceptance criteria (commands + expected output) - "SPEC changes needed" → apply via spec skill on OK 3. Build (build skill) - Failing test first (named after the §V it guards) - Implement; no domain logic in UI; injectable randomness/clock - Failure → backprop: my bug / spec wrong / unspecified edge case? (b)/(c) → §B entry, link the §V that should catch it (existing or new), add or strengthen its guard test, then continue - Stuck twice on setup/tooling → read official docs, cite URL 4. Verify (show output, don't assert) - lint clean + full test suite green (paste the tails) - UI touched → run the real app: click/tap through, screenshots, READ them 5. Land - Same change first: SPEC §T status, ARCHITECTURE, docs/live/ - Stage the task's files by explicit path; `git diff --cached --stat` shows only this task - Shared tree → verify the staged snapshot alone (git archive HEAD + apply the cached diff in a scratch dir, then lint + test) - First-pass review of the staged diff (review script with --cached, other model, read-only); verify each finding. Any fix → re-stage, re-test, re-review - Commit exactly what was reviewed; message says why (backprop §B.x + §V.y: … for bug fixes) - Push the branch; open a PR: §T row, what changed, checks run (paste the tails) - PR review: other model family, Copilot, you; findings arrive as PR comments - Post the reviewer's report on the PR (gh pr comment <n> --body-file <report>) - Each comment: verify against the code → fix as a NEW commit on the same branch, push, reply "fixed in <sha>"; or reply "dismissed: <reason>" - Re-review the whole PR diff after fixes. Done: one round with no high/med findings and CI green (lows → issues). Stop after 3 rounds: list what's open, hand to the human - Merge only when: branch up to date with main, CI green on it, every comment answered, required reviews done - Found-not-fixed bugs → gh issue create --label bug; link from the PR and the handoff 6. Record (after milestones) - journal entry incl. Process notes (AI collaboration) + prompting reflection - check skill → no 🔴 ``` ## What goes wrong most often? Trusting the agent's own account of what it did. Almost every failure in both projects was a confident statement nobody checked: a research report, a spec line, a "done", a review finding. The rest were safety slips around secrets. Each item below happened at least once. **Do** - Verify every agent report by grep before building on it; keep reports short so that's cheap - Pin behavior with a characterization test before writing the invariant - Check that each new test fails without the change, and that each guard calls production code - Test a restore right after a backup, checking the property that matters - Split files over 400 lines along real flows; agents edit giant files badly - Write tool launch recipes down with the date and version they were verified on **Don't** - Write up a blocked or failed action as done; re-check the real state first - Apply reviewer findings unverified (two were wrong; one flagged Dart `%` falsely) - Rely on deny rules alone (they covered Read but not `git show`) - Run secret-handling scripts where `bash -x` can print tokens - Mass-reformat code in the same diff as a refactor - Keep found-but-unfixed bugs only in handoff notes; file issues **Prompt shapes that worked well:** - "Assert actual behavior, flag surprises, don't fix them" (characterization tests). - "Trace the cause and decide whether an invariant should catch recurrence" instead of "just fix it". - Read-only researchers with a word cap, then writers with separate file ownership. - Any agent-loop brief as **goal + machine-checkable "done" + how to check + hard stop**. - Reviewer prompts that name the contract, ask for vacuous tests and leaks, and forbid style nits and praise. ## So, what does a repo need before you hand it to an agent? Less trust in the agent and more checks in the repo. What surprised me most wasn't that the agents made mistakes. It was how confident and plausible every wrong claim looked. The agent still does most of the work; the repository decides whether it's done. If you want to try it, open a repo, install the [prep-repo skill](#install-the-prep-repo-skill) or point your agent at the [raw guide](https://frequently-asking-questions.com/guides/agentic-repo-prep/GUIDE.md), and start with [step 1](#step-by-step-preparing-a-repo). [^best-practices]: Anthropic, [Best practices for Claude Code](https://code.claude.com/docs/en/best-practices): verification, writer/reviewer pattern, hooks vs CLAUDE.md, adversarial review. [^memory]: Anthropic, [How Claude remembers your project](https://code.claude.com/docs/en/memory): CLAUDE.md size, AGENTS.md import, path-scoped rules, auto memory. [^agentsmd]: [AGENTS.md — a simple, open format for guiding coding agents](https://agents.md/). [^speckit]: GitHub Blog, [Spec-driven development with AI: Get started with a new open source toolkit](https://github.blog/ai-and-ml/generative-ai/spec-driven-development-with-ai-get-started-with-a-new-open-source-toolkit/). [^permissions]: Anthropic, [Configure permissions](https://code.claude.com/docs/en/permissions) and [Sandboxing](https://code.claude.com/docs/en/sandboxing). [^sandboxing]: Anthropic, [Configure the sandboxed Bash tool](https://code.claude.com/docs/en/sandboxing): `filesystem.denyRead`, child processes, protected paths, `failIfUnavailable`, `allowUnsandboxedCommands`, what runs outside the sandbox; path prefixes and Linux wildcard expansion in the [settings reference](https://code.claude.com/docs/en/settings-reference). [^codex-auth]: OpenAI, [Codex authentication](https://learn.chatgpt.com/docs/auth) (formerly developers.openai.com/codex/auth): ChatGPT sign-in vs API key billing, `~/.codex/auth.json`, `cli_auth_credentials_store` (`file`, `keyring`, `auto`, `ephemeral`). [^settings]: Anthropic, [Settings files and precedence](https://code.claude.com/docs/en/settings). [^hooks]: Anthropic, [Hooks reference](https://code.claude.com/docs/en/hooks): PreToolUse, exit code 2, `permissionDecision`. [^skills]: Anthropic, [Extend Claude with skills](https://code.claude.com/docs/en/skills). [^teams]: Anthropic, [Orchestrate teams of Claude Code sessions](https://code.claude.com/docs/en/agent-teams). [^effective-agents]: Anthropic, [Building effective agents](https://www.anthropic.com/engineering/building-effective-agents) (2024-12-19): evaluator-optimizer workflow, stopping conditions, ground truth from the environment. [^react]: Yao, Zhao, Yu et al., [ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/abs/2210.03629) (2022).