Agent Reliability Kit — Stop Your Coding Agent From Lying About “Done”
Four hooks and two CLIs that stop your coding agent from telling you it's done when it isn't. Your agent finishes and says: "Done. It's live at github.com/acme/thing, build passed." Three of those claims were never checked. Sometimes the repo doesn't exist. You find out later, by hand, after you've already acted on it. Asking the agent to double-check doesn't fix it — that's the same model with the same prior, confirming its own hallucination. The fix has to be structural: something outside the model that touches the outside world before the turn is allowed to end. What's in the box done_gate.py — Stop hook. Blocks the turn when the agent asserts completion tied to a repo / URL / build claim and no verification actually ran. verify — CLI. Hard-checks every repo (gh api) and URL (curl) in a claim. Exit 0/1/2, composes in shell. Optional adversarial model panel. qcheck — CLI. Fails any deliverable whose quality_check.md is missing a dimension or repeats a claim you already ruled out. task_gate.py + subagent_gate.py — inject canonical project facts into both sides of a subagent spawn. Context doesn't flow downhill on its own. session_digest.py — opens every session grounded in what's actually true. PROJECT_TRUTH.json + KNOWN_FAILURES.md — the fact registry and failure ledger the gates render from, live, so nothing carries a stale copy. The Five Failure Modes of Coding Agents — 7-page PDF. The patterns, why each gate is shaped the way it is, and the adoption order. Built from production use, not theory Every design decision here traces to a specific failure. The trigger excludes weak verbs because the first version false-blocked ordinary status messages and got disabled within a week. The evidence scan excludes the agent's own prose because for three weeks it didn't — and an agent could satisfy the gate just by typing "all checks passed." The playbook is honest about all of it, including the mistakes. Details Python 3.8+, stdlib only. No pip install, no dependencies, no telemetry. One-command installer. Merges into settings.json, backs it up first, idempotent. Every hook fails open — on any error it exits cleanly. A crashed gate never wedges your session. Turn any of it off instantly with an env var. gh CLI and curl optional; missing tools report "skipped", never a false pass. Single-user commercial license: use it on your machines and in your company's work, modify it freely, ship products built with it. Don't resell the kit itself. Shift The Culture Works with the free tool The same operation ships whats-running-mcp — free, MIT, no telemetry — an MCP server that hands your agent live machine state (sessions, listening ports, daemons, load) straight from the OS. That server answers what is running. This kit answers is what the agent just told me actually true. Neither requires the other, and the free one stays free and complete — nothing was held back from it to build this. Running several agents on one machine? The companion is the Agent Fleet Ops Kit ($29) — collisions, port ownership, and orphaned sessions. Both kits and the free server are laid out at shifttheculture.media/agent-tools.Where this comes from. We ship three free, MIT-licensed MCP servers for running agent fleets: whats-running-mcp (what is actually running on the machine), whats-loaded-mcp (what is eating your context before a session starts), and whats-inherited-mcp (what a checkout tells your agent to do before you run it). All three stay free with no telemetry — read them before deciding whether this kit is worth $29.
Get it → stcai.gumroad.com