For about a day this month, one of our production databases could not accept a new entry in its vocabulary table. Every insert failed with the same error: a column that no longer existed. No alert fired. No test went red. The system that governs which words our tools are allowed to be built from had stopped learning, and the only way anyone found out was by trying to teach it something.
The cause was ordinary. A schema migration split one column into two and dropped the original. It added a new version of a helper function for the new shape, but left the old version in place, still reading the dropped column, and a trigger was still calling the old one. The interesting question is not how the bug got in. It is why nothing caught it.
The answer is that three migrations had been applied to the live database by hand and registered nowhere: not in the code's migration list, not in the ledger. The automated test fixture is built from that list. So the fixture never reached the shape the live database was actually in, and every test that passed was passing against a database that did not exist. A fixture can prove logic. It cannot prove state. This was the third time we had recorded that exact class of failure.
The pattern underneath
Five other near-misses came out of the same session, and they rhyme.
An AI agent working in the codebase ran a query that counted registered capabilities and got zero. It could see related rows that plainly existed, so it concluded the query was wrong. The query was right; the rows were in the test fixture, not the live database. The correct answer was never one number. It was "0 in live, 2 in fixture," and the gap was the finding.
The same agent hand-wrote a bug report and numbered it B-124. B-124 already existed, in the database that issues bug numbers. The file collided with a live identifier.
Every one of these was corrected the same way: not by someone with a better opinion, but by re-deriving a measurement with its context attached. Which query. Which database. Which value. When measurement and judgment compete, measurement won every time that day.
The characteristic failure of a capable model, in our experience, is not ignorance. The agent knew what a fixture was. It knew identifiers come from the system of record. The failure is confident generalization from a single measurement, and you do not fix that by asking the model to be less confident. You fix it by making the measurement carry its own context whether or not anyone remembers to ask, and by making the identifier come from a writer that will not accept a number from its caller.
Rules that are mechanisms
We run a set of guard programs that inspect every tool call the agent makes before it runs. That same day, one of them refused an inline python3 -c call with a message that named the rule and the alternative: anything repeatable becomes a proper program; use the file editor for a one-off; otherwise build the tool. The agent changed approach in a single turn.
Then a second, cruder layer refused the same call after the guard had been taught to allow an attributed exception. The second layer runs first, cannot express "allowed with attribution," and returns no message. The guard that knew about the exception never got to speak.
Two lessons came out of that pair. First, a refusal that names the replacement costs one turn; a silent denial costs several and teaches nothing, because it tells the agent only that one route is closed. Second, two decision layers with no shared evaluation order will eventually disagree, and the blunt one will win.
The chain that inspects a tool call is already shaped like a packet filter: a coarse list, then finer guards, then the tool, then a log. So we are configuring it like one, on the model of OpenBSD's pf: an ordered ruleset where the last matching rule decides, a quick keyword that lets the dangerous rules opt out of that ordering so no later loosening can reach them, and tags so a category is classified once and referred to afterwards. The grammar is written and has 49 passing tests. It is not yet wired to anything; the two layers are still two layers today.
What we took from it
A rule written in prose that a model is asked to remember is not a rule. A mechanism it cannot act outside of is. Memory cannot block a merge; a check can.
A count without the name of its database is not a count.
Identifiers come from the system of record. An author who mints one has forked the record.
Guards should teach. Name the rule, name the replacement, cite the line.
And every rule should be able to name the incident that earned it. One that cannot is a convention wearing a rule's clothes.
None of the fixes that day came from asking the agent to try harder. All of them came from building something it could not act outside of, and making sure that when it was stopped, it was told why.