I call this a fail-open check: a check whose “did not run” is indistinguishable from “passed.” Over one month I logged twenty-four named instances in one small stack of scripts, CI jobs, and agent pipelines. Not, I think, because the stack is unusually bad, but because I gave the class a name and started writing instances down.

Green is ambiguous by construction

Three from the same month:

• A wrapper around a nightly embedding job logged success unconditionally and discarded stderr. The job had been dying mid-batch, at exit 0, behind months of healthy-looking logs.
• A style lint reported clean on its first run because a comment line broke the grep over its own pattern file. Zero banned patterns were matched against anything. The check had not passed; it had not run. The recorded outcome was identical either way.
• A link checker printed every finding to stdout and exited 0. Three downstream consumers read the exit code. None read the text.

The absence of a result gets recorded as the presence of a lesser one. A checker has three honest outcomes: found problems, found nothing after actually looking, and could not look. Most tooling gives it two exit states, so “could not look” gets folded into whichever side the error handling happens to land on. When it lands on green, the result is strictly worse than no check at all: a fail-open check produces a false record that closes the question, and the record is what stops anyone from looking again. A missing check, by contrast, is a visible gap. Anyone who asks “what verifies this?” finds nothing and knows the question is open.

The artifact is the camouflage

These survive for months because the failure produces an artifact, and the artifact conceals it. A test suite in that same stack wrote its “real build” case to the shipped deliverable instead of a fixture, which makes it an unreviewed production change wearing a green checkmark. Run to confirm an unrelated change was safe, it passed, and in passing silently rebuilt the deliverable against different inputs, evicting real content. Nothing failed, nothing warned; the output was a plausible, well-formed, freshly-dated file.

A curated export, built on a cloud machine, sat in the repo vouching for its build script, but an artifact is not evidence its producer runs here: the script used tomllib (Python 3.11+) against a local 3.9.6 and had never once executed locally. Dead at import. Its own test suite failed 9 of 14 cases the same way, and nobody read past the “5 passed” to ask out of how many.

This is not a tooling quirk; it is how oversight fails generally, because headcount measures effort applied, not failures found. Buffett's 2008 shareholder letter describes OFHEO, a regulator created to oversee exactly two companies, staffed with more than a hundred people with no other assignment. It published a glowing review of its own first decade and, on Buffett's telling, entirely missed that both companies had spent years misstating their earnings.

The ranking error underneath is universal: crashes and blank fields at the bad end of the severity scale, mostly-correct output at the good end. Ranked by expected damage, the order inverts. A crash routes immediately to a human who now knows something is wrong; the output that looks complete routes to acceptance, consuming exactly the attention budget that would have caught it.

The sneakiest variant: the check that cannot see the defect

The instances above fail by not running or not examining. The subtler ones run perfectly, against a scope that excludes the defect.

My own site's CI config carried .github/** in its paths-ignore list, alongside **.md and LICENSE. Reasonable on its face: editing a workflow file doesn't change the application. It meant the one change class able to delete a job, loosen a trigger, or drop a required check was the only class that merged with nothing run against it; a weakened gate and an intact one leave identical clean history, so nothing in the record would ever have surfaced it.

The same scope failure arises with nobody configuring it: a frontmatter parser whose regex read only the first item of every YAML block list computed a health metric over a universe 16% smaller than it claimed, and printed roughly 78% either way. A list API returned the first 20 items with has_more: true, then returned the identical page when handed its own continuation cursor. Nothing consumed the one field that contradicted the roster, and I nearly filed a report that four scheduled jobs had vanished. They were on page two.

And git log --since='7 days ago' --diff-filter=A --name-only on a fresh CI clone reported 1,618 files added that week; the true figure was 32. The clone was shallow, so its boundary commit appeared to add the entire repo at once.

The design-time test for all of these: if this control were wrong, what would tell me? If the honest answer is “the same green output I get when it is right,” you do not have a check so much as a green light wired to the wall.

The field guide

Rules that have held up, each earned by at least one instance above:

1. Three outcomes, never two. Passed / failed / could-not-run, and could-not-run must be loud. This applies per item, too: an input your checker cannot parse must land in undetermined, never in a benign bucket like “no findings.”
2. Assert the condition, not the reaching of the line. Emit success only after verifying the thing you claim, and match the invariant's shape to what consumers depend on. An embedding backfill satisfied its count-shaped post-condition (pending reached zero) while writing the same vector for forty different documents. A count cannot see a content defect.
3. Every emitted marker needs a named consumer. A has_more flag, a truncation marker, a warning line: a signal nobody reads is worse than no signal, because it looks like coverage.
4. Test the check against known-dirty input, from the position it actually runs in. Both checks I wrote to catch this class missed their own motivating cases on first run; only fixtures with known answers caught it. And a detector validated on a laptop can still be unrunnable at its scheduled call site; testing the detector is not testing the detection.
5. A control's scope must include the control. Whatever decides what gets checked (a path filter, a sampling rule, a pagination default) is the best-hidden place for this defect.
6. Spend verification on the fix. The patch that closes a fail-open is itself fresh, unverified check code, written under pressure with attention on the old defect. In my log, remediation builds are where new instances concentrate.

“So write more checks” is the wrong response. The fix for this class is a contract on the checks you have, not a headcount of new detectors, and every new detector is new surface for the same defect. “We have a runbook for this” is worse: a written policy with no enforcing mechanism is not a missing check, it is a fail-open one, occupying the slot where verification would report while readers take its existence as evidence the boundary holds. Often the strongest move is not a better check at all: restructure so the bad output cannot be produced. Will Larson describes catching an agent mis-forwarding alerts and, rather than adding an eval he knew would work, moving the filtering into a deterministic script so the agent never sees what it might mishandle. The check would have left the failure mode alive behind a gate; the restructure removed it from the system.

The same month produced two defects I deliberately did not log: they failed closed. Costly, but they never certified a falsehood, and logging them anyway would have blurred what the ledger measures.

A clean report from a check that cannot say “I could not run” is not evidence of anything. Build checks that cannot fail quietly, or their green eventually becomes the thing that hides the failure they were hired to catch.

So the vault of 1,700 claims stays in git, and OB1 projects it into Supabase behind MCP, so Claude, ChatGPT, and my phone all retrieve the same judgment. A claim is one assertable proposition, named by a filename that states it, hand-typed as claim, pattern, tension, or anti-pattern. Nothing derives that type, which is why it carries judgment: a thing is an anti-pattern because someone decided it was. Retrieval embeds the query, ranks with pgvector, then expands one hop along the wiki-links, so a claim reached by a parallels edge surfaces even when its similarity is low; 2,777 of the 8,281 edges cross domains, the ones worth the hop. On 15 August a reduce pass was about to generalize a second claim about legibility inverting felt value. Retrieval returned prevented-loss-invisibility, already established on an external source, and the output became an enrichment instead of a duplicate. The judgment you have to remember to look up is judgment you do not have.

What actually carries a constraint is position, not volume. Attention has a measurable front-and-back bias: RoPE, the positional encoding most current models use, decays in a way that puts tokens far from both ends of the sequence into a systematically lower-attention zone, and retrieval accuracy for a fact placed mid-context drops by more than 20 points compared to the same fact at the start or end. A constraint's location is load-bearing in a way its wording is not. So the rewrite was tiered rather than shortened: identity and non-negotiable constraints at the edges, task detail in the middle, and reinforcement anchors placed to survive attention decay across a long multi-turn conversation rather than only the first exchange. The other half of the compression was subtraction. Salesforce's corollary is that anything you can draw as a flowchart belongs in code, not in a prompt, because code executes identically every time and no wording does. A context window is an attention budget for the run, not a junk drawer for everything that once went wrong.

Management needs a unit of account, and session-level thinking is the wrong one. The right unit is the agent run: it begins at delegation, contains the tool calls, branches, and corrections, and ends in acceptance or rejection. That framing makes the work measurable: completion rate, correction rate, and whether your approval gates ever actually reject anything (a gate that always approves is not a control, it's theater). It also surfaces a free asset: every correction you make to agent output is a labeled evaluation you wrote by acting, the natural test set for the next run. This is the same discipline as my receipts rule: done without an attached artifact is self-attestation by the party most motivated to claim success. Getting the machine to do the work is the easy part now. Deciding the work is trustworthy is the job.

The corrective is ownership, and it decomposes into four responsibilities I now require for any agent near production work. Define the job narrowly: a vague agent is an unowned agent waiting to happen. Curate the diet: what it reads, which examples it learns from, including rejected outputs so it learns what not to do. Manage permissions proportional to stakes: draft-only and write access are different categories, and write access is earned inside a narrow job, not granted because a demo looked good. And run the review loop, where one-off failures get fixed at the output level but recurring failures get fixed at the system level. Team agents fail by tragedy of the commons (the pain is collective, the maintenance is nobody's job), so the owner follows the work. An agent is not a feature you ship. It is a service you operate.

Building production systems with Cursor and Claude Code has restructured where my own hours go. My leverage stopped correlating with how fast I can type and started correlating with how precisely I can state three things: the goal, the boundary, and what "done" has to prove. The human stays at the two endpoints, specification in and satisfaction judgment out, and everything between is increasingly the machine's. This also explains why AI amplifies experts instead of equalizing them: it equalizes execution speed, but execution was already cheap. What it amplifies is specification quality, and specification quality is a direct function of domain depth. If the agent keeps disappointing you, the uncomfortable first question is no longer about the model. It's whether you actually specified the thing you wanted.

The good news: distributed systems solved these coordination problems decades ago, we just have to notice the mapping. A visible CLAIMED state on a task is a lease, revalidated when the worker returns. "Done" without an attached receipt is self-attestation by the party with the strongest incentive to declare success, so the receipt (the diff, the test run, the artifact link) is non-negotiable, the same way you require an acknowledgement instead of trusting a fire-and-forget write. And the issue tracker you already run is the natural control plane: it has owners, statuses, comments, links, and history built in. Reliability is engineered into the loop, not summoned from the model. Make the loop less ambiguous before you ask for a smarter agent.

The more interesting find while doing this wasn't a feature, it was a bug. My static-site adapter's SPA fallback page and the prerendered root route both wanted the filename index.html, and the fallback was winning the write, silently replacing the real homepage (title, description, Open Graph tags, all of it) with an empty shell at build time. Every crawler and every link preview had been getting nothing. The fix was a one-line rename, but the lesson generalizes: a static site's build output is not implied by its source, so verify what actually ships, especially at the config layer nobody re-reads after initial setup.

As the person who sets AI-adoption standards for my org, I spend almost none of my time on prompting technique. I spend it on the guardrails: what an agent can touch unsupervised, what requires a human review gate before it ships, and what "done" has to prove before I believe it. This site's agent-readable rewrite was built the same way: the notes, the structured data, and the build-output bug above were scoped to specific files, verified against the existing type-check, lint, test, and build gates before anything shipped, with the plan surfaced for review rather than pushed silently. "I use AI" stopped being the differentiator. Whether you can hand an agent a boundary and a review bar, instead of still typing every line yourself, is the one that's left.

What changed for me was the filter. When a first version costs little enough, an idea stops being killed at the worth-building stage and starts being killed by contact with something running, and the second filter is both later and far more informative, because a running version answers questions a plan cannot. The compounding lives in the attempt count rather than in any single task, which is exactly what a stopwatch on one task cannot see: it measures the task in front of it and says nothing about how many got tried. METR's own February 2026 update reports that developers are likely more sped up now than in early 2025, while selection effects leave the size of that increase uncertain. I have no measured attempt count of my own to put against theirs. Count the things you tried, not the speed you felt.

The browser sits behind a network nobody controls, so a request that times out has not necessarily failed, and a payment the reader submits twice still has to land once. The IETF's httpapi working group took an Idempotency-Key header draft to revision 07 across four years and let it expire there, which says more about the problem than a finished standard would. What it encodes is that a client can repeat a request without the server applying the work twice. Rendering at the edge is a blast-radius decision before it is a performance one, and every time I have watched a region degrade during a market event, what mattered was that traffic routed around it instead of queueing behind it. None of this is new work. It is replication and failure domains, arriving in a codebase where nobody uses those words. You cannot reuse a solved problem you do not recognize.

The fix is to stop treating arrival and display as the same event. I buffer incoming chunks in a mutable useRef, which accepts writes synchronously without scheduling anything, then flush to state on a requestAnimationFrame tick, so the DOM is written at most once per frame and only when the browser is about to paint. Throughput and render frequency come apart: the stream arrives as fast as the model emits, and the interface still paints on a steady cadence. This separation is older than React and older than LLMs, since a game loop makes the same split between simulation and draw. When the producer is faster than the consumer, the answer is a buffer and a clock, not a faster consumer.