Twenty-three releases and 164 commits between August 12 and August 29. The last instalment of this series said most commits were chasing one defect in different costumes. Two weeks later I can report that the defect has an excellent wardrobe.
A thing that did not happen must not look like a thing that did.
A check that never ran, reporting as passed. A cost nobody measured, rendering as $0.00. A screenshot from two releases ago, looking current. A pipeline installed in seventeen projects and able to run in four.
Here are the ten that matter, roughly in the order a user meets them.
1. Verification stopped being a suggestion
The dispatcher did not verify anything. It printed a sentence — VERIFY FIRST: run … — and a printed sentence is followed when the reader feels like it.
Measured rather than assumed: 31 agent runs, 10 scores, and most of those ten were run by hand. The pipeline's quality gate was, in practice, my own conscientiousness on a given afternoon.
Verification now runs. It is not advice.
Why it matters. A check that can be skipped is not a check, it is a preference. The difference between the two becomes visible on exactly one day, and you do not get to choose which.
2. The judge got 48× cheaper, which is why #1 was possible
Verification stayed optional partly because it was expensive. So the price came down first:
| context | in / out per M | one stage | 8-stage pipeline | |
|---|---|---|---|---|
| previous judge | 1.0M | $3.00 / $15.00 | $0.0252 | $0.201 |
| current judge | 1.31M | $0.075 / $0.25 | $0.0005 | $0.004 |
Cheap is worthless if it judges badly, so it was measured on two fixtures with known answers, three runs each: right 6 of 6, no disagreement between runs, matching the model it replaced. One free candidate abstained on the passing fixture, which is a creative way to never be wrong and disqualifying for a judge.
Why it matters. You can only make mandatory what costs less than skipping it. #1 does not exist without #2.
3. An agent's report about itself is not evidence
The pipeline handed one agent's output to the next on the strength of a line the agent had written about itself. architect | APPROVED | arch=docs/… is a self-report, and every downstream stage treated it as fact — so a wrong one was not caught at the next stage. It was built on.
The parts needed to check it were already in the repository, pointed at nothing:
- The router to a second model was **declared by 19 agents and had never been
invoked once.** The router worked. Verified with a live call. Nineteen agents carried it in their context, and zero used it.
- The acceptance script had exactly one caller: its own test — because no
document in the repository had the section it looked for.
Why it matters. This is the most expensive failure mode in any multi-agent system. A wrong result is not discarded at the next step; it becomes the foundation.
4. The pipeline was installed, wired, and unable to run
The path to the pipeline map resolved against the current working directory — the project, not the plugin. So only a project that happened to carry its own copy of the map could dispatch anything.
Measured across the registry: of seventeen projects, four had a copy and thirteen did not. In those thirteen the hook reached a silent exit and said nothing. No dispatch, no verdict, no error.
Why it matters. A tool that silently does nothing is indistinguishable from a tool with nothing to do. Three quarters of installs were in the first state and looked like the second.
5. The cost log stopped reporting $0.00 over spent money
Four links in one chain: a hook measures token usage from the transcript, prices it, appends a line to cost-history.log; the board reads that line and enriches the verdict, which carries no cost of its own because agents do not measure themselves.
All four links existed. Three were broken and one was mislabelled. The result looked exactly like a project nobody had ever measured: 26 verdicts in the window, zero with a cost, and a budgets screen reading unmeasured. True about what it had read. False about what existed.
A related repair landed this week: the same log holds two kinds of row — one run, and a session's running total. The benchmark collector added the running totals together, counting the same dollars once per snapshot. It reported $90,026 for a series whose largest single reading is $3,912. The correct figure is $7,420. A running total contributes its increment; a reading lower than the one before it is a new session, not a refund.
Why it matters. A budget that cannot fire is not a budget, and $0.00 next to the word measured is worse than an honest "I don't know". A 23× over-count wearing the same word is worse still.
6. Every stage declares what it produces
A stage that declared nothing was the cheapest possible way to pass verification, because there was nothing to check. Three stages out of seven returned unverifiable — not the check failed, but there is nothing here to check.
Stages now declare their artefact in the pipeline map, and the declarations are counted.
Why it matters. A declaration nobody tallies is a suggestion. That is exactly how you end up with nineteen agents declaring a tool that none of them calls.
7. The board's appearance is checked by a machine, not only by a person
It previously had exactly one check on its appearance: someone looking at it.
Now there are three in CI.
- Colour contrast. Every text token measured against every surface it can
land on, against the WCAG AA floor. Arithmetic over declared tokens — no browser, milliseconds.
- The rendered page against its own design system. A real browser opens the
page, and every font size and colour it paints must be one the stylesheet declares. This catches what source-level checks cannot see: the cascade, the media queries, the JavaScript.
- Colours that bypass the token system. Twenty-nine of them, invisible to the
contrast audit precisely because they were hardcoded.
The defect that bought these: a token used in 61 places and defined in none. Every use fell through to a light-theme fallback, and the densest block on the page rendered at 2.62:1 against a 4.5:1 floor. A person found it. Months later. By looking.
Why it matters. Accessibility guarded only by attention is accessibility until the first inattentive day.
8. A document goes stale on a date its author chose
Freshness was the file's modification time. Fixing a typo therefore rejuvenated a document that had stopped being true six months earlier — the documentation equivalent of shaving before a job interview.
Authors now declare when a document expires. Three verdicts — fresh, stale, unknown — and each names the rule that produced it.
Why it matters. "When was this file last touched" and "is this still true" are different questions. Answering the first while appearing to answer the second is the same defect as everything else on this list, wearing a cardigan.
9. The board starts itself and opens on one ranked queue
Six blocks of status sat above the single row an operator could act on. Four headed lists carried five rows between them, and the reason a row was there lived in the heading rather than the row. And the board did not start on its own; you had to remember it existed.
Now:
- Installing or updating brings it up. If it is already running on its port,
nothing is spawned on top of it.
- The Inbox opens on one queue, ordered by what needs you first, with the
reason on the row.
- The headline says whether the project is alive — it no longer wishes good
morning to a project that has not moved in eleven days.
- An empty queue says which kind of empty it is. Nothing to do and nothing
being read are different states. The old empty state congratulated you identically for finishing the work and for the machine having stopped.
Why it matters. A dashboard that shows everything shows nothing. And a blank screen that is pleased with you either way is misinformation with a smile.
10. Documentation knows what links to it, and screenshots know their own age
The Docs screen counted 188 documents where there are 156 — it was counting machine-written summaries and translated copies as separate documents. It also said nothing about how any document connected to the rest.
Each row now shows how many other documents cite it, and the list sorts by that. The record that eleven documents reference no longer looks identical to one nobody has ever linked.
Then the screenshots. A board shot from v2.73.1 sat in eleven READMEs while the board shipped 3.16.0 — two months and one redesign out of date, and nothing in the repository could say so. Now:
- Shots are taken against a seeded sample project, never the operator's own
board, which lists every project on the machine and prints paths containing a username.
- Every PNG carries its version inside the file. A sidecar manifest would
diverge from the image the first time someone moved it; a chunk inside the PNG cannot.
- Three states: current, stale, and unstamped. An image that was never
stamped is not an image that passed.
It fired on the first release after it was built, naming all six files and both versions. Being caught by your own trap is the cheapest possible confirmation that it is closed.
The through-line
If you take one thing from a fortnight of release notes, take this:
Every check needs three states, not two.
Not passed / failed, but passed / failed / not checked. Almost every defect above is the third state rendered as the first: an unverified stage as verified, an unmeasured cost as zero, an unreadable file as empty, an unstamped screenshot as current.
The second lesson is less obvious and more expensive: a declaration nobody counts is a wish. A tool declared by nineteen agents and invoked by none behaved precisely as if it did not exist, while costing nineteen agents' worth of context to keep saying so.
Install or update:
npx great-ctoFull detail in the repository's CHANGELOG.md — including the parts that were too specific to be interesting.