← All posts

Blog

What the weekly verify caught this time

Every Monday the four Core 4 repositories are verified from fresh clones, three ways each. Direct, through the Codex CLI, and through the Muse CLI. Twelve runs. This week all twelve passed, and the interesting findings were not in the products.

Twelve runs, four products

Gearbox passed its 26 tests. Context Firewall passed 45 of 45 with conformance over 33 fixtures. Affected Verification passed 126 with 1 documented skip, determinism 12 of 12, conformance 15. Visible Value passed 7 of 7. The agent-driven runs matched the direct runs count for count, which is the point of running them. A verify that only works when a human drives it proves less than it claims.

The warnings were in the tester

Every Codex log carried a deprecation warning. The Codex CLI had upgraded from 0.163.0 to 0.169.0, and a config setting it used to accept is now deprecated. Every context-mode plugin hook also reported failure after the plugin auto-updated that morning. The same hook script exits clean standalone. The products were fine. The harness around them had drifted.

This is the failure shape worth remembering. Nothing failed, so nothing looked broken. The warnings sat in logs that a passing run gives nobody a reason to open.

Fix the source, not the symptom

The standing rule here is that any error or warning gets fixed at its source, and no impact is not a dismissal. The deprecated setting was replaced in the tester config with its current equivalent. The failing hooks were disabled at the same layer, since the plugin update needs a trust review before its hooks run again. A probe run after the fix showed zero warnings and zero hook failures.

The same pattern showed up again the same day in a different layer. The research repository has two self-hosted runners. Both services were stopped, and a pull request sat queued for two days with no error anywhere. Starting the services fixed the queue. No log complained, because a stopped runner produces no log.

What a verify cannot prove

A verify run proves the documented commands pass from a fresh clone. It does not prove the fleet around the products is alive. Runner hooks, runner services, and tester configs all sit outside the verify boundary, and each one failed quietly at some point this month while every product check stayed green.

The deploy layer told the same story last week. Deploy jobs were refused for weeks because a hook compared against a GitHub organization that no longer exists. The fix was one word in a hook file. The products never changed.

The limits, stated plainly

Passing counts are evidence about the products, not about the whole system. The weekly run now has company. The autopilot checks runner state and deploy health alongside the products, because that is where the last three failures actually lived.

Only show intelligence evidence that requires intelligence.

Context Firewall is one of the four Opsle theories. The theory page states the principle, and the repository holds the reducer, the protocol, and the limitations.

Read the theory