Your coding agent doesn't know what it broke

Short version: a coding agent is a local optimizer. It makes the change you asked for and has no standing memory of the hundred things that used to work. That's not a prompt problem or a model-size problem — it's structural. The only reliable fix is to put your app's invariants outside the agent, in checks that run without it.

Ask Claude Code or Cursor to "add a discount field to checkout" and it will. It reads the relevant files, edits them, and the field appears. What it does not do — cannot do, by construction — is ask "did I just break the guest-checkout redirect I wrote three sessions ago?" That behavior isn't in its context, isn't in its objective, and isn't anywhere it can see. The agent isn't careless. It's aimed at one target, and yesterday's working feature isn't it.

Why agents regress, structurally

Three things make an agent blind to its own regressions, and none of them get better with a smarter model:

1. Local optimization. The objective is "satisfy this prompt," not "satisfy this prompt without disturbing every prior invariant." Those are different functions, and the second one is not what you asked for or what it's scored on.
2. No durable memory of invariants. Whatever worked before this session lives outside the context window. The agent can re-read the code, but code tells it what the app does now — including the bug it just introduced — not what the app is supposed to do.
3. It grades its own homework. Ask the same agent to test the feature and it writes a test that passes against the code it just wrote. The test encodes the current behavior, bug and all. A green check from the author of the change is not evidence.

Put together: the entity making the change is the least qualified thing in your stack to certify that nothing else broke.

A bigger model doesn't fix a missing contract

The tempting response is "this improves as models get better." It doesn't, because the gap isn't capability — it's information. Even a perfect reasoner can't protect an invariant it was never told about and can't see. You can stuff more of the codebase into context, but the codebase is a description of the present, not a specification of what must remain true. The requirement — a guest who starts checkout keeps their cart — exists only in your head until you write it down somewhere the machine can check.

That's the actual fix, and it's boring: externalize the invariants. Write the behaviors that must survive as checks that live in your repo and run without the agent — so "still works" is decided by something that has no incentive to see green and no amnesia about last week.

What "external" has to mean

Not every check counts as external. Two properties matter:

• It runs without a model. If your regression gate calls an LLM at runtime, you've reintroduced the same non-determinism and the same "make it pass" pressure you were trying to escape. A gate should be plain, deterministic code — green means the same thing every time, and costs nothing to run.
• You own it, and it outlives the session. The contract can't be a transcript in a chat that's gone tomorrow. It has to be a durable artifact — for us, a plain @playwright/test spec in your git repo — that the next agent, and the one after that, has to pass.

This is the whole idea behind defining the behavior first and letting the AI fit the code to it: the spec is the fixed point, the code is the variable. The agent can rewrite the implementation as many times as it likes, as fast as it likes — which is the whole appeal of vibe-coding — precisely because something it doesn't control is holding the line on what must stay true.

The uncomfortable version

Vibe-coding didn't remove the need for tests. It made tests more load-bearing, by removing the human who used to remember what the code was for. When a person wrote the checkout flow, a person carried its invariants in their head and flinched when a change threatened them. Hand the writing to an agent and that memory goes with it — unless you wrote it down.

So the question for anyone shipping AI-written code isn't "is my agent good enough to not break things." It's: what, that isn't the agent, is allowed to tell you it broke? If the answer is "nothing," you're not moving fast. You're just finding out slowly.

FAQ

Isn't this just "write tests"? Yes — with two constraints most AI-testing setups drop: the tests must run without a model (deterministic, free, un-gameable) and you must own them (a durable file, not a cloud recording).

Can't I just prompt the agent to not break things? You can, and it will still break things, because the invariant it needs to protect isn't in its context. Prompting improves intent, not information.

Who writes the tests, then, if not the agent? An agent can absolutely author them from a real run — the key is that they then run in CI without any model in the loop, and that the requirement was declared before the code, not reverse-engineered from it.

Where does Hover fit? Hover is the regression-test suite for exactly this: declare the behavior, let an agent crystallize it into a plain spec you own, run it in CI with zero AI, and triage each failure as drift vs. bug. It's the external contract, made concrete.