How to test a vibe-coded app before you ship it

Short version: don't test everything — lock the handful of flows that would embarrass you if they broke (sign-up, login, checkout), turn each into a test you own that runs in CI with no AI, and let the next agent edit prove itself against them. The goal isn't coverage. It's a contract the AI can't quietly violate.

Vibe-coding is a genuinely new way to build: you describe what you want, an agent writes it, and you have a working screen in minutes. The catch shows up on the second feature. The agent optimizes for the prompt in front of it — "add a discount code field" — with no durable memory that checkout used to work. It ships the field and, somewhere along the way, breaks the thing that was fine yesterday. You don't notice, because you clicked through the new feature, not the old one.

So "testing a vibe-coded app" is really one question: which behaviors must survive the next hundred AI edits, and how do you make them un-break-able? Here's the setup that works.

1. Name the flows that must never break

Skip the urge to test everything — for a young app that's a maintenance tax you'll abandon in a week. List the three-to-five flows where a silent failure costs you a customer or your credibility:

• Can a new user sign up and land logged in?
• Can they do the one thing your app is for (post, upload, generate, book)?
• Can they pay?

That's your launch-critical set. Everything else can wait.

2. Declare the behavior, not the implementation

Before you touch selectors, write down what "working" means for each flow, in plain language: a guest who starts checkout is sent to login, and their cart survives the round-trip. This is the contract. It doesn't mention a button color or a CSS class, so it survives redesigns — it only fails when the behavior is actually wrong.

Writing the contract first is the whole trick. If you let the AI write the test after it writes the code, the test just re-states whatever the code happens to do — including the bug. The requirement has to come first, and the code gets fit to it.

3. Turn each contract into a test you own

Now make it executable. The fast path is to let a coding agent drive the real flow once and crystallize it into a plain @playwright/test spec — the kind of file you'd have written by hand, sitting in your own repo. With Hover, that's one MCP prompt:

/mcp__hover__guard    # record the contract: what must be true
/mcp__hover__test_app checkout   # the agent drives the flow, saves the spec

The important property isn't that an agent wrote it — it's record == replay: the agent acts through grounded tools (by role and name), so the selector that drove each click is the exact one saved. The spec is deterministic, semantic, and yours. Delete Hover tomorrow and it still runs.

You could also write these by hand. The point of this step is only that the output is a standard test file in your repo, not a recording trapped in someone's cloud.

4. Run it in CI on every change — with zero AI

This is where regressions actually get caught. Wire the specs into CI (npx playwright test on every pull request) so they run on every AI edit, not just when you remember. Crucially, no model runs at this stage: it's plain Playwright, so a green run is deterministic, costs no tokens, and means the same thing every time. The agent that wrote the feature doesn't get to grade its own homework.

Now the loop is closed. The agent adds the discount field; CI replays checkout; if the cart no longer survives login, the pull request goes red before it reaches your users.

5. When a spec drifts, heal it locally — don't auto-rewrite in CI

Real UIs change on purpose, so a spec will eventually fail because the behavior legitimately moved, not because there's a bug. The wrong fix is to let CI self-heal by calling a model mid-run — now your gate is nondeterministic and you're paying tokens to make red things green, which is exactly how real regressions get papered over.

The right fix is to triage: is this drift (the app changed, update the test) or a bug (the app broke, fix the app)? Hover Cloud makes that call on each failure and routes a heal hint back to your editor, where you fix it locally and review the diff like any other change. The contract only moves when you move it.

What not to do

• Don't rely on the agent to check its own work. It has no memory of what worked before this prompt (more on that in why your coding agent doesn't know what it broke).
• Don't lean on click-through QA. You test the feature you just built; the regression hides in the feature you didn't reopen.
• Don't chase 100% coverage on day one. Five un-break-able flows beat five hundred flaky ones.
• Don't put an AI in the CI critical path. Green should be free, fast, and deterministic.

FAQ

Do I need to know Playwright to do this? No. An agent authors the specs from the real flow; they land as standard Playwright files you can read and hand-edit later if you want.

Won't the tests break every time the UI changes? Rarely — they target roles and labels, not CSS. They fail when the behavior changes, which is exactly when you want to know.

Isn't running tests slow for a vibe-coding pace? The launch-critical set is small and runs in your normal CI in seconds to a couple of minutes — the point is it runs automatically, so speed of iteration stays high and the safety net is always on.

Does any of this send my code to a third party? No. The specs run in your own CI, and the authoring tools run locally. The behavior contract, not your source, is what leaves your machine.

Vibe-coding moved the cost of building down to almost nothing. The cost that's left is knowing it still works — and that's the one part you shouldn't hand back to the AI that built it.