You Shouldn’t Have to Hold Your Breath Every Time Your Product Ships

Every change your developer pushes to your product is a bet that it didn’t break something else. You probably have no idea what odds you are getting on that bet, and asking feels embarrassing, because “do we have enough tests?” sounds like something you should already know.

You are not supposed to know. Nobody explained it to you, and the people who could have were busy shipping.

When I first opened up a venue-booking marketplace that a non-technical founder had built on Replit with AI coding agents, it had live customers, real bookings, and payouts moving through Stripe. It also had 32 automated tests, which for a product with money flowing through it is close to none, and there was no reason he would have known that.

So the short answer first, since that is what you came for.

The short answer

Your startup needs enough automated testing that two things are true.

First, every part of your product that touches money, customer data, or login gets checked by a machine every single time something changes, rather than by a person clicking around when they remember to. Second, when a check fails, you (the founder, not only your developer) can read what broke in plain language and understand what it means for your customers.

Neither of those is a percentage of “coverage” or a total count of tests. That marketplace had 32 tests and needed roughly 2,600 to be safe. On GoFixly (more on that below) 165 tests cover one component, and that is the right number for it. Those two conditions are what you are paying for. How many tests it takes to get there depends on what your product does with money and data.

Why this matters now

If you built your first version with Lovable, Base44, Replit, Bolt.new, v0, or Cursor, the product works and customers showed up, and the codebase came with zero tests, because AI builders are optimized to make something work in front of you. Proving it keeps working after the next change was never their job. You are the only person who has ever “tested” it, by clicking through the screens you happened to be thinking about that day.

That was the marketplace founder’s position when I met him. Real product, real bookings, and almost nothing checking it.

The same AI tools cut both ways, though. Claude Code and Cursor will write tests all day long, cheerfully, for a fraction of what it used to cost, so “there is no time for tests” has mostly stopped being true as an excuse. What replaced it is what those same tools do to a test they cannot pass, and I will spend a whole section on that below.

What automated tests actually do for you

Strip away the vocabulary and an automated test is a small program that uses your product the way a customer would and complains loudly if the answer changes. “When a venue owner connects their Stripe account, the payout goes to that venue owner and nobody else.” Write that down as code, run it every time anything changes, and you have a check that does not get bored.

Everything else is detail about where those checks sit. Your developer will use a few words for the kinds.

Unit tests check one small piece in isolation, which means “does the function that calculates the booking fee return the right number for a three-night stay.” On a Python codebase these usually run in pytest, on a JavaScript or TypeScript one, Vitest. They are fast and cheap and you will end up with hundreds.

End-to-end tests drive a real browser through your product the way a customer does: open the site, log in, search for a venue, book it, pay, see the confirmation. Playwright is the tool I reach for. They are slow (honestly, I don’t mind, the slow ones catch the failure a customer would actually notice) and you will have far fewer of them.

Architecture tests are the odd one out, and most startups don’t need them yet. On Mira, an exam-prep product I built as the sole engineer, the codebase is split into eleven separate areas, and there are tests whose only job is to fail if code in one area reaches into another where it shouldn’t. So the structure of the product is enforced by a machine instead of by hoping everyone remembers the rules.

One distinction, because founders mix these up. Tests run before a change reaches customers and tell you whether you broke something. Monitoring runs after and tells you whether customers are hitting a problem right now. You need both eventually. This article is about the first one, because it is the one your AI builder never gave you.

Isn’t 100% coverage the goal?

No, and the reason takes a minute, because coverage is the number your developer will quote if you ask “how tested are we,” and you will ask for a number. Everybody does.

Coverage is the percentage of your code that gets exercised by at least one test when the suite runs (the suite being the whole pile of tests, run together). It tells you which lines ran. It does not tell you whether anyone checked the answer. I can write a test that calls the payout function and never looks at what comes back, and coverage goes up. I can also have a product at 60% coverage where the 60% is the login flow, the checkout, and the permissions logic, and that product is safer than one at 95% where the tests are all on the parts that format dates.

The marketplace ended up around 98%. (Around. I am not going to pretend the last decimal matters.) I am telling you that number mostly so you know what it took to get there, which was asking one question over and over: what are the things in this product that must never break, and is there a test standing in front of every one of them? Coverage went up on its own.

If your developer says “we have good coverage,” ask “on what?” If the answer includes the words money, login, and customer data, you are in decent shape. If the answer is a number, ask again.

How much testing your startup actually needs, in practice

Think of it in three rings.

Ring one is non-negotiable: anything that touches money, customer data, or who is allowed to see what. Payments, payouts, refunds, and subscriptions; sign-up, login, and password reset; and every place where user A must not be able to see or change user B’s data. Every one of these needs a test before the next change ships, and for a marketplace that ring alone runs to a few hundred tests.

Ring two is the workflow that makes you money. For a venue marketplace, that is search, view, book, confirm. For a SaaS tool, it is whatever a customer does in the first five minutes that convinces them to keep paying. This is where end-to-end tests matter most, because the failure here looks like “the button still exists but the thing behind it quietly stopped working.”

Ring three is everything else, and everything else gets tested as it gets touched. When a developer or an agent changes a feature, the tests for that feature get written in the same change, rather than as a cleanup task that gets pushed to next week and then the week after.

On GoFixly, an Office add-in where I was the fractional CTO, the document-generation engine has 165 tests. It is the part that would embarrass us if it broke, so it got tested until I stopped thinking about it, and other parts got less, on purpose, because they would embarrass us less.

The signal you are looking for is whether your developer can ship a change on a Tuesday afternoon without anybody holding their breath. Counting tests will not tell you that. If releases still feel like an event, you don’t have enough tests in ring one.

What about the app you built with Lovable, Replit, or Cursor?

If your product was built by an AI agent, you have one advantage and one specific danger.

The advantage is speed. Point Claude Code at the booking flow and ask for tests, and you will have a few dozen inside the hour. That is genuinely new, and it is why a non-technical founder can get from 32 tests to 2,600 without hiring a QA team.

The danger is what the agent does when a test fails.

Ask an agent to fix a failing test and the shortest path to passing is very often to change the test. It loosens the check, or marks the test as skipped, or deletes it outright. The agent did exactly what you asked, which was make the failure go away, and nobody notices until the bug that test was guarding shows up in front of a customer. So I write one rule into the standards of every codebase I work on (four production codebases at the moment), in the file the agent reads before every change:

No test may be deleted, weakened, or skipped.

One sentence. It means the agent has to make the code pass the test, never the reverse, and when it cannot, it has to stop and tell you. That is the moment a human should be looking anyway. Of everything I set up on an AI-built product, this rule does the most work, and it is one line in a text file.

The second thing I did on the marketplace matters just as much for a founder who does not read code, and it is the part I would copy first if I were you. We wrote 74 testing standards in plain English, the kind of sentence you would put on a whiteboard rather than in a codebase, and every test in the suite is tagged to the standard it protects. The suite runs every night and reports per standard, so when something fails the founder does not get a wall of error text. He sees which standard failed, and he knows what that standard says because it is his list. There is a failure ledger too, which means every failure gets written down and has to be resolved rather than waved away.

He ships his own fixes now, with Claude Code, and he knows they are verified because the nightly run tells him so. He never became an engineer. He did not have to, because the report comes back in sentences he wrote himself.

What to ask your developer (or your agent)

You do not need to read a test to evaluate one. You need five questions and a feel for what a good answer sounds like.

“Which parts of the product have tests, and which don’t?” A good answer names areas: payments, auth, the booking flow. A bad answer is a percentage, or “most of it.”

“If someone changed the payout code tomorrow, what would catch a mistake?” A good answer is “these tests, and they run automatically before anything ships.” A bad answer involves the word “review” or the phrase “we’d notice.”

“When was the last time a test failed, and what happened next?” A good answer is a specific bug the test caught. If no test has ever failed, the tests are either brand new or checking nothing.

“What stops the AI from editing a test it can’t pass?” If your team uses Copilot, Cursor, or Claude Code, this is the one that matters most. A good answer is a written rule in the codebase. “We’d catch that in review” is a bad answer, because you can then ask how many agent-written changes were reviewed line by line last week.

“Can I see the results without asking you?” A nightly run that reports somewhere you can read it, in language you understand. If the only way you learn a test failed is by asking, ask instead for the nightly Slack message.

Is it worth the time and money?

The honest version of “we don’t have time for tests” is “we are choosing to find out about bugs from customers instead of from a machine.” For a weekend prototype nobody is paying for, fine. It stops being fine the day someone enters a credit card.

And the cost has collapsed. The 2,600 tests on that marketplace were written with Claude Code under written standards, with me reviewing the standards and the results rather than every line, and no human typed them out. The expensive part of testing was always the developer hours, and that is the part AI took away.

What you are buying with the time that remains is the ability to let a developer or an agent you don’t fully trust yet touch your product, because the tests will tell you if they got it wrong. If you don’t have a technical co-founder, this is how you hand off work you cannot personally check.

Anti-patterns to avoid

Chasing a coverage number. A team told to hit 80% will hit 80%, and the fastest way there is to test the easy parts. Ask where the tests are before you ask how many.

Testing only the happy path, which means the case where everything goes right. A test that checks “booking succeeds with a valid card” is fine. The test that matters is “booking fails cleanly with a declined card and the customer is never charged.” Most of the damage in production comes from the second kind, and an agent asked for tests will write the first kind unless you tell it to write the second.

Letting a failing test get deleted, weakened, or skipped. I am repeating it on purpose. A suite that has been quietly loosened over six months is more dangerous than no suite, because every check still shows as passing.

Treating “I clicked through it and it worked” as tested. That is a demo. A demo proves the product can work once, with you driving, on the path you were thinking about. A test runs the same check again after every change, whoever made it and whatever file they touched. A product that has only been tested the demo way can carry a problem for months, because nobody has clicked on that path yet, and that is the starting point for any product an AI builder wrote.

How to start, gently

You can do all of this this week without writing a line of code.

1. Write down, in plain English, the five things in your product that must never break. Money, login, and “user A can’t see user B’s data” will be three of them. This is the start of your own standards list. The marketplace founder’s list is 74 items long now, and nobody starts at 74.
2. Ask your developer or your agent for one automated test per item, and ask to see it fail once before it passes. A test that has never failed has never proven it checks anything.
3. Ask for the suite to run every night and report somewhere you can read it. A Slack message with a pass or fail per standard is plenty.
4. Paste this sentence into the file your AI tools read before every change (CLAUDE.md, .cursorrules, whatever your setup uses): “No test may be deleted, weakened, or skipped. If a test cannot be made to pass by fixing the code, stop and report it.”
5. Next month, add five more standards.

None of this requires you to understand the tests. It requires you to own the list of what they protect, and only you can write that list, because you are the one who knows which failure would cost you a customer.

If the part of your product that takes money broke tonight, who would find out first, you or a customer? That is the first test to write. And if you would rather talk through where your product actually stands before you ask your team, book a clarity call at https://lambdalynx.dev/schedule/ and we will go through it together.

#FractionalCTO #StartupTech #StartupFounder #SoftwareArchitecture