QA for vibe-coded apps: what breaks in AI-written code

The demo is impressive. Someone described an internal expense tool in a paragraph, and four hours later there is a working app with login, a dashboard, file upload, and a settings page. It looks finished.

Then you log in as a viewer and open /admin/users directly in the address bar. The page loads. All 1,400 employee records are there.

That is not an unlucky bug. It is the most common bug in AI-written applications, and it is one of about six patterns that repeat across almost every project.

Short version

Vibe coding means building software mostly by describing it to a model and accepting the generated code.
The generated code usually handles the happy path well and the edges badly.
Permission checks are commonly in the UI only, not on the server.
Error handling is often missing, or present but silent.
Models invent functions and API parameters that do not exist. Some fail loudly, some quietly.
Test the server, not the screen. Most of these bugs are invisible from the interface.

Why the failure patterns are predictable

A model writes code by predicting what usually follows. Tutorials, documentation examples, and blog posts are the bulk of what it learned from. Those sources are written to explain one idea clearly, so they skip the parts that make software safe: the second permission check, the failure branch, the rate limit, the migration.

The result resembles a working application in the way a film set resembles a street. The front is convincing. There is nothing behind it. So you do not need a new testing discipline — you need to aim your existing one at the places where that front is thinnest.

Pattern 1: authorisation that only exists in the UI

This is the big one. The generated app hides the admin link from non-admins, and stops there.

What you will find:

The API route has an authentication check ("are you logged in?") but no authorisation check ("are you allowed to do this?")
The user's role is read from something the client sends, such as a request body field or a localStorage value
A record is fetched by ID with no check that it belongs to the requesting user
The "delete" endpoint checks permissions but "update" does not

How to test it in five minutes:

Log in as a low-privilege user, such as [email protected].
Open DevTools, go to the Network tab, and copy a request that an admin makes as a curl command.
Replace the admin's token with the viewer's token and send it.
If you get 200 OK with real data, you have found it.
Repeat for one resource belonging to a different account: change /api/invoices/1042 to /api/invoices/1043.

Step 5 catches what the industry calls insecure direct object reference: reading someone else's record just by changing a number in the URL. It appears in AI-generated CRUD code constantly.

Pattern 2: error handling that is missing or silent

Generated code assumes calls succeed. When they do not, you get one of two behaviours, and the second is worse.

Loud failure: the page goes blank and the console shows TypeError: Cannot read properties of undefined (reading 'map') because the fetch returned an error object instead of an array.

Silent failure: the code catches the error, logs nothing, and shows an empty list. The user believes they have no invoices.

Silent failure is the expensive one because nobody reports it. Things to check on every screen that loads or saves data:

What appears while loading? Often nothing, so a slow request looks like an empty page.
What appears when the request returns 500? Test it by blocking the endpoint in DevTools.
What appears when the request returns 200 with an empty array? Empty state is frequently missing entirely.
Is the error message the raw server response? Generated code often prints internal detail straight to the screen.
Does a failed save leave the form cleared, so the user loses what they typed?

A related habit worth naming: try { ... } catch (e) {} with an empty block. Search the codebase for it. Every instance is a bug waiting to be reported as "it just doesn't work sometimes".

Pattern 3: invented APIs and parameters

A model will confidently call stripe.charges.createWithRetry() or pass an option that a library never supported. Some of these crash immediately during development and get fixed. The dangerous ones are the plausible near-misses:

A real function called with an option name that does not exist, so the option is silently ignored. A timeout that never applies. A sanitize: true that does nothing.
A real function whose behaviour was misremembered, such as treating a returned promise as a value.
A configuration key spelled slightly wrong, so a security setting silently stays at its default.

These do not throw. They just quietly do less than the code claims.

How to test: take every third-party call in the changed files and check each named option against the current documentation. It is dull work and it finds real problems. Focus first on anything security-related: session options, CORS settings, cookie flags, password hashing parameters.

Pattern 4: stale patterns and versions

Models write what was common when their training data was collected. In fast-moving ecosystems that can be two or three years behind.

Symptoms:

A deprecated authentication method that still works but has known weaknesses
An old data-fetching approach mixed into a codebase that uses a newer one
Package versions old enough to have published vulnerabilities
Config files written for a previous major version of the framework

Run a dependency audit, then check the framework's current recommended patterns. If the app mixes two eras of the same framework, expect bugs at the seams: hydration mismatches, duplicate requests, and state that resets unexpectedly.

Pattern 5: data layer shortcuts

Generated database code favours the shortest thing that works in a demo.

What you find	What it causes
No unique constraint on email	Two accounts, same address, login picks one at random
No transaction around multi-step writes	Order created, payment row missing, after a mid-request error
Query inside a loop	Page takes 9 seconds once there are 300 rows
No pagination	Endpoint returns 40,000 records and the browser freezes
Cascade delete everywhere	Deleting a user silently removes their team's shared documents
Timestamps stored without timezone	Reports off by hours for anyone outside the server's timezone

Test these with volume, not with three rows. Seed a few thousand records before you judge any list screen.

Pattern 6: input handling that trusts the client

Generated forms validate in the browser, which is good for user experience and worthless for safety. Check that the same rules exist on the server by sending a request that skips the form entirely.

Send an order with quantity: -5. Send a profile update with a 50,000-character bio. Send a price field as a string. Send a field the form never shows, such as "role": "admin", and see whether it is saved.

That last one — mass assignment — is common in generated code, because writing "save all the fields from the request body" is shorter than listing the allowed ones.

A review checklist

Work through this on any AI-written feature before it ships.

Auth: every endpoint checked as a logged-out user, a wrong-role user, and a different-account user
IDs: at least three endpoints tested by changing the resource ID to another account's
Errors: every data screen tested with a blocked endpoint and with an empty response
Empty states: every list screen viewed with zero rows
Silent catches: codebase searched for empty catch blocks
Third-party calls: every option name checked against current documentation
Dependencies: audit run, outdated majors listed
Volume: list and report screens tested with a few thousand rows
Server validation: three requests sent that bypass the form entirely
Concurrency: the same action submitted twice quickly, such as a double-clicked payment
Secrets: repository searched for keys, and the client bundle checked for anything server-only
Migrations: feature tested against data created before it existed

Items 1, 2, and 9 find the most severe issues. If you only have an hour, do those.

When you file what you find, the reports need the network call, the exact response, and the console output to be convincing — "the viewer can load the admin page" gets argued with, while a captured 200 response containing 1,400 records does not. A browser-based tool like Crosscheck captures those from the page as you report, which is useful when you are moving quickly through a checklist like this one.

How to work with the team that built it

The person who generated the app is usually not being careless. They are moving at a speed where reviewing every line defeats the purpose, and the code looked right.

Two framings work better than "this code is bad". Show the request and response rather than the code: curl output from a viewer account reading admin data ends the discussion in one message. And ask for the checks to be generated too — "add a permission check to every route and a test that a viewer gets 403" works well, and the same tooling that created the problem can fix it in bulk.

The goal is not to slow the generation down. It is to point it at the parts the tutorials left out.

Frequently asked questions

Is AI-written code worse than human-written code? It fails differently. Human code tends to have inconsistent quality across a file; generated code tends to be uniformly polished on the happy path and uniformly thin at the edges. That predictability is an advantage for testers, because you know where to look first.

Do I need to read all the generated code? No. Read the auth middleware, the database queries, and any third-party integration. Test the rest from outside through the API. Reading 8,000 lines is not a good use of your day.

What is the single highest-value test? Call a privileged endpoint with an unprivileged token. It takes two minutes and finds the most severe class of bug in this kind of codebase.

Can I ask the model to review its own output? Yes, and it helps more than you would expect, especially with a specific prompt such as "list every endpoint in this file that lacks an authorisation check". It will still miss things it never knew about, so it supplements your review rather than replacing it.

Should the team stop generating code? That is rarely the right answer or a realistic one. The workable version is a short, non-negotiable checklist applied to anything that reaches production, plus tests for permissions and error paths written deliberately rather than generated.