Vettd case study · 2026

We scanned 1,036 vibe-coded apps.
Most look finished. The invisible parts decide what happens next.

We ran 1,036 live, AI-built web apps through Vettd's audit — 82 automated checks per app across eight categories (Security, SEO, AEO, Reliability, Performance, Legal, CRO, and Completeness). Nearly 85,000 individual pass / warn / fail verdicts. This is what we found, and the pattern that explains it.

The short version

The headline isn't "vibe coding is unsafe." It's more specific and more useful than that:

Vibe-coded apps reliably pass the things their platform sets up by default, and reliably fail the things a human has to know to ask for.

The work looks finished — it loads over HTTPS, it's mobile-responsive, it doesn't leak secrets, it isn't blocking search engines. But the invisible production layer — error tracking, security headers, legal pages, analytics, server-rendered content — is missing on the majority of apps, because nothing in the build loop ever prompted for it.

Across all 1,036 apps and 82 checks, here's the overall split:

The failures aren't evenly spread. They cluster — hard — in exactly the places you'd expect if no one was thinking past "does it work." And, as the per-platform section shows below, they cluster differently depending on which AI builder made the app.

What we scanned

1,036 apps. All live and deployed, scanned at their public URLs. Every verdict comes from what the deployed app actually serves to a browser and a bot — not a guess from the source code. Builder breakdown:

82 checks per app — 2 Critical, 27 High, 39 Medium, 14 Low. (A subset of AEO checks ran on fewer apps where pages weren't reachable.) The whole dataset is reproducible: drop any URL into the directory and run the same 82 checks against any app you doubt.

The good: what vibe coding genuinely gets right

This is the part that doesn't get written about enough. On the fundamentals that used to trip up beginner developers, AI-built apps are now excellent — because the modern tool-and-host stack handles them automatically. The two Critical checks pass 99.8% of the time. The High-severity "table stakes" tier is nearly spotless:

The "AI ships dangerously broken garbage" narrative is mostly outdated. Secrets aren't leaking in bundles at scale. HTTPS is universal. Apps don't ship with placeholder lorem ipsum or framework boilerplate. The floor is much higher than it was even two years ago, and that's a real, underappreciated win for AI-assisted building.

The bad: what vibe coding quietly skips

The other half. Several High-severity checks fail on the majority of apps — three of them on the overwhelming majority:

Group these failures and three blind spots emerge clearly.

1. No operational instrumentation

96% of apps have no error tracking. 83% have no analytics. When something breaks for a real user, the builder has no idea — there's no Sentry, no logging, no funnel data. The app works the day it ships and goes dark the moment it doesn't.

2. Missing security headers (beyond what the host auto-adds)

This is the cleanest illustration of the whole pattern. Two headers, near-mirror-image results:

• HSTS: 93.9% pass. Hosts like Vercel and Netlify add this automatically. ✅
• Clickjack / X-Frame-Options: 93.9% fail. This one you have to configure yourself. ❌

Same category, opposite outcomes — the only difference is whether the platform set it up or the developer had to. X-Content-Type-Options sits at exactly 50.6% fail — and as the next section shows, which half an app lands in is decided almost entirely by which builder made it.

3. No legal pages

Roughly two-thirds of apps have no Terms of Service (71%) and no Privacy Policy (69%). Many are taking emails, payments, or sign-ups — which in most jurisdictions makes a privacy policy a legal requirement, not a nicety.

There's a fourth, quieter cost: discoverability. 80% of apps render their content only after JavaScript runs, and ~78% ship no structured data. To a search crawler or an AI assistant trying to read and cite the page, a lot of these apps are effectively blank. In an AI-search world, that's traffic the app will never know it didn't get.

Per-platform fingerprints — same failures, different patterns

Here's where it gets interesting. The collective numbers above make the failures look uniform — as if every vibe-coded app were broken the same way. Split the same data by builder and a sharper truth appears: an app's scorecard is mostly a fingerprint of its platform's defaults. Where a builder scaffolds something, nearly all its apps pass. Where it doesn't, nearly all fail. The "average app" doesn't exist — there are six distinct profiles hiding inside it.

Hard-fail rates by builder (lower is better):

Five anomalies tell the whole story.

Bolt apps tell every AI assistant to leave

Bolt apps block AI crawlers — GPTBot, ClaudeBot, PerplexityBot, and the rest — 100% of the time (186 of 186). Replit blocks them ~20% of the time; everyone else is at or near zero. This isn't a missing header or a forgotten config — it's the platform's default robots.txt opting every app it ships out of AI search. If you launched on Bolt, your product is invisible to ChatGPT, Claude, Perplexity, and Google's AI Overviews on day one — not because no one has indexed you yet, but because you told them not to.

v0 server-renders, so it wins the one check almost everyone else fails

Only 32% of v0 apps fail "content readable without JavaScript," versus 98% for Bolt and 92% for Lovable — because v0 builds on Next.js, which renders content on the server by default. But that same architectural choice cuts the other way: v0 is the worst platform for the metadata it doesn't scaffold — sitemaps (94% fail), structured data (96%), social images (94%), even favicons (62%). v0 hands you a page a crawler can read and then gives it almost nothing to index.

Base44 scaffolds metadata, so its apps quietly win SEO

Base44 posts the lowest fail rates anywhere for sitemaps (1.1%), social preview images (1.1%), favicons (1.1%), and structured data (44% — roughly half the rate of its peers). It clearly auto-generates SEO and social metadata. Yet it's simultaneously the worst platform for legal pages (privacy 93%, terms 92%) and still fails content-without-JS 77% of the time. A platform optimizes what its makers thought to optimize, and nothing else.

Security headers are a coin flip decided by the platform, not the developer

X-Content-Type-Options fails ~98% of the time on v0, Bolt, and Replit — and essentially 0% on Lovable (0.4%) and Base44 (0%). Same one-line header, opposite outcome, purely because some builders inject it and some don't. And note the limit of even the "good header" platforms: Lovable ships X-Content-Type-Options and HSTS, but still fails clickjack protection 100% of the time. Setting some headers is not the same as setting the right set.

The universal floor is exactly the stuff no platform scaffolds

Error tracking (88–100% fail), analytics (52–100%), and legal pages (35–93%) fail on every platform, because no builder sets them up for you. These aren't platform quirks — they're the root cause of the collective pattern. The differences between builders are about defaults; the similarities are about the decisions every builder leaves to you.

One note on "Other," the 281-app unattributed bucket: this is whatever the URL-pattern backfill couldn't classify — mostly apps that have moved off the builder's default subdomain to a custom domain. It almost certainly contains Lovable, Bolt, and Replit apps the pattern matcher missed, so its numbers should be read as a noisy view of those same platforms, not a distinct population.

The practical upshot: knowing the platform tells you which holes to expect before you even scan. A v0 app almost certainly needs metadata and a sitemap; a Bolt or Replit app almost certainly needs security headers and server-rendered content; a Base44 app needs legal pages. The fix list is short and the same categories recur — but the priority order is set by the tool you built on.

The middle: the "looks fine, isn't quite" layer

Beyond hard fails, a large band of checks sits at warn almost everywhere — borderline debt that won't show up in a fail count but is real:

• Content Security Policy: 98.8% warn — almost no app has a meaningful CSP.
• Permissions Policy: 97.1% warn.
• Author / Organization schema: 93.8% warn.
• llms.txt for AI agents: 85.5% warn.
• Custom 404 page: 78.6% warn.
• Canonical URL: 78.5% warn.
• Meta descriptions: 50.7% warn.
• Dead links: 49.9% warn.

None of these will take an app down. All of them are the difference between "a prototype someone vibe-coded" and "a product." They accumulate silently because nothing in the build loop flags them.

Why this pattern happens

The split is not random, and it's not about AI writing "bad code." It maps perfectly onto a single line:

Defaults get done. Decisions get skipped.

If a thing is handled by the framework or the host — TLS, indexability, responsive meta tags, HSTS, not leaking secrets into the bundle — the AI-built app gets it for free, every time. That's the 65% pass rate. And as the per-platform data shows, the list of "free" things is set by your builder: v0 gives you server rendering for free; Base44 gives you metadata for free; Lovable gives you two security headers for free.

If a thing requires someone to know it exists and ask for it — "add Sentry," "set an X-Frame-Options header," "write a privacy policy," "server-render this so crawlers can read it" — it doesn't happen, because the person describing the app in natural language doesn't know to describe those, and the AI won't volunteer infrastructure no one requested. Those are the checks that fail on every platform.

Vibe coding compresses the build. It doesn't compress the judgment about what a production app needs. The gap between those two is exactly what this scan measures.

What it means if you ship with AI

You don't need to stop vibe coding. You need a pre-ship pass for the handful of things the tools won't do for you. In priority order, weighted by how often it's missing and how much it matters:

1. Add error tracking and analytics. (Missing on ~96% / ~83%, on every platform.) You cannot run what you cannot see.
2. Set your security headers — X-Frame-Options, X-Content-Type-Options, and a real CSP. (Failing or warning on 51–99%.) These are configuration, not code — and your builder may have done half of them already.
3. Add a Terms of Service and Privacy Policy if you collect any user data. (Missing on ~two-thirds.) This is a legal exposure, not a polish item.
4. Make your content readable without JavaScript and add basic structured data. (Failing on ~78–79%, unless you're on v0.) This is your visibility to Google and to AI assistants.
5. Sweep the warnings — canonical URLs, custom 404, meta descriptions, dead links — before you call it done.

The encouraging read of this data: every one of these is cheap and well-understood. None requires rewriting the app. The problem was never that the work is hard — it's that nobody was prompted to do it.

Methodology & caveats

• Sample: 1,036 live AI-generated web apps. Builder breakdown: Lovable (249), Bolt (186), Replit (157), Base44 (94), v0 (69), Other (281). Apps were scanned as deployed at their public URLs. A subset of AEO checks ran on fewer apps where pages weren't reachable.
• Checks: 82 automated checks across 8 categories, each returning pass / warn / fail and a severity (Critical / High / Medium / Low).
• What this is: an external, black-box view of what each app serves to browsers and bots. It is strong evidence for missing headers, pages, instrumentation, and rendering behavior.
• What this isn't: a source-code or auth-logic audit. Issues like broken access control or missing row-level security on a database don't fully surface in a surface scan, so the true "needs work before launch" rate is likely higher than the failures here, not lower.
• "Other" is a mixed bucket of apps not attributable to a single named builder; read its numbers as a heterogeneous group, not a single platform.
• One verdict ≠ one severity-weighted risk. A 5% failure on a Critical check matters more than a 50% warn on a Low one; read the tables with severity in mind.
• Verify any of this yourself. Every scan in this dataset is reproducible. Drop a URL into the directory and run the same 82 checks against any app you doubt.

Scan data: Vettd · 1,036 apps · 2026. Every figure in this document is computed directly from the raw check results.