AI-Generated Code Security: The Risks of Vibe Coding

"Vibe coding" means building software mainly by prompting AI assistants such as GitHub Copilot, Cursor or Claude Code, instead of writing every line by hand. It speeds up delivery dramatically, but it hands the team a new problem: much of that code lands unreviewed, reproduces insecure patterns, and drags in dependencies that sometimes do not even exist. For any organization that handles sensitive data, understanding these risks is no longer optional. It is part of quality control and of your attack surface.

This article explains the real security risks of AI-generated code and how to govern them without killing productivity. We cover why models produce insecure code by default, what slopsquatting is and how hallucinated dependencies become a supply-chain attack, secret leakage, prompt injection in coding agents, and licensing concerns. Then we get concrete about controls: SAST, SCA, secret scanning in CI/CD, human review, pinning with an allowlist, SBOM, and least privilege for agents. The stance is not anti-AI. It is "trust but verify".

Key takeaways

• Models learn from public code, including vulnerable code, so they reproduce insecure patterns by default.
• Hallucinated dependencies (packages the AI invents) enable slopsquatting, a cheap supply-chain attack to set up.
• The volume of unreviewed code overwhelms human review and lets bugs slip through that used to be caught.
• Hardcoded keys and leaked secrets become more frequent when code is generated at high speed.
• Coding agents with repository access widen the surface for prompt injection and poisoned context.
• Effective control treats AI code as untrusted: automated scanners, mandatory review, and pinned, verified dependencies.

What vibe coding is and why it changes the risk model

The term describes a way of working where the developer describes what they want in natural language and the assistant generates the code, which is often accepted with little or no critical reading. In personal projects it is a legitimate way to prototype fast. The problem appears when that same flow reaches production code that handles authentication, payments, personal data or infrastructure.

The underlying shift is one of responsibility. In traditional development, whoever writes a line ideally understands what it does. In vibe coding, the person who commits may not fully understand the code they just accepted. That gap between authorship and comprehension is exactly where security flaws slip in, because a review only catches what the reviewer understands.

This is not an argument against AI. AI assistance delivers real productivity gains and is here to stay. It is an argument for treating its output with the same skepticism we would apply to code copied from a public forum: useful as a starting point, untrusted as a finished product.

Risk 1: insecure by default

Language models are trained on vast amounts of public code. That corpus includes both good practices and a considerable amount of insecure, outdated or outright vulnerable code. A model optimizes to produce what is statistically likely, not what is secure, so it tends to reproduce the patterns it has seen most, and many of those patterns are bad.

Various studies on AI code generation find that a significant portion of responses contains exploitable vulnerabilities. The patterns that recur most are recognizable to anyone who has done pentesting:

• Injection: SQL queries built by concatenation, system commands with unsanitized input, unescaped templates.
• Weak cryptography: outdated algorithms, insecure modes, non-cryptographic randomness for tokens or passwords.
• Missing authorization: endpoints that check authentication but not authorization, so any valid user can reach another user's resources.
• Poor validation: trusting client input, no size limits, insecure deserialization.

The aggravating factor is subtle: generated code usually looks correct. It compiles, passes the happy-path tests and does what the prompt asked. The vulnerability is not in what it does, but in what it fails to do (it does not check, does not escape, does not limit). That makes it especially hard to catch in a shallow review, which is precisely the kind of review that vibe coding encourages. This is why it helps to lean on the risk catalog in the OWASP LLM Top 10 when integrating models into the development flow.

Risk 2: hallucinated dependencies and slopsquatting

This is the most specific and, in many ways, the most dangerous risk. When a model does not know the exact name of a library to solve a task, it sometimes makes one up. It generates an import or an install command pointing to a package with a plausible name that does not exist in the registry (npm, PyPI, etc.).

The exploitation chain is direct and cheap:

1. The model consistently suggests a nonexistent package, for instance a name that "sounds" right for the desired function.
2. An attacker observes those recurring hallucinations (names tend to repeat because the model reproduces recurring patterns from its training data, not because its output is deterministic) and registers that name with a malicious package.
3. The next developer who receives the same suggestion and runs the install blindly downloads and executes the attacker's code.

This pattern has come to be called slopsquatting, by analogy with typosquatting, but exploiting the "slop" that models generate instead of human typos. The key difference is one of scale and trust: typosquatting depends on the victim mistyping; slopsquatting depends on the victim trusting the AI, which is far more systematic.

The damage from a malicious package during the build phase can be total: theft of CI environment credentials, exfiltration of source code, backdoors planted in the final artifact. It is, in essence, a software supply-chain attack with a new entry vector. The central defense is never to install a dependency without verifying that it exists, who maintains it, and why it is needed.

Risk 3: secret leakage and hardcoded keys

When code is generated at high speed, secrets slip in easily. The assistant proposes an example with a "sample" API key inside the code, the developer swaps it for the real one to test quickly, and that value ends up in a commit. It also happens in reverse: when pasting context into the prompt so the AI "understands" the project, it is easy to include real credentials in the conversation.

The result is hardcoded keys, tokens and connection strings in the repository, sometimes in git history that no one remembers. Secret scanning in the pipeline is the minimum safety net, and it must cover both the current working tree and history.

Risk 4: overtrust and unreviewed volume

There is a human risk that amplifies all the others. When code comes "for free" and appears to work, the temptation to accept it without reading it is enormous. Two effects combine:

• Automation bias: we tend to trust the output of an automated system more than a colleague's, even without reason to. An AI-generated snippet is perceived as more "neutral" and reviewed with less rigor.
• Review saturation: AI generates code far faster than a human reviews it. If the pace of generation overwhelms review capacity, review stops being a real control and becomes a formality. Giant pull requests are the classic symptom.

The net effect is that volume erodes the quality of human control exactly when that control is most needed, because the code was not written by a person who understood it.

Risk 5: prompt injection and poisoned context in agents

The more advanced assistants no longer just complete lines: they are agents that read files, run commands, browse documentation and apply changes to the repository. That multiplies their usefulness and also their attack surface.

Prompt injection happens when an attacker plants malicious instructions in content the agent will read as part of its context: a GitHub issue, a code comment, a dependency file, an external documentation page. The agent does not reliably distinguish between "data I should process" and "instructions I should obey", so it can end up executing the attacker's order (for example, exfiltrating an environment variable or introducing a subtle backdoor).

Poisoned context is the variant where the attacker contaminates the sources the agent learns from or relies on during the task. These vectors map to those described in the OWASP LLM Top 10 and are especially relevant when the agent has write permissions or access to secrets. If the assistant is part of a corporate suite, it is also worth reviewing the implications of Microsoft Copilot 365 security for business.

Risk 6: licensing and intellectual property

A risk that is not technical security but is legal exposure: generated code can reproduce fragments with restrictive licenses present in the training data, or introduce doubts about the ownership of the resulting code. In regulated sectors or in mergers-and-acquisitions due diligence, unclear code provenance can become a real problem. It is worth having an explicit policy on this, though its handling goes beyond the technical scope of this article.

Controls: trust but verify

The answer is not to ban AI. It is to treat its code exactly as we would treat code from an unverified external contributor: useful, welcome, and subject to control. The following controls reinforce each other.

Automated scanners in CI/CD

No AI code should reach the main branch without passing a battery of automated analysis:

• SAST (static analysis) to detect insecure patterns in the code itself: injection, weak cryptography, missing authorization.
• SCA (software composition analysis) to check dependencies against known vulnerabilities and problematic licenses.
• Secret scanning to block hardcoded keys and tokens before they merge, covering both the current tree and history.

These controls must be pipeline gates, not reports no one reads. A solid foundation of CI/CD pipeline hardening is what turns these scanners into a real control rather than noise.

Mandatory human review

No amount of automation replaces a human who understands the code before accepting it. The practical rule is simple: if you do not understand it, you do not approve it. This means keeping changes reviewable in size (huge PRs generated in one shot are an anti-pattern) and paying special attention to authentication logic, authorization, handling of sensitive data and calls to external systems.

Verify and pin every dependency

Against slopsquatting, the defense is concrete:

• Verify that each dependency actually exists and that whoever claims to maintain it really does, before installing it. Do not trust that a name "sounds right".
• Pin versions with hashes or lockfiles, so a name cannot resolve to a package other than the validated one.
• Use an allowlist of approved dependencies when the context allows, instead of permitting arbitrary installs.

Complement this with an SBOM (software bill of materials) to improve visibility into what actually enters the artifact, to the extent the inventory is complete and accurate, making it faster to respond when a vulnerability appears in a dependency.

Guardrails and least privilege for agents

When the assistant is an agent that can act, permissions matter as much as the code. Apply least privilege: read-only access by default, short-lived credentials, no direct access to production secrets, and human approval for destructive actions. Limit the sources the agent can read and treat all external content as potentially hostile, to reduce the prompt-injection surface.

SBOM and training

Two cross-cutting controls close the set. The SBOM provides supply-chain traceability and answers the question of "what is actually in our software". And training changes the culture: teams must understand that AI code is untrusted by default, recognize the vulnerable patterns, and know why pinning and review are not bureaucracy but defense.

Frequently asked questions

Is AI-generated code always insecure?

Not always, but more often than it seems. Models reproduce patterns from their training, which includes vulnerable code, so a significant portion of responses contains exploitable flaws. The problem is that this code usually looks correct and passes the happy-path tests, which makes the vulnerability hard to catch in a shallow review. The takeaway is not to avoid AI, but to never trust its output without verifying it.

What exactly is slopsquatting?

It is a supply-chain attack that exploits models' hallucinated dependencies. When an AI recurrently invents the name of a package that does not exist, an attacker registers that name with a malicious package. The developer who receives the same suggestion and installs it blindly runs the attacker's code. It is analogous to typosquatting, but it exploits trust in the AI rather than human typos.

How do I avoid installing a hallucinated dependency?

Always verify that the dependency really exists, who maintains it, and why you need it, before installing. Pin versions with lockfiles or hashes so a name cannot resolve to an unexpected package, use an allowlist of approved dependencies when possible, and rely on SCA and an SBOM for visibility into everything that enters the build.

Is a SAST scanner enough to secure AI code?

No. SAST detects many insecure patterns in the code, but it does not cover hallucinated dependencies (which are addressed by verifying package provenance and with an allowlist and pinning, not primarily by SCA), hardcoded secrets (secret scanning), or authorization-logic flaws that require business context (that is human review). Effective defense combines several controls in the pipeline and does not rely on a single tool.

Are coding agents more dangerous than autocomplete?

Yes, because they widen the attack surface. An agent that reads files, runs commands and applies changes can be manipulated through prompt injection: malicious instructions hidden in an issue, a comment or external documentation that the agent processes as part of its context. That is why agents need least privilege, short-lived credentials, and human approval for sensitive actions.

Does this approach slow down the productivity AI brings?

Not if you automate it well. Scanners in CI/CD, pinning and the allowlist work mostly transparently in the pipeline, though they too carry costs of integration, maintenance, and false-positive triage. The cost that cannot be automated away is human review, which is precisely the control AI cannot replace. The goal is "trust but verify": harness AI's speed without inheriting its flaws.

Related resources

• CI/CD pipeline hardening
• SBOM and software supply-chain security
• Software supply-chain attacks and DevSecOps
• OWASP LLM Top 10 explained
• Microsoft Copilot 365 security for business

Verify before you trust your AI code

AI-generated code is an extraordinary help and, at the same time, a new attack surface. Trust but verify means placing real controls between the prompt and production: scanners in the pipeline, human review, and verified, pinned dependencies.

At Secra we are an offensive security company with our own vulnerability research program. We have discovered and published real CVEs, such as CVE-2025-40652 in CoverManager and CVE-2023-3512 in Setelsa ConacWin CB, coordinated through NVD and INCIBE-CERT. We apply that offensive mindset to audit your SDLC, review the code and dependencies your team generates with AI, and help you set up the right guardrails. If you want an assessment of your supply chain and your AI-assisted development process, get in touch with us.