In Veracode's July 2026 benchmark, 44% of AI code-generation tasks failed at least one covered security check, and Python did best of the tested languages at 63% passing. That is a result on controlled tasks, not a rate for code that reached production, and it is still a good reason to check generated code before merge.

Veracode released its 2026 GenAI Code Security Report in late July 2026 (press release dated 28 July). Its headline: the average security pass rate across more than 100 models it has tracked is 56%, essentially unchanged from the previous year, even as syntax correctness keeps improving. The leading model in its latest test, GPT-5.5, passed 68%. By language, coverage of the report lists Python highest at 63% and Java lowest at 30% (SD Times, eSecurity Planet).

Those numbers justify verification. They don't justify turning benchmark samples into claims about deployed code.

What Veracode actually tested

Veracode's Spring 2026 methodology update describes the harness:

  • 80 coding tasks;
  • Java, JavaScript, C# and Python;
  • four vulnerability classes: SQL injection (CWE-89), cross-site scripting (CWE-80), log injection (CWE-117) and insecure cryptographic algorithms (CWE-327);
  • prompts without security-specific guidance; and
  • Veracode SAST checks on the generated code.

The tests run against raw models, not agents with tools, guardrails or a human reviewer in the loop. They don't measure whole repositories, deployed applications, secrets, authorization design, cloud permissions or every vulnerability class. The failure rate also depends on the prompts, model versions, rules and scoring.

So a generation that fails one covered check is a failed benchmark task. It isn't evidence that a team accepted, merged, deployed or exposed the code.

ResultScopeCorrect interpretation
56% pass / 44% failAverage across Veracode's benchmark program, July 2026 reportA benchmark average across models and languages, not a production rate
63% pass for Python (Java 30%)Per-language results reported with the July 2026 reportThe best of the four tested languages on these tasks; not a universal Python rate
68% pass for the leading modelLatest test in the July 2026 reportStill limited to the tested tasks and vulnerability classes

What the benchmark says about Python security

The tested classes map to real Python review concerns.

SQL injection

Python web and data code creates SQL injection when untrusted input reaches a query built from strings. A data-flow-aware rule can trace supported paths from request input to database execution and tell parameter binding from string building. It still needs framework coverage: a clean result for one ORM says little about a custom query layer the scanner doesn't understand.

Cross-site scripting and log injection

Both depend on context. A value can be safe in one output position and dangerous in another. Scanners trace supported sources into template, response and logging sinks and check whether the right escaping happens on the way. Simple text matching is weak here.

Insecure cryptographic algorithms

Weak hashes and ciphers are usually visible from the API name, such as MD5 or SHA-1 in security-sensitive code. The limit is intent: a weak hash used as a checksum is different from one used for passwords.

Package hallucination is a separate measurement

The USENIX Security 2025 paper We Have a Package for You! (Spracklen et al.) studied a different risk with a different denominator.

The researchers generated 576,000 Python and JavaScript code samples with 16 models. Of 2.23 million package references, 440,445 (19.7%) were hallucinated: 15.8% of Python references and 21.3% of JavaScript ones.

That doesn't mean 15.8% of AI-generated Python files import a fake package. It is hallucinated package references divided by all package references, under the paper's prompts, models and extraction method.

The practical control: treat every newly suggested package as its own review decision. Confirm it exists and is the intended project, add it through the normal lockfile workflow, and check it for known vulnerabilities. A vulnerability database knows about flaws in recognized packages; it can't tell you a newly registered package is legitimate (how to detect hallucinated packages, slopsquatting explained).

What static analysis can and can't tell you

RiskA suitable check can detectIt can't establish by itself
SQL injectionSupported input-to-query flows and unsafe query constructionSafety inside an unknown query builder or runtime-generated SQL
XSS and log injectionSupported flows into response, template and log sinksCorrect escaping in every custom rendering or logging layer
Weak cryptographyKnown weak algorithms, modes and insecure randomness APIsWhether every cryptographic design choice fits the threat model
New dependenciesPackages that don't exist, undeclared imports, known vulnerabilitiesPackage legitimacy, maintainer identity or unpublished malware
Removed security checksA named route guard, validator or decorator that disappeared in a diffThe complete authorization policy of an arbitrary application
Dead codeUnused code under the scanner's language and framework modelWhether apparently unused code is activated at runtime

A clean scan is evidence about the rules and code paths that ran. It isn't a certificate that the code is secure.

AI review and static analysis do different jobs

An AI reviewer can explain a complex change, notice inconsistencies and suggest tests. Its output can vary with model, context and prompt. A rule-based check gives the same result for the same code, configuration and scanner version. Keep repeatable checks in the required merge gate, use model review to widen the investigation, and test the security behaviour neither can prove from source.

The research doesn't compare AI-written and human-written code under the same tasks and scoring. It supports checking generated code carefully; it doesn't give a multiplier for how much less secure it is.

A practical Python merge gate

  1. Scan the changed code. Run deterministic rules on the diff with enough repository context to understand sources, sinks, routes and imports.
  2. Check concrete security classes. Injection paths, unsafe cryptography, secrets, dangerous configuration, and changes to known authorization or validation code.
  3. Review new dependencies separately. Verify identity and provenance, pin through a lockfile, run vulnerability checks.
  4. Run security-focused tests. Authorization boundaries, invalid input, output encoding, logging, failure paths.
  5. Keep a human decision at the boundary. Review what the tools covered, what they skipped, and whether the change keeps the application's security requirements.

With Skylos, steps 1 to 3 look like this:

skylos . --diff-base origin/main --danger --quality --ai-defects --format json
skylos . --sca --format json

The first reports security, quality and AI-defect findings in the files changed since origin/main, including made-up or undeclared packages (SKY-D222, SKY-D223) and removed security decorators such as @login_required (SKY-L021). In CI, check out with fetch-depth: 0 so origin/main exists. The second adds dependency vulnerability checks. Neither fails a build on its own: use --strict or a configured --gate, and see the removed-controls job for failing on SKY-L021, which the default gate doesn't. If an AI agent wrote the change, also run skylos done --base origin/main to check it didn't delete or skip tests (how).

The evidence-based conclusion

AI-generated Python should pass the same security checks as any other code before merge, and plausible output still needs verification.

Veracode's 56% average shows that functional-looking benchmark answers often fail its covered security checks, and Python's 63% is better than other languages but far from clean. The USENIX study shows that suggested package names create a separate supply-chain problem.

Neither measures how much vulnerable AI code reaches production, and neither has a matched human baseline. The practical response doesn't need either claim: run repeatable static checks, validate dependencies, test security behaviour and review what's left before merge.

The free Skylos CLI runs these checks locally and in CI. If several people or agents open pull requests, Skylos Cloud keeps the results in one place: the Free plan adds a pass/fail check on pull requests and scan history for one project, and the Workspace plan adds up to 10 projects, approvals for findings you decide not to fix, pull request comments, and audit exports. For a builder's checklist, see Vibe coding security checklist for Python apps; for reviewers, the AI code review security PR checklist.

Sources