Your AI coding assistant wrote this line:

from huggingface_cli import login

It looks fine. It looks like something that should exist. You run pip install huggingface-cli, the install succeeds, your tests pass, and you merge.

In March 2024, that exact package was a proof-of-concept attack by Bar Lanyado at Lasso Security. He'd noticed GPT-based assistants repeatedly recommending huggingface-cli to developers — a package that didn't exist on PyPI. He registered an empty placeholder package under that name and waited.

Three months later, it had been downloaded over 30,000 times. An Alibaba research repository was among the adopters — it recommended the install in its README. (Lasso Security, March 28 2024)

This is slopsquatting: the class of software supply chain attack where an attacker registers a package name that AI coding assistants repeatedly hallucinate, then waits for devs to pip install it into production.

Who named it, and why it's its own category

The term was coined by Seth Larson, the Python Software Foundation's Security Developer-in-Residence. "Slop" is the common pejorative for low-quality generative-AI output; "squatting" comes from typosquatting, the long-standing attack where malicious actors register names one keystroke away from real packages (reqeusts, numpi, djnago).

The distinction matters:

TyposquattingSlopsquatting
Attacker needsA real, popular package with typo-prone spellingAn LLM-hallucinated name
Who "types" the bad nameHuman developerAI assistant
Catch pointSpellcheckers, eye-catching diffsAlmost nothing — the name looks plausible
RepeatabilityRelies on human errorRelies on model determinism

Typosquatting has existed for decades. Slopsquatting is new because its delivery channel — the LLM — is new, and because LLMs are consistent enough that attackers can pre-compute which hallucinated names are worth registering.

The data: Spracklen et al., USENIX Security 2025

The foundational empirical study is "We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs" by Joseph Spracklen, Raveen Wijewickrama, A H M Nazmus Sakib, Anindya Maiti, Bimal Viswanath, and Murtuza Jadliwala. It was accepted to USENIX Security 2025.

The numbers, directly from the paper's USENIX/arXiv abstract:

  • 16 LLMs tested, spanning commercial and open-source models
  • 576,000 Python and JavaScript code samples generated
  • 205,474 unique hallucinated package names observed across those samples
  • At least 5.2% hallucination rate across commercial models (the paper's stated floor)
  • 21.7% hallucination rate across open-source models

That's the headline. But the more interesting question is what happens when the same prompt is run more than once.

Why recurrence is the load-bearing fact

If hallucinated names were random — if every generation produced a fresh nonexistent package — slopsquatting wouldn't be economically viable. An attacker would have to register tens of thousands of variants and hope some unlucky dev's LLM happens to emit one on some given day.

The Spracklen study dismantled that defense. When the same prompt was run ten times through the same model, the researchers observed a sharply split recurrence pattern:

  • 43% of hallucinated package names appeared in every single run
  • 39% never reappeared at all
  • 58% were repeated more than once

(Spracklen et al., arXiv v3 full text; Socket.dev, April 8 2025, summarizing the same finding)

Almost half the hallucinations are stable. The model invents the same fake package every time you ask. That's all an attacker needs — run a popular prompt 100 times, take the top 10 hallucinated names, register them, and let the users come to you.

Which models hallucinate most

The Spracklen paper breaks per-model performance down in Appendix G. For this Python-focused article, the clearest comparison is the paper's package-level Python total, not secondary "output" shorthand:

Model or cohortMetricHallucination rate
GPT-4 TurboPython package-level total3.59%
Commercial modelsAverage stated in the abstractat least 5.2%
Open-source modelsAverage stated in the abstract21.7%
CodeLlama 7BPython package-level total26.12%
CodeLlama 34BPython package-level total21.15%

CodeLlama matters here because Llama-family code models have historically shipped in local-first coding assistants and self-hosted pair programmers. In the Python results, CodeLlama 7B's 26.12% package hallucination rate is about 7.3x GPT-4 Turbo's 3.59% rate. The broader commercial/open-source averages show the same direction of travel, but do not turn this into a universal multiplier for every model, language, or prompt set.

One caveat worth stating up front: the Spracklen lineup reflects the models available for the 2024/2025 study. GPT-4 Turbo, CodeLlama, WizardCoder, DeepSeek-Coder, Mistral, and friends — not GPT-4o, Claude 3.5/4, Llama 3.x, or Qwen-Coder 2.5. Treat the numbers above as a published benchmark baseline, not a live leaderboard for current frontier models.

An interesting footnote from the Spracklen paper: only 133 of 12,871 deleted PyPI package names, or 0.17%, appeared in the hallucination set. The vast majority of hallucinations are pure invention — names the model constructed from learned patterns, not faint memories of deleted packages.

Why Python is a particularly exposed target

Three structural reasons.

1. PyPI's namespace is flat

Unlike npm's @org/package scoped packages, PyPI uses globally unique project names under PEP 541. Subject to PyPI's policies, an unregistered hallucinated name can therefore be registered without an organization-owned namespace such as @huggingface/cli reserving it for one publisher.

2. Python is common in AI development

Python is widely used in machine learning, data science, LLMOps, and agent frameworks. Libraries such as langchain, llama-index, transformers, autogen, and crewai can also change module layouts between releases. That combination makes it especially important to verify generated package names and import paths against the version a project actually uses.

3. Installing a package is one command

Python makes adding a dependency easy: pip install X. That convenience shortens the path from a plausible generated import to installing a project with the same name. Review the project page, publisher, release history, source repository, and declared dependency change before adding it.

What a hallucinated import actually looks like

There is more than one import failure mode, and current Skylos keeps them separate:

Pure hallucination. The package simply doesn't exist anywhere:

from cryptoutils import secure_hash        # no such package
from flask_permissions import require      # no such package

Stale module path. The package exists, but the model remembers an older layout:

from langchain.chat_models import ChatAnthropic
# Pre-0.1 location. LangChain 0.1 (January 2024) split integrations out
# into separate partner packages — the modern import is:
#     from langchain_anthropic import ChatAnthropic
# The old top-level shim is deprecated and fails outright on fresh installs
# that don't carry the legacy compatibility layer.

Alias confusion or an undeclared dependency. The package exists on PyPI but its import root differs from its distribution name, or the dependency is absent from the repository manifest:

import sklearn
# sklearn is the import name; the distribution on PyPI is scikit-learn.
# If your requirements.txt declares neither, this import fails at runtime
# no matter how obvious the name looks. The same trap catches cv2/opencv-python
# and yaml/PyYAML — three of the most misremembered import/distribution splits
# in Python.

Current SKY-D222 covers the first case when package-registry evidence supports that the top-level import package does not exist. SKY-D223 reports a recognized third-party package that is undeclared. Neither rule validates an internal submodule path such as langchain.chat_models; use dependency-aware tests or import checking for that case.

Catching it at lint time, not install time

Existing tooling tends to be install-time:

  • pip-audit (PyPA) checks installed packages against known vulnerabilities. Useless against a hallucinated name, because the name isn't in any advisory database — there's no CVE for "this package doesn't exist."
  • Lockfiles (uv.lock, poetry.lock, hash-pinned requirements.txt) pin what you've already installed. If a dev ran pip install cryptoutils to make the AI-generated import work, the lockfile now enshrines that decision.
  • Trusted publishing / Sigstore (PyPI docs) guarantees provenance for packages you know you want. It can't tell you that cryptoutils shouldn't be on your want-list to begin with.

Every one of these layers runs too late. By the time lockfile hashing kicks in, the slopsquatted package is already resolved as a legitimate dependency.

The cheap detection layer is static. Parse every import X and from X import Y in the diff. Resolve against the declared dependency graph — requirements.txt, pyproject.toml, uv.lock, whatever you use. If an import has no matching distribution, fail the PR.

Skylos reports evidence-backed dependency hallucination candidates as SKY-D222:

pip install skylos
skylos . --select SKY-D222

--select SKY-D222 enables the AI-defect family and reports only dependency-hallucination results backed by registry evidence. Use --select SKY-D223 when the question is whether a real third-party package is missing from the repository's declared dependencies.

A workflow you can drop in today

A minimal GitHub Action that blocks a PR when hallucinated imports are present:

name: skylos scan
on: [pull_request]

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install skylos
      - run: skylos . --select SKY-D222 --strict

--select SKY-D222 enables and reports the dependency-hallucination rule, while --strict makes any matching finding block the job. If you prefer a shared policy, set max_ai_defects = 0 under [tool.skylos.gate] and use --gate. Use skylos defend . separately when the repository builds an LLM application or agent and you want to verify its runtime guardrails.

Pair it with a lockfile — uv.lock, poetry.lock, or pip-tools-generated requirements.txt --generate-hashes — so that any pip install a dev might run to "fix" the failing import also has to pass code review. The lockfile catches the second-order supply chain risk; the static scan catches the first-order hallucination.

Bottom line

  • LLMs hallucinate Python imports. Commercial models hallucinate package names at about a 5% floor; open-source models average above 20%.
  • Roughly 43% of those hallucinations recur on every re-run of the same prompt. That determinism is what makes pre-computing attack targets profitable. (Spracklen et al.)
  • The attack is not hypothetical. Bar Lanyado demonstrated 30,000+ downloads of a single hallucinated package name in three months, including an Alibaba research repo recommending the install in its README.
  • PyPI's globally unique project names under PEP 541 leave unregistered hallucinated names available for registration, subject to PyPI policy.
  • Lockfiles and pip-audit catch known vulnerable packages after install. They do not catch nonexistent names at lint time. Static import resolution against your declared dependencies is the cheap layer that does.

Run it on a repo you care about:

pip install skylos
skylos . -a

If Skylos flags an unresolved import in an AI-generated diff, nothing was lost — you caught the exact class of bug that makes slopsquatting possible.

(Disclosure: we build Skylos. The Spracklen, Lasso, and Socket findings cited above are independent third-party research.)

Sources