Cybersecurity researchers have identified a new type of supply chain attack called ‘Slopsquatting,’ which exploits hallucinations in generative AI models used for code generation. A research team from the University of Texas at San Antonio, Virginia Tech, and the University of Oklahoma analyzed 16 code-generation models, including GPT-4, GPT-3.5, CodeLlama, DeepSeek, and Mistral, and found that approximately 20% of the recommended packages were non-existent.
The Slopsquatting Threat
The researchers noted that the reliance on centralized package repositories and open-source software in programming languages like Python and JavaScript, combined with AI-generated code, creates a new threat vector. Threat actors can register hallucinated package names and distribute malicious code, potentially leading to widespread compromise if a single widely-recommended hallucinated package is registered by an attacker.
Key Findings
- 19.7% (205,000) of recommended packages in test samples were found to be non-existent
- Open-source models like DeepSeek and WizardCoder hallucinated more frequently (21.7% on average) compared to commercial models (5.2%) like GPT-4
- CodeLlama was the worst offender, hallucinating over a third of its outputs
- GPT-4 Turbo was the best performer with only 3.59% hallucinations
Characteristics of Hallucinations
The study found that these hallucinations were persistent, repetitive, and semantically convincing. When researchers reran 500 prompts that had previously produced hallucinated packages, 43% of hallucinations reappeared consistently across 10 successive runs. The hallucinated package names showed moderate string similarity to real packages, making them more believable.
Protective Measures
To mitigate this threat, security experts recommend that developers implement dependency scanners before production and runtime to detect malicious packages. The study emphasizes that rushing through security testing increases the risk of AI model hallucinations, highlighting the need for rigorous validation of AI-generated code recommendations.
