The Model Is the Malware: Runtime Behavioral Detection of Malicious ML Artifacts

Abstract

Agentic AI systems and automated ML pipelines increasingly rely on third-party model artifacts from public hubs, loading them through framework APIs such as torch.load() and AutoModel.from_pretrained(). These systems may select, evaluate, fine-tune, or deploy models without human review before execution. Attackers embed payloads in these artifacts that trigger during deserialization or inference, spawning processes, opening network connections, or exfiltrating host data. Platform scanners use static denylists and miss attacks that avoid denylisted functions, exploit unsupported loading paths, or abuse legitimate framework APIs. To detect such malware, we present ModelShield, the first runtime behavioral detection system that attributes system-level behavior to specific model operations. ModelShield combines application-level instrumentation of framework APIs with eBPF-based syscall tracing, preserving attribution across threads and child processes to separate model-triggered activity from environment noise. Attributed syscall sequences are classified using 193 behavior patterns mapped to MITRE ATT&CK techniques. We evaluated ModelShield on 145,000 models from Hugging Face, ModelScope, and Kaggle across 17 ML frameworks and 36 formats. On Hugging Face, it detected 41 malicious models, including a live reverse shell downloaded 108 times without platform-scanner detection before disclosure, and a payload performing host reconnaissance and outbound exfiltration; all detections were manually confirmed. It detected all 29 evasion variants spanning function substitution, loading-path obfuscation, and TensorFlow API abuse, outperforming static scanners. Analysis completes in a median of 12.3 seconds per model, making ModelShield practical for hub-scale screening. This poster demonstrates that detecting malicious models requires observing what they do, not just inspecting what they contain.