Quick answer: The OWASP Machine Learning Security Top 10 is a reference that lists the ten most critical security risks to machine learning systems. It differs from the OWASP LLM Top 10, which focuses on LLM applications: this list covers attacks that target the model itself, the training data and the ML pipeline. It ranks ML specific risks, from input manipulation and data poisoning to model theft and model drift, with defensive recommendations.

Why is the OWASP ML Top 10 a separate list?

OWASP has two different AI security lists that get confused. The LLM Top 10 covers risks specific to large language model applications like ChatGPT (prompt injection, insecure output). The ML Security Top 10 targets broader machine learning systems, that is, classifiers, prediction models and their training pipeline.

The difference matters because attacks against an image classifier or a fraud detection model differ from attacks against an LLM chatbot. An organization should reference the list appropriate to the type of AI it uses.

Prominent ML risks

Risk Description
Input manipulation (evasion) Misleading the model by altering the input at inference
Data poisoning Corrupting the model by injecting malicious samples into training
Model inversion Recovering training data from outputs
Membership inference Determining whether a data point was in the training set
Model theft Copying the model through API queries
Supply chain attack A flaw via a pretrained model or library

How are these risks reduced?

Defense spans every phase of the ML lifecycle. The source and integrity of training data should be verified, rate limiting and access control should be applied to the model API, input validation should catch anomalous inputs, and model behavior should be continuously monitored in production. In addition, pretrained models and ML libraries should be used with signature and provenance verification against supply chain risk.

The KAOS and DSET approach

DSET references the OWASP ML Top 10 in the security review of machine learning systems and applies the correct list based on the type of AI the organization uses. Our KAOS security engine evaluates risks such as model theft and input manipulation on ML APIs in an evidence based way. Our goal is not a generic checklist but showing the real attack path specific to that model.

Related reading: prompt injection and OWASP LLM Top 10, AI data poisoning, AI model security and model theft, adversarial ML and evasion attacks, MLSecOps machine learning pipeline security, MITRE ATLAS AI threat matrix.

Frequently asked questions

What is the difference between the OWASP ML Top 10 and the LLM Top 10? The LLM Top 10 covers risks specific to large language model applications (chatbots, assistants), such as prompt injection and insecure output. The ML Security Top 10 covers attacks targeting general machine learning systems such as classifiers and prediction models, and the training pipeline. An organization should reference the list appropriate to the type of AI it uses.

What is a membership inference attack? Membership inference is when an attacker tries to determine whether a specific data point was used in the model's training set. It is a privacy attack: for example, revealing that a specific patient's data was in the training of a health model is a privacy violation. For defense, model outputs should not leak excessive confidence information.

Does using a pretrained model carry risk? Yes. Pretrained models and ML libraries downloaded from the internet carry supply chain risk: they may have a backdoor embedded or be poisoned. That is why models and dependencies should be obtained from a trusted source, with signature and provenance verification where possible.

Sources

For an OWASP ML Top 10 based security review of your machine learning systems, you can talk to our team at Ankara Hacettepe Technopark. Contact DSET.