OWASP Top 10 LLM Vulnerabilities

LLM01: Prompt Injection

What is Prompt Injection? 

When attackers manipulate the operations of a trusted large language model by utilizing crafted input prompts, either directly or indirectly, via multiple channels, then prompt injection vulnerabilities occur. This manipulation of LLM goes unidentified because of inherent trust in its output.

The results of these types of LLM vulnerabilities may include exposing their sensitive information and unauthorized actions without alerting the user security system. 

Solution for Prompt Injection

1. Limiting the privileges of an LLM to the minimum use for its functionality. 
2. Enhancing input validation by implementing methods to restrict potential unauthorized prompt inputs from unknown sources. 
3. Building trustworthy relationships between the LLM, external source, and functionality. 

LLM02: Insecure Output Handling

What is Insecure Output Handling? 

An insecure output handling happens when an application accepts LLM output without scrutiny, allowing it to reach directly to its backend systems. As Large Language Model-generated content can be operated by prompt input, this action is similar to providing users indirect access to additional functionality. In some scenarios, it results in XSS, CSRF, SSRF, and remote code execution on backed systems. 

Solution for Insecure Output Handling

1. Treating the model output like any other untrusted user content and applying rightful input validation on responses.
2. Encoding output from the model back to users to mitigate undesired code interpretations. 

LLM03: Training Data Poisoning

What is Training Data Poisoning? 

Training data poisoning happens when the manipulation of data is generated to introduce vulnerabilities and backdoors within the Large Language Model (LLM)

This could illegibly compromise the ML model’s security, effects, and ethical behavior by exposing users to false results or information.

Solution for Training Data Poisoning

1. Verifying the supply chain of the training data and legitimacy of data, both external and internal. 
2. Assuring that enough sandboxing is available to prevent the model from scraping unintended data sources.
3. Run LLM vulnerabilities scan into the test phases of the LLM’s lifecycle.

LLM04: Denial of Service

What is Denial of Service?

Denial of Service happens when attackers interact with an LLM so strategically that it can result in heavy resource consumption, causing service degradation or a hike in price. There is an increased use of LLMs for various applications and utilization of intensive resources, making the Denial of Service more critical.

Solution for Denial of Service

1. Limit the total number of actions in a system reacting to LLM responses. 
2. Limiting the number of queued actions.
3. Cap resource use per request.

LLM05: Supply Chain

What is Supply Chain?

The supply chain in large language models can be vulnerable, impacting the integrity of training data, machine learning models, and deployment platforms. All this can lead to influenced outcomes, system failure, and security breaches. Generally, supply chain vulnerabilities are concentrated on third-party software components. However, they are now extended to artificial intelligence. (AI)

Solution for Supply Chain

1. Code signing
2. Auditing
3. To detect extraction queries, implement adversarial robustness training.
4. Vetting of sources and suppliers

LLM06: Permission Issues

What are Permission Issues?

When authorization is not tracked down between the plugins and treats each LLM content, it understands that it’s being utterly created by a user and starts performing any command without requiring authorization. However, this might lead to privilege escalation, loss of confidentiality, and dependency on available plugins.

Solution for Permission Issues

1. Prevent sensitive plugins from being called after any other plugins.
2. Enable manual authorization of any action taken by sensitive plugins. 
3. Not calling more than one plugin with each user input.

LLM07: Data Leakage

What is Data Leakage?

Data Leakage or sensitive information disclosure occurs when LLMs unintentionally disclose confidential data and proprietary algorithms through their responses. It causes privacy invasion and security breaches. That’s why LLM application users need to know how to interact and identify potential risks with an LLM.

Solution for Data Leakage

1. Integrate adequate data sanitization and scrubbing techniques. 
2. Maintain ongoing supply chain mitigation of risk through techniques such as SAST and SBOM.
3. Perform LLM vulnerability scanning.
4. Implementation of robust input validation and sanitization methods.

LLM08: Excessive Agency

What is an Excessive Agency?

Excessive Agency vulnerability, as the name suggests, is caused by excessive authorizations and functionality. Without any restriction barrier, any unwanted operation of LLM will result in undesirable actions.

Solution for Excessive Agency

1. Reduction in the permissions to only the minimum necessary for the LLMs.
2. Implement human approval for every major action.
3. Implement rate-limiting to minimize the number of unnecessary actions. 

LLM09: Overreliance

What is Overreliance?

Overreliance is when a system depends on LLMs for decision-making and content generation without proper oversight and validation mechanism. LLMs are capable of creating informative content, but they might have factual errors, leading to misinformation and misguidance.

Solution for Overreliance

1. Frequent monitoring and reviewing of the outputs of the LLM. 
2. Implement automatic validation mechanisms.
3. Verification of the information generated by LLMs before using it in the decision-making process.

LLM10: Insecure Plugins

What are Insecure Plugins?

Plugins are designed to connect Large Language Models to an external resource, allowing the use of unstructured text as input instead of structured and validated inputs. This allows potential cyber attackers to create a malicious request to the plugin. That could lead to several unwanted outcomes. 

Solution for Insecure Plugins

1. The plugin should be aimed at a least privileged perspective, exposing little functionality. 
2. Whenever possible, plugin calls should be strictly parameterized.