---
title: 'LLM05: Improper Output Handling'
description: OWASP LLM Top 10 reference for improper output handling vulnerabilities including injection via LLM-generated content
---

# 05 Improper Output Handling

Identifier: LLM05:2025
Category: Output Safety

## Description

Improper Output Handling refers to insufficient validation, sanitization, and handling of
the outputs generated by large language models before they are passed downstream to other
components and systems. Since LLM-generated content can be controlled by prompt input, this
behavior is similar to providing users indirect access to additional functionality.

Improper Output Handling differs from Overreliance in that it deals with LLM-generated
outputs before they are passed downstream whereas Overreliance focuses on broader concerns
around overdependence on the accuracy and appropriateness of LLM outputs.

Successful exploitation can result in XSS and CSRF in web browsers, SSRF, privilege
escalation, or remote code execution on backend systems.

The following conditions increase the impact of this vulnerability: granting the LLM
privileges beyond what is intended for end users, vulnerability to indirect prompt injection
attacks, third-party extensions that do not adequately validate inputs, lack of proper output
encoding for different contexts, insufficient monitoring and logging of LLM outputs, and
absence of rate limiting or anomaly detection for LLM usage.

## Risk

* Cross-site scripting (XSS) and CSRF in web browsers from unescaped LLM-generated content.
* Server-side request forgery (SSRF) through unvalidated output used in server requests.
* Privilege escalation through LLM capabilities exceeding intended end user access.
* Remote code execution on backend systems via output passed to system shells or eval
  functions.
* SQL injection through unparameterized LLM-generated queries executed against databases.
* Path traversal through unsanitized file path construction from LLM output.
* Phishing attacks through unescaped LLM-generated content used in email templates.
* Insecure code generation leading to vulnerabilities when developers integrate unverified
  LLM code suggestions.

## Vulnerability checklist

* LLM output is entered directly into system shells or functions such as exec or eval
  without validation.
* JavaScript or Markdown generated by the LLM is returned to users and interpreted by the
  browser without encoding.
* LLM-generated SQL queries are executed without proper parameterization.
* LLM output is used to construct file paths without proper sanitization.
* LLM-generated content is used in email templates without proper escaping.
* The application grants the LLM privileges beyond what is intended for end users.
* Third-party extensions do not adequately validate LLM-generated inputs.
* No proper output encoding is applied for different downstream contexts such as HTML,
  JavaScript, or SQL.
* Monitoring and logging of LLM outputs is insufficient to detect exploitation attempts.
* No Content Security Policy is in place to mitigate XSS from LLM-generated content.

## Prevention controls

1. Treat the model as any other user, adopting a zero-trust approach, and apply proper input
   validation on responses coming from the model to backend functions.
2. Follow the OWASP ASVS (Application Security Verification Standard) guidelines to ensure
   effective input validation and sanitization.
3. Encode model output back to users to mitigate undesired code execution by JavaScript or
   Markdown. OWASP ASVS provides detailed guidance on output encoding.
4. Implement context-aware output encoding based on where the LLM output will be used, such
   as HTML encoding for web content and SQL escaping for database queries.
5. Use parameterized queries or prepared statements for all database operations involving
   LLM output.
6. Employ strict Content Security Policies (CSP) to mitigate the risk of XSS attacks from
   LLM-generated content.
7. Implement robust logging and monitoring systems to detect unusual patterns in LLM outputs
   that might indicate exploitation attempts.

## Example attack scenarios

### Scenario A — Extension shutdown via unvalidated output

An application utilizes an LLM extension to generate responses for a chatbot feature. The
extension also offers administrative functions accessible to another privileged LLM. The
general purpose LLM directly passes its response without proper output validation to the
extension, causing the extension to shut down for maintenance.

### Scenario B — Data exfiltration via summarizer

A user utilizes a website summarizer tool powered by an LLM to generate a concise summary
of an article. The website includes a prompt injection instructing the LLM to capture
sensitive content from either the website or from the user's conversation. The LLM encodes
the sensitive data and sends it without any output validation or filtering to an
attacker-controlled server.

### Scenario C — SQL execution via chat

An LLM allows users to craft SQL queries for a backend database through a chat-like feature.
A user requests a query to delete all database tables. If the crafted query from the LLM is
not scrutinized, all database tables are deleted.

### Scenario D — XSS via unsanitized content

A web app uses an LLM to generate content from user text prompts without output sanitization.
An attacker submits a crafted prompt causing the LLM to return an unsanitized JavaScript
payload, leading to XSS when rendered on a victim's browser.

### Scenario E — Malicious email template injection

An LLM is used to generate dynamic email templates for a marketing campaign. An attacker
manipulates the LLM to include malicious JavaScript within the email content. If the
application does not properly sanitize the LLM output, this leads to XSS attacks on
recipients who view the email in vulnerable email clients.

### Scenario F — Insecure code generation

An LLM is used to generate code from natural language inputs in a software company. The
approach risks exposing sensitive information, creating insecure data handling methods, or
introducing vulnerabilities like SQL injection. The AI may also hallucinate non-existent
software packages, potentially leading developers to download malware-infected resources.

## Detection guidance

* Review code for direct use of LLM output in system commands, SQL queries, or file path
  construction.
* Test for XSS by examining LLM-generated content rendered in browsers for unescaped
  JavaScript or Markdown.
* Inspect Content Security Policies for adequacy against LLM-generated content injection.
* Monitor logging systems for unusual patterns in LLM outputs indicating exploitation
  attempts.
* Check all database operations involving LLM output for parameterized query usage.
* Audit third-party extensions for adequate input validation of LLM-generated content.

## Remediation

* Replace all direct use of LLM output in system commands with properly validated and
  sanitized alternatives.
* Implement context-aware output encoding for all LLM output based on the downstream
  consumer.
* Replace all unparameterized queries involving LLM output with parameterized queries or
  prepared statements.
* Deploy strict Content Security Policies to mitigate XSS from LLM-generated content.
* Add robust logging and monitoring to detect and respond to exploitation attempts in
  LLM outputs.
* Conduct thorough code review and verification of all LLM-suggested packages and code
  before integration.

---

*🤖 Crafted with precision by ✨Copilot following brilliant human instruction, then carefully refined by our team of discerning human reviewers.*
