MCP Security Top 10 - Part 5: Unvalidated Tool Responses

This is the fifth article in our series about the top 10 security risks associated with the Model Context Protocol (MCP). This post focuses on Unvalidated Tool Responses, which occurs when AI systems trust and act upon potentially manipulated or malicious data returned by MCP tools without proper verification.

Introduction

When AI systems leverage MCP to interact with external tools and data sources, they typically trust that the responses they receive are accurate and untampered. However, this trust creates a significant security vulnerability. If an MCP tool returns compromised, manipulated, or malicious data, an AI system might incorporate this data into its reasoning and actions without validation, potentially leading to security breaches, incorrect decisions, or harmful outcomes.

MCP Security Top 10 Series

This article is part of a comprehensive series examining the top 10 security risks when using MCP with AI agents:

0. MCP Security Top 10 Series: Introduction & Index
1. MCP Overview
2. Over-Privileged Access
3. Prompt Injection Attacks
4. Malicious MCP Servers
5. Unvalidated Tool Responses (this article)
6. Command Injection
7. Resource Exhaustion
8. Cross-Context Data Leakage
9. MITM Attacks
10. Social Engineering
11. Overreliance on AI

What Are Unvalidated Tool Responses?

Unvalidated tool responses occur when:

1. An AI system requests data or functionality from an MCP tool
2. The tool returns a response that is malicious, manipulated, or incorrect
3. The AI system accepts and acts upon this response without proper validation
4. The compromised data influences the AI's reasoning or actions

Attack Vectors

1. Tool Response Manipulation

External actors might compromise tool responses through:

• Man-in-the-middle attacks intercepting and altering responses
• Compromised or malicious MCP servers
• Exploitation of vulnerabilities in tools or their backends
• Direct manipulation of data sources that tools access

2. False Information Injection

Attackers can use manipulated responses to:

• Feed false information to the AI to influence its conclusions
• Insert harmful instructions that might be executed by the AI
• Return malformed data that causes unexpected behavior
• Bypass security controls by returning false validation results

Real-World Example

Consider the following vulnerable pattern in an AI application using MCP:

// AI system code handling the MCP tool response
async function handleUserQuery(userQuery) {
  if (userQuery.includes('financial data')) {
    // Get financial data from MCP tool
    const financialData = await mcpClient.invokeTool('get_financial_data', {
      report: 'quarterly',
      year: '2025',
      quarter: 'Q1'
    });
    
    // VULNERABLE: No validation of the returned data
    // The AI system blindly trusts that the data is correct
    
    return `Based on the financial data, our revenue for Q1 2025 was 
      ${financialData.revenue} million, showing a 
      ${financialData.growth}% growth compared to last year.`;
  }
  
  // Handle other queries...
}

In this example, the AI system blindly trusts the financial data returned by the MCP tool. If the tool or its data source is compromised, attackers could manipulate the financial figures, potentially causing the AI to report false information that might impact business decisions or mislead users.

Detection Methods

1. Data Validation Checks

Implement validation for tool responses:

• Schema validation to ensure responses match expected formats
• Range checks for numerical values
• Type checking for all fields
• Consistency checks across multiple data points
• Pattern matching for expected structures

2. Response Integrity Verification

Verify the integrity of tool responses:

• Implement digital signatures for critical data
• Use checksums to detect tampering
• Compare responses with trusted sources when possible
• Verify timestamps and sequence numbers
• Check for unexpected or anomalous response patterns

3. Anomaly Detection

Monitor for unusual patterns in tool responses:

• Track historical response patterns to establish baselines
• Detect statistical anomalies in data
• Look for unexpected changes in response format or structure
• Monitor for suspicious timing patterns or response sizes
• Implement monitoring for unusual output from the AI after processing tool responses

Mitigation Strategies

1. Implement Response Validation

Add validation logic for all tool responses:

// IMPROVED: With response validation
import { z } from 'zod'; // Schema validation library

// Define an expected schema for the response
const FinancialDataSchema = z.object({
  revenue: z.number().positive().lt(10000), // Must be positive and reasonable
  growth: z.number().gt(-100).lt(1000), // Must be within reasonable range
  reportDate: z.string().regex(/^\d{4}-\d{2}-\d{2}$/), // YYYY-MM-DD format
  source: z.string(),
  // Additional fields...
});

async function handleUserQuery(userQuery) {
  if (userQuery.includes('financial data')) {
    // Get financial data from MCP tool
    const rawFinancialData = await mcpClient.invokeTool('get_financial_data', {
      report: 'quarterly',
      year: '2025',
      quarter: 'Q1'
    });

    // Validate the response against our schema
    try {
      const validatedData = FinancialDataSchema.parse(rawFinancialData);

      // Additional custom validation logic
      if (new Date(validatedData.reportDate) > new Date()) {
        throw new Error('Report date is in the future');
      }

      // Cross-check with another source for sensitive data
      if (Math.abs(validatedData.revenue - getExpectedRevenue()) > 100) {
        throw new Error('Revenue differs significantly from expected value');
      }

      return `Based on the validated financial data, our revenue for Q1 2025 was
        ${validatedData.revenue} million, showing a
        ${validatedData.growth}% growth compared to last year.`;
    } catch (error) {
      // Handle validation failure
      logSecurityEvent(`Financial data validation failed: ${error.message}`);
      return "I'm unable to provide accurate financial information at this time due to data validation issues.";
    }
  }

  // Handle other queries...
}

2. Implement Cryptographic Verification

For critical data, use cryptographic methods to ensure integrity:

import { createVerify } from 'crypto';

async function verifySignedToolResponse(response, publicKey) {
  // Extract data and signature
  const { data, signature } = response;

  // Verify the signature
  const verify = createVerify('SHA256');
  verify.update(JSON.stringify(data));
  const isValid = verify.verify(publicKey, signature, 'base64');

  if (!isValid) {
    throw new Error('Response signature verification failed');
  }

  return data;
}

3. Use Multiple Sources for Critical Data

Cross-validate important information with multiple sources:

• Compare tool responses with data from different providers
• Implement a voting system for critical decisions
• Use cached or historical data as a reference point
• Cross-check with user input when appropriate
• Maintain a trusted database of expected values for comparison

4. Implement Response Processing Guardrails

Add guardrails to how tool responses are processed:

• Set strict limits on what actions can be taken based on tool responses
• Require additional confirmation for high-impact actions
• Implement timeouts for response processing
• Sanitize tool responses before incorporating them into AI reasoning
• Add clear error handling for validation failures

Design Patterns for Secure Tool Response Handling

When developing AI systems that use MCP tools, consider these design patterns:

1. Decorator Pattern: Wrap tool invocation with validation logic
2. Chain of Responsibility: Create a pipeline of validators for different aspects of the response
3. Strategy Pattern: Apply different validation strategies based on the data type or context
4. Facade Pattern: Create a simplified interface that handles validation behind the scenes

Conclusion

Unvalidated tool responses represent a significant security risk in MCP implementations. By implementing proper validation, cryptographic verification, cross-checking with multiple sources, and adding processing guardrails, you can significantly reduce the risk of AI systems being compromised through manipulated tool responses.

Remember that the security of an AI system depends not just on its internal logic but also on how it validates and processes external inputs. Always assume that external data sources could be compromised, and design your AI systems accordingly.

In the next article in this series, we'll explore the risks of command injection in MCP tools and how to prevent unauthorized command execution.