---
title: "LLM Data Reducers"
slug: "polarity-mcp-llm-reducers"
description: "Learn how Polarity LLM data reducers condense large integration lookup results to optimize token usage and reduce latency for AI applications."
tags: ["AI", "data reducers", "integration results", "LLM context", "LLM reducers", "response summarization", "token optimization", "VirusTotal"]
updated: 2026-07-29T18:50:37Z
published: 2026-07-29T18:50:37Z
canonical: "knowledge.threatconnect.com/polarity-mcp-llm-reducers"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://knowledge.threatconnect.com/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Data Reducers

## What Are Reducers?

Integration lookup results can be large. A VirusTotal response for a single IP may include hundreds of fields spanning vendor verdicts, behavioral tags, network graphs, and historical submissions. Sending the full raw response to an LLM wastes context tokens, increases latency, and can cause models to struggle with noise.

**Polarity's reducer system addresses this.**

A reducer is a JSON-defined transformation pipeline (built on the `polarity-reducer-ex` DSL) that is attached to a specific integration. When a lookup flows through the MCP and reducers are enabled, the reducer is applied to the raw response before it is returned to the LLM client. The result is a condensed, relevant payload that preserves the data the LLM needs and discards the rest.

## How Reducers Are Enabled

Reducers are controlled by the `mcp\_reducers\_enabled` application configuration flag in `polarity\_integrations`. By default, this is set to `true` in production environments:

```plaintext
config :polarity\_integrations, 
  mcp\_reducers\_enabled: true
```

When enabled, every call to `do\_integration\_lookup` passes results through the reducer pipeline for that integration if a details reducer is registered.

If no reducer is registered for an integration, the full raw response is returned without modification.

## How Reducers Are Registered

Reducers are defined at the integration level in the integration's configuration.

When an integration starts, Polarity reads the `\_reducers` key from the integration's configuration and stores it in the integration's database record under `reducers`. The `LookupReducerService` then looks for a `details`" key within that map when processing MCP lookups.

Integration developers define reducers in the integration's `config.json` or equivalent configuration file. A reducer is a JSON DSL pipeline.

**Example operations available in the DSL:**

| **Operation** | **Description** |
| --- | --- |
| drop | Removes specified fields from the result. |
| project | Keeps only the specified fields, discarding everything else. |
| rename | Renames fields to more descriptive or LLM-friendly names. |
| prune | Recursively removes null, empty, or low-value fields. |
| Date/time operations | Formats epoch timestamps and ISO dates into readable strings. |
| Array processing | Trims or transforms nested arrays and objects. |

## Fallback Behavior

The reducer service is designed to be safe. If the integration is not found, if no details reducer is registered, or if reducer execution fails for any reason, the service falls back to returning the original raw data.

> MCP lookups never fail because of a reducer error.

## Measuring Reducer Effectiveness

**When a reducer runs, Polarity logs the original and reduced data sizes, along with the reduction percentage:**

```json
message: "Reduced lookup data for LLM" 
integration\_id: "virustotal\_3\_1\_2\_node\_18\_abc123" 
original\_size\_bytes: 48320 
reduced\_size\_bytes: 4210 
reduction\_percentage: 91
```

Review these log entries to evaluate whether reducers are effectively trimming integration responses for your deployment's specific integrations and use cases.