> For the complete documentation index, see [llms.txt](https://docs.groundcover.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.groundcover.com/collect-data/data-pipelines/traces-pipeline/filter-traces.md).

# Filter Traces

## Filter Traces

#### Overview

Filter out noisy, irrelevant, or low-value spans before they're stored. By dropping spans at the pipeline level, you can significantly reduce storage costs while focusing on the traces that matter most.

#### Why Filter Traces?

Not all spans provide equal value. Filtering traces allows you to:

* **Reduce storage costs** by removing noise before ingestion
* **Focus on critical traffic** without distractions from health checks and internal probes
* **Improve query performance** by reducing the dataset size
* **Eliminate redundant data** such as duplicate internal spans

{% hint style="success" %}
Spans are dropped **in the pipeline** before they're sent to storage, achieving maximum savings in terms of networking and ingestion.
{% endhint %}

#### Verifying the Impact

Use the **Ingestion** screen in the [Data Explorer](/investigate/querying-your-groundcover-data/explore-and-monitors-query-builder.md#ingestion) to confirm how much volume a filter rule removed. Select the **Traces** signal, then switch the **Stage** selector between:

* **Ingested** - volume that entered the pipeline
* **Filtered** - volume your filter rules removed (the amount dropped)
* **Stored** - volume that remained and was persisted

Read **Filtered** directly to see a rule's savings, or compare **Ingested** with **Stored**. Use the `Volume` / `Count` toggle to measure in bytes or number of records.

{% hint style="info" %}
**Filtered** is the volume that was dropped, not what remained. **Filtered %** is `Filtered / (Filtered + Stored) × 100` - the share of entering volume your rules dropped, so a higher value means more was dropped.
{% endhint %}

#### Best Practices

1. **Be specific with conditions** - Use precise conditions to avoid accidentally dropping important spans
2. **Start conservative** - Begin by dropping obvious noise, then expand gradually
3. **Monitor the impact** - Track how many spans are being dropped and adjust as needed
4. **Document your rules** - Use clear `ruleName` values to explain what each rule does
5. **Avoid dropping errors** - Unless absolutely necessary, preserve error spans for debugging
6. **Consider performance** - Dropping spans early saves processing and storage costs

#### How Span Filtering Works

To drop a span, you define a condition (e.g., span name, workload, protocol type) and use a `set(drop, true)` statement. If the condition matches, the span is removed from the pipeline before storage.

**Key Concepts:**

* `drop` - A reserved field. If set to `true`, the span is filtered out
* `where` - Used to specify when the dropping should apply
* `IsMatch()` - A function for matching regex or substrings within fields

#### Common Use Cases

**Drop Health Check Spans**

Health check endpoints often generate high-volume, low-value spans.

```yaml
ottlRules:
  - ruleName: drop_health_checks
    conditions:
      - protocol_type == "http"
    statements:
      - set(drop, true) where IsMatch(span_name, "/health")
      - set(drop, true) where IsMatch(span_name, "/healthz")
      - set(drop, true) where IsMatch(span_name, "/readyz")
      - set(drop, true) where IsMatch(span_name, "/livez")
```

💡 **What it does:** Drops all HTTP spans for common health check, readiness, and liveness probe endpoints.

**Drop Spans by Protocol Type**

Filter out spans for specific protocols you don't need to monitor.

```yaml
ottlRules:
  - ruleName: drop_dns_traces
    conditions:
      - protocol_type == "dns"
    statements:
      - set(drop, true)
```

💡 **What it does:** Drops all DNS resolution spans to focus on application-level traces.

**Drop Spans Based on Attributes**

Filter based on custom span attributes.

```yaml
ottlRules:
  - ruleName: drop_synthetic_traffic
    conditions:
      - attributes["synthetic"] == "true"
    statements:
      - set(drop, true)
```

💡 **What it does:** Drops spans flagged as synthetic/test traffic.

**Conditional Dropping with Complex Logic**

Combine multiple conditions for fine-grained control.

```yaml
ottlRules:
  - ruleName: "drop_successful_health_checks"
    conditions:
      - protocol_type == "http"
      - status == "Ok"
    conditionLogicOperator: "and"
    statements:
      - set(drop, true) where IsMatch(span_name, "/health") or IsMatch(span_name, "/ready")
```

💡 **What it does:** Drops only **successful** health check spans. Failed health checks are preserved for debugging.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.groundcover.com/collect-data/data-pipelines/traces-pipeline/filter-traces.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.