> For the complete documentation index, see [llms.txt](https://docs.groundcover.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.groundcover.com/use-groundcover/data-pipelines/traces-pipeline/filter-traces.md).

# Filter Traces

## Filter Traces

#### Overview

Filter out noisy, irrelevant, or low-value spans before they're stored. By dropping spans at the pipeline level, you can significantly reduce storage costs while focusing on the traces that matter most.

#### Why Filter Traces?

Not all spans provide equal value. Filtering traces allows you to:

* **Reduce storage costs** by removing noise before ingestion
* **Focus on critical traffic** without distractions from health checks and internal probes
* **Improve query performance** by reducing the dataset size
* **Eliminate redundant data** such as duplicate internal spans

{% hint style="success" %}
Spans are dropped **in the pipeline** before they're sent to storage, achieving maximum savings in terms of networking and ingestion.
{% endhint %}

#### Best Practices

1. **Be specific with conditions** - Use precise conditions to avoid accidentally dropping important spans
2. **Start conservative** - Begin by dropping obvious noise, then expand gradually
3. **Monitor the impact** - Track how many spans are being dropped and adjust as needed
4. **Document your rules** - Use clear `ruleName` values to explain what each rule does
5. **Avoid dropping errors** - Unless absolutely necessary, preserve error spans for debugging
6. **Consider performance** - Dropping spans early saves processing and storage costs

#### How Span Filtering Works

To drop a span, you define a condition (e.g., span name, workload, protocol type) and use a `set(drop, true)` statement. If the condition matches, the span is removed from the pipeline before storage.

**Key Concepts:**

* `drop` - A reserved field. If set to `true`, the span is filtered out
* `where` - Used to specify when the dropping should apply
* `IsMatch()` - A function for matching regex or substrings within fields

#### Common Use Cases

**Drop Health Check Spans**

Health check endpoints often generate high-volume, low-value spans.

```yaml
ottlRules:
  - ruleName: drop_health_checks
    conditions:
      - protocol_type == "http"
    statements:
      - set(drop, true) where IsMatch(span_name, "/health")
      - set(drop, true) where IsMatch(span_name, "/healthz")
      - set(drop, true) where IsMatch(span_name, "/readyz")
      - set(drop, true) where IsMatch(span_name, "/livez")
```

💡 **What it does:** Drops all HTTP spans for common health check, readiness, and liveness probe endpoints.

**Drop Spans by Protocol Type**

Filter out spans for specific protocols you don't need to monitor.

```yaml
ottlRules:
  - ruleName: drop_dns_traces
    conditions:
      - protocol_type == "dns"
    statements:
      - set(drop, true)
```

💡 **What it does:** Drops all DNS resolution spans to focus on application-level traces.

**Drop Spans Based on Attributes**

Filter based on custom span attributes.

```yaml
ottlRules:
  - ruleName: drop_synthetic_traffic
    conditions:
      - attributes["synthetic"] == "true"
    statements:
      - set(drop, true)
```

💡 **What it does:** Drops spans flagged as synthetic/test traffic.

**Conditional Dropping with Complex Logic**

Combine multiple conditions for fine-grained control.

```yaml
ottlRules:
  - ruleName: "drop_successful_health_checks"
    conditions:
      - protocol_type == "http"
      - status == "Ok"
    conditionLogicOperator: "and"
    statements:
      - set(drop, true) where IsMatch(span_name, "/health") or IsMatch(span_name, "/ready")
```

💡 **What it does:** Drops only **successful** health check spans. Failed health checks are preserved for debugging.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.groundcover.com/use-groundcover/data-pipelines/traces-pipeline/filter-traces.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.