Log Filtering using OTTL

Filter out noisy, irrelevant, or non-critical logs at ingestion using OpenTelemetry Transformation Language (OTTL) in groundcover. By leveraging simple conditions and the drop flag, you can fine-tune which logs are retained for analysis and which are safely discarded.

Overview

In groundcover, you can apply filtering logic directly to logs using OTTL. This allows you to:

• Reduce data volume and storage costs.

• Focus on the logs that matter most.

• Clean up noisy logs from specific services, containers, or patterns.

Filtering is achieved by setting a special attribute called drop to true based on specific conditions.

How Log Filtering Works

To filter a log, you define a condition (e.g., log message content, container name, log level) and use a set(drop, true) statement. If the condition matches, the log is dropped before ingestion.

Logs are dropped in the sensor - the earliest point possible - achieving maximum savings in terms of networking and ingestion.

Key Concepts:

• drop: A reserved field. If set to true, the log is filtered out.

• where: Used to specify when the filtering should apply.

• IsMatch(): A function for matching regex or substrings within fields (like body).

• statementsErrorMode: How to handle transformation errors (propagate, skip, fail).

• conditionLogicOperator: Used when you define multiple conditions (or, and).

Common Examples

The examples below can be applied by following the relevant docs.

Example 1: Drop health check logs

ottlRules:
- ruleName: "drop_health_checks"
  conditions:
    - 'container_name == "nginx"'
  statements:
    - 'set(drop, true) where IsMatch(body, "GET /healthz")'

💡 Filters out logs with requests to the /healthz endpoint from nginx containers.

Example 2: Drop logs with debug level

ottlRules:
- ruleName: "drop_debug_logs"
  conditions:
    - 'level == "debug"'
    - 'workload == "groundcover-demo"'
  statements:
    - 'set(drop, true)'
  conditionLogicOperator: "and"

💡 Filters out debug logs from groundcover-demo service.