Drop Logs

Overview

Filter out noisy, irrelevant, or non-critical logs at ingestion. By dropping logs at the sensor level—before they're sent to storage—you can significantly reduce costs while focusing on the logs that matter most.

Why Drop Logs?

Not all logs provide equal value. Dropping logs allows you to:

Reduce data volume and storage costs by filtering out noise
Focus on critical information without distractions
Improve query performance by reducing the dataset size
Save on storage costs since logs are dropped at the sensor

Logs are dropped in the sensor - the earliest point possible - achieving maximum savings in terms of networking and ingestion.

How Log Dropping Works

To drop a log, you define a condition (e.g., log message content, container name, log level) and use a set(drop, true) statement. If the condition matches, the log is dropped before ingestion.

Key Concepts:

drop - A reserved field. If set to true, the log is filtered out
where - Used to specify when the dropping should apply
IsMatch() - A function for matching regex or substrings within fields

Best Practices

Be specific with conditions - Use precise conditions to avoid accidentally dropping important logs
Test before deploying - Use the Parsing Playground to verify your drop rules
Start conservative - Begin by dropping obvious noise, then expand gradually
Monitor the impact - Track how many logs are being dropped and adjust as needed
Document your rules - Use clear ruleName values to explain what each rule does
Consider performance - Dropping logs early saves processing, networking, and storage costs
Avoid dropping errors - Unless absolutely necessary, preserve error and warning logs

Common Use Cases

Drop Health Check Logs

Health check endpoints often generate high-volume, low-value logs.

ottlRules:
  - ruleName: "drop_health_checks"
    conditions:
      - 'container_name == "nginx"'
    statements:
      - 'set(drop, true) where IsMatch(body, "GET /healthz")'

💡 What it does: Filters out logs with requests to the /healthz endpoint from nginx containers.

Drop Debug Logs from Specific Services

Development or verbose logging can overwhelm production logs.

ottlRules:
  - ruleName: "drop_debug_logs"
    conditions:
      - 'level == "debug"'
      - 'workload == "groundcover-demo"'
    conditionLogicOperator: "and"
    statements:
      - 'set(drop, true)'

💡 What it does: Filters out debug logs from the groundcover-demo service.

Drop Logs Based on Multiple Patterns

Filter out various monitoring and internal checks.

ottlRules:
  - ruleName: "drop_monitoring_logs"
    conditions:
      - 'namespace == "production"'
    statements:
      - 'set(drop, true) where IsMatch(body, "/metrics")'
      - 'set(drop, true) where IsMatch(body, "/readyz")'
      - 'set(drop, true) where IsMatch(body, "/livez")'

💡 What it does: Drops logs containing metrics, readiness, or liveness probe requests in production.

Drop Logs by Status Code

Filter out successful requests to focus on errors.

ottlRules:
  - ruleName: "drop_successful_requests"
    conditions:
      - 'workload == "api-gateway"'
      - 'attributes["status_code"] != nil'
    conditionLogicOperator: "and"
    statements:
      - 'set(drop, true) where Int(attributes["status_code"]) >= 200 and Int(attributes["status_code"]) < 300'

💡 What it does: Drops logs with 2xx status codes from the API gateway to focus on errors and unusual behavior.

Drop Logs from Test Environments

Reduce noise from non-production environments.

ottlRules:
  - ruleName: "drop_test_logs"
    conditions:
      - 'namespace == "test" or namespace == "staging"'
    conditionLogicOperator: "or"
    statements:
      - 'set(drop, true)'

💡 What it does: Drops all logs from test and staging namespaces.

Conditional Dropping with Complex Logic

Combine multiple conditions for fine-grained control.

ottlRules:
  - ruleName: "drop_low_priority"
    conditions:
      - 'workload == "background-worker"'
    statements:
      - 'set(drop, true) where level == "info" and IsMatch(body, "job completed")'
      - 'set(drop, true) where level == "debug"'

💡 What it does: Drops info-level "job completed" messages and all debug logs from background workers.

Key Functions

set(drop, true)

Marks a log for dropping. Can be used with or without conditions.

# Drop all logs matching the rule
- 'set(drop, true)'

# Drop only when specific condition is met
- 'set(drop, true) where IsMatch(body, "pattern")'

IsMatch()

Checks if a field matches a regex pattern or substring.

# Simple substring match
- 'set(drop, true) where IsMatch(body, "/healthz")'

# Regex pattern match
- 'set(drop, true) where IsMatch(body, "^ERROR.*timeout")'

Combining Conditions

Use logical operators to create complex filtering rules.

# Using conditionLogicOperator at rule level
ottlRules:
  - ruleName: "example"
    conditions:
      - 'condition1'
      - 'condition2'
    conditionLogicOperator: "and"  # or "or"
    statements:
      - 'set(drop, true)'

# Using inline conditions in statements
- 'set(drop, true) where condition1 and condition2'
- 'set(drop, true) where condition1 or condition2'

Last updated 5 hours ago