Custom data retention

Custom Data Retentions

groundcover allows you to customize retention policies for your data to better manage storage and compliance requirements. You can define specific retention periods for logs, traces, and events based on various criteria such as cluster, log level, and namespace values or just default.

Metrics retention is based solely on time retention.

Default Retention Values

  • Traces - 24 hours

  • Metrics - 7 days

  • Logs - 3 days

  • Events - 7 days

YAML Retention Configuration Structure

To customize the retention on the groundcover platform, either create a new custom-values.yaml or edit your existing values.yaml with the overrides defined below and redeploy groundcover.

Retention Field Format

Retention value format is: {amount}[h(ours), d(ays), w(eeks), y(ears)] .

For example: 4h, 30d, 6w, 1y

Custom Retention Overrides

The custom_retention_overrides list allows you to define specific retention periods for data based on conditions. Each override has a retention field and a conditions field.

  • Retention: Specifies the duration for which the data should be retained.

  • Conditions: These specify the criteria that the retention policy applies to. When multiple conditions are set, they are connected by an AND condition, meaning all conditions must be met.

Default Retention

The retention field under each data type (traces, logs, events) specifies the default retention period for that data type.

Example Configuration File

global:
  traces:
    retention: 24h # default traces retention
    custom_retention_overrides:
      - retention: 7d
        conditions:
          cluster: 'prod'
          namespace: 'app'
      - retention: 14d
        conditions:
          env: 'staging'
  logs:
    retention: 3d # default logs retention
    custom_retention_overrides:
      - retention: 20d
        conditions:
          cluster: 'prod'
          level: 'info'
      - retention: 30d
        conditions:
          cluster: 'prod'
          level: 'error'
  events:
    retention: 7d # default events retention
    custom_retention_overrides:
      - retention: 15d
        conditions:
          type: 'Warning'
          cluster: 'dev'

victoria-metrics-single:
  server:
    retentionPeriod: 7d # {amount}[h(ours), d(ays), w(eeks), y(ears)]

Available Retention Fields

Logs

  • cluster

  • source

  • env

  • env_type

  • workload

  • namespace

  • level

Traces

  • cluster

  • source

  • env

  • env_type

  • protocol_type

Events

  • cluster

  • source

  • env_name

  • entity_workload

  • entity_namespace

  • type

Important Notes

  • If a default retention is not set for a certain datatype, groundcover will apply its own default retention.

  • In instances of overlapping overrides, the override with the shorter retention interval will be used.

Last updated