Custom data retention

Data Retention Strategies

There are two ways to define retention in groundcover:

1. Simple - each type of data has a global retention period

2. Advanced - data is retained based on various criteria such as cluster, log level, namespace, etc.

Controlling Retention

To customize the retention on the groundcover platform, either create a new custom-values.yaml or edit your existing values.yaml with the overrides defined below and redeploy groundcover.

Retention Field Format

Retention value format is: {amount}[h(ours), d(ays), w(eeks), y(ears)] .

For example: 4h, 30d, 6w, 1y

Simple Data Retention

The most common and simple way to configure retention in groundcover. Based solely on data type, without exceptions.

Below is an example configuration for setting data retention values:

Default retention values

Below are the default retention values for our BYOC solution:

• Traces - 7 Days

• Metrics - 30 days

• Logs - 30 days

• Events - 7 days

Advanced Data Retention

groundcover allows you to customize retention policies for your data to better manage storage and compliance requirements. You can define specific retention periods for logs, traces, and events based on various criteria such as cluster, log level, namespace and more.

Custom Retention Overrides

The custom_retention_overrides list allows you to define specific retention periods for data based on conditions. Each override has a retention field and a conditions field.

• Retention: Specifies the duration for which the data should be retained.

• Conditions: Specify the criteria that the retention policy applies to. When multiple conditions are set, they are connected by an AND condition, meaning all conditions must be met.

Default Retention

The retention field under each data type (traces, logs, events) specifies the default retention period for that data type, meaning, anything that doesn't match the custom conditions set.

Example Advanced Configuration File

The configuration below implements the following logic:

• Traces

  • Traces with labels cluster: prod, namespace: app will be retained for 7d

  • Traces with label env: staging will be retained for 14d

Available Retention Fields

Logs

• cluster

• source

• env

Traces

• cluster

• workload

• container

Events

• cluster

• source

• env_name