Enrich logs and traces with pod labels & annotations

groundcover allows you to add custom labels and annotations from Kubernetes pods and Docker containers to the traces and logs generated by those pods or containers.

This feature is available in groundcover sensor versions ≥ 1.9.127

Specifying what to collect

To apply custom labels and annotations collection, you need to update your groundcover deployment values.yaml file.

To collect a pod label named app.kubernetes.io/name and an annotation named app.kubernetes.io/other-name you will need to add the following configuration:

agent:
  sensor:
    customEntityTags:
      labels:
        keys: ["app.kubernetes.io/name"]
        collectAll: false
      annotations:
        keys: ["app.kubernetes.io/other-name"]
        collectAll: false

To collect all pod labels and/or annotations, use the following configuration:

agent:
  sensor:
    customEntityTags:
      labels:
        keys: []
        collectAll: false
      annotations:
        keys: []
        collectAll: true

We prefix the labels/annotations with the following (according to pod/docker):

• k8s.pod.label

• k8s.pod.annotation

• docker.container.label

• docker.container.annotation

pod labels in a span attributes

Searching labels and annotations

Once you've setup the configuration to the sensor, you will be able to view and search for specific labels and annotation like any other attribute. For example, to search for all traces that has the label app.kubernetes.io/part-of = groundcover you can do the following search: k8s.pod.label.app.kubernetes.io/part-of:groundcover in the search bar.

auto-complete label values from traces