Metrics

Learn how to ingest OTEL metrics with groundcover

groundcover fully supports the ingestion of metrics in the Open Telemetry format, displaying them as custom metrics in our platform, allowing to build custom dashboards and setting up alerts.

Ingestion from Kubernetes Pods

OTLP Metrics generated from Kubernetes pods can be ingested directly by our Custom Metrics deployment. Ingestion is supported by changing the exporter endpoint to the Custom Metrics Service Endpoint, sending it directly to our Victoria Metrics backend.

Finding the groundcover Custom Metrics Service Endpoint

Use the instructions here to locate the endpoint for the Custom Metrics service, referenced below as {GROUNDCOVER_CUSTOM_METRICS_ENDPOINT}.

Setting Up The Required Environment Variables

Change your deployments environments variables to the endpoint found above.

groundcover versions from 1.8.245

Choose the type of exporting below based on what your code uses - gRPC or HTTP exporters.

OTLP/HTTP

env:
  - name: OTEL_EXPORTER_OTLP_METRICS_ENDPOINT
    value: "http://groundcover-sensor.groundcover.svc.cluster.local:4318/v1/metrics"
  - name: OTEL_EXPORTER_OTLP_METRICS_PROTOCOL
    value: "http/protobuf"

OTLP/gRPC

groundcover versions before 1.8.245

For versions before 1.8.245, it is required to turn on custom-metrics to enable the OTLP metrics endpoint. See docs here. The method below only supports HTTP protocol with protobuf payloads. For gRPC please upgrade sensor version to >1.8.245

 env:
  - name: OTEL_EXPORTER_OTLP_METRICS_ENDPOINT
    value: "http://{GROUNDCOVER_CUSTOM_METRICS_ENDPOINT}:8429/opentelemetry/api/v1/push"
  - name: OTEL_EXPORTER_OTLP_METRICS_PROTOCOL
    value: "http/protobuf"

Shipping from an OpenTelemetry Collector

Shipping Data to the sensors

Starting from version 1.8.216, the recommended method to ship metrics from an OpenTelemetry Collector is the same as other deployments - directly to the groundcover sensor endpoint.

Using an earlier version? Upgrade your installation or contact us on Slack!

Finding the groundcover Sensor Service Endpoint

Use the instructions here to locate the endpoint for the Sensor service, referenced below as {GROUNDCOVER_SENSOR_ENDPOINT}.

Defining the OTLP exporter and pipelines

Ingestion is supported for both OTLP/HTTP and OTLP/gRPC

OTLP/HTTP

exporters:
  otlphttp/groundcover:
    endpoint: http://{GROUNDCOVER_SENSOR_ENDPOINT}:4318

pipelines:
  metrics:
    exporters:
    - otlphttp/groundcover

OTLP/gRPC

Shipping Data directly to inCloud Managed endpoint

This feature is only supported for inCloud Managed installations as part of our Enterprise offering. See here for more details.

groundcover exposes an OpenTelemetry interface as part of our inCloud Managed endpoints, which can be used to ingest data in all standard OTLP protocols for workloads which are not running alongside sensors.

Use the instructions here to locate the endpoint, referenced below as {GROUNDCOVER_MANAGED_OPENTELEMETRY_ENDPOINT}.

These endpoints require authentication using an {INGESTION_KEY} of type 3rd Party (Settings -> Access -> Ingestion Keys)

Additional notes

1. Both gRPCs and HTTPs are supported.

2. The list of supported authentication methods can be found here.

Ingestion is supported for both OTLP/HTTP and OTLP/gRPC

OTLP/HTTP

exporters:
  otlphttp/groundcover:
    endpoint: https://{GROUNDCOVER_MANAGED_OPENTELEMETRY_ENDPOINT}:443
    headers: 
      apikey: {INGESTION_KEY}

pipelines:
  metrics:
    exporters:
    - otlphttp/groundcover

OTLP/gRPC