Search
⌃K

Multi-Cluster installation

By default, when you install groundcover on several clusters, each cluster will contain its own independent set of traces, metrics and logs databases (the groundcover backend). The following guide will walk you through how to install groundcover on multiple clusters while using a single centralized instance of each of these databases.

Requirements

Architecture Overview

In this installation mode, we will deploy the following components separately:

Deploying the groundcover backend

  1. 2.
    Add the groundcover helm repository and/or make sure it's up to date
    helm repo add groundcover https://helm.groundcover.com # only for the first time
    helm repo update groundcover
  2. 3.
    Create the following backend-values.yaml file and fill the required values accordingly
    global:
    otlp:
    tls:
    enabled: false
    multipleClusterIds:
    # a list of the the cluster names that will be monitored
    # - dev
    # - staging
    # - prod
    opentelemetry-collector:
    ingress:
    enabled: true
    annotations:
    ## ALB
    # kubernetes.io/ingress.class: alb
    # alb.ingress.kubernetes.io/target-type: ip
    # alb.ingress.kubernetes.io/certificate-arn: {cert-arn}
    hosts:
    - host: {host}
    paths:
    - path: /health
    port: 13133
    pathType: Exact
    - path: /loki
    pathType: Prefix
    port: 3100
    additionalIngresses:
    - name: otlp-grpc
    annotations:
    ## ALB
    # kubernetes.io/ingress.class: alb
    # alb.ingress.kubernetes.io/target-type: ip
    # alb.ingress.kubernetes.io/certificate-arn: {cert-arn}
    hosts:
    - host: {host}
    paths:
    - path: /
    pathType: Prefix
    port: 4317
    # optional - if custom-metrics is enabled
    victoria-metrics-single:
    server:
    ingress:
    enabled: true
    annotations:
    ## ALB
    # kubernetes.io/ingress.class: alb
    # alb.ingress.kubernetes.io/target-type: ip
    # alb.ingress.kubernetes.io/certificate-arn: {cert-arn}
    hosts:
    - name: {host}
    path: "/"
    port: http
    shepherd:
    ## ALB
    config:
    ingestor:
    TLSCertFile: ""
    TLSKeyFile: ""
    httpIngress:
    enabled: true
    annotations:
    ## ALB
    # kubernetes.io/ingress.class: alb
    # alb.ingress.kubernetes.io/target-type: ip
    # alb.ingress.kubernetes.io/certificate-arn: {cert-arn}
    hosts:
    - path: /
    # if host is pre-known (e.g not generated upon ingress creation)
    # host: {the shepherd host that will be used by the agents outside of the cluster}
    grpcIngress:
    enabled: true
    annotations:
    ## ALB
    # kubernetes.io/ingress.class: alb
    # alb.ingress.kubernetes.io/backend-protocol-version: GRPC
    # alb.ingress.kubernetes.io/target-type: ip
    # alb.ingress.kubernetes.io/certificate-arn: {cert-arn}
    hosts:
    - path: /
    # if host is pre-known (e.g not generated upon ingress creation)
    # host: {the shepherd host that will be used by the agents outside of the cluster}
  3. 4.
    Run the following installation command
    helm upgrade -i groundcover-backend groundcover/groundcover --set global.groundcover_token=<apikey>,clusterId={cluster id of backend cluster} -f backend-values.yaml --create-namespace -n groundcover
    Now, make sure the ingresses you've just created are accessible from the clusters you intend to deploy the groundcover agent on.

Deploying the groundcover agent

  1. 1.
    Obtain the addresses that are used by the created ingresses, using
    kubectl get ingress -n groundcover
  2. 2.
    Create the following agent-values.yaml, and fill the required values accordingly
    global:
    logs:
    # for example https://opentelemtry-collector-http-ingress.net
    overrideUrl: {opentelemtry-collector-http-ingress-endpoint}
    otlp:
    # for example opentelemtry-collector-grpc-ingress.net:443
    overrideGrpcURL: {opentelemtry-collector-grpc-ingress-endpoint}
    overrideHttpURL: {opentelemtry-collector-http-ingress-endpoint}
    backend:
    enabled: false
    # optional if custom-metrics enabled
    custom-metrics:
    remoteWriteUrls:
    # for example https://victoria-metrics-ingress.net/api/v1/write
    - {victoria-metrics-ingress-endpoint}/api/v1/write
    shepherd:
    service:
    grpcPort: 443
    # for example https://shepherd-ingress.net
    overrideHttpURL: {shepherd-http-ingress-endpoint}
    # otel is grpc server, so the format is {shepherd.net}:{443}
    overrideGrpcURL: {shepherd-grpc-ingress-endpoint}:443
  3. 3.
    Run the following installation command on each of the target clusters
    helm upgrade -i groundcover groundcover/groundcover --set global.groundcover_token=<apikey>,clusterId={cluster id} -f agent-values.yaml --create-namespace -n groundcover
    Make sure the clusterId parameter corresponds to one of the cluster ids listed in the backend deployment.
  4. 4.
    Once the installation completes - you can browse to groundcover
    🎉