Ingest CloudWatch Metrics

groundcover supports ingesting CloudWatch metrics directly into our platform, allowing you to visualize them using dashboards and create alerts.

How does it work

CloudWatch integration is done by deploying a service called integrations-agent which is responsible for pulling metrics from CloudWatch using periodic polling of these APIs:

• ListMetrics

• GetMetricData

• GetMetricStatistics

The integration is setup using the following steps:

1. Create a role with permissions to access the metrics on the target AWS account

2. Provide the needed permissions to allow the integrations-agent service account to assume the new role

3. Select the AWS namespaces you wish to collect metrics from

4. Deploy the integrations-agent service and start collecting metrics

The actual steps differ for inCloud Managed and inCloud deployments. Follow the instructions below depending on your deployment type.

Things to know

Ingestion interval

The integration pulls data from CloudWatch every five minutes.

Data storage

Data fetched is stored in the Victoria Metrics database, meaning metrics are queried via the CloudWatch API only one time per data point.

Metric Statistics

Each metric has a label called stat which denotes the AWS statistic used during querying. Some metrics have multiple stats which are useful for different cases.

Supported AWS services

Click to Open

AWS/AOSS
AWS/ApiGateway
AWS/AppRunner
AWS/AppStream
AWS/AppSync
AWS/ApplicationELB
AWS/Athena
AWS/AutoScaling
AWS/Backup
AWS/Bedrock
AWS/Cassandra
AWS/CertificateManager
AWS/CloudFront
AWS/Cognito
AWS/DDoSProtection
AWS/DMS
AWS/DX
AWS/DocDB
AWS/DynamoDB
AWS/EBS
AWS/EC2
AWS/EC2Spot
AWS/ECR
AWS/ECS
AWS/EFS
AWS/ELB
AWS/ES
AWS/ElastiCache
AWS/ElasticBeanstalk
AWS/ElasticMapReduce
AWS/FSx
AWS/Firehose
AWS/GameLift
AWS/GlobalAccelerator
AWS/Glue
AWS/IoT
AWS/KMS
AWS/Kafka
AWS/Kinesis
AWS/KinesisAnalytics
AWS/Lambda
AWS/MWAA
AWS/MediaConnect
AWS/MediaConvert
AWS/MediaLive
AWS/MediaPackage
AWS/MediaTailor
AWS/MemoryDB
AWS/NATGateway
AWS/Neptune
AWS/Network Manager
AWS/NetworkELB
AWS/NetworkFirewall
AWS/PrivateLinkEndpoints
AWS/PrivateLinkServices
AWS/RDS
AWS/Redshift
AWS/Route53
AWS/S3
AWS/SES
AWS/SNS
AWS/SQS
AWS/SageMaker
AWS/Sagemaker/ModelBuildingPipeline
AWS/Sagemaker/Endpoints
AWS/Sagemaker/ProcessingJobs
AWS/Sagemaker/TrainingJobs
AWS/Sagemaker/TransformJobs
AWS/States
AWS/StorageGateway
AWS/TransitGateway
AWS/TrustedAdvisor
AWS/VPN
AWS/WorkSpaces
ECS/ContainerInsights

Resource Discovery Methods

groundcover seamlessly integrates both methods below to avoid duplicate metric fetching.

The integration uses two methods to discover the AWS resources to fetch metrics for:

1. Tagging-based discovery - this method uses the AWS tagging mechanism to discover resources across all metric namespaces. This method supports all AWS namespaces but only works for resources which are tagged with at least one AWS tag.

2. List-based discovery - this method uses standard AWS APIs to list the resources in each namespace. It works for all resources regardless of tags, but the coverage is limited to specific namespaces as listed below:

   1. AWS/RDS

   2. AWS/S3

   3. AWS/SQS

   4. AWS/Lambda

   5. AWS/ElastiCache

If you're not seeing metrics for a specific resource, it likely has no tags and is not in the list of services above. Contact us on Slack to help with resolving the issue.

Setting up the integration

The integration is supported for both inCloud Managed and inCloud deployments, by following slightly different steps. Make sure to select the deployment relevant for you.

Not sure? Contact us on slack!

inCloud Managed

In this setup, the integrations-agent is deployed inside the managed backend. A service account is automatically provisioned inside the managed account, so the installation mainly requires creating a role in the target AWS account with the correct trust relationship.

Create an IAM role and policy

The following part requires two parameters:

• YOUR_GROUNDCOVER_ACCOUNT_ID - the AWS account id hosting the groundcover backend, created during onboarding

• YOUR_GROUNDCOVER_SITE_ID - your unique groundcover endpoint ID:

  • Fetch your inCloud Site from these docs It will look like <SITE_ID>.platform.grcv.io

  • The ID is the first part marked above as <SITE_ID>

1. Go to Amazon IAM

2. Click on Roles in the side bar

3. Click on Create Role

   1. Select Custom trust policy

   2. Paste the following policy:

      {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Principal": {
              "AWS": "arn:aws:iam::<YOUR_GROUNDCOVER_ACCOUNT_ID>:role/groundcover-integrations-agent-<YOUR_GROUNDCOVER_SITE_ID>-sa"
            },
            "Action": "sts:AssumeRole"
          }
        ]
      }
      

   3. Click on Next twice (we'll attach permissions later)

   4. Provide a name for the role

   5. Click on Create Role

4. Go to your newly created role

   1. In the Permissions section, click on Add permissions and then Create inline policy

   2. Click on JSON and paste the following:

      {
          "Version": "2012-10-17",
          "Id": "groundcover-integrations-agent",
          "Statement": [
              {
                  "Action": [
                      "tag:GetResources",
                      "storagegateway:ListTagsForResource",
                      "storagegateway:ListGateways",
                      "shield:ListProtections",
                      "iam:ListAccountAliases",
                      "ec2:DescribeTransitGatewayAttachments",
                      "ec2:DescribeSpotFleetRequests",
                      "dms:DescribeReplicationTasks",
                      "dms:DescribeReplicationInstances",
                      "cloudwatch:ListMetrics",
                      "cloudwatch:GetMetricStatistics",
                      "cloudwatch:GetMetricData",
                      "autoscaling:DescribeAutoScalingGroups",
                      "aps:ListWorkspaces",
                      "apigateway:GET",
                      "s3:ListAllMyBuckets",
                      "s3:GetBucketLocation",
                      "s3:GetBucketTagging",
                      "sqs:ListQueues",
                      "sqs:GetQueueAttributes",
                      "rds:DescribeDBInstances",
                      "rds:DescribeDBClusters",
                      "lambda:ListFunctions",
                      "elasticache:DescribeCacheClusters",
                      "elasticache:DescribeServerlessCaches",
                      "dynamodb:ListTables",
                      "dynamodb:ListTagsOfResource",
                      "dynamodb:DescribeTable"
                  ],
                  "Effect": "Allow",
                  "Resource": "*"
              }
          ]
      }

   3. Click on Next

   4. Give the policy a name

   5. Click on Create Policy

Choosing namespaces to ingest

Select the AWS namespaces you wish to ingest metrics from. The list of supported services is here.

Share details with groundcover

Share the following details with the groundcover team to complete the integration:

1. The ARN of the role created above

2. The list of namespaces you wish to ingest

3. The region of the account you wish to ingest metrics from

inCloud

In this setup, the integrations-agent is deployed inside your cluster. The deployment comes with a built-in service account, and permissions will be provided to it using IRSA. Afterwards you will need to create a role + trust policy to allow the integrations-agent service account to assume the role and query the metrics.

Create an identity provider (OIDC) for your EKS cluster

If you already have an identity provider set up in the account where the EKS cluster is set up, skip this stage

The recommended way to give the integrations-agent the ability to interact with AWS resources inside an EKS cluster is using IRSA . The deployment comes in with a built-in service account, which can be used alongside OIDC to provide it with AWS permissions.

Create an OIDC provider for the EKS cluster where groundcover resides in. Once created, the details you will need in the next steps are:

• Provider Name

• Provider ARN

• Namespace - the namespace where groundcover is installed (default: groundcover)

Create an IAM role and policy

1. Go to Amazon IAM

2. Click on Roles in the side bar

3. Click on Create Role

   1. Select Custom trust policy

   2. Paste the following policy:

      {
          "Version": "2012-10-17",
          "Statement": [
              {
                  "Effect": "Allow",
                  "Principal": {
                      "Federated": "{Provider ARN}"
                  },
                  "Action": "sts:AssumeRoleWithWebIdentity",
                  "Condition": {
                      "StringLike": {
                          "{Provider Name}:sub": "system:serviceaccount:{Namespace}:integrations-agent"
                      }
                  }
              }
          ]
      }
      

   3. Click on Next twice (we'll attach permissions later)

   4. Provide a name for the role

   5. Click on Create Role

4. Go to your newly created role

   1. In the Permissions section, click on Add permissions and then Create inline policy

   2. Click on JSON and paste the following:

      {
          "Version": "2012-10-17",
          "Id": "groundcover-integrations-agent",
          "Statement": [
              {
                  "Action": [
                      "tag:GetResources",
                      "storagegateway:ListTagsForResource",
                      "storagegateway:ListGateways",
                      "shield:ListProtections",
                      "iam:ListAccountAliases",
                      "ec2:DescribeTransitGatewayAttachments",
                      "ec2:DescribeSpotFleetRequests",
                      "dms:DescribeReplicationTasks",
                      "dms:DescribeReplicationInstances",
                      "cloudwatch:ListMetrics",
                      "cloudwatch:GetMetricStatistics",
                      "cloudwatch:GetMetricData",
                      "autoscaling:DescribeAutoScalingGroups",
                      "aps:ListWorkspaces",
                      "apigateway:GET",
                      "s3:ListAllMyBuckets",
                      "s3:GetBucketLocation",
                      "s3:GetBucketTagging",
                      "sqs:ListQueues",
                      "sqs:GetQueueAttributes",
                      "rds:DescribeDBInstances",
                      "rds:DescribeDBClusters",
                      "lambda:ListFunctions",
                      "elasticache:DescribeCacheClusters",
                      "elasticache:DescribeServerlessCaches",
                      "dynamodb:ListTables",
                      "dynamodb:ListTagsOfResource",
                      "dynamodb:DescribeTable"
                  ],
                  "Effect": "Allow",
                  "Resource": "*"
              }
          ]
      }

   3. Click on Next

   4. Give the policy a name

   5. Click on Create Policy

Choosing namespaces to ingest

Select the AWS namespaces you wish to ingest metrics from. The list of supported services is here. These will be provided as a list in the next stage.

Deploy the changes

Add the following overrides to your groundcover installation. You will need the following:

• Region - The region of the account you wish to ingest metrics from

• NamespaceA, NamespaceB... - the AWS namespaces you chose

• RoleARN - the ARN of the role created above

Note: the roleArn section in the configuration below is optional, and only required when using cross-account scraping. If you are scraping metrics inside the same account as the EKS cluster where groundcover is deployed, do not add the roleArn part.

In that scenario, you will need to define a role in each account with the permissions above, and enable cross-account access to assume the role from the RoleARN created above. The cross-account role appears below as otherAccountRole

global:
  integrations:
    agent:
      enabled: true
      
integrationsAgent:
  serviceAccount:
      annotations:
        eks.amazonaws.com/role-arn: {RoleARN}
  targets:
    cloudwatch:
    - stsRegion: "{Region}"
      regions:
        - "{Region}"
      namespaces:
        - "{NamespaceA}"
        - "{NamespaceB}"
      # optional - only for cross-account scraping, see note above
      roleArn: "{otherAccountRole}"

For example:

global:
  integrations:
    agent:
      enabled: true
      
integrationsAgent:
  serviceAccount:
      annotations:
        eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/groundcover-cw-metrics
  targets:
    cloudwatch:
    - stsRegion: "us-east-1"
      regions:
        - "us-east-1"
      namespaces:
        - "AWS/Lambda"
        - "AWS/ApiGateway"