Ingest CloudWatch Metrics

groundcover supports ingesting CloudWatch metrics directly into our platform, allowing you to visualize them using dashboards and create alerts.

How does it work

CloudWatch integration is done by deploying a service called integrations-agent which is responsible for pulling metrics from CloudWatch using periodic polling of these APIs:

• ListMetrics

• GetMetricData

• GetMetricStatistics

The integration setup is done directly through the App by following these steps:

1. Navigate to Settings > Integrations > Data Sources or following this link. Note that only users with Admin permissions can navigate to this page.

2. Select Amazon Web Services and follow the 3 steps in the wizard. Note that in step 1 you'll need to provide an ARN, granting groundcover with permissions to poll metrics. To do that, please follow the guidelines in this section.

Things to know

Ingestion interval

The integration pulls data from CloudWatch according to this interval. The lower the interval, the higher the polling rate and as a result, the overall costs will be higher.

Data storage

Data fetched is stored in the Victoria Metrics database, meaning metrics are queried via the CloudWatch API only one time per data point.

Metric Statistics

Each metric has a label called stat which denotes the AWS statistic used during querying. Some metrics have multiple stats which are useful for different cases.

Supported AWS services

Click to Open

/aws/sagemaker/Endpoints
/aws/sagemaker/ProcessingJobs
/aws/sagemaker/TrainingJobs
/aws/sagemaker/TransformJobs
AWS/AOSS
AWS/AmazonMQ
AWS/ApiGateway
AWS/AppRunner
AWS/AppStream
AWS/AppSync
AWS/ApplicationELB
AWS/Athena
AWS/AutoScaling
AWS/Backup
AWS/Bedrock
AWS/Cassandra
AWS/CertificateManager
AWS/CloudFront
AWS/CloudWatchSynthetics
AWS/Cognito
AWS/DDoSProtection
AWS/DMS
AWS/DX
AWS/DocDB
AWS/DynamoDB
AWS/EBS
AWS/EC2
AWS/EC2Spot
AWS/ECR
AWS/ECS
AWS/EFS
AWS/EKS
AWS/ELB
AWS/ES
AWS/ElastiCache
AWS/ElasticBeanstalk
AWS/ElasticMapReduce
AWS/FSx
AWS/Firehose
AWS/GameLift
AWS/GlobalAccelerator
AWS/IPAM
AWS/IoT
AWS/KMS
AWS/Kafka
AWS/Kinesis
AWS/KinesisAnalytics
AWS/Lambda
AWS/MWAA
AWS/MediaConnect
AWS/MediaConvert
AWS/MediaLive
AWS/MediaPackage
AWS/MediaTailor
AWS/MemoryDB
AWS/NATGateway
AWS/Neptune
AWS/Network Manager
AWS/NetworkELB
AWS/NetworkFirewall
AWS/PrivateLinkEndpoints
AWS/PrivateLinkServices
AWS/RDS
AWS/Redshift
AWS/Route53
AWS/S3
AWS/SES
AWS/SNS
AWS/SQS
AWS/SageMaker
AWS/Sagemaker/ModelBuildingPipeline
AWS/States
AWS/StorageGateway
AWS/TransitGateway
AWS/TrustedAdvisor
AWS/Usage
AWS/VPN
AWS/WAF
AWS/WAFV2
AWS/WorkSpaces
ECS/ContainerInsights
Glue

Resource Discovery Methods

groundcover seamlessly integrates both methods below to avoid duplicate metric fetching.

The integration uses two methods to discover the AWS resources to fetch metrics for:

1. Tagging-based discovery - this method uses the AWS tagging mechanism to discover resources across all metric namespaces. This method supports all AWS namespaces but only works for resources which are tagged with at least one AWS tag.

2. List-based discovery - this method uses standard AWS APIs to list the resources in each namespace. It works for all resources regardless of tags, but the coverage is limited to specific namespaces as listed below:

   1. AWS/RDS

   2. AWS/S3

   3. AWS/SQS

   4. AWS/Lambda

   5. AWS/ElastiCache

   6. AWS/DynamoDB

   7. AWS/ELB

   8. AWS/NetworkELB

   9. AWS/ApplicationELB

If you're not seeing metrics for a specific resource, it likely has no tags and is not in the list of services above. Contact us on Slack to help with resolving the issue.

Create an IAM role and policy

The following part requires two parameters:

• YOUR_GROUNDCOVER_ACCOUNT_ID - the AWS account id hosting the groundcover backend, created during onboarding

• GROUNDCOVER_SITE_ID - the groundcover site ID as extracted from your inCloud site endpoint:

  • Fetch your inCloud Site from these docs It will look like <SITE_ID>.platform.grcv.io

  • The GROUNDCOVER_SITE_ID is the first part marked above as <SITE_ID>

  • For example, if your inCloud Site address is m234r1.platform.grcv.io, then the GROUNDCOVER_SITE_ID will be m234r1.

1. Go to Amazon IAM

2. Click on Roles in the side bar

3. Click on Create Role

   1. Select Custom trust policy

   2. Paste the following policy:

      {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Principal": {
              "AWS": "arn:aws:iam::<YOUR_GROUNDCOVER_ACCOUNT_ID>:role/groundcover-integrations-agent-<GROUNDCOVER_SITE_ID>-sa"
            },
            "Action": "sts:AssumeRole"
          }
        ]
      }
      

   3. Click on Next twice (we'll attach permissions later)

   4. Provide a name for the role

   5. Click on Create Role

4. Go to your newly created role

   1. In the Permissions section, click on Add permissions and then Create inline policy

   2. Click on JSON and paste the following:

      {
          "Version": "2012-10-17",
          "Id": "groundcover-integrations-agent",
          "Statement": [
              {
                  "Action": [
                      "tag:GetResources",
                      "storagegateway:ListTagsForResource",
                      "storagegateway:ListGateways",
                      "shield:ListProtections",
                      "iam:ListAccountAliases",
                      "ec2:DescribeTransitGatewayAttachments",
                      "ec2:DescribeSpotFleetRequests",
                      "dms:DescribeReplicationTasks",
                      "dms:DescribeReplicationInstances",
                      "cloudwatch:ListMetrics",
                      "cloudwatch:GetMetricStatistics",
                      "cloudwatch:GetMetricData",
                      "autoscaling:DescribeAutoScalingGroups",
                      "aps:ListWorkspaces",
                      "apigateway:GET",
                      "s3:ListAllMyBuckets",
                      "s3:GetBucketLocation",
                      "s3:GetBucketTagging",
                      "sqs:ListQueues",
                      "sqs:GetQueueAttributes",
                      "rds:DescribeDBInstances",
                      "rds:DescribeDBClusters",
                      "lambda:ListFunctions",
                      "elasticache:DescribeCacheClusters",
                      "elasticache:DescribeServerlessCaches",
                      "dynamodb:ListTables",
                      "dynamodb:ListTagsOfResource",
                      "dynamodb:DescribeTable"
                  ],
                  "Effect": "Allow",
                  "Resource": "*"
              }
          ]
      }

   3. Click on Next

   4. Give the policy a name

   5. Click on Create Policy