Workflows

circle-exclamation

Migrating from Workflows to Notification Routes

The new notification system (Connected Apps + Notification Routes) offers several advantages:

  • Simpler configuration - No YAML required, use the UI wizard

  • Better reliability - Improved notification delivery and link handling

  • Automatic label passing - Labels are automatically included without complex templating

  • Testing capabilities - Test notifications before enabling on production monitors

  • Terraform support - Fully supported via Terraform provider

Migration Steps

  1. Create Connected Apps (Settings → Connected Apps)

    • Set up your Slack webhook, PagerDuty, OpsGenie, or other destinations

    • Admin permissions required for creating Connected Apps

  2. Create Notification Routes (Monitors → Notification Routes)

    • Define scope using gcQL (e.g., env:prod AND severity:S1)

    • Add rules for Firing and/or Resolved states

    • Select your Connected Apps as destinations

  3. Disable old Workflows

    • To disable a workflow without deleting it, add a filter that matches nothing (e.g., key: _never_match_)

Template Variable Changes

The new system uses a simplified variable syntax:

Old Workflows Syntax
New Connected Apps Syntax

{{ alert.labels.workload }}

{{ labels.workload }}

{{ alert.annotations._gc_severity }}

{{ severity }}

{{ alert.alertname }}

{{ alertname }}

{{ alert.fingerprint }}

{{ fingerprint }}

circle-info

Note: The old alert.labels.* format is still supported in Connected Apps, so you don't need to update your existing monitor templates when migrating. However, the new simplified syntax is recommended for new configurations.

Common Migration Issues

Issue links not working: Old workflows generate links that may not resolve correctly. Add ?resolve_timerange=true to your workflow URL templates, or migrate to Notification Routes which handle this automatically.

Labels not appearing: Ensure you're using the correct variable syntax with the labels. prefix.


Legacy Workflows Documentation

Workflows are YAML-based configurations that are executed whenever a monitor is triggered. They enable you to integrate with third-party systems, apply custom logic to format and transform data, and set different conditions to handle your monitoring alerts intelligently.

Workflow components

Triggers

Triggers apply filtering conditions that determine whether a specific workflow is executed. In groundcover, the trigger type is always set to "alert".

Example: This trigger ensures that only monitors fired with telemetry data from the Prod environment will actually execute the workflow. Note that the "env" attribute needs to be provided as a context label from the monitor:

Note: Workflows are "pull based" which means they will try to match monitors even when these monitors did not explicitly add a specific workflow. Therefore, the filters need to accurately define the condition to be used for a monitor.

Consts

Consts is a section where you can declare predefined attributes based on data provided with the monitor context. A set of functions is available to transform existing data and format it for propagation to third-party integrations. Consts simplify access to data that is needed in the actions section.

Example: The following example shows how to map a predefined set of severity values to the monitor severity as defined in groundcover - here, any potential severity in groundcover is translated into one of P1-P5 values.

The function keep.dictget gets a value from a map (dictionary) using a specific key. In case the key is not found - P3 will be the default severity:

Actions

Actions specify what happens when a workflow is triggered. Actions typically interface with external systems (like sending a Slack message). Actions can be an array of actions, they can be executed conditionally and include the integration in their config part as well as a payload block which is typically dependent on the exact integration used for the notification.

Actions include:

  1. Provider part (provider:) - Configures the integration to be used

  2. Payload part (with:) - Contains the data to submit to the integration based on its actual structure

Example: In this example you can see a typical Slack notification. Note that the actual integration is referenced through the 'providers' context attribute. The integration name is the exact string used to create the integration (in this case "groundcover-alerts").

Last updated