Create a new Monitor

Learn how to create and configure monitors using the Wizard, Monitor Catalog, or Import options. The following guide will help you set up queries, thresholds, and alert routing for effective monitoring.

In the Monitors section (left navigation bar), navigate to the Issues page or the Monitor List page to create a new Monitor. Click on the blue “Create Monitor” button and select one of the following options from the dropdown:

Using the Monitor Wizard

Overview

The Monitor Wizard is a guided, user-friendly approach to creating and configuring monitors tailored to your observability needs. By breaking down the process into simple steps, it ensures consistency and accuracy.

Section 1: Information

Set up the basic information for the monitor.

Monitor Title (Required):

Add a title for the monitor. The title will appear in notifications and in the Monitor List page.

Description (Optional):

Add description for your monitor, The description will appear when viewing monitor details, you can also use this for your alerts.

Section 2: Query

Select the data source, build the query and define thresholds for the monitor.

• Data Source (Required):

  • Select the type of data (Metrics, Infra Metrics, Logs, or Traces).

• Query Functionality:

  • Choose how to process the data (e.g., average, count).

  • Add group-by clauses if applicable.

• Time Window (Required):

  • Specify the period over which data is aggregated.

  • Example: “Over the last 5 minutes.”

• Threshold Conditions (Required):

  • Define when the monitor triggers. You can use:

    • Greater Than - Trigger when the value exceeds X.

    • Lower Than - Trigger when the value falls below X.

    • Within Range - Trigger when the value is between X and Y.

    • Outside Range - Trigger when the value is not between X and Y.

  • Example: “Trigger if disk space usage is greater than 10%.”

• Visualization Type (Optional):

  • Preview data using Stacked Bar or Line Chart for better clarity. This is just for helping visualize while building the monitor.

Section 3: Display

Customize how the Monitor’s Issues will appear. This section also includes a live preview of the way it will appear in the Issues page.

Issue Header (required):

Define a name for issues that this Monitor will raise. It's useful to use labels that can include information from the query.

For example, adding {{ alert.labels.statusCode }} to the header will inject the status code to the name of the issue - this becomes especially useful when one Monitor raises multiple issues and you want to quickly understand their content without having to open each one.

Severity (required):

Use severity to categorize alerts by importance.

Select a severity level (S1-S4).

Resource Labels (optional):

The labels here should give you team the context of what is the subject of the issue.

Examples:

• span_name for an API based monitor

• pod_name for a Pod Crash monitor

Context Labels (optional):

The labels here should give you team the context of where the issue happened.

• Examples:

  • cluster

  • namespace

Section 4: Metadata Labels

Organize and categorize monitors, you can use these to route issues using advanced workflows.

• Labels (optional):

  • Add key-value pairs for metadata.

Section 5: Evaluation Settings

Define how often the monitor evaluates its conditions.

Evaluation Interval (Required):

Specify how often the monitor evalutes the query

Example: “Evaluate every 1 minute.”

Pending Period (Required):

This ensures that transient conditions do not trigger alerts, reducing false positives. For example, setting this to 10 minutes ensures the condition must persist for at least 10 minutes before firing.

Example: “Wait for 10 minute before alerting.t

Section 6: Routing

Set up how issues from this monitor will be routed.

Select Workflow (Optional):

Route alerts to existing workflows only, this means that other workflows will not process them. Use this to send alerts for a critical application to Slack or PagerDuty.

No Routing (Optional):

This means that any workflow (without filters), will process the issue.

Quick tips to create effective Monitors

Use the Monitor Catalog as much as you can

Whenever possible, use our carefully crafted monitors from the Monitor Catalog. This will save you time, ensure the Monitors are built effectively, and help you align your alerting strategy with best practices. If you can't find one that perfectly matches your needs, use them as your starting point and edit their properties to customize them to your needs.

Give the Monitor a short and clear title

Give the Monitor a clear, short name, that describes its function at a high level.

Examples:

• “Workload High API Error Rate”

• “Workload Pods High Memory”

Use a Descriptive Issue Header

Choose a clear name for the Issue header, offering a bit more details and going into a more specific description of the monitor name. A Header is a specific property of an issue, so you can add templated dynamic values here. For example, you can use dynamic label values in the header name.

Examples:

• “HTTP API Error {{ alert.labels.status_code }}”,

• “Workload {{ alert.labels.workload }} Pod Restart”

• “{{ alert.labels.customer }} APIs High Latency”.

Use up to 3 Resource Labels

We recommend using up to 3 ResourceHeaderLabels. The labels here should give your team the context of what is the subject of the issue.

Examples:

span_name , pod_name

Use up to 3 Context Labels

We recommend using up to 3 ContextHeaderLabels. The labels here should give you team the context of where the issue happened.

Examples:

cluster, namespace , workload

Using the Import option

Here you can add multiple Monitors using an array of Monitors that follows the Monitor YAML structure.

monitors:
- title: HTTP API Errors Monitor
  ...
- title: K8s Node Low Disk Space Monitor
  ...

Click on "Create Monitors" to create them.