Pipeline Operations

Overview

Pipeline operations transform query results using the pipe operator |. Chain multiple operations together to filter, sort, limit, rename fields, and more. Operations are executed sequentially from left to right.

Syntax

<query> | <operation1> | <operation2> | <operation3>

Each operation processes the output of the previous step.

sort

Order results by one or more fields.

Syntax:

| sort by (field1 [asc|desc], field2 [asc|desc], ...)

Single field:

* | stats by (workload) count() as total | sort by (total desc)

Multiple fields:

* | stats by (workload) count() | sort by (workload asc, count desc)

limit

Limit the number of results returned.

Syntax:

| limit N

* | limit 100

offset

Skip a number of results (for pagination).

Syntax:

| offset N

* | sort by (timestamp desc) | offset 20 | limit 10

Records 21-30 (page 3)

fields

Select specific fields to include in results (similar to SQL SELECT).

Note: The fields pipe does not work with time-series visualizations. For time-series queries, all fields from the aggregation are included automatically.

Syntax:

| fields field1, field2, field3, ...

* | fields timestamp, level, message

rename

Rename fields for clarity or compatibility.

Syntax:

| rename old_name1 as new_name1, old_name2 as new_name2, ...

* | rename aa as service_name, bb as region

format

Format output strings using a template with field values.

Syntax:

| format "template with <field1> and <field2>"

* | format "request from <ip>:<port> is sent"

uniq

Get unique combinations of specified fields (similar to SQL DISTINCT + GROUP BY).

Syntax:

| uniq (field1, field2, ...) [limit N]

* | uniq (workload)

Unique workloads

filter

Apply additional filters after aggregations or transformations.

Syntax:

| filter <filter_expression>

Aliases: | <filter_expression> (implicit)

* | stats by (workload) count() as total | filter total:>1000

Workloads with more than 1000 logs

math

Perform mathematical calculations on fields and create new calculated fields.

Syntax:

| math <expression> <result_field>

Arithmetic Operators:

• + - Addition

• - - Subtraction

• * - Multiplication

• / - Division

• % - Modulo

• ^ - Power

Functions:

• max(a, b) - Maximum of two values

• min(a, b) - Minimum of two values

• abs(x) - Absolute value

• exp(x) - Exponential (e^x)

• ln(x) - Natural logarithm

• round(x) - Round to nearest integer

• ceil(x) - Ceiling (round up)

• floor(x) - Floor (round down)

• rand() - Random number

• now() - Current timestamp

Basic Arithmetic

* | stats by (resource) count() requests, sum(errors) errors | math errors / requests error_rate

Calculate error rate

Complex Expressions

Operator precedence follows standard math rules (multiplication/division before addition/subtraction).

* | count() a, count_uniq(workload) b | math a / b - 3 result

Expression with precedence: (a / b) - 3

* | stats by (node) avg(cpu) cpu, avg(memory) mem | math cpu * 0.7 + mem * 0.3 health_score

Weighted health score

With Functions

* | stats by (workload) avg(value) avg_val | math round(avg_val) rounded_value

Round to integer

Chaining Math Operations

* | count() a, count_uniq(workload) b | math a / b ratio | math ratio * 100 percentage

Chain multiple math operations

With Join

level:info | stats by (workload) count() info_count 
| join by (workload) (level:error | stats by (workload) count() error_count) 
| math error_count / info_count error_ratio

Calculate ratio from joined data