# Configuring Pipelines

groundcover supports the configuration of logs and traces pipelines, to further process and customize the data being collected, using [Vector](https://vector.dev/) transforms. This enables full flexibility to manipulate the data as it flows into the platform.

{% hint style="success" %}
See [this page](/~/revisions/1mY0CQn12Klyi0QiItHv/architecture/overview.md#detailed-architecture) for more information about how Vector is being used in the groundcover platform's architecture.
{% endhint %}

## Using Vector transforms

groundcover uses Vector as an aggregator and transformer deployed into each monitored environment. It is an open-source, highly performant service, capable of supporting many manipulations on the data flowing into groundcover's backend.

Pipelines are configured using [Vector transforms](https://vector.dev/docs/reference/configuration/transforms/), where each transform defines one step in the pipeline. There are many types of transforms, and all of them can be natively used within the groundcover deployment to achieve full flexibility.

{% hint style="info" %}
The most common transform is the [`remap`](https://vector.dev/docs/reference/configuration/transforms/remap/) transform - allowing to write arbitrary logic using Vector's [VRL ](https://vector.dev/docs/reference/vrl/)syntax. There are many pre-defined functions to parse, filter and enrich data, and we recommend experimenting with it to fit your needs.

\
For testing out VRL before deployment we recommend the [VRL playground](https://playground.vrl.dev/).
{% endhint %}

## Deploying groundcover with Pipelines

groundcover's deployment supports adding a list of transforms for logs and traces independently. These steps will be automatically appended into the default pipeline, eliminating the need to understand the inner workings of grouncover's setup. Instead, you only need to configure the steps you wish to execute, and after redeploying groundcover you will see them take effect immediately.

Each step requires two attributes:

* **name**: must be unique across all pipelines
* **transform**: the transform itself, passed as-is to Vector.

### Logs Pipeline

The following is a template for a logs pipeline with two remap stages:

```
vector:
  logsPipeline:
    extraSteps: 
    - name: stepA
      transform:
        type: remap
        source: |-
          ...
    - name: stepB
      transform:
        type: remap
        source: |-
          ...
```

#### [View an example with real inputs ->](https://github.com/groundcover-com/docs/blob/main/use-groundcover/configuring-pipelines/broken-reference/README.md)

### Traces Pipeline

The following is a template for a traces pipeline with one filter stage:

```
vector:
  tracesPipeline:
    extraSteps: 
    - name: stepA
      transform:
        type: filter
        condition: |- 
           ...
```

#### [View an example with real inputs ->](/~/revisions/1mY0CQn12Klyi0QiItHv/use-groundcover/configuring-pipelines/traces-pipeline-examples.md)

### Custom Logs to Events Pipeline

Logs to Events pipelines allow creating of custom events from incoming logs. Unlike the logs and traces pipelines, they do not affect the original logs, and are meant to create parallel, distinguished events for future analytics.

The following is a template for a custom event pipeline with a filter stage and an extraction step.

{% hint style="info" %}
The `inputs` fields below will connect the events pipeline with the default incoming logs pipelines.
{% endhint %}

```yaml
vector:
  eventsPipelines:
    my_event_name:
      inputs:
        - logs_from_logs
        - json_logs
      extraSteps:
        - name: filter_step
          transform:
            type: filter
            condition: |-
              ...
        - name: extraction_step
          transform:
            type: remap
            source: |-
              ...
          
```

#### [View an example with real inputs](/~/revisions/1mY0CQn12Klyi0QiItHv/use-groundcover/configuring-pipelines/logs-to-events-pipeline-examples.md)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.groundcover.com/~/revisions/1mY0CQn12Klyi0QiItHv/use-groundcover/configuring-pipelines.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.