# Custom logs collection

By default, groundcover stores logs from all namespaces and workloads in your cluster. However there are multiple ways to modify this behavior.

### Filtering Rules

groundcover allows you to add **logs filtering rules** using LogQL syntax by creating a custom `values.yaml` file.\
The available labels to filter are: `namespace, workload, pod, level, container.`\
Example of filtering out all logs coming from namespace `demo` with level `info`:\
`{namespace="demo",level="info"}`\
In addition, we enable the use of the optional log stream pipeline in order filter the log lines.\
Example of filtering out all logs coming from container `my-container` which contain the word `fifo` or `handler`:\
`{container="my-container"} |~ "fifo|handler"`

More info on LogQL syntax can be found [here](https://grafana.com/docs/loki/latest/logql/log_queries/).

{% hint style="info" %}
Rules are applied sequentially and independently. Therefore, rules which are meant to specify multiple values of the same label should be written as one rule with multiple options, and not many rules with one option each.\
\
For example, a rule to drop logs from all namespaces except `prod` and `dev` should be written as:\
\
`{namespace!="prod", namespace!="dev"}`
{% endhint %}

## Usage

#### values.yaml example

```yaml
logsDropFilters:
 - '{namespace="demo-ng",workload="loadgenerator"} |~ ".*GET.*"'
 - '{namespace="demo-ng",workload="currencyservice"} !~ "received"'
```

#### Using CLI on New or Existing Installation

```bash
groundcover deploy --values values.yaml
```

#### Using Helm

```bash
helm upgrade -i  \
    groundcover \
    groundcover/groundcover \
    -n groundcover \
    -i \
    -f values.yaml
```

{% hint style="info" %}
If upgrading an existing installation with different values, consider passing `--reuse-values`.
{% endhint %}

## Configure Journal Logs

groundcover collects kubelet logs on Kubernetes clusters and docker logs on host machines.\
You can customize this behavior through additional configuration options.

### Usage

{% tabs %}
{% tab title="Helm Values" %}

```yaml
journalScraper:
  serviceNames: ["kubelet", "kernel"]  # List of systemd service units to capture logs from
  historyDuration: 10m                 # How far back in time to collect logs when discovering new files
  journalFilter: "system.journal"      # Journal file pattern to match (e.g., "*.journal")
  includeRemoteJournals: true          # Enable collection of journals beyond the current machine-id
```

{% endtab %}

{% tab title="Host Overrides" %}
{% code title="/etc/opt/groundcover/overrides.yaml" %}

```yaml
logs:
  scraper:
    journalConfig:
      - serviceNames: ["docker", "sshd", "kernel", "groundcover-sensor"]
        journalFilter: "*.journal"
        includeRemoteJournals: false
```

{% endcode %}
{% endtab %}
{% endtabs %}

## Configure Log File Targets

groundcover can collect logs from specific files on your host machine.\
You can define paths to monitor and add custom labels to the collected logs.

### Usage

{% tabs %}
{% tab title="Helm Values" %}

```yaml
logFileTargets:
  - path: "/var/log/syslog.*" # Path pattern to match log files
    excludedPath: "*.gz"      # Pattern to exclude from matched files
    labels:                   # Custom labels to attach to collected logs
      label1: "value1"
    workload: "syslog"        # Optional: Set a custom workload name
```

{% endtab %}

{% tab title="Host Overrides" %}
{% code title="/etc/opt/groundcover/overrides.yaml" %}

```yaml
logs:
  scraper:
    logFileTargets:
      - path: /var/log/auth.log
```

{% endcode %}
{% endtab %}
{% endtabs %}

## Customize logs decolorization

This feature enable removing ANSI color codes from logs' body.

### Usage

```yaml
decolorizeLogs: true
```

### Example

```
[2023-10-31 20:38:57.123] \033[0;32HTTP\033[0m: GET /_healthz (1 ms) 204
```

Will be stripped into:

```
[2023-10-31 20:38:57.123] HTTP: GET /_healthz (1 ms) 204
```

## Control logs truncation

{% hint style="warning" %}
Use this customization carefully as it might heavily effect the performance of the groundcover sensors.
{% endhint %}

### Body VS Content:

During log parsing groundcover generates two attributes named `content` and `body`:

1. `body` - contains the full log line
2. `content` - contains the message field of structured logs (from msg/message attribute) or the full log line for unstructured logs

In the platform UI the attribute displayed is the `content`, while `body` is available in the DB.

#### Example:

Formatted log with message:\
`{"time": "Jun 09 2023 15:28:14", "severity": "info", "msg": "Hello World"}`

{% code fullWidth="false" %}

```yaml
Body: {"time": "Jun 09 2023 15:28:14", "severity": "info", "msg": "Hello World"}
Content: "Hello World"
```

{% endcode %}

Unformatted log:\
`[Jun 09 2023 15:28:14], Hello World`

```yaml
Body: "[Jun 09 2023 15:28:14], Hello World"
Content: "[Jun 09 2023 15:28:14], Hello World"
```

### Truncation Config:

The following values contain the default truncation size for `body` and `content` respectively:

{% hint style="warning" %}
Up to sensor version 1.11.113 the max content size was 5KB, and increased to 15KB in version 1.11.114
{% endhint %}

```yaml
maxLogSize: 102400 # 100KB
maxLogContentSize: 15360 # 15KB
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.groundcover.com/customization/customize-usage/custom-logs-collection.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.