# Shipping from the DataDog Agent

## Overview

groundcover supports ingestion of data from the DataDog agent, you have two options:

1. **Dual Shipping** - continue using DataDog normally while ingesting your traces and metrics in groundcover.
2. **Single Shipping** - Send from the DataDog agent to groundcover exclusively.

### When to use this feature

This feature is mostly used in two scenarios:

1. Evaluation of groundcover - showcasing all data available in the groundcover platform while maintaining your existing stack. In this case you likely need the **dual shipping** method.
2. Post evaluation of groundcover - as you turn off other solutions. Also, for supporting teams migrating from DataDog SDKs to open source standards like OpenTelemetry. You can work on the new SDKs while still using the old ones, and even compare them side-by-side in the platform.

### What data will be received in groundcover?

1. Traces - allowing you to use the DataDog distributed traces generated by your applications
2. APM metrics - allowing you to use DataDog APM metrics (e.g `trace.`) common in monitors and dashboards
3. Custom metrics - sent by your applications using DogStatsD

### About Sampling

No sampling is done when data is sent from the DataDog agent.

## Setting up the Datadog Agent

### Kubernetes

The recommended method to Dual Ship data in Kubernetes is by sending it to groundcover components inside the cluster

* Traces and APM will be forwarded to the `groundcover-sensor` service
* Custom metrics will be forwarded to the `groundcover-custom-metrics` service

#### Turning on Custom Metrics

Add the following values to your deployment configuration:

```yaml
custom-metrics:
  enabled: true
```

#### Configuring the DataDog Kubernetes/Host Agent

Add the following to the DataDog agent configuration:

{% hint style="success" %}
Leave `groundcover-nokeyneeded` as is - do not replace with a real API key
{% endhint %}

#### Dual Shipping

```yaml
datadog:  
  env:
    - name: "DD_APM_ADDITIONAL_ENDPOINTS"
      value: "{\"http://groundcover-sensor.groundcover.svc.cluster.local:8126\": [\"groundcover-nokeyneeded\"]}"
    - name: "DD_ADDITIONAL_ENDPOINTS"
      value: "{\"http://groundcover-custom-metrics.groundcover.svc.cluster.local:8429/datadog\": [\"groundcover-nokeyneeded\"]}"
```

#### Single Shipping

```yaml
datadog:  
  env:
    - name: "DD_APM_DD_URL"
      value: "http://groundcover-sensor.groundcover.svc.cluster.local:8126"
    - name: "DD_DD_URL"
      value: "http://groundcover-custom-metrics.groundcover.svc.cluster.local:8429/datadog"
    - name: "DD_API_KEY"
      value: "groundcover-nokeyneeded"
```

### Non-Kubernetes

#### Get your BYOC site endpoint

Find your BYOC endpoint in the [ingestion keys tab](https://app.groundcover.com/settings?selectedTab=ingestion-keys).

#### Create an ingestion key

See how in the [ingestion key docs](https://docs.groundcover.com/use-groundcover/remote-access-and-apis/ingestion-keys).

#### Configuring the DataDog Non-Kubernetes Agent

Add the following to the DataDog agent configuration:

{% hint style="success" %}
Don't forget to replace "{groundcover-ingestion-key}" with the real ingestion key created.
{% endhint %}

#### Dual Shipping

```bash
DD_APM_ADDITIONAL_ENDPOINTS='{\"https://{BYOC_ENDPOINT}\": [\"{groundcover-ingestion-key}\"]}'
DD_ADDITIONAL_ENDPOINTS='{\"https://{BYOC_ENDPOINT}/datadog\": [\"{groundcover-ingestion-key}\"]}'
```

#### Single Shipping

```bash
DD_APM_DD_URL='https://{BYOC_ENDPOINT}'
DD_DD_URL='https://{BYOC_ENDPOINT}/datadog'
DD_API_KEY='{groundcover-ingestion-key}'
```