Obfuscate Traces

Obfuscate Traces

Overview

Protect sensitive data in your traces by masking or removing it before storage. By integrating data obfuscation directly into your traces pipeline, you maintain privacy and meet compliance requirements while still retaining the necessary operational details.

Why Obfuscate Traces?

Traces often contain sensitive information in payloads, headers, and attributes:

• Personal Identifiable Information (PII) - emails, names, addresses in request/response bodies

• Credentials - API keys, tokens, passwords in headers or payloads

• Financial data - credit card numbers, account numbers in payment spans

• Internal system details - internal IPs, service tokens in headers

Obfuscating this data helps you:

• Meet compliance requirements (GDPR, PCI-DSS, HIPAA, etc.)

• Protect customer privacy

• Reduce security risks from leaked credentials

• Maintain audit trails while removing sensitive details

Obfuscation Approaches

There are four approaches to obfuscating sensitive data in traces:

1. Automatic PII Detection with obfuscate_pii

Automatically detect and redact sensitive data, no regex required. This is the recommended approach for broad coverage with minimal configuration.

Best for: Broad PII protection across many pattern types with zero regex effort

2. Masking with replace_pattern

Replace parts of a string with a masking token (e.g., replacing email characters with asterisks). Use this when you want to preserve the field structure while hiding the sensitive value.

Best for: Custom patterns not covered by obfuscate_pii, partial masking with capture groups

3. Removing with delete_key

Remove fields that contain sensitive data entirely. Use this when the field is not required for downstream analysis.

Best for: API keys, passwords, tokens, unnecessary PII in attributes

4. Value masking by key pattern with obfuscate_matching_keys

Mask the values of every entry in a map (e.g. request_headers) whose key matches a regex, while leaving the keys themselves in place. Unlike obfuscate_pii, which matches on value content, this matches on key name — so one rule covers every new key spelling that matches the regex (api-key, api_key, X-Api-Key, …), without a redeploy. A spelling outside the pattern still won't be caught, so keep the regex broad enough for the naming conventions you expect.

Best for: Header or attribute maps where the set of sensitive key names keeps growing and you don't want to enumerate them one by one

Best Practices

1. Apply obfuscation to the right scope - Target specific workloads, headers, or body fields rather than applying broadly

2. Be specific with patterns - Avoid over-matching by using precise regex patterns

3. Test thoroughly - Review rules carefully before deploying

4. Document your rules - Use clear ruleName values to explain what each rule protects

5. Balance utility and privacy - Mask data in a way that preserves operational value

6. Combine approaches - Use obfuscate_pii for broad coverage and replace_pattern for custom patterns

7. Order matters - Place obfuscation rules after transformation rules so useful fields are extracted first

Automatic PII Obfuscation

The obfuscate_pii function detects and redacts sensitive data across 17 built-in patterns — without writing any regex. It scans the specified field and replaces any detected PII in-place.

obfuscate_pii is designed for zero allocations when no PII is detected, making it safe for high-throughput pipelines.

Supported Patterns

Pattern	Category	Example	Min Match Length
email	personal_info	[e-mail]	6
credit_card	credit_card	4111-1111-1111-1111	13
ipv4_address	network_info	192.168.1.1	7
ipv6_address	network_info	::1	3
mac_address	network_info	11:22:33:44:55:66	17
url	network_info	https://example.com/path	10
jwt	auth_token	eyJhbGciOi...	20
bearer_token	auth_token	Bearer abc123xyz	10
basic_auth	auth_token	Basic dXNlcjpwYXNz	4
aws_credential	cloud_credential	AKIAIOSFODNN7EXAMPLE	20
azure_credential	cloud_credential	azure_key=ABCDE...	15
github_token	api_token	ghp_xxxx...	40
gitlab_token	api_token	glpat-xxxx...	26
slack_token	api_token	xoxb-xxxx...	15
google_api_key	api_token	AIzaXXXX...	39
stripe_key	api_token	sk_live_xxxx...	24
private_key	private_key	-----BEGIN RSA PRIVATE KEY-----	50

basic_auth behavior:

• Supported from groundcover version 1.12.104 and above.

• Matching is case-insensitive (for example Basic, basic, or BASIC).

• The canonical Basic prefix is preserved in the output.

• Only the Base64 credential token is masked.

• Detection sets pii_basic_auth_detected to "true".

Usage

obfuscate_pii(field, replacement, patterns)

Arguments:

Argument	Required	Description
field	Yes	The field to scan and obfuscate (e.g. request_body)
replacement	Yes	The string used to redact detected PII. A single ASCII character (e.g. "*") enables mask mode; a multi-character string uses replacement mode
patterns	Yes	Comma-separated list of pattern names to enable

Two replacement modes:

• Mask mode (single ASCII character, e.g. "*") — overwrites each byte of the match with the mask char, preserving the original length. Recommended for structured payloads since the surrounding JSON stays byte-aligned.

• Replacement mode (multi-character, e.g. "[R]") — replaces each match with the literal string and compacts the buffer (output is shorter than input). Length must be ≤ the shortest enabled pattern's min match length, or compilation fails.

Non-ASCII single runes (e.g. "•") are rejected since mask mode is byte-level.

Behavior change for single-character replacement (obfuscate_pii v1.11.814+)

A single ASCII character now selects mask mode (length-preserving) instead of replacement mode. If you were previously using replacement: "*" and relied on the buffer being compacted (each match collapsed to a single *), the output will now be a run of *s the same length as the original match.

To keep the old buffer-shrinking behavior, use a multi-character ASCII replacement that satisfies the min-match-length constraint (≤ the shortest enabled pattern), for example "**" or "[R]".

# Before (old behavior: each match collapsed to a single "*")
- 'obfuscate_pii(request_body, "*", "email")'

# After — same string, NEW behavior: each byte of the match overwritten with "*"
- 'obfuscate_pii(request_body, "*", "email")'

# To keep old behavior (multi-character → replacement mode + buffer compaction)
- 'obfuscate_pii(request_body, "**", "email")'

The same contract applies to logs — see Obfuscate Logs for details.

Mode Comparison: Before / After

The same input run through each mode produces different outputs. Given this request body:

{"user":"[e-mail]","card":"4111-1111-1111-1111"}

Mask mode with "*" — every PII byte is overwritten in place; quotes, commas, and braces stay at the same byte offsets, so the result is still valid JSON of the same length:

{"user":"*****************","card":"*******************"}

Replacement mode with "[R]" — each match is replaced by the literal string and the buffer compacts (output is shorter):

{"user":"[R]","card":"[R]"}

For trace bodies and headers that share storage with attribute strings, prefer mask mode so byte offsets are preserved and JSON parsing downstream is unaffected.

When PII is detected, obfuscate_pii automatically:

1. Replaces the matched content in the target field

2. Sets pii_<pattern>_detected = "true" as a span attribute

3. Sets is_pii = true on the span

Common Use Cases

Obfuscate Request Body (mask mode — recommended for JSON payloads)

Scan request payloads for emails and cloud credentials. Single-character replacement ("*") selects mask mode, so the body's byte length is preserved and any JSON structure stays parseable downstream.

ottlRules:
  - ruleName: obfuscate_request_body
    conditions:
      - request_body != nil
    statements:
      - obfuscate_pii(request_body, "*", "email,aws_credential")

💡 What it does: Scans the request body for emails and AWS credentials, overwriting each matched byte with *. Automatically sets is_pii = true and detection attributes.

Obfuscate Response Body (replacement mode example)

Same idea, but using a multi-character replacement ("[R]") — this is replacement mode, so each match collapses to the literal string [R] and the buffer compacts. Use this only when you don't need to preserve byte offsets or JSON structure.

ottlRules:
  - ruleName: obfuscate_response_body
    conditions:
      - response_body != nil
      - workload == "user-service"
    conditionLogicOperator: "and"
    statements:
      - obfuscate_pii(response_body, "[R]", "email,credit_card,ipv4_address")

💡 What it does: Scans response bodies from the user service for emails, credit cards, and IP addresses. Each match is replaced with [R] and the buffer is compacted (output shorter than input).

Obfuscate Authorization Headers

Redact bearer tokens and JWTs in request headers (mask mode).

ottlRules:
  - ruleName: obfuscate_auth_header
    conditions:
      - request_headers["authorization"] != nil
    statements:
      - obfuscate_pii(request_headers["authorization"], "*", "bearer_token,jwt")

💡 What it does: Detects and redacts bearer tokens and JWTs in the Authorization header, byte-for-byte.

Obfuscate Basic Authentication in Request Headers

Redact HTTP Basic Authentication credentials across all request headers while preserving the Basic scheme prefix:

ottlRules:
  - ruleName: obfuscate_basic_auth_headers
    conditions:
      - request_headers != nil
    statements:
      - obfuscate_pii(request_headers, "*", "basic_auth")

💡 What it does: Scans all request header values for Basic Auth credentials. Matching is case-insensitive; only the Base64 credential token is masked and pii_basic_auth_detected is set to "true".

Scan All Request Headers

Scan every entry in request headers for sensitive data (mask mode).

ottlRules:
  - ruleName: "scan_all_headers"
    conditions:
      - request_headers != nil
    statements:
      - obfuscate_pii(request_headers, "*", "bearer_token,jwt,aws_credential")

💡 What it does: Scans all request header values for tokens and credentials, without needing to specify individual header names.

Obfuscate Span Attributes

Redact sensitive values stored in span attributes (mask mode).

ottlRules:
  - ruleName: "obfuscate_user_email"
    conditions:
      - attributes["user_email"] != nil
    statements:
      - obfuscate_pii(attributes["user_email"], "*", "email")

💡 What it does: Detects and redacts email addresses in the user_email attribute, preserving the attribute's byte length.

---

Manual Obfuscation with replace_pattern

For custom patterns not covered by obfuscate_pii, use replace_pattern with regex.

Mask SSN in Attributes

ottlRules:
  - ruleName: mask_ssn
    conditions:
      - attributes["ssn"] != nil
    statements:
      - set(is_pii, true)
      - replace_pattern(attributes["ssn"], "[0-9]{3}-[0-9]{2}-[0-9]{4}", "XXX-XX-XXXX")

💡 Example: 123-45-6789 → XXX-XX-XXXX

Redact Passwords in Request Body

ottlRules:
  - ruleName: redact_passwords
    conditions:
      - request_body != nil
      - IsMatch(request_body, "password")
    conditionLogicOperator: "and"
    statements:
      - replace_pattern(request_body, "(\"password\"\\s*:\\s*\")[^\"]+\"", "$1[REDACTED]\"")

💡 Example: {"password": "secret123"} → {"password": "[REDACTED]"}

Redact Entire Response from Sensitive Server

ottlRules:
  - ruleName: redact_secret_server_responses
    conditions:
      - workload == "vault-service"
    statements:
      - set(response_body, "<REDACTED>")
      - set(is_pii, true)

💡 What it does: Completely replaces the response body for all spans from the vault service.

Redact Cookie Headers

ottlRules:
  - ruleName: redact_cookies
    statements:
      - set(request_headers["cookie"], "[REDACTED]") where request_headers["cookie"] != nil
      - set(response_headers["set-cookie"], "[REDACTED]") where response_headers["set-cookie"] != nil

💡 What it does: Removes cookie values from both request and response headers.

Removing Sensitive Attributes

Use delete_key to completely remove sensitive fields.

ottlRules:
  - ruleName: remove_sensitive_attributes
    conditions:
      - workload == "auth-service"
    statements:
      - delete_key(attributes, "api_key")
      - delete_key(attributes, "secret_token")
      - delete_key(attributes, "password_hash")

💡 What it does: Removes the entire api_key, secret_token, and password_hash attributes from auth service spans.

---

Masking by Key Pattern with obfuscate_matching_keys

For maps where the sensitive entries are identified by their key rather than their value — like headers — use obfuscate_matching_keys to mask the value of every entry whose key matches a regex, keeping the key itself untouched. It's the value-preserving sibling of the standard delete_matching_keys: same key-regex matching, but it redacts the value instead of deleting the entry.

Usage

obfuscate_matching_keys(target_map, key_pattern)
obfuscate_matching_keys(target_map, key_pattern, keep_prefix_n)

Arguments:

Argument	Type	Required	Description
target_map	Map	Yes	The map to scan, e.g. request_headers, response_headers, or attributes
key_pattern	String	Yes	Regex matched against each key in target_map. Compiled once when the rule is saved, so an invalid pattern fails immediately instead of on every record
keep_prefix_n	Int	No	Number of characters to keep from the start of the value before masking the rest. Defaults to 0, which masks the entire value

obfuscate_matching_keys supports request_headers, response_headers, and attributes. tags is read-only and can't be targeted.

Mask All Credential-Bearing Headers

Mask the value of any header whose name looks like an API key, secret, token, or passphrase — regardless of casing or separator — without listing each header individually:

ottlRules:
  - ruleName: obfuscate_credential_headers
    statements:
      - obfuscate_matching_keys(request_headers, "(?i).*(api[-_]?key|secret|token|passphrase)$")

💡 What it does: Matches header names such as X-Api-Key, api_key, Auth-Token, or client-secret and masks each matching value, leaving the header names in place.

Partially Mask Matching Attributes

Keep the first few characters of the value visible for debugging while masking the rest:

ottlRules:
  - ruleName: obfuscate_internal_tokens
    statements:
      - obfuscate_matching_keys(attributes, "(?i)_token$", 4)

💡 What it does: For every attribute whose key ends in _token, keeps the first 4 characters of the value and masks the remainder (e.g. sk_l****************).

Key Functions

obfuscate_pii

Automatically detects and redacts sensitive data across built-in PII patterns.

Syntax:

- 'obfuscate_pii(field, "replacement", "pattern1,pattern2,...")'

Supported targets:

• request_body / response_body — scan payloads

• attributes["key"] — scan a specific attribute

• attributes — scan ALL attributes

• request_headers["key"] — scan a specific header

• request_headers / response_headers — scan ALL headers

Available patterns: email, credit_card, ipv4_address, ipv6_address, mac_address, url, jwt, bearer_token, basic_auth, aws_credential, azure_credential, github_token, gitlab_token, slack_token, google_api_key, stripe_key, private_key

replace_pattern

Replaces text matching a pattern with a replacement string.

Syntax:

- 'replace_pattern(target_field, "pattern", "replacement")'

With capture groups:

- 'replace_pattern(target_field, "(keep_this)remove_this(keep_this)", "$1****$2")'

delete_key

Completely removes a field from the attributes.

Syntax:

- 'delete_key(attributes, "field_name")'

obfuscate_matching_keys

Masks the values of map entries whose keys match a regex, keeping the keys in place.

Syntax:

- 'obfuscate_matching_keys(target_map, "key_pattern")'
- 'obfuscate_matching_keys(target_map, "key_pattern", keep_prefix_n)'

Supported targets: request_headers, response_headers, attributes