# Obfuscate Traces ## Obfuscate Traces **Overview** Protect sensitive data in your traces by masking or removing it before storage. By integrating data obfuscation directly into your traces pipeline, you maintain privacy and meet compliance requirements while still retaining the necessary operational details. **Why Obfuscate Traces?** Traces often contain sensitive information in payloads, headers, and attributes: * **Personal Identifiable Information (PII)** - emails, names, addresses in request/response bodies * **Credentials** - API keys, tokens, passwords in headers or payloads * **Financial data** - credit card numbers, account numbers in payment spans * **Internal system details** - internal IPs, service tokens in headers Obfuscating this data helps you: * **Meet compliance requirements** (GDPR, PCI-DSS, HIPAA, etc.) * **Protect customer privacy** * **Reduce security risks** from leaked credentials * **Maintain audit trails** while removing sensitive details **Obfuscation Approaches** There are three approaches to obfuscating sensitive data in traces: **1. Automatic PII Detection with obfuscate\_pii** Automatically detect and redact sensitive data, no regex required. This is the **recommended approach** for broad coverage with minimal configuration. **Best for:** Broad PII protection across many pattern types with zero regex effort **2. Masking with replace\_pattern** Replace parts of a string with a masking token (e.g., replacing email characters with asterisks). Use this when you want to preserve the field structure while hiding the sensitive value. **Best for:** Custom patterns not covered by `obfuscate_pii`, partial masking with capture groups **3. Removing with delete\_key** Remove fields that contain sensitive data entirely. Use this when the field is not required for downstream analysis. **Best for:** API keys, passwords, tokens, unnecessary PII in attributes **Best Practices** 1. **Apply obfuscation to the right scope** - Target specific workloads, headers, or body fields rather than applying broadly 2. **Be specific with patterns** - Avoid over-matching by using precise regex patterns 3. **Test thoroughly** - Review rules carefully before deploying 4. **Document your rules** - Use clear `ruleName` values to explain what each rule protects 5. **Balance utility and privacy** - Mask data in a way that preserves operational value 6. **Combine approaches** - Use `obfuscate_pii` for broad coverage and `replace_pattern` for custom patterns 7. **Order matters** - Place obfuscation rules after transformation rules so useful fields are extracted first **Automatic PII Obfuscation** The `obfuscate_pii` function detects and redacts sensitive data across 16 built-in patterns — without writing any regex. It scans the specified field and replaces any detected PII in-place. {% hint style="success" %} `obfuscate_pii` is designed for **zero allocations** when no PII is detected, making it safe for high-throughput pipelines. {% endhint %} **Supported Patterns** | Pattern | Category | Example | Min Match Length | | ------------------ | ----------------- | --------------------------------- | ---------------- | | `email` | personal\_info | `user@example.com` | 6 | | `credit_card` | credit\_card | `4111-1111-1111-1111` | 13 | | `ipv4_address` | network\_info | `192.168.1.1` | 7 | | `ipv6_address` | network\_info | `::1` | 3 | | `mac_address` | network\_info | `11:22:33:44:55:66` | 17 | | `url` | network\_info | `https://example.com/path` | 10 | | `jwt` | auth\_token | `eyJhbGciOi...` | 20 | | `bearer_token` | auth\_token | `Bearer abc123xyz` | 10 | | `aws_credential` | cloud\_credential | `AKIAIOSFODNN7EXAMPLE` | 20 | | `azure_credential` | cloud\_credential | `azure_key=ABCDE...` | 15 | | `github_token` | api\_token | `ghp_xxxx...` | 40 | | `gitlab_token` | api\_token | `glpat-xxxx...` | 26 | | `slack_token` | api\_token | `xoxb-xxxx...` | 15 | | `google_api_key` | api\_token | `AIzaXXXX...` | 39 | | `stripe_key` | api\_token | `sk_live_xxxx...` | 24 | | `private_key` | private\_key | `-----BEGIN RSA PRIVATE KEY-----` | 50 | **Usage** ```yaml obfuscate_pii(field, replacement, patterns) ``` **Arguments:** | Argument | Required | Description | | ------------- | -------- | ----------------------------------------------------- | | `field` | Yes | The field to scan and obfuscate (e.g. `request_body`) | | `replacement` | Yes | The string to replace detected PII with | | `patterns` | Yes | Comma-separated list of pattern names to enable | {% hint style="warning" %} The **replacement string** must be shorter than or equal to the minimum match length across all enabled patterns. For example, if `ipv6_address` is enabled (min match length 3), the replacement must be <= 3 characters. {% endhint %} **When PII is detected, `obfuscate_pii` automatically:** 1. Replaces the matched content in the target field 2. Sets `pii__detected = "true"` as a span attribute 3. Sets `is_pii = true` on the span **Common Use Cases** **Obfuscate Request Body** Scan request payloads for emails and cloud credentials. ```yaml ottlRules: - ruleName: obfuscate_request_body conditions: - request_body != nil statements: - obfuscate_pii(request_body, "[R]", "email,aws_credential") ``` 💡 **What it does:** Scans the request body for emails and AWS credentials, replacing them with `[R]`. Automatically sets `is_pii = true` and detection attributes. **Obfuscate Response Body** Protect sensitive data in API responses. ```yaml ottlRules: - ruleName: obfuscate_response_body conditions: - response_body != nil - workload == "user-service" conditionLogicOperator: "and" statements: - obfuscate_pii(response_body, "***", "email,credit_card,ipv4_address") ``` 💡 **What it does:** Scans response bodies from the user service for emails, credit cards, and IP addresses. **Obfuscate Authorization Headers** Redact bearer tokens and JWTs in request headers. ```yaml ottlRules: - ruleName: obfuscate_auth_header conditions: - request_headers["authorization"] != nil statements: - obfuscate_pii(request_headers["authorization"], "[R]", "bearer_token,jwt") ``` 💡 **What it does:** Detects and redacts bearer tokens and JWTs in the Authorization header. **Scan All Request Headers** Scan every entry in request headers for sensitive data. ```yaml ottlRules: - ruleName: "scan_all_headers" conditions: - request_headers != nil statements: - obfuscate_pii(request_headers, "[R]", "bearer_token,jwt,aws_credential") ``` 💡 **What it does:** Scans all request header values for tokens and credentials, without needing to specify individual header names. **Obfuscate Span Attributes** Redact sensitive values stored in span attributes. ```yaml ottlRules: - ruleName: "obfuscate_user_email" conditions: - attributes["user_email"] != nil statements: - obfuscate_pii(attributes["user_email"], "[R]", "email") ``` 💡 **What it does:** Detects and redacts email addresses in the `user_email` attribute. *** **Manual Obfuscation with replace\_pattern** For custom patterns not covered by `obfuscate_pii`, use `replace_pattern` with regex. **Mask SSN in Attributes** ```yaml ottlRules: - ruleName: mask_ssn conditions: - attributes["ssn"] != nil statements: - set(is_pii, true) - replace_pattern(attributes["ssn"], "[0-9]{3}-[0-9]{2}-[0-9]{4}", "XXX-XX-XXXX") ``` 💡 **Example:** `123-45-6789` → `XXX-XX-XXXX` **Redact Passwords in Request Body** ```yaml ottlRules: - ruleName: redact_passwords conditions: - request_body != nil - IsMatch(request_body, "password") conditionLogicOperator: "and" statements: - replace_pattern(request_body, "(\"password\"\\s*:\\s*\")[^\"]+\"", "$1[REDACTED]\"") ``` 💡 **Example:** `{"password": "secret123"}` → `{"password": "[REDACTED]"}` **Redact Entire Response from Sensitive Server** ```yaml ottlRules: - ruleName: redact_secret_server_responses conditions: - workload == "vault-service" statements: - set(response_body, "") - set(is_pii, true) ``` 💡 **What it does:** Completely replaces the response body for all spans from the vault service. **Redact Cookie Headers** ```yaml ottlRules: - ruleName: redact_cookies statements: - set(request_headers["cookie"], "[REDACTED]") where request_headers["cookie"] != nil - set(response_headers["set-cookie"], "[REDACTED]") where response_headers["set-cookie"] != nil ``` 💡 **What it does:** Removes cookie values from both request and response headers. **Removing Sensitive Attributes** Use `delete_key` to completely remove sensitive fields. ```yaml ottlRules: - ruleName: remove_sensitive_attributes conditions: - workload == "auth-service" statements: - delete_key(attributes, "api_key") - delete_key(attributes, "secret_token") - delete_key(attributes, "password_hash") ``` 💡 **What it does:** Removes the entire `api_key`, `secret_token`, and `password_hash` attributes from auth service spans. **Key Functions** **obfuscate\_pii** Automatically detects and redacts sensitive data across built-in PII patterns. **Syntax:** ```yaml - 'obfuscate_pii(field, "replacement", "pattern1,pattern2,...")' ``` **Supported targets:** * `request_body` / `response_body` — scan payloads * `attributes["key"]` — scan a specific attribute * `attributes` — scan ALL attributes * `request_headers["key"]` — scan a specific header * `request_headers` / `response_headers` — scan ALL headers **Available patterns:** `email`, `credit_card`, `ipv4_address`, `ipv6_address`, `mac_address`, `url`, `jwt`, `bearer_token`, `aws_credential`, `azure_credential`, `github_token`, `gitlab_token`, `slack_token`, `google_api_key`, `stripe_key`, `private_key` **replace\_pattern** Replaces text matching a pattern with a replacement string. **Syntax:** ```yaml - 'replace_pattern(target_field, "pattern", "replacement")' ``` **With capture groups:** ```yaml - 'replace_pattern(target_field, "(keep_this)remove_this(keep_this)", "$1****$2")' ``` **delete\_key** Completely removes a field from the attributes. **Syntax:** ```yaml - 'delete_key(attributes, "field_name")' ```