> For the complete documentation index, see [llms.txt](https://docs.groundcover.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.groundcover.com/collect-data/data-sources/azure/collect-application-insights-logs.md). # Collect Azure logs Forward telemetry from existing Azure Application Insights resources, optional Azure subscription Activity Logs, and other Azure resource logs to groundcover by using Azure diagnostic settings, Event Hubs, and the groundcover Azure ingester. ```mermaid flowchart LR AI1["Application Insights
resource 1"] --> DS1["Diagnostic setting 1"] AI2["Application Insights
resource 2"] --> DS2["Diagnostic setting 2"] AIN["Application Insights
resource N"] --> DSN["Diagnostic setting N"] DS1 --> EH["Shared Event Hubs namespace
Event Hub: appinsights-logs"] DS2 --> EH DSN --> EH SUB["Azure subscription"] -.-> ADS["Optional Activity Log
diagnostic setting"] ADS -.-> EH EH --> CG["Consumer group:
azure-function"] CG --> FN["Linux Azure Function App"] FN -->|"OTLP/gRPC"| GC["groundcover"] ``` ## Overview The complete deployment template is recommended for most users. It creates the entire forwarding pipeline downstream of one or more existing Application Insights resources, including the shared Event Hub infrastructure and Azure Function. The template does not create Application Insights. All listed Application Insights resources share one Event Hub and one Function App. You can also forward Activity Logs from the deployment subscription through the same pipeline. Activity Log export is disabled by default; see [Collect subscription Activity Logs](#collect-subscription-activity-logs). Only telemetry generated after each diagnostic setting becomes active is forwarded. Existing telemetry is not backfilled. The ingester routes supported telemetry as follows: * `AppTraces`, `AppExceptions`, and `AppEvents` appear in groundcover Logs. * `AppRequests` and `AppDependencies` appear in groundcover Traces. * Other nonblank string categories, such as Cosmos DB `DataPlaneRequests`, appear in Logs under workload `azure-resource-logs`, with the complete original record as a JSON body. * Subscription Activity Logs in the `Administrative`, `Security`, `ServiceHealth`, `ResourceHealth`, `Alert`, `Autoscale`, `Recommendation`, and `Policy` categories appear in groundcover Logs under workload `azure-activity-logs`. Application Insights telemetry, Activity Logs, and other Azure resource logs can share the same Event Hub and message batch. Application Insights diagnostic settings using `allLogs` also export categories such as `AppMetrics` and `AppPerformanceCounters`. These are forwarded as JSON logs under workload `azure-resource-logs`; they are not converted to native groundcover metrics. Existing deployments using `allLogs` can see increased log volume after upgrading an ingester that previously skipped these categories. ## Other Azure resource logs The ingester forwards categories without a dedicated mapping as generic logs through the same Event Hub. This works with the default public image in the deployment templates and requires no additional ingester flag. For example, Cosmos DB `DataPlaneRequests` and `ControlPlaneRequests` are collected this way. Configure a diagnostic setting on each source resource, such as a Cosmos DB account, to send the desired log categories to the Event Hub. The templates automate diagnostic settings for Application Insights and optional subscription Activity Logs only. Keep `applicationInsightsNames` limited to Application Insights resource names. For regional source resources, the destination Event Hub must be in the same Azure region as the source. Sources in different regions need Event Hubs in their respective regions. See [Azure Monitor diagnostic-setting destination requirements](https://learn.microsoft.com/en-us/azure/azure-monitor/platform/diagnostic-settings#destinations). For resource-log-only collection, use the [partial template with an existing Event Hub](#advanced-connect-to-an-existing-event-hub), set `applicationInsightsNames=[]`, and leave both `deployDiagnosticSetting` and `deployActivityLogDiagnosticSetting` set to `false`. Configure the source resources' diagnostic settings separately. For tag enrichment, grant the Function identity **Reader** on the source resources or their resource group, or use `grantActivityLogTagReader` for subscription-wide access. This parameter also covers tag lookups for generic resource logs and can be enabled independently of Activity Log export. Unknown categories use the following mapping: | Field | Value | | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Body | Complete original record as JSON, preserving nested and unfamiliar fields. | | `azure.category` | Original category string as a log attribute. | | `azure.resource.id` | `resourceId` or `ResourceId`, when a non-empty string, as a resource attribute. | | `service.name` | `azure-resource-logs`, shown as the workload. | | Timestamp | First present key among `time`, `Time`, `timeStamp`, and `Timestamp`. Accepts RFC3339 with fractional seconds and the Azure layout `2006-01-02 15:04:05Z07:00`; missing or invalid values use receive time. | | Severity | Unspecified. The original severity remains in the JSON body. | | `azure.resource.tags.*` | Resource tags, when enrichment is enabled and the identity has read access. | Routing checks `category`, then `Category`, then `Type`. The first present key wins, even when its value is invalid. Records with missing, null, empty, whitespace-only, or non-string categories are skipped. A malformed record does not discard the rest of its batch. Known categories retain their existing mappings and accept common capitalized Azure field aliases, with canonical keys taking precedence. Generic records keep their original fields without service-specific mapping. ### Verify other Azure resource logs 1. Confirm that the source resource's diagnostic setting exports the desired categories to the ingester's Event Hub. 2. Generate new activity for an enabled category after configuring export, such as requests to a Cosmos DB account. 3. Check the Event Hubs incoming-message metrics and the Function App's runtime logs. 4. In groundcover Logs, filter by `workload:azure-resource-logs`. If configured, also filter by `env:`. 5. Inspect `azure.category` and `azure.resource.id`, and confirm that the JSON body contains the source event's fields. If tag enrichment is enabled, check `azure.resource.tags.*` against tags directly attached to the source resource. If no logs arrive, check that the source resource actually exports the selected category and that the diagnostic setting targets the correct Event Hub. Generic category support does not automatically enable diagnostic export on Azure resources. Records without a valid category are skipped; missing tags do not block forwarding. ## Azure resource tags on logs The ingester enriches `AppTraces`, `AppExceptions`, `AppEvents`, Activity Logs, and generic Azure resource logs with tags attached directly to the resource or scope identified by `azure.resource.id`. Tag enrichment is enabled by default in the deployment templates, which grant access to the listed Application Insights resources. Activity Log and other resource-log tag access requires an additional [permission opt-in or separately managed Reader assignments](#collect-subscription-activity-logs). Traces are not enriched. Tags appear as OTLP resource attributes under `azure.resource.tags`. Tag keys are lowercased, and characters other than `a-z`, `0-9`, and `_` are replaced with `_`: | Azure resource tag | groundcover resource attribute | | ------------------ | --------------------------------------- | | `Environment=QC` | `azure.resource.tags.environment="QC"` | | `Cost-Center=123` | `azure.resource.tags.cost_center="123"` | Only tags directly assigned to the event's resource or scope are included. Parent resource-group or subscription tags are not inherited. Tags describe the current lookup or cached state, not the tags at the event timestamp. Deleted or inaccessible resources can produce logs without tags. The Function App reads tags from Azure Resource Manager by using its system-assigned managed identity. Lookups are best effort: a missing permission, throttling, or an Azure Resource Manager outage does not interrupt log forwarding. Results are cached in each Function process, so tag changes can take several minutes to appear. Some child resources, such as Event Hub consumer groups and diagnostic settings, can return HTTP `405 Method Not Allowed` during tag lookup because their scope does not support the tag operation. These tag-lookup warnings can occur even with subscription-wide **Reader** access. Logs continue forwarding without tags from that lookup; granting additional permissions does not necessarily resolve an unsupported operation. Check the scope and operation in the warning before treating it as a permissions issue. To disable enrichment, set the `enableAzureResourceTags` template parameter to `false`. This also prevents the template from creating the associated **Reader** role assignments. ## Prerequisites For the complete deployment, make sure you have: * One or more existing Application Insights resources that receive telemetry. * All listed Application Insights resources in the deployment resource group. * All listed Application Insights resources in the same Azure region. * A dedicated **Third Party** [groundcover ingestion key](/collect-data/ingestion-keys.md). Create it in [Settings > Access > Ingestion Keys](https://app.groundcover.com/settings?selectedTab=ingestion-keys). * Your groundcover OTLP/gRPC hostname. Enter the hostname only, without `https://`, a port, or a path. Find it in [Settings > Access > Ingestion Keys](https://app.groundcover.com/settings?selectedTab=ingestion-keys). * Sufficient Azure App Service EP1 quota in the selected region. * Event Hubs public network access. The current template requires it. * Network access from the Function App to: * Amazon ECR Public, to pull the groundcover Azure ingester image. * The groundcover OTLP/gRPC endpoint. * The Event Hubs namespace. ### Azure permissions The identity that deploys the template must have one of the following permission combinations: * **Owner** at the deployment scope. * **Contributor** plus **Role Based Access Control Administrator**. * **Contributor** plus **User Access Administrator**. For Application Insights collection, assign these permissions at the target resource-group scope or inherit them from a broader scope. Activity Log deployment options also require [subscription-scope permissions](#activity-log-permissions). **Contributor alone is insufficient.** The template creates a `Microsoft.Authorization/roleAssignments` resource and therefore requires the following Azure action: ``` Microsoft.Authorization/roleAssignments/write ``` This action is required to grant the Function App's managed identity the **Azure Event Hubs Data Receiver** role and, when Azure resource-tag enrichment is enabled, the **Reader** role on each listed Application Insights resource. For more information, see [Steps to assign an Azure role](https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-steps). ### Network connectivity The complete template creates an Event Hubs namespace with public network access. If you add an Event Hubs firewall while public access remains enabled, enable **Allow trusted Microsoft services to bypass this firewall** so Azure Monitor can publish diagnostic data. The template does not configure regional VNet integration, an Event Hubs private endpoint, private endpoint approval, or a `privatelink.servicebus.windows.net` private DNS zone. If your organization requires private networking, use the [advanced deployment](#advanced-connect-to-an-existing-event-hub) and provision the required networking separately. For Azure destination requirements, see [Azure Monitor diagnostic settings](https://learn.microsoft.com/en-us/azure/azure-monitor/platform/diagnostic-settings#destinations). ## Recommended deployment: Azure portal [![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fgroundcover-public-cloudformation-templates.s3.us-east-1.amazonaws.com%2FBicep_groundcover_azure_ingester_complete.json) The button opens an Azure custom deployment with the public [complete ARM template](https://groundcover-public-cloudformation-templates.s3.us-east-1.amazonaws.com/Bicep_groundcover_azure_ingester_complete.json). 1. Select the subscription. 2. Select the resource group containing all the Application Insights resources. 3. Set `applicationInsightsNames` to a JSON array of Application Insights resource names. For example: ```json ["orders-appinsights", "payments-appinsights"] ``` 4. Set `GCIngestionKey` to the dedicated Third Party ingestion key. 5. Set `GCEndpoint` to the groundcover OTLP/gRPC hostname only. 6. Optionally set `GCEnvironment`. 7. Leave `applicationInsightsRegion` set to the resource-group location, or override it when the Application Insights resources use a different region. 8. Leave `enableAzureResourceTags` enabled to add Application Insights resource tags to logs. 9. To also collect subscription Activity Logs, set `deployActivityLogDiagnosticSetting` to `true`. Optionally set `grantActivityLogTagReader` to `true` for subscription-wide tag lookup. Review [Activity Log permissions](#activity-log-permissions) first. 10. Select **Review + create**, and then select **Create**. ### Complete template parameters | Parameter | Required | Description | | ------------------------------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `applicationInsightsNames` | Yes | JSON array of existing Application Insights resource names. Use names, not resource IDs. Every resource must be in the selected resource group and region. | | `GCIngestionKey` | Yes | Dedicated Third Party groundcover ingestion key. | | `GCEndpoint` | Yes | groundcover OTLP/gRPC hostname without a scheme, port, or path. | | `GCEnvironment` | No | Environment label attached to forwarded telemetry, such as `production`. | | `applicationInsightsRegion` | No | Region containing every listed Application Insights resource. Defaults to the resource-group location. | | `enableAzureResourceTags` | No | Enable resource-tag enrichment for logs. Defaults to `true` and grants the Function identity **Reader** on each listed Application Insights resource. Activity Log and other resource-log tag access is configured separately. | | `deployActivityLogDiagnosticSetting` | No | Export all eight supported Activity Log categories from the deployment subscription to the shared Event Hub. Defaults to `false`. | | `grantActivityLogTagReader` | No | Grant the Function identity **Reader** across the deployment subscription for Activity Log and other Azure resource-log tag lookup. Defaults to `false` and takes effect only when `enableAzureResourceTags` is `true`. | ## Recommended deployment: Azure CLI Download the public [complete ARM template](https://groundcover-public-cloudformation-templates.s3.us-east-1.amazonaws.com/Bicep_groundcover_azure_ingester_complete.json) and [example parameters](https://groundcover-public-cloudformation-templates.s3.us-east-1.amazonaws.com/Bicep_groundcover_azure_ingester_complete.parameters.example.json): ```bash az login export AZURE_RESOURCE_GROUP="" curl --fail --location \ --output Bicep_groundcover_azure_ingester_complete.json \ https://groundcover-public-cloudformation-templates.s3.us-east-1.amazonaws.com/Bicep_groundcover_azure_ingester_complete.json umask 077 curl --fail --location \ --output Bicep_groundcover_azure_ingester_complete.parameters.json \ https://groundcover-public-cloudformation-templates.s3.us-east-1.amazonaws.com/Bicep_groundcover_azure_ingester_complete.parameters.example.json chmod 600 Bicep_groundcover_azure_ingester_complete.parameters.json ``` The ARM JSON template includes its deployment modules, so no separate Bicep module downloads are required. Open `Bicep_groundcover_azure_ingester_complete.parameters.json` in a text editor. Replace the example Application Insights names, ingestion key, and endpoint. Add `GCEnvironment`, `applicationInsightsRegion`, or `enableAzureResourceTags` only when you need to override their defaults. To also collect Activity Logs, set `deployActivityLogDiagnosticSetting` to `true`; optionally set `grantActivityLogTagReader` to `true` for tags. Keep the default public container image for normal deployments. Deploy the template into the resource group containing the Application Insights resources: ```bash az deployment group create \ --name groundcover-azure-ingester \ --resource-group "$AZURE_RESOURCE_GROUP" \ --template-file Bicep_groundcover_azure_ingester_complete.json \ --parameters @Bicep_groundcover_azure_ingester_complete.parameters.json ``` {% hint style="warning" %} Do not commit the populated parameters file or run the deployment with `--debug`. The file contains your ingestion key, and debug output can expose it. {% endhint %} After deployment, securely store or delete the populated parameters file. ## Resources created The complete template creates: * One Event Hubs Standard namespace. * One Event Hub named `appinsights-logs`. * One consumer group named `azure-function`. * One Azure Monitor namespace authorization rule. * One diagnostic setting per Application Insights resource, with `allLogs` enabled. * One Linux Function App using the public groundcover container. * One Elastic Premium EP1 App Service plan. * One Function storage account. * One system-assigned managed identity on the Function App. * One **Azure Event Hubs Data Receiver** RBAC assignment for the managed identity. * One **Reader** RBAC assignment per Application Insights resource when Azure resource-tag enrichment is enabled. With Activity Log options enabled, the template also creates a subscription-scoped deployment containing a subscription diagnostic setting and/or a subscription-wide **Reader** role assignment. It reuses the same Event Hub and Function App. Diagnostic settings, Event Hubs, consumer groups, and RBAC assignments are nested or extension resources. They might not appear as separate rows in the resource group's resource list. Find them in the Azure portal at: * **Application Insights > Monitoring > Diagnostic settings**. * **Event Hubs namespace > Entities > Event Hubs > `appinsights-logs`**. * **`appinsights-logs` > Consumer groups > `azure-function`**. * **Subscriptions > your subscription > Activity log > Export Activity Logs** for subscription diagnostic settings. ## Collect subscription Activity Logs Both the complete and partial templates support two independent options: | Collection setup | `deployActivityLogDiagnosticSetting` | `grantActivityLogTagReader` | | -------------------------------------------------------------------- | ------------------------------------ | --------------------------- | | Application Insights only (default) | `false` | `false` | | Add Activity Log export; manage tag access separately or omit tags | `true` | `false` | | Add Activity Log export and subscription-wide tag access | `true` | `true` | | Activity Log export already exists; add subscription-wide tag access | `false` | `true` | Keep `enableAzureResourceTags=true` when you want tags. `grantActivityLogTagReader` has no effect when enrichment is disabled. Exporting Activity Logs does not itself require granting the Function identity subscription-wide Reader. For example, merge these entries into your deployment parameters file's existing `parameters` object, keeping the other deployment values: ```json { "parameters": { "deployActivityLogDiagnosticSetting": { "value": true }, "grantActivityLogTagReader": { "value": true }, "enableAzureResourceTags": { "value": true } } } ``` If Activity Logs already reach the shared Event Hub, leave `deployActivityLogDiagnosticSetting=false` to avoid creating a second export route and duplicate events. The ingester automatically recognizes the supported categories. For Activity Log-only collection, use the [partial template with an existing Event Hub](#advanced-connect-to-an-existing-event-hub). Set `applicationInsightsNames=[]`, `deployDiagnosticSetting=false`, and `deployActivityLogDiagnosticSetting=true`. No Application Insights resource is required for this setup. ### Activity Log permissions Activity Log options create a nested deployment at subscription scope. In addition to the resource-group deployment permissions, the deploying identity needs: * `Microsoft.Resources/deployments/write` at subscription scope when either Activity Log option creates resources. * `Microsoft.Insights/diagnosticSettings/write` at subscription scope when enabling export. * `Microsoft.Authorization/roleAssignments/write` at subscription scope when enabling the subscription-wide Reader assignment. The role combinations listed in [Azure permissions](#azure-permissions), assigned at subscription scope, cover these operations. Resource-group-only permissions are insufficient for these options. The subscription-wide **Reader** assignment grants read access across the subscription, including resources and resource groups; it is broader than tag-only access. For narrower access, leave `grantActivityLogTagReader=false` and grant the Function identity **Reader** separately on selected resources or resource groups. Events for resources outside those assignments still forward, but their tags may be absent. ### Configure Activity Log export separately To manage export yourself, leave `deployActivityLogDiagnosticSetting=false`. In the Azure portal, open **Subscriptions > your subscription > Activity log > Export Activity Logs**, add a diagnostic setting, select the eight supported categories, and choose **Stream to an event hub**. Select the ingester's Event Hub and a namespace authorization rule with **Manage**, **Send**, and **Listen** permissions, then save. You can also use the [Azure CLI subscription diagnostic-settings command](https://learn.microsoft.com/en-us/cli/azure/monitor/diagnostic-settings/subscription#az-monitor-diagnostic-settings-subscription-create): ```bash export AZURE_SUBSCRIPTION_ID="" export EVENTHUB_NAME="" export EVENTHUB_RULE_ID="" az monitor diagnostic-settings subscription create \ --subscription "$AZURE_SUBSCRIPTION_ID" \ --name groundcover-activity-logs \ --event-hub-name "$EVENTHUB_NAME" \ --event-hub-auth-rule "$EVENTHUB_RULE_ID" \ --logs '[ {"category":"Administrative","enabled":true}, {"category":"Security","enabled":true}, {"category":"ServiceHealth","enabled":true}, {"category":"ResourceHealth","enabled":true}, {"category":"Alert","enabled":true}, {"category":"Autoscale","enabled":true}, {"category":"Recommendation","enabled":true}, {"category":"Policy","enabled":true} ]' ``` Use an existing namespace authorization rule, or create one as described in [Create diagnostic settings manually](#create-diagnostic-settings-manually). Tag permissions can still be managed by `grantActivityLogTagReader` independently of export. ### Verify Activity Logs 1. Generate a new management event after enabling export, such as updating a tag on a test resource. 2. Confirm the event appears in the subscription's **Activity log**. 3. Check Event Hubs incoming-message metrics and the Function App's runtime logs. 4. In groundcover Logs, filter by `workload:azure-activity-logs`. If configured, also filter by `env:`. 5. Inspect `azure.category`, `azure.operation.name`, `azure.result.type`, `azure.correlation.id`, and `azure.resource.id` where present. Identity and additional event details are preserved in `azure.identity` and `azure.properties`. 6. If tag access is enabled, check `azure.resource.tags.*` against tags directly attached to the event's target resource or scope. The log body summarizes the message or operation, outcome, and target resource when available. Only newly exported events are collected; existing Activity Log history is not backfilled. If Activity Logs are missing while Application Insights works, check the subscription diagnostic setting separately: Application Insights `allLogs` settings do not export subscription Activity Logs. Confirm the selected categories, destination Event Hub, and that the Function App runs an ingester image that supports Activity Logs. If only tags are missing, check `enableAzureResourceTags` and Reader access to the event's target resource or scope. Deleted resources may no longer have tags available for lookup. ### Disable Activity Log collection Setting an option to `false` in a later incremental deployment does not delete an existing subscription diagnostic setting or revoke a previously created Reader assignment. Remove the diagnostic setting explicitly to stop export, and remove the subscription Reader assignment explicitly if it is no longer needed. Setting `enableAzureResourceTags=false` disables runtime tag lookup but does not revoke existing permissions. ## Verification For Activity Logs, follow [Verify Activity Logs](#verify-activity-logs). For Application Insights: 1. Generate new Application Insights telemetry after deployment. 2. Confirm that the new telemetry appears in Application Insights. 3. Open the Event Hubs namespace's **Metrics** page and check **Incoming Messages** and **Incoming Bytes**. 4. Open the Function App's runtime logs and check for Event Hub connection or OTLP export errors. 5. Find the telemetry in groundcover Logs or Traces. 6. Open an `AppTraces`, `AppExceptions`, or `AppEvents` log and confirm that its resource attributes include the expected `azure.resource.tags.*` fields. Use these groundcover filters to narrow the results: * `env_type:azure` * `source:EventHub` * `workload:` If you configured `GCEnvironment`, also apply `env:`. {% hint style="info" %} A newly created Azure diagnostic setting can take up to 90 minutes before data starts flowing to Event Hub. Event Hub and groundcover remain empty until Azure Monitor starts exporting. For details, see [Expected latency after creating a diagnostic setting](https://learn.microsoft.com/en-us/azure/azure-monitor/platform/diagnostic-settings#expected-latency-after-creating-a-diagnostic-setting). Early Event Hub authorization errors can also occur briefly while the managed-identity RBAC assignment propagates. {% endhint %} ## Troubleshooting If telemetry does not appear, verify that: * A diagnostic setting exists on each Application Insights resource and has the `allLogs` category group enabled. * Each diagnostic setting targets the `appinsights-logs` Event Hub in the namespace created by the deployment. * Application Insights and the Event Hub are in the same Azure region. * You generated fresh telemetry after the diagnostic setting was enabled. * The initial diagnostic-setting activation delay has elapsed (up to 90 minutes). * The Function App's managed identity has **Azure Event Hubs Data Receiver** on the Event Hub. * When tag enrichment is enabled, the Function App's managed identity has **Reader** on every Application Insights resource whose logs reach the Event Hub. * The Function App uses the `azure-function` consumer group. * The Event Hubs firewall allows trusted Microsoft services where applicable. * The Function App can pull the public image from Amazon ECR Public. * The Function App can reach the groundcover OTLP/gRPC endpoint. * `GCEndpoint` contains no scheme, port, or path. * The ingestion key is active and has the **Third Party** type. * The subscription has sufficient EP1 quota in the selected region. If logs arrive without the expected Azure tags, also verify that: * The record is an Application Insights log, Activity Log, or generic Azure resource log with a resource ID. Trace categories are not enriched. * The tags are assigned directly to the resource or scope in `azure.resource.id` rather than inherited from a parent scope. * `enableAzureResourceTags` is `true`, or `AZURE_RESOURCE_TAGS_ENABLED` is `true` for a manually configured Function App. * You allowed several minutes for a tag update to refresh in the Function process cache. ## Advanced: connect to an existing Event Hub Use the partial template when you already have an Event Hubs namespace and Event Hub, need to split resources across resource groups, or must provision networking separately. This flow deploys the Function App and connects it to your existing Event Hub. It can also create a consumer group and diagnostic settings when the relevant resources are in the deployment resource group. ### Create an Event Hubs namespace manually Skip this step if you already have a Standard Event Hubs namespace. When collecting Application Insights telemetry, the namespace must be in the same Azure region as Application Insights. For Activity Log-only collection, choose the region for your forwarding infrastructure. Its namespace name must be unique across Azure. #### Azure portal 1. Open your Application Insights resource and note its **Resource group** and **Location**. 2. In the Azure portal, search for **Event Hubs** and select **Create**. 3. Select the target subscription and resource group, and use the same region as Application Insights. 4. Enter a unique namespace name and select the **Standard** pricing tier. 5. Configure public or private network access according to your networking design, and then create the namespace. #### Azure CLI: public network access The following example creates a namespace with public network access. For a private deployment, configure the resources described in [Network restrictions and firewalls](#network-restrictions-and-firewalls) before disabling public access. ```bash az login export AZURE_RESOURCE_GROUP="" export EVENTHUB_NAMESPACE="" export APPLICATION_INSIGHTS_REGION="" az eventhubs namespace create \ --resource-group "$AZURE_RESOURCE_GROUP" \ --name "$EVENTHUB_NAMESPACE" \ --location "$APPLICATION_INSIGHTS_REGION" \ --sku Standard \ --public-network-access Enabled ``` ### Create an Event Hub manually Create a non-compacted Event Hub. You can keep the Azure defaults for partitions and retention. #### Azure portal 1. Open the Event Hubs namespace. 2. Select **Event Hubs > + Event Hub**. 3. Enter a name such as `appinsights-logs`, and then create the Event Hub. #### Azure CLI ```bash export AZURE_RESOURCE_GROUP="" export EVENTHUB_NAMESPACE="" export EVENTHUB_NAME="appinsights-logs" az eventhubs eventhub create \ --resource-group "$AZURE_RESOURCE_GROUP" \ --namespace-name "$EVENTHUB_NAMESPACE" \ --name "$EVENTHUB_NAME" ``` ### Deploy the Function App with the partial template [![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fgroundcover-public-cloudformation-templates.s3.us-east-1.amazonaws.com%2FBicep_groundcover_azure_ingester.json) Use the public [partial ARM template](https://groundcover-public-cloudformation-templates.s3.us-east-1.amazonaws.com/Bicep_groundcover_azure_ingester.json) in the Azure portal, or download it for Azure CLI. Deploy the partial template to the resource group containing the Event Hub. Configure these parameters for your environment: | Parameter | Description | | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `GCIngestionKey` | Dedicated Third Party groundcover ingestion key. | | `GCEndpoint` | groundcover OTLP/gRPC hostname without a scheme, port, or path. | | `GCEnvironment` | Optional environment label. | | `eventHubNamespaceName` | Existing Event Hubs namespace name. | | `eventHubName` | Existing Event Hub receiving Application Insights, Activity Logs, or other Azure resource logs. | | `eventHubConsumerGroup` | Consumer group used by the ingester. Defaults to `azure-function`. | | `createEventHubConsumerGroup` | Set to `true` to create the consumer group, or `false` to use an existing consumer group. | | `useManagedIdentityForEventHub` | Set to `true` to authenticate with the Function App's managed identity. | | `eventHubConnectionString` | Event Hubs connection string used only when managed-identity authentication is disabled. | | `applicationInsightsNames` | Application Insights resource names used for automatic diagnostic-setting creation and, when tag enrichment is enabled, **Reader** role assignments. For example, `["orders-appinsights", "payments-appinsights"]`. The resources must be in the template's deployment resource group. | | `deployDiagnosticSetting` | Set to `true` to create diagnostic settings, or `false` when you will create them separately. | | `enableAzureResourceTags` | Enable resource-tag enrichment for logs. Defaults to `true` and grants the Function identity **Reader** on each listed Application Insights resource. | | `deployActivityLogDiagnosticSetting` | Export all eight Activity Log categories from the deployment subscription to the existing Event Hub. Defaults to `false`; independent of `deployDiagnosticSetting`. | | `grantActivityLogTagReader` | Grant the Function identity subscription-wide **Reader** for Activity Log and other Azure resource-log tag lookup. Defaults to `false`; requires `enableAzureResourceTags=true` but does not require template-managed export. | Do not change image-related parameters unless you are intentionally using a different container image. When `createEventHubConsumerGroup` is `false`, make sure the consumer group named by `eventHubConsumerGroup` already exists. Do not share this consumer group with another independent consumer. For an Azure CLI deployment, download the partial ARM template and its [example parameters file](https://groundcover-public-cloudformation-templates.s3.us-east-1.amazonaws.com/Bicep_groundcover_azure_ingester.parameters.example.json). Protect and edit the parameters file as described in the recommended CLI flow, and then run: ```bash az deployment group create \ --name groundcover-azure-ingester \ --resource-group "$AZURE_RESOURCE_GROUP" \ --template-file Bicep_groundcover_azure_ingester.json \ --parameters @Bicep_groundcover_azure_ingester.parameters.json ``` Do not commit the populated parameters file or run the deployment with `--debug`. After deployment, securely store or delete the populated parameters file: ```bash rm -f Bicep_groundcover_azure_ingester.parameters.json ``` #### Managed identity or connection string Managed identity is recommended. Keep `useManagedIdentityForEventHub` set to `true` and leave `eventHubConnectionString` empty. The template enables a system-assigned identity and grants it **Azure Event Hubs Data Receiver** on the Event Hub. When tag enrichment is enabled, it also grants the identity **Reader** on each Application Insights resource listed in `applicationInsightsNames`. The deploying identity needs permission to create role assignments, as described in [Azure permissions](#azure-permissions). If your organization requires connection-string authentication, set `useManagedIdentityForEventHub` to `false` and provide `eventHubConnectionString`. Treat the connection string as a secret and use a policy that has only the permissions the Function needs to receive events. ### Create diagnostic settings manually The partial template can create diagnostic settings only for Application Insights resources in its deployment resource group. When Application Insights and Event Hubs are in different resource groups, set `deployDiagnosticSetting` to `false` and configure the settings separately. Keep `applicationInsightsNames` limited to Application Insights resources in the template's deployment resource group. If all Application Insights resources are external, explicitly set `applicationInsightsNames=[]`, replacing any example names. For a mix of local and external resources, include only the local names. Setting `deployDiagnosticSetting=false` does not disable **Reader** assignments: when enrichment is enabled, the template still creates them for every listed name in the deployment resource group. Grant access to external resources separately as described in [Grant access for resource-tag enrichment](#grant-access-for-resource-tag-enrichment). The identity that configures them needs these Azure permissions on the relevant resources: * `Microsoft.EventHub/namespaces/read` * `Microsoft.EventHub/namespaces/eventhubs/read` * `Microsoft.EventHub/namespaces/authorizationRules/read` * `Microsoft.EventHub/namespaces/authorizationRules/write` * `Microsoft.EventHub/namespaces/authorizationRules/listKeys/action` * `Microsoft.Insights/components/read` * `Microsoft.Insights/diagnosticSettings/write` To configure each Application Insights resource in the Azure portal: 1. Open the Event Hubs namespace and select **Settings > Shared access policies**. 2. Add a policy named `groundcover-app-insights-diagnostics` with **Manage**, **Send**, and **Listen** permissions. 3. Open the Application Insights resource. 4. Select **Monitoring > Diagnostic settings**. 5. Select **Add diagnostic setting** and enter a name, such as `groundcover-app-insights-logs`. 6. Select **allLogs**. 7. Select **Stream to an event hub**. 8. Select the subscription, Event Hubs namespace, Event Hub, and authorization rule. 9. Select **Save**. Repeat the diagnostic-setting steps for every Application Insights resource. Alternatively, create the authorization rule and one diagnostic setting with Azure CLI. Repeat the final command with the resource ID of each Application Insights resource: ```bash export EVENTHUB_RESOURCE_GROUP="" export EVENTHUB_NAMESPACE="" export EVENTHUB_NAME="" export APP_INSIGHTS_RESOURCE_GROUP="" export APPLICATION_INSIGHTS_NAME="" az eventhubs namespace authorization-rule create \ --resource-group "$EVENTHUB_RESOURCE_GROUP" \ --namespace-name "$EVENTHUB_NAMESPACE" \ --name groundcover-app-insights-diagnostics \ --rights Manage Send Listen EVENTHUB_RULE_ID="$(az eventhubs namespace authorization-rule show \ --resource-group "$EVENTHUB_RESOURCE_GROUP" \ --namespace-name "$EVENTHUB_NAMESPACE" \ --name groundcover-app-insights-diagnostics \ --query id \ --output tsv)" APP_INSIGHTS_ID="$(az resource show \ --resource-group "$APP_INSIGHTS_RESOURCE_GROUP" \ --name "$APPLICATION_INSIGHTS_NAME" \ --resource-type Microsoft.Insights/components \ --query id \ --output tsv)" az monitor diagnostic-settings create \ --name groundcover-app-insights-logs \ --resource "$APP_INSIGHTS_ID" \ --event-hub "$EVENTHUB_NAME" \ --event-hub-rule "$EVENTHUB_RULE_ID" \ --logs '[{"categoryGroup":"allLogs","enabled":true}]' ``` ### Grant access for resource-tag enrichment When an Application Insights resource is outside the partial template's deployment resource group, the template cannot create its **Reader** assignment. Grant the Function App's system-assigned identity **Reader** on each external Application Insights resource whose logs reach the Event Hub: ```bash export FUNCTION_RESOURCE_GROUP="" export FUNCTION_APP_NAME="" export APP_INSIGHTS_RESOURCE_GROUP="" export APPLICATION_INSIGHTS_NAME="" FUNCTION_PRINCIPAL_ID="$(az functionapp identity show \ --resource-group "$FUNCTION_RESOURCE_GROUP" \ --name "$FUNCTION_APP_NAME" \ --query principalId \ --output tsv)" APP_INSIGHTS_ID="$(az resource show \ --resource-group "$APP_INSIGHTS_RESOURCE_GROUP" \ --name "$APPLICATION_INSIGHTS_NAME" \ --resource-type Microsoft.Insights/components \ --query id \ --output tsv)" az role assignment create \ --assignee-object-id "$FUNCTION_PRINCIPAL_ID" \ --assignee-principal-type ServicePrincipal \ --role Reader \ --scope "$APP_INSIGHTS_ID" ``` Repeat the role assignment for every external Application Insights resource. Skip this step when `enableAzureResourceTags` is `false`. ### Network restrictions and firewalls For a namespace with public network access and an Event Hubs firewall, enable **Allow trusted Microsoft services to bypass this firewall** so Azure Monitor can publish telemetry. Also allow the Function App to reach the Event Hubs namespace, Amazon ECR Public, and the groundcover OTLP/gRPC endpoint. For private access, provision Function App regional VNet integration, an Event Hubs private endpoint and approval, and a `privatelink.servicebus.windows.net` private DNS zone linked to the Function App's VNet. Confirm name resolution and routing from the Function App, and then disable public network access before troubleshooting the ingester itself. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.groundcover.com/collect-data/data-sources/azure/collect-application-insights-logs.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.