--- title: "Set up OpenTelemetry Collector for Application Observability | OpenTelemetry documentation" description: "Set up the upstream community supported OpenTelemetry Collector for Application Observability." --- > For a curated documentation index, see [llms.txt](/llms.txt). For the complete documentation index, see [llms-full.txt](/llms-full.txt). # Set up OpenTelemetry Collector for Application Observability The OpenTelemetry project maintainers and the Cloud Native Computing Foundation maintain the upstream OpenTelemetry Collector. For production observability, Grafana Labs recommends [Grafana Alloy](/docs/grafana-cloud/monitor-applications/application-observability/collector/grafana-alloy/), an OpenTelemetry Collector distribution that packages upstream OpenTelemetry Collector components and Prometheus exporters. Alloy provides stability, support, and an integrated experience with Grafana Application Observability and other Grafana products. ## Recommended setup For production, run an OpenTelemetry Collector on every host to seamlessly correlate data between infrastructure and application observability. Grafana Labs recommends using the **OpenTelemetry Collector** [Grafana Cloud integration tile](/docs/grafana-cloud/send-data/otlp/send-data-otlp/#recommended-opentelemetry-setup-via-grafana-cloud-integrations) to configure the OpenTelemetry Collector. If you already have an OpenTelemetry Collector per host, refer to the [advanced manual setup](#advanced-manual-setup) and [configure your application](#configure-your-application) sections. ## Advanced manual setup For advanced use cases, manually configure the OpenTelemetry `config.yaml` configuration file: YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```yaml # Tested with OpenTelemetry Collector Contrib v0.98.0 receivers: otlp: # https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver protocols: grpc: http: hostmetrics: # Optional. Host Metrics Receiver added as an example of Infra Monitoring capabilities of the OpenTelemetry Collector # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver scrapers: load: memory: processors: batch: # https://github.com/open-telemetry/opentelemetry-collector/tree/main/processor/batchprocessor resourcedetection: # Enriches telemetry data with resource information from the host # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourcedetectionprocessor detectors: ["env", "system"] override: false system: resource_attributes: host.id: enabled: true host.name: enabled: true transform/drop_unneeded_resource_attributes: # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor error_mode: ignore trace_statements: - context: resource statements: - delete_key(attributes, "k8s.pod.start_time") - delete_key(attributes, "os.description") - delete_key(attributes, "os.type") - delete_key(attributes, "process.command_args") - delete_key(attributes, "process.executable.path") - delete_key(attributes, "process.pid") - delete_key(attributes, "process.runtime.description") - delete_key(attributes, "process.runtime.name") - delete_key(attributes, "process.runtime.version") metric_statements: - context: resource statements: - delete_key(attributes, "k8s.pod.start_time") - delete_key(attributes, "os.description") - delete_key(attributes, "os.type") - delete_key(attributes, "process.command_args") - delete_key(attributes, "process.executable.path") - delete_key(attributes, "process.pid") - delete_key(attributes, "process.runtime.description") - delete_key(attributes, "process.runtime.name") - delete_key(attributes, "process.runtime.version") log_statements: - context: resource statements: - delete_key(attributes, "k8s.pod.start_time") - delete_key(attributes, "os.description") - delete_key(attributes, "os.type") - delete_key(attributes, "process.command_args") - delete_key(attributes, "process.executable.path") - delete_key(attributes, "process.pid") - delete_key(attributes, "process.runtime.description") - delete_key(attributes, "process.runtime.name") - delete_key(attributes, "process.runtime.version") transform/add_resource_attributes_as_metric_attributes: # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor error_mode: ignore metric_statements: - context: datapoint statements: - set(attributes["deployment.environment"], resource.attributes["deployment.environment"]) - set(attributes["service.version"], resource.attributes["service.version"]) exporters: otlphttp/grafana_cloud: # https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlphttpexporter endpoint: "${env:GRAFANA_CLOUD_OTLP_ENDPOINT}" auth: authenticator: basicauth/grafana_cloud extensions: basicauth/grafana_cloud: # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/basicauthextension client_auth: username: "${env:GRAFANA_CLOUD_INSTANCE_ID}" password: "${env:GRAFANA_CLOUD_API_KEY}" service: extensions: [ basicauth/grafana_cloud, ] telemetry: metrics: level: detailed address: 0.0.0.0:8888 pipelines: traces: receivers: [otlp] processors: [resourcedetection, transform/drop_unneeded_resource_attributes, batch] exporters: [otlphttp/grafana_cloud] metrics: receivers: [otlp, hostmetrics] processors: [ resourcedetection, transform/drop_unneeded_resource_attributes, transform/add_resource_attributes_as_metric_attributes, batch, ] exporters: [otlphttp/grafana_cloud] logs: receivers: [otlp] processors: [ resourcedetection, transform/drop_unneeded_resource_attributes, batch, ] exporters: [otlphttp/grafana_cloud] ``` Set the following environmental variables in the configuration file: Expand table | Environment Variable | Description | Example | |-------------------------------|----------------------------------------------------------------------------|----------------------------------------------| | `GRAFANA_CLOUD_API_KEY` | API key generated above | `eyJvSomeLongStringJ9fQ==` | | `GRAFANA_CLOUD_OTLP_ENDPOINT` | **OTLP endpoint** from **Grafana Cloud > OpenTelemetry > Configure** | `https://otlp-endpoint-xyz.grafana.net/otlp` | | `GRAFANA_CLOUD_INSTANCE_ID` | **Instance ID** from **Grafana Cloud > OpenTelemetry > Configure** | | ### Host identification for Application Observability Application Observability identifies hosts automatically from standard OpenTelemetry resource attributes attached to your telemetry. The minimal configuration above doesn’t include explicit host identifier configuration because proper host identification depends on your deployment environment. Application Observability evaluates the following resource attributes in priority order and uses the **first match** as the host’s billing identifier: Expand table | Priority | Attributes | Typical environment | |----------|-------------------|----------------------------------------| | 1 | `k8s.node.name` | Kubernetes | | 2 | `host.id` | Cloud VMs, bare metal, on-premises VMs | | 3 | `grafana.host.id` | Docker, or any host without `host.id` | For environment-specific configuration guidance, refer to the [Application Observability host-hours pricing documentation](/docs/grafana-cloud/monitor-applications/application-observability/pricing/#how-application-observability-identifies-hosts), which provides detailed examples for: - Kubernetes deployments using the `k8sattributes` processor - Cloud VM and bare-metal deployments emitting `host.id` (cloud or system resource detector) - Docker deployments using `grafana.host.id` > Note > > If host identification isn’t configured correctly for your environment, Application Observability will guide you to the appropriate resolution within the product. ### Collector self-monitoring The collector telemetry configuration enables monitoring of the collector itself. This exposes collector performance metrics at `:8888/metrics` including pipeline throughput, queue sizes, and resource usage. Use these metrics to troubleshoot collector performance issues and ensure your observability pipeline is healthy. ### Data pipelines OpenTelemetry Collector receives OTLP data and processes it with the following pipelines: **Traces:** The `traces` pipeline receives traces with the `otlp` receiver and exports them to the Grafana Cloud Tempo with the [otlp](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlpexporter) exporter. The `traces` pipeline uses the [`resourcedetection`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourcedetectionprocessor) processor to enrich telemetry data with resource information from the host. Consult the resource detection processor [README.md](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourcedetection) for a list of configuration options. **Metrics:** The `metrics` pipeline receives traces from the `otlp` receiver and exports metrics to the Grafana Cloud Metrics with the [`prometheusremotewrite`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/prometheusremotewriteexporter) exporter. The `metrics` pipeline uses the [`transform`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor) processor to add `deployment.environment`, and `service.version` labels to metrics. **Logs:** The `logs` pipeline receives logs with the `otlp` receiver and exports them to the Grafana Cloud Loki with the [loki](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/lokiexporter) exporter. ## Run OpenTelemetry Collector Create the `config.yaml` file, set the necessary environment variables, and [run the OpenTelemetry Collector](https://opentelemetry.io/docs/collector/quick-start/). ## Configure your application Set the following environment variables to configure your application to use the OpenTelemetry Collector: Expand table | Configuration | Options | Result | |-------------------------------------------------|----------------------------------------------|-----------------------------------------------------------| | `export OTEL_EXPORTER_OTLP_ENDPOINT=` | `http://localhost:4318`, remote host address | The default local host address, or a remote host address. | | `export OTEL_EXPORTER_OTLP_PROTOCOL=` | `grpc`, `http/protobuf` | The default `http/protobuf` protocol or `grpc` | For example, for a local OpenTelemetry Collector set the following environment variables: shell ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```shell export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 export OTEL_EXPORTER_OTLP_PROTOCOL=grpc ``` Then restart your application. ## Troubleshoot host identification If Application Observability displays a notification that host information isn’t being received, verify that at least one of the priority tiers is satisfied: 1. **Check resource attributes**: Use Explore to query your service’s spans and verify which host attributes are present: ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```none {service.name="your-service"} | json ``` Look for `k8s.node.name`, `host.id`, or `grafana.host.id` in the resource attributes. 2. **Verify resourcedetection processor**: Ensure the `resourcedetection` processor is active in your traces pipeline and configured to detect `host.id` and `host.name`. 3. **For Kubernetes**: Add the [`k8sattributes` processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/k8sattributesprocessor) to stamp `k8s.node.name` on your telemetry. 4. **For cloud VMs and bare metal**: Enable `host.id` in the `resourcedetection` processor. For cloud VMs, add your provider’s detector to the `detectors` list (`ec2`, `gcp`, or `azure`) so `host.id` is set to the cloud instance ID. For bare metal and on-premises VMs, the `system` detector derives `host.id` from the machine ID. 5. **For Docker**: Set `grafana.host.id` to the host’s identifier via `OTEL_RESOURCE_ATTRIBUTES`, since containers have no stable per-host `host.id`. For detailed configuration examples for each environment, refer to the [Application Observability host-hours pricing documentation](/docs/grafana-cloud/monitor-applications/application-observability/pricing/#make-sure-your-hosts-are-identified). ## Next steps [Observe your services in Application Observability](/docs/grafana-cloud/monitor-applications/application-observability/manual/)