--- title: "Configure GCP logs | Grafana Cloud documentation" description: "How to configure GCP logs" --- > For a curated documentation index, see [llms.txt](/llms.txt). For the complete documentation index, see [llms-full.txt](/llms-full.txt). # Configure GCP logs Complete the following steps to configure GCP Logs as shown in the following diagram. [Installation steps for GCP Logs](/media/docs/grafana-cloud/cloud-provider/GCP-logsinstall.png) ## Before you begin Make sure you have the following roles and access: - **Grafana Cloud:** You need the GCP Writer plugin role, Admin basic role, or Grafana Admin basic role to configure GCP logs in Cloud Provider Observability. If you use a plugin role without a Grafana Cloud basic role, you also need the `Datasources:Reader` fixed role to view dashboard data. For details, refer to [RBAC roles and permissions](/docs/grafana-cloud/observe-and-act/monitor-infrastructure/monitor-cloud-provider/rbac/). - **Google Cloud:** The account you use to create the log sink, Pub/Sub topic, and subscription needs the `roles/pubsub.editor` and `roles/logging.configWriter` IAM roles. The service account that Alloy uses needs the `roles/pubsub.subscriber` role on the subscription. - **Tools:** Install the [`gcloud` CLI](https://cloud.google.com/sdk/docs/install) if you plan to use the command-line setup method. ## Select your platform Select the platform from the drop-down menu. ## Install Grafana Alloy [Grafana Alloy](/docs/grafana-cloud/send-data/alloy/) reads the logs from the Pub/Sub subscription. 1. If you have not already installed Alloy where you intend to run GCP Logs, click **Run Grafana Alloy**. 2. At the **Alloy configuration** screen, either create a token or use an existing token. 1. To create a token, enter a token name, then click **Create token**. The token displays on the screen and is added to the command for running Alloy. Copy the command and paste it into the terminal. 2. To use an existing token, paste it into the token field, then click **Proceed to install integration**. 3. Copy the command and paste it into the terminal. 4. Click **Proceed to install integration**. ## Set up GCP logs In this step, you create the GCP resources that route logs to Grafana Cloud. You can create these resources with either the `gcloud` CLI or Terraform. Choose the method that best fits how you manage your GCP infrastructure. To set up GCP logs: 1. Click **Setup instructions**. 2. At the **Set up GCP resources** screen, create the required resources for GCP Log collection. A customizable logging sink routes logs from Cloud Logging to Pub/Sub. Grafana Alloy then pulls these logs from the Pub/Sub topic. 3. Select either the `gcloud` CLI or Terraform configuration. ### Configure with gcloud CLI Complete these steps in the `gcloud` CLI, then click **Close** to return to the next step. To configure with gcloud CLI: 1. Set up topic in Pub/Sub. This topic acts as the queue to persist log messages so Grafana Alloy can read them. For example: `gcloud pubsub topics create cloud-logs` Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash gcloud pubsub topics create @@@TOPIC_ID@@@ ``` 2. Set up a [log sink](https://cloud.google.com/logging/docs/export/configure_export_v2#creating_sink) to forward logs into the topic you created in the previous step. This command adds a `log-filter` option, which controls the type of logs that reaches the destination topic. For example: `gcloud logging sinks create cloud-logs pubsub.googleapis.com/projects/my-project/topics/cloud-logs --log-filter='resource.type=("gcs_bucket")' --description="Cloud logs"` Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash gcloud logging sinks create @@@SINK_NAME@@@ @@@SINK_LOCATION@@@ @@@OPTIONAL_FLAGS@@@ ``` 3. Find the writer identity service account of the log sink. For example: `gcloud logging sinks describe --format='value(writerIdentity)' cloud-logs` Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash gcloud logging sinks describe --format='value(writerIdentity)' @@@SINK_NAME@@@ ``` 4. Grant a Pub/Sub publisher role to the log sink by creating an IAM policy binding. This allows the log sink to publish messages to the topic. For example: `gcloud pubsub topics add-iam-policy-binding cloud-logs --member=serviceAccount:pxxxxxxxxx-xxxxxx@gcp-sa-logging.iam.gserviceaccount.com --role=roles/pubsub.publisher` Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash gcloud pubsub topics add-iam-policy-binding @@@TOPIC_ID@@@ --member=@@@WRITER_IDENTITY@@@ --role=roles/pubsub.publisher` ``` 5. Create a Pub/Sub subscription for log delivery. Alloy uses the subscription to consume log messages. For more options, refer to `gcloud pubsub subscriptions --help`. For example: `gcloud pubsub subscriptions create cloud-logs --topic=projects/my-project/topics/cloud-logs --ack-deadline=10 --message-retention-duration=7d` Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash gcloud pubsub subscriptions create cloud-logs --topic=@@@TOPIC_ID@@@ --ack-deadline=@@@ACK_DEADLINE@@@ --message-retention-duration=@@@RETENTION_DURATION@@@ ``` ### Configure with Terraform > Tip > > If you need help working with Terraform and GCP, refer to [this HashiCorp Developer Build infrastructure tutorial](https://developer.hashicorp.com/terraform/tutorials/gcp-get-started/google-cloud-platform-build). Complete the following steps with Terraform, then click **Close** to return to the next set of steps. To configure with Terraform: 1. Copy the following code snippet into a TF file: hcl ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```hcl // Provider module provider "google" { project = "$GCP_PROJECT_ID" } // Topic resource "google_pubsub_topic" "main" { name = "cloud-logs" } // Log sink variable "inclusion_filter" { type = string description = "Optional GCP Logs query which can filter logs being routed to the pub/sub topic and Alloy" } resource "google_logging_project_sink" "main" { name = "cloud-logs" destination = "pubsub.googleapis.com/${google_pubsub_topic.main.id}" filter = var.inclusion_filter unique_writer_identity = true } resource "google_pubsub_topic_iam_binding" "log-writer" { topic = google_pubsub_topic.main.name role = "roles/pubsub.publisher" members = [ google_logging_project_sink.main.writer_identity, ] } // Subscription resource "google_pubsub_subscription" "main" { name = "cloud-logs" topic = google_pubsub_topic.main.name } ``` 2. Create the new resources by running the following after replacing the required variable ``. hcl ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```hcl terraform apply \ -var="inclusion_filter=@@@@@@" ``` ## Set up GCP service account In the `gcloud` CLI, [create a service account](https://cloud.google.com/iam/docs/service-account-overview) with the `pubsub.subscriber` permission. The service account and its permission enables Grafana Alloy to read log entries from the Pub/Sub subscription. ## Configure Alloy to consume GCP Pub/Sub Now that the GCP resources and service account are in place, configure Grafana Alloy to read logs from the Pub/Sub subscription and forward them to Grafana Cloud. Alloy uses two components to do this: - `loki.source.gcplog` pulls log entries from the Pub/Sub subscription you created. It retrieves logs from cloud resources such as GCS buckets, load balancers, or Kubernetes clusters running on GCP. - `discovery.relabel` defines any relabeling rules applied to the logs before Alloy sends them to Loki. To configure Alloy, add these components to your configuration file and update them with the values from your GCP setup. To configure Alloy to consume GCP Pub/Sub: 1. [Navigate](/docs/grafana-cloud/send-data/alloy/configure/linux/) to the configuration file for your Alloy instance. 2. Copy the following code snippet and append it to your Alloy configuration file: ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```none discovery.relabel "logs_integrations_integrations_gcp" { targets = [] rule { source_labels = ["__gcp_logname"] target_label = "logname" } rule { source_labels = ["__gcp_resource_type"] target_label = "resource_type" } } loki.source.gcplog "logs_integrations_integrations_gcp" { pull { project_id = "@@@@@@" subscription = "@@@@@@" labels = { job = "integrations/gcp", } } forward_to = [loki.write.grafana_cloud_loki.receiver] relabel_rules = discovery.relabel.logs_integrations_integrations_gcp.rules } ``` 3. Find the following values and replace them as follows with values you created when you set up GCP Logs: - `gcp_project_id`: replace with the GCP project ID - `gcp_pubsub_subscription_name`: replace with the subscription name 4. To view additional configuration examples, click **Advanced configurations**. ### Examples of advanced configuration You can further modify the [`loki.source.gcplog` component](/docs/agent/latest/flow/reference/components/loki.source.gcplog/#lokisourcegcplog) for more advanced usages. #### Multiple consumers, single subscription ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```none loki.source.gcplog "logs_integrations_integrations_gcp" { pull { project_id = "project-1" subscription = "subscription-1" labels = { job = "integrations/gcp", } } forward_to = [loki.write.grafana_cloud_loki.receiver] relabel_rules = discovery.relabel.logs_integrations_integrations_gcp.rules } loki.source.gcplog "logs_integrations_integrations_gcp" { pull { project_id = "project-1" subscription = "subscription-1" labels = { job = "integrations/gcp2", } } forward_to = [loki.write.grafana_cloud_loki.receiver] relabel_rules = discovery.relabel.logs_integrations_integrations_gcp.rules } ``` #### Multiple subscriptions ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```none loki.source.gcplog "logs_integrations_integrations_gcp" { pull { project_id = "project-1" subscription = "subscription-2" labels = { job = "integrations/gcp", } } forward_to = [loki.write.grafana_cloud_loki.receiver] relabel_rules = discovery.relabel.logs_integrations_integrations_gcp.rules } loki.source.gcplog "logs_integrations_integrations_gcp" { pull { project_id = "project-1" subscription = "subscription-1" labels = { job = "integrations/gcp2", } } forward_to = [loki.write.grafana_cloud_loki.receiver] relabel_rules = discovery.relabel.logs_integrations_integrations_gcp.rules } ``` ## Restart Grafana Alloy Run the command appropriate for your platform to restart Grafana Alloy so your changes can take effect. ## Test connection Click **Test connection** to test that Grafana Alloy is collecting data and sending it to Grafana Cloud. ## View your logs Click the **Logs** tab to view your logs.