Set up on Grafana Labs

GET hardware requirements

Mon, 24 Mar 2025 20:44:45 +0000

GET hardware requirements

This page outlines the current hardware requirements for running Grafana Enterprise Traces (GET). Grafana Labs reserves the right to mark a support issue as ‘unresolvable’ if these requirements are not followed. Refer to the Grafana Labs Enterprise Support SLA for more details.

Note
As of February 2025, you must run GET on the managed Kubernetes services offered by AWS (Amazon Elastic Kubernetes Service), GCP (Google Kubernetes Engine), or Azure (Azure Kubernetes Service). Additionally, you must use the managed object storage services offered by AWS (S3), GCP (GCS), or Azure (Blob Storage).

No other deployment modes are supported.

CPU and memory

Deploy on machines with a 1:4 ratio of CPU to memory. For every CPU core, you should have 4GB of memory. Grafana Labs recommends deploying onto machines with at least 16 CPU cores and 64GB of memory. All the nodes in the Kubernetes cluster running GET should be of the same type.

Network

All components require fast network access. Nodes the software runs on should be connected by 10 gigabit/second or faster network connection speed.

Storage

GET requires fast storage to run. Any storage solutions you use have to have adequate performance. Object storage (for example AWS S3 buckets, Google Cloud GCS or Microsoft Azure Storage) buckets or blobs are used to persist trace data for the required retention period. Your object storage should be large enough to contain 30 days worth of data, the default block retention for GET.

However, various components of GET (for example, the ingester) also require fast, persistent block storage to be available to the host machine. For example, in the case of the ingester component, all incoming data is sent to a write-ahead log (WAL) to aggregate trace data into blocks before storing them in object storage, and to help withstand unexpected node termination.

For every deployment, you need both object and block storage. Block storage must be memory-state storage, like SSD or NVMEs. Hard drive (platter-based) disk storage is not supported.

The supported configurations for AWS, GCP, and Azure are in the following sections.

Block storage

Platform	Storage type	Configuration option
Amazon Web Services	Block	`gp3` General Purpose SSD EBS volumes
Google Cloud Platform	Block	`pd-ssd` SSD persistent storage
Microsoft Azure	Block	`Premium SSD` SSD persistent storage

Object storage

GET requires object storage for configuration storage, as well as long-term data storage. Amazon AWS S3, GCP GCS, and Azure Blob storage are the only supported object stores for running GET.

Platform	Storage type	Configuration option	Comments
Amazon Web Services	Object	S3 Standard	S3 object storage service using the Standard storage class.
Google Cloud Platform	Object	STANDARD	GCS object storage service using the STANDARD storage class in both regional and dual regional storage locations.
Microsoft Azure	Object	Standard performance with LRS replication	Blob Storage object storage service with Standard performance and replication type Locally Redundant Storage (LRS)

Deploy GET with Helm

Thu, 17 Apr 2025 13:08:48 +0000

Deploy GET with Helm

The Helm charts for Grafana Enterprise Traces (GET) and Grafana Tempo allow you to configure, install, and upgrade Grafana Tempo or Grafana Enterprise Traces within a Kubernetes cluster.

The recommended method for installing GET is to use the tempo-distributed Helm chart, which deploys Tempo or GET in microservices mode.

To deploy GET using the tempo-distributed Helm chart, read the Get started with Grafana Tempo using Helm.

Note
Monitoring with Grafana Cloud is required for all GET installations. Professional services will assist you in configuring monitoring for your GET installation.

Set up the GET plugin for Grafana

Wed, 12 Mar 2025 12:54:02 +0000

Set up the GET plugin for Grafana

The Grafana Enterprise Traces (GET) plugin lets you create new tenants and then use the access page to create new roles to publish traces. These commands are completed using the Grafana user interface in lieu of commands that would normally be completed using the GET CLI.

This procedure explains how to enable and configure the GET plugin.

Before you begin

To configure the GET plugin, you need:

Grafana Enterprise 8.4.0 or higher
GET cluster access token
Microservice deployments require the GET gateway URL, for example: https://gateway.enterprise-traces.svc.cluster.local:3100

Refer to Deploy Grafana Enterprise on Kubernetes if you are using Kubernetes. Otherwise, refer to Install Grafana for more information.

Install the plugin in your Grafana Enterprise instance

There are multiple ways to install the plugin to your local Grafana Enterprise instance. For more information, refer to Grafana Enterprise Traces app installation (requires login).

After installing the plugin, you will need to restart the Grafana pod for the plugin to be loaded on startup. If using the “Deploy Grafana Enterprise on Kubernetes” instructions, you will need to access the shell for the Grafana pod that you have just deployed to use the GET plugin installation instructions “Installing on a local Grafana”.

Enable and configure the plugin

To enable and configure the plugin:

Log in to Grafana Enterprise.
From the top-left menu, select Configuration and then select Plugins.
Search for Grafana Enterprise Traces plugin. This option is only available if you have GET and Grafana Enterprise licenses installed.
From the Connection settings tab of the plugin, enable the plugin by clicking on Enable plugin.
Provide the necessary API settings so that the plugin can connect to your cluster:
1. Access token: Enter the admin-scoped access token that you generated when setting up your GET cluster. This is available in the logs output (or other configured output) from the tokengen job for the GET installation. The token is prefixed by the text Token:.
2. Grafana Enterprise Traces URL: Enter the URL of your GET cluster. For single-process clusters, this is any node in the cluster. For microservice deployments, this URL is the GET gateway (for example, http://gateway.enterprise-traces.svc.cluster.local:3100).
3. Admin API version: Choose the relevant API version. For a new installation, select v3.
Click Test and save configuration.
Navigate to Grafana Enterprise Traces (select the GET, the stylized ‘T’ icon, on the menu bar).
Verify that the plugin loads and can communicate with the GET admin API endpoints. You can quickly test this by selecting the ‘Licenses’ tab from the GET plugin and ensure that you see something like this:
```
Issuer:
https://grafana.com
Issued at:
2022-06-23 12:37:48
Expires:
2023-06-23 12:37:32
Max number of users:
0
Products:
grafana-enterprise-traces
```
Look at the default access policy under the Access Policies tab and ensure that there is a default access policy for the admin scope.

The next step is to set up a GET tenant and visualize your data.

Set up GET tenants

Wed, 12 Mar 2025 12:54:02 +0000

Set up a tenant in your Grafana Enterprise Traces cluster

Tenants provide a mechanism for trace isolation. Access policies may be set on a per-tenant basis. Authorization of requests is based on specified access policies.

These instructions assume that you have the Grafana Enterprise Traces (GET) plugin installed. Use this plugin to create tenants, access policies, and tokens for your GET cluster. The steps include:

Create a GET tenant.
Create an access policy.
Configure remote-write to your tenant.
Set up your tenant as a Grafana data source.
Visualize your data.

Create a GET tenant

Once a cluster is running, you can create new tenants. To create a GET tenant:

Within Grafana Enterprise, navigate to Apps > Grafana Enterprise Traces > Tenants.
Click on Create tenant.
Choose a name for this tenant. For demonstration purposes, use the name dev-tenant.
Select the cluster.
Click Create tenant.

Create an access policy

Access policies are used to authorize actions and operations by specified tenants. Access policies have a realm, which defines the set of tenants they apply to, and a scope, which defines the set of actions upon which they confer permissions to use.

Grafana Enterprise requires a data source access policy and token to access the traces in the tenant named dev-tenant.

Navigate to Apps > Grafana Enterprise Traces > Access policies.
Click Create access policy.
Choose a name for the policy. For demonstration purposes, use dev-read-write-policy.
Under Scopes, select the toggle next to the question Are you planning on creating a data source that uses this access policy?. This enables the traces:read policy.
Enable the scope traces:write to create an access policy that grants access for both reading and writing trace data (as the previous step toggles the traces:read policy automatically).
Select the tenant dev-tenant.
Click on Create.
From the newly created access policy, click Add token.
Name the token dev-token and click on Create.
In the next window, copy the token by selecting Copy to clipboard.

At this point, you can add a data source to your Grafana Enterprise instance by using the Create a data source option on this dialog or by manually adding one using the procedure below.

Select Create a data source. The data source fields are automatically configured based upon your instance.
Verify the created data source by opening the Configuration option in the Grafana menu and then selecting Data sources.

Set up remote-write to your tenant

To enable writes to your cluster, add a remote-write configuration snippet to the configuration file of an existing Grafana Alloy instance. If you do not have an existing traces collector, refer to Set up with Grafana Alloy. For Kubernetes, refer to the Grafana Alloy Kubernetes quick start guide.

Caution
Grafana Alloy is the new name for our distribution of the OTel collector. Grafana Agent has been deprecated and is in Long-Term Support (LTS) through October 31, 2025. Grafana Agent will reach an End-of-Life (EOL) on November 1, 2025. Read more about why we recommend migrating to Grafana Alloy.

The sample below is an example Alloy configuration below for the Kubernetes Helm chart (the values.yaml manifest):

alloy:
  extraPorts:
    - name: otlp-grpc
      port: 4317
      targetPort: 4317
      protocol: TCP
  configMap:
    create: true
    content: |-
      // Creates a receiver for OTLP gRPC.
      // You can easily add receivers for other protocols by using the correct component
      // from the reference list at: https://grafana.com/docs/alloy/latest/reference/components/
      otelcol.receiver.otlp "otlp_receiver" {
        // Listen on all available bindable addresses on port 4317 (which is the
        // default OTLP gRPC port) for the OTLP protocol.
        grpc {
          endpoint = "0.0.0.0:4317"
        }

        // Output straight to the OTLP gRPC exporter. We would usually do some processing
        // first, most likely batch processing, but for this example we pass it straight
        // through.
        output {
          traces = [
            otelcol.exporter.otlp.get.input,
          ]
        }
      }

      // Define an OTLP gRPC exporter to send all received traces to GET.
      // The unique label 'get' is added to uniquely identify this exporter.
      otelcol.exporter.otlp "get" {
          // Define the client for exporting.
          client {
              // Use Header authentication, defined below.
              auth = otelcol.auth.headers.credentials.handler
              // Send to the locally running Tempo instance, on port 4317 (OTLP gRPC).
              endpoint = "http://get-cluster-tempo-enterprise-gateway.get.svc.cluster.local:3100"
              // Configure TLS settings for communicating with the endpoint.
              tls {
                  // The connection is insecure.
                  insecure = true
                  // Do not verify TLS certificates when connecting.
                  insecure_skip_verify = true
              }
          }
      }

      // A default GET install doesn't expose a TLS-encrypted gateway.
      // Grafana Alloy expects gRPC (HTTP/2) traffic to be TLS-encrypted, and
      // therefore fails if the "basic_auth" authorization block is used. Because of
      // this, use the generic "headers" block to authenticate a GET
      // tenant and token.
      // The authorization must be exposed as a '<tenant>:<writeToken>' pair that
      // has been base64 encrypted.
      otelcol.auth.headers "credentials" {
        header {
          key = "Authorization"
          value = "Basic ZGV2LXRlbmFudDpaR1YyTFhKbFlXUXRkM0pwZEdWeUxYQnZiR2xqZVMxa1pXMXZMWFJ2YTJWdU9qWS9lemR1TVRWaEpEUXZQR012THpRMVN6Z3NKakZiTVE9PQo="
        }
      }

In the above Flow configuration, the token for authentication was generated using the dev-tenant user and writeable token via:

echo 'dev-tenant:ZGV2LXJlYWQtd3JpdGVyLXBvbGljeS1kZW1vLXRva2VuOjY/ezduMTVhJDQvPGMvLzQ1SzgsJjFbMQ==' | base64

Create a Grafana data source

To allow Grafana to read traces from GET, you must create a Tempo data source with the proper credentials.

Navigate to Connections > Data Sources.
Click on Add a data source.
Select Tempo.
Set the URL to http://<get-gateway-host>:<http_listen_port>/<http_api_prefix>, filling in the path to your gateway and the configured HTTP API prefix.
Enable Basic Auth. Use User dev-tenant and the token from your clipboard as the Password.
Click Save & Test.

You should see a message that says Data source is working.

Visualize your data

Once you have created a data source, you can visualize your traces in the Grafana Explore page.

Test your configuration using the Intro to MLTP application

The Intro to MLTP application provides an example five-service application that generates data for Tempo, Mimir, Loki, and Pyroscope. This procedure installs the application on your cluster so you can generate meaningful test data.

Navigate to https://github.com/grafana/intro-to-mltp to get the Kubernetes manifests for the Intro to MLTP application.

Clone the repository using commands similar to the ones below:

  git clone git+ssh://github.com/grafana/intro-to-mltp
  cp intro-to-mltp/k8s/mythical/* ~/tmp/intro-to-mltp-k8s

Change to the cloned repository: cd intro-to-mltp/k8s/mythical
In the mythical-beasts-deployment.yaml manifest, alter each TRACING_COLLECTOR_HOST environment variable instance value to point to the Alloy location. For example, based on an Alloy installation in the default namespace called and with a Helm installation called test:
YAML
```
 	- env:
     ...
     - name: TRACING_COLLECTOR_HOST
       value: test-grafana-alloy.default.svc.cluster.local
```

Deploy the Intro to MLTP application. It deploys into the default namespace.

    kubectl apply -f mythical-beasts-service.yaml,mythical-beasts-persistentvolumeclaim.yaml,mythical-beasts-deployment.yaml

Once the application is deployed, go to Grafana Enterprise and select the Explore menu item.
Select the GET data source from the list of data sources.
Select the Search Query type for the data source.
Select Run query.
Traces from the application are displayed in the traces Explore panel.

Monitor GET using Grafana Cloud

Thu, 17 Apr 2025 13:08:48 +0000

Monitor GET using Grafana Cloud

Note
Monitoring with Grafana Cloud is required for all GET installations. Professional services will assist you in configuring monitoring for your GET installation.

Grafana Labs champions the value of observability for your software applications and infrastructure. This extends to the Database solutions offered as part of the Grafana Enterprise Stack: namely Grafana Enterprise Metrics, Logs, and Traces. Successful operation of these databases requires comprehensive monitoring and alerting. This allows the teams managing these Databases to swiftly identify issues and diagnose their root causes. Grafana Labs itself maintains robust observability for the Grafana Enterprise Metrics, Logs, and Traces Databases it operates to power Grafana Cloud, our hosted observability-as-a-service offering.

Our recommended approach for monitoring your Grafana Enterprise Databases is to use Grafana Cloud. This approach is the easiest, most robust way to observe your database in accordance with Grafana Labs best practices and is included free of charge with your purchase of a Grafana Enterprise Database.

How it works

Each Grafana Enterprise Database is instrumented for observability by default. Each Database component exposes metrics on a scrapable Prometheus compatible /metrics endpoint and emits logs and traces.

The Helm chart for each Enterprise Database includes an option for meta monitoring. When enabled, the Helm chart deploys Grafana Alloy alongside the Enterprise Database. Alloy is configured to collect metrics and logs from all database components and apply additional metadata, such as extra labels to indicate where the metrics or logs were scraped from. Alloy ONLY collects logs and metrics relevant to the internal system state of the Grafana Enterprise Databases; no other information is collected.

From there, Grafana Alloy forwards the telemetry data to your Grafana Cloud account, where it can be stored and queried to help you understand the health of your Grafana Enterprise Database. You can access this account at any time, and have complete control over who has access to it using Grafana Cloud’s identity and access management functionality. Availability and uptime adhere to the Grafana Cloud SLA.

Additionally, this Grafana Cloud account is preconfigured with Grafana Labs’ recommended dashboards for visualizing the telemetry data collected from your Grafana Enterprise Database. Should you choose, you can also create alerts on the data in this account to notify you when the telemetry data collected from your Grafana Enterprise Database is no longer within expected parameters. Grafana Labs can provide a recommended set of alerts to configure.

If you need to file a support escalation, you can choose to give the Grafana Labs support team access to the telemetry data in your Grafana Cloud account. Historically, Grafana Labs has found that faster access to telemetry data radically reduces the time needed to resolve escalations.