APIs on Grafana Labs

Grafana Mimir HTTP API

Tue, 22 Apr 2025 17:05:43 +0000

Grafana Mimir HTTP API

Grafana Mimir exposes an HTTP API that you can use to write and query time series data, and operate the cluster.

This document groups API endpoints by service. Note that the API endpoints are exposed when you run Grafana Mimir in microservices mode, monolithic mode, and read-write mode:

Microservices mode: Each service exposes its own endpoints.
Monolithic mode: The Grafana Mimir instance exposes all API endpoints.
Read-write mode: The component services are exposed on the endpoint that they are contained within. Either Mimir read, Mimir write, or Mimir backend. Refer to Deployment modes for the grouping of components.

Endpoints

API	Service	Endpoint
Index page	All services	`GET /`
Configuration	All services	`GET /config`
Status Configuration	All services	`GET /api/v1/status/config`
Status Flags	All services	`GET /api/v1/status/flags`
Runtime Configuration	All services	`GET /runtime_config`
Services’ status	All services	`GET /services`
Readiness probe	All services	`GET /ready`
Metrics	All services	`GET /metrics`
Pprof	All services	`GET /debug/pprof`
Fgprof	All services	`GET /debug/fgprof`
Build information	All services	`GET /api/v1/status/buildinfo`
Memberlist cluster	All services	`GET /memberlist`
Get tenant limits	All services	`GET /api/v1/user_limits`
Remote write	Distributor	`POST /api/v1/push`
OTLP	Distributor	`POST /otlp/v1/metrics`
Tenants stats	Distributor	`GET /distributor/all_user_stats`
HA tracker status	Distributor	`GET /distributor/ha_tracker`
Flush chunks / blocks	Ingester	`GET,POST /ingester/flush`
Prepare for Shutdown	Ingester	`GET,POST,DELETE /ingester/prepare-shutdown`
Shutdown	Ingester	`POST /ingester/shutdown`
Prepare Partition Downscale	Ingester	`GET,POST,DELETE /ingester/prepare-partition-downscale`
Prepare Instance Ring Downscale	Ingester	`GET,POST,DELETE /ingester/prepare-instance-ring-downscale`
Ingesters ring status	Distributor,Ingester	`GET /ingester/ring`
Ingester tenants	Ingester	`GET /ingester/tenants`
Ingester tenant TSDB	Ingester	`GET /ingester/tsdb/{tenant}`
Instant query	Querier, Query-frontend	`GET,POST <prometheus-http-prefix>/api/v1/query`
Range query	Querier, Query-frontend	`GET,POST <prometheus-http-prefix>/api/v1/query_range`
Exemplar query	Querier, Query-frontend	`GET,POST <prometheus-http-prefix>/api/v1/query_exemplars`
Get series by label matchers	Querier, Query-frontend	`GET,POST <prometheus-http-prefix>/api/v1/series`
Get active series by selector	Query-frontend	`GET, POST <prometheus-http-prefix>/api/v1/cardinality/active_series`
Get label names	Querier, Query-frontend	`GET,POST <prometheus-http-prefix>/api/v1/labels`
Get label values	Querier, Query-frontend	`GET <prometheus-http-prefix>/api/v1/label/{name}/values`
Get metric metadata	Querier, Query-frontend	`GET <prometheus-http-prefix>/api/v1/metadata`
Remote read	Querier, Query-frontend	`POST <prometheus-http-prefix>/api/v1/read`
Label names cardinality	Querier, Query-frontend	`GET, POST <prometheus-http-prefix>/api/v1/cardinality/label_names`
Label values cardinality	Querier, Query-frontend	`GET, POST <prometheus-http-prefix>/api/v1/cardinality/label_values`
Build information	Querier, Query-frontend, Ruler	`GET <prometheus-http-prefix>/api/v1/status/buildinfo`
Format query	Querier, Query-frontend	`GET, POST <prometheus-http-prefix>/api/v1/format_query`
Get tenant ingestion stats	Querier	`GET /api/v1/user_stats`
Query-scheduler ring status	Query-scheduler	`GET /query-scheduler/ring`
Ruler ring status	Ruler	`GET /ruler/ring`
Ruler rules	Ruler	`GET /ruler/rule_groups`
List Prometheus rules	Ruler	`GET <prometheus-http-prefix>/api/v1/rules`
List Prometheus alerts	Ruler	`GET <prometheus-http-prefix>/api/v1/alerts`
List rule groups	Ruler	`GET <prometheus-http-prefix>/config/v1/rules`
Get rule groups by namespace	Ruler	`GET <prometheus-http-prefix>/config/v1/rules/{namespace}`
Get rule group	Ruler	`GET <prometheus-http-prefix>/config/v1/rules/{namespace}/{groupName}`
Set rule group	Ruler	`POST <prometheus-http-prefix>/config/v1/rules/{namespace}`
Delete rule group	Ruler	`DELETE <prometheus-http-prefix>/config/v1/rules/{namespace}/{groupName}`
Delete namespace	Ruler	`DELETE <prometheus-http-prefix>/config/v1/rules/{namespace}`
Delete tenant configuration	Ruler	`POST /ruler/delete_tenant_config`
Alertmanager status	Alertmanager	`GET /multitenant_alertmanager/status`
Alertmanager configs	Alertmanager	`GET /multitenant_alertmanager/configs`
Alertmanager ring status	Alertmanager	`GET /multitenant_alertmanager/ring`
Alertmanager UI	Alertmanager	`GET <alertmanager-http-prefix>`
Build Information	Alertmanager	`GET <alertmanager-http-prefix>/api/v1/status/buildinfo`
Alertmanager Delete Tenant Configuration	Alertmanager	`POST /multitenant_alertmanager/delete_tenant_config`
Get Alertmanager configuration	Alertmanager	`GET /api/v1/alerts`
Set Alertmanager configuration	Alertmanager	`POST /api/v1/alerts`
Delete Alertmanager configuration	Alertmanager	`DELETE /api/v1/alerts`
Store-gateway ring status	Store-gateway	`GET /store-gateway/ring`
Store-gateway tenants	Store-gateway	`GET /store-gateway/tenants`
Store-gateway tenant blocks	Store-gateway	`GET /store-gateway/tenant/{tenant}/blocks`
Prepare for Shutdown	Store-gateway	`GET,POST,DELETE /store-gateway/prepare-shutdown`
Compactor ring status	Compactor	`GET /compactor/ring`
Start block upload	Compactor	`POST /api/v1/upload/block/{block}/start`
Upload block file	Compactor	`POST /api/v1/upload/block/{block}/files?path={path}`
Complete block upload	Compactor	`POST /api/v1/upload/block/{block}/finish`
Check block upload	Compactor	`GET /api/v1/upload/block/{block}/check`
Tenant delete request	Compactor	`POST /compactor/delete_tenant`
Tenant delete status	Compactor	`GET /compactor/delete_tenant_status`
Compactor tenants	Compactor	`GET /compactor/tenants`
Compactor tenant planned jobs	Compactor	`GET /compactor/tenant/{tenant}/planned_jobs`
Overrides-exporter ring status	Overrides-exporter	`GET /overrides-exporter/ring`

Path prefixes

The following table provides usage of placeholder path prefixes, for prefixes that are configurable.

Prefix	Default	CLI flag	YAML configuration
`<prometheus-http-prefix>`	`/prometheus`	`-http.prometheus-http-prefix`	`api > prometheus_http_prefix`
`<alertmanager-http-prefix>`	`/alertmanager`	`-http.alertmanager-http-prefix`	`api > alertmanager_http_prefix`

Authentication

Endpoints that require authentication must be called with the X-Scope-OrgID HTTP request header specified to the tenant ID.

If you disable multi-tenancy, Grafana Mimir doesn’t require any request to include the X-Scope-OrgID header.

Multi-tenancy can be enabled and disabled via the -auth.multitenancy-enabled flag or its respective YAML configuration option.

For more information about authentication and authorization, refer to Authentication and Authorization.

All services

The following API endpoints are exposed by all services.

Index page

GET /

This endpoint displays an index page with links to other web pages exposed by Grafana Mimir.

Configuration

GET /config

This endpoint displays the configuration currently applied to Grafana Mimir including default values and settings via CLI flags. This endpoint provides the configuration in YAML format and masks sensitive data.

Note
The exported configuration doesn’t include the per-tenant overrides.

Different modes

GET /config?mode=diff

This endpoint displays the differences between the Grafana Mimir default configuration and the current configuration.

GET /config?mode=defaults

This endpoint displays the default configuration values.

Status Configuration

GET /api/v1/status/config

This endpoint displays empty configuration settings, it exists only to be compatible with the Prometheus /api/v1/status/config API.

Status Flags

GET /api/v1/status/flags

This endpoint displays empty configuration flags, it exists only to be compatible with the Prometheus /api/v1/status/flags API.

Runtime Configuration

GET /runtime_config

This endpoint displays the runtime configuration currently applied to Grafana Mimir, in YAML format, including default values. The endpoint is only available if Grafana Mimir is configured with the -runtime-config.file option.

Different modes

GET /runtime_config?mode=diff

This endpoint displays the differences between the Grafana Mimir default runtime configuration and the current runtime configuration.

Services’ status

GET /services

This endpoint displays a web page with the status of internal Grafana Mimir services.

Readiness probe

GET /ready

This endoint returns 200 when Grafana Mimir is ready to serve traffic.

Metrics

GET /metrics

This endpoint returns the metrics for the running Grafana Mimir service in the Prometheus exposition format.

Pprof

GET /debug/pprof/heap
GET /debug/pprof/block
GET /debug/pprof/profile
GET /debug/pprof/trace
GET /debug/pprof/goroutine
GET /debug/pprof/mutex

These endpoints return runtime profiling data in the format expected by the pprof visualization tool. There are many things that can be profiled using this endpoint, including heap, trace, goroutine, and so on.

For more information about pprof, refer to pprof.

Fgprof

GET /debug/fgprof

This endpoint returns the sampling Go profiling data that you can use to analyze On-CPU and Off-CPU (for example, I/O) time.

For more information about fgprof, refer to fgprof.

Build information

GET /api/v1/status/buildinfo
GET <prometheus-http-prefix>/api/v1/status/buildinfo
GET <alertmanager-http-prefix>/api/v1/status/buildinfo

This endpoint returns in JSON format information about the build and enabled features. The format returned is not identical, but is similar to the Prometheus Build Information endpoint.

Format query

GET <prometheus-http-prefix>/api/v1/format_query?query={query}
POST <prometheus-http-prefix>/api/v1/format_query

Formats the PromQL query.

This endpoint is compatible with the Prometheus format query endpoint.

For more information about formatting queries, refer to Prometheus’ documentation.

Memberlist cluster

GET /memberlist

This admin page shows information about Memberlist cluster (list of nodes and their health) and KV store (keys and values in the KV store).

If memberlist message history is enabled, this page also shows all received and sent messages stored in the buffers. This can be useful for troubleshooting memberlist cluster. To enable message history buffers use -memberlist.message-history-buffer-bytes CLI flag or the corresponding YAML configuration parameter.

Get tenant limits

GET /api/v1/user_limits

Returns realtime limits for the authenticated tenant, in JSON format. This API is experimental.

Requires authentication.

The endpoint is only available if Grafana Mimir is configured with the -runtime-config.file option.

Distributor

The following endpoints relate to the distributor.

Remote write

POST /api/v1/push

Entrypoint for the Prometheus remote write.

This endpoint accepts an HTTP POST request with a body that contains a request encoded with Protocol Buffers and compressed with Snappy. You can find the definition of the protobuf message in pkg/mimirpb/mimir.proto. The HTTP request must contain the header X-Prometheus-Remote-Write-Version set to 0.1.0.

To skip the label name validation, perform the following actions:

Enable API’s flag -api.skip-label-name-validation-header-enabled=true
Ensure that the request is sent with the header X-Mimir-SkipLabelNameValidation: true

This feature supports the writes from non-standard downstream clients that have metric name not Prometheus compliant.

For more information, refer to Prometheus Remote storage integrations.

Requires authentication.

OTLP

POST /otlp/v1/metrics

Note
To send OTLP data to Grafana Cloud, refer to Send data using OpenTelemetry Protocol (OTLP).

Entrypoint for the OTLP HTTP.

This endpoint accepts an HTTP POST request with a body that contains a request encoded with Protocol Buffers and optionally compressed with GZIP. You can find the definition of the protobuf message in metrics.proto.

Requires authentication.

Distributor ring status

GET /distributor/ring

This endpoint displays a web page with the distributor hash ring status, including the state, and the health and last heartbeat time of each distributor.

Tenants stats

GET /distributor/all_user_stats

This endpoint displays a web page that shows per-tenant statistics updated in real time, including the total number of active series across all ingesters and the current ingestion rate displayed in samples per second.

Note
This endpoint requires all ingesters to be ACTIVE in the ring for a successful response.

HA tracker status

GET /distributor/ha_tracker

This endpoint displays a web page with the current status of the HA tracker, including the elected replica for each Prometheus HA cluster.

Ingester

The following endpoints relate to the ingester.

Flush chunks / blocks

GET,POST /ingester/flush

This endpoint triggers a flush of the in-memory series time series data to the long-term storage. This endpoint also triggers the flush when -blocks-storage.tsdb.flush-blocks-on-shutdown is disabled.

This endpoint accepts a tenant parameter to specify the tenant whose blocks are compacted and shipped. This parameter might be specified multiple times to select more tenants. If no tenant is specified, all tenants are flushed.

The flush endpoint also accepts a wait=true parameter, which makes the call synchronous, and only returns a status code after flushing completes.

Note
The returned status code doesn’t reflect the result of flush operation.

Prepare for Shutdown

GET,POST,DELETE /ingester/prepare-shutdown

This endpoint inspects or changes in-memory ingester configuration to prepare for permanently stopping an ingester instance but does not actually stop any part of the ingester.

After a POST to the prepare-shutdown endpoint returns, when the ingester process is stopped with SIGINT / SIGTERM, the ingester will be unregistered from the ring and in-memory time series data will be flushed to long-term storage. This endpoint causes the ingester to be unregistered from the ring when stopped even if you disable -ingester.ring.unregister-on-shutdown.

A GET to the prepare-shutdown endpoint returns the status of this configuration, either set or unset.

A DELETE to the prepare-shutdown endpoint reverts the configuration of the ingester to its previous state (with respect to unregistering on shutdown and flushing of in-memory time series data to long-term storage).

This API endpoint is usually used by Kubernetes-specific scale down automations such as the rollout-operator.

Shutdown

POST /ingester/shutdown

This endpoint flushes in-memory time series data from ingesters to the long-term storage, and then shuts down the ingester service. After the shutdown endpoint returns, the operator or any automation that’s used terminates the process with a SIGINT / SIGTERM signal. During this time, /ready does not return 200. This endpoint unregisters the ingester from the ring even if you disable -ingester.ring.unregister-on-shutdown.

This API endpoint is usually used by scale down automations.

Prepare partition downscale

GET,POST,DELETE /ingester/prepare-partition-downscale

This endpoint prepares the ingester’s partition for downscaling by setting it to the INACTIVE state.

A GET call to this endpoint returns a timestamp of when the partition was switched to the INACTIVE state, or 0, if the partition is not in the INACTIVE state.

A POST call switches this ingester’s partition to the INACTIVE state, if it isn’t INACTIVE already, and returns the timestamp of when the switch to the INACTIVE state occurred.

A DELETE call sets the partition back from the INACTIVE to the ACTIVE state.

If the ingester is not configured to use ingest-storage, any call to this endpoint fails.

This API endpoint is usually used by scale down automation, e.g. rollout-operator.

Prepare instance ring downscale

GET,POST,DELETE /ingester/prepare-instance-ring-downscale

This endpoint prepares the ingester for downscaling by setting it to read-only mode.

A GET call to this endpoint returns a timestamp of when the ingester was switched to read-only mode, or 0, if the ingester is not in read-only mode.

A POST call switches this ingester’s partition to read-only mode, if it isn’t read-only already, and returns the timestamp of when the switch to read-only mode occurred.

A DELETE call sets the ingester back to read-write mode.

If the ingester is configured to use ingest-storage, any call to this endpoint fails.

This API endpoint is usually used by scale down automation, e.g. rollout-operator.

Prepare for unregister

GET,PUT,DELETE /ingester/unregister-on-shutdown

This endpoint controls whether an ingester should unregister from the ring on its next termination, that is, the next time it receives a SIGINT or SIGTERM signal. Via this endpoint, Mimir operators can dynamically control an ingester’s -ingester.ring.unregister-on-shutdown state without having to restart the ingester.

A PUT sets the ingester’s unregister state. When invoked with the PUT method, the endpoint takes a request body:

{"unregister": true}

A GET returns the ingester’s current unregister state.

A DELETE resets the ingester’s unregister state to the value that was passed via the -ingester.ring.unregister-on-shutdown configuration option.

Regardless of the HTTP method used, the endpoint always returns a response body with the ingester’s current unregister state:

{"unregister": true}

TSDB Metrics

GET /ingester/tsdb_metrics

This endpoint returns low-level metrics exposed by per-tenant TSDB. This can be useful for troubleshooting. Difference from /metrics is that metrics returned in /metrics are aggregated across all open TSDBs, while this endpoint returns TSDB metrics only for specific tenant.

Requires authentication, authenticated tenant is one whose TSDB metrics are returned.

Ingesters ring status

GET /ingester/ring

This endpoint displays a web page with the ingesters hash ring status, including the state, health, and last heartbeat time of each ingester.

Ingester tenants

GET /ingester/tenants

Displays a web page with the list of tenants with open TSDB on given ingester.

Ingester tenant TSDB

GET /ingester/tsdb/{tenant}

Displays a web page with details about tenant’s open TSDB on given ingester.

Querier / Query-frontend

The following endpoints are exposed both by the querier and query-frontend.

Instant query

GET,POST <prometheus-http-prefix>/api/v1/query

This endpoint is compatible with the Prometheus instant query endpoint.

For more information about Prometheus instant queries, refer to Prometheus instant query.

Requires authentication.

Range query

GET,POST <prometheus-http-prefix>/api/v1/query_range

This endpoint is compatible with the Prometheus range query endpoint. When a client sends a request through the query-frontend, the query-frontend uses caching and execution parallelization to accelerate the query.

For more information about Prometheus range queries, refer to Prometheus range query.

Requires authentication.

Exemplar query

GET,POST <prometheus-http-prefix>/api/v1/query_exemplars

This endpoint is compatible with the Prometheus exemplar query endpoint.

For more information about Prometheus exemplar queries, refer to Prometheus exemplar query.

Requires authentication.

Get series by label matchers

GET,POST <prometheus-http-prefix>/api/v1/series

For more information, refer to Prometheus series endpoint.

Requires authentication.

Get active series by selector

GET,POST <prometheus-http-prefix>/api/v1/cardinality/active_series

Returns the label sets of all active series matching a PromQL selector.

This endpoint is similar to the series endpoint but operates on the set of series considered active at the time of query processing. A series is considered active if any data has been written for it within the period specified by -ingester.active-series-metrics-idle-timeout.

This endpoint is disabled by default; you can enable it via the -querier.cardinality-analysis-enabled CLI flag (or its respective YAML configuration option).

Requires authentication.

Query parameters

selector - mandatory - PromQL selector used to filter the result set.

Headers

Sharding-Control - optional - Integer value specifying how many shards to use for request execution.

Response format

The response format is a subset of the series endpoint format including only the data field. The following shows an example request/response pair for this endpoint. Each item in the data array corresponds to a matched series.

$ curl 'http://localhost:9090/api/v1/cardinality/active_series' \
    --header 'Sharding-Control: 4' \ # optional
    --data-urlencode 'selector=up'

{
   "data" : [
      {
         "__name__" : "up",
         "job" : "prometheus",
         "instance" : "localhost:9090"
      },
      {
         "__name__" : "up",
         "job" : "node",
         "instance" : "localhost:9091"
      },
      {
         "__name__" : "process_start_time_seconds",
         "job" : "prometheus",
         "instance" : "localhost:9090"
      }
   ]
}

Caching

Responses for the active series endpoint are never cached.

Get label names

GET,POST <prometheus-http-prefix>/api/v1/labels

For more information, refer to Prometheus get label names.

Requires authentication.

Caching

The query-frontend can return a stale response fetched from the query results cache if -query-frontend.cache-results is enabled and -query-frontend.results-cache-ttl-for-labels-query set to a value greater than 0.

Get label values

GET <prometheus-http-prefix>/api/v1/label/{name}/values

For more information, refer to Prometheus get label values.

Requires authentication.

Caching

Get metric metadata

GET <prometheus-http-prefix>/api/v1/metadata

Prometheus-compatible metric metadata endpoint.

For more information, refer to Prometheus metric metadata.

Requires authentication.

Remote read

POST <prometheus-http-prefix>/api/v1/read

Prometheus-compatible remote read endpoint.

For more information, refer to Prometheus Remote storage integrations.

Requires authentication.

Label names cardinality

GET,POST <prometheus-http-prefix>/api/v1/cardinality/label_names

Returns label names cardinality across all ingesters, for the authenticated tenant, in JSON format. It counts distinct label values per label name.

The items in the field cardinality are sorted by label_values_count in DESC order and by label_name in ASC order. The count of items is limited by limit request param.

This endpoint is disabled by default and can be enabled via the -querier.cardinality-analysis-enabled CLI flag (or its respective YAML config option).

Requires authentication.

Count series by `inmemory` or `active`

Two methods of counting are available: inmemory and active. To choose one, use the count_method parameter.

The inmemory method counts the labels in currently opened TSDBs in Mimir’s ingesters. Two subsequent calls might return completely different results if an ingester cut a block between calls. This method of counting is most useful for understanding ingester memory usage.

The active method also counts labels in currently opened TSDBs in Mimir’s ingesters, but filters out values that have not received a sample within a configurable duration of time. To configure this duration, use the -ingester.active-series-metrics-idle-timeout parameter. This method of counting is most useful for understanding what label values are represented in the samples ingested by Mimir in the last -ingester.active-series-metrics-idle-timeout. Two subsequent calls will likely return similar results, because this window of time is not related to the block cutting on ingesters. Values will change only as a result of changes in the data ingested by Mimir.

Caching

Request params

selector - optional - specifies PromQL selector that will be used to filter series that must be analyzed.
count_method - optional - specifies which series counting method will be used. (default=“inmemory”, available options=[“inmemory”, “active”])
limit - optional - specifies max count of items in field cardinality in response (default=20, min=0, max=500)

Response schema

{
  "label_values_count_total": <number>,
  "label_names_count": <number>,
  "cardinality": [
    {
      "label_name": <string>,
      "label_values_count": <number>
    }
  ]
}

Label values cardinality

GET,POST <prometheus-http-prefix>/api/v1/cardinality/label_values

Returns label values cardinality associated to request param label_names[] across all ingesters, for the authenticated tenant, in JSON format. It returns the series count per label value associated to request param label_names[].

The items in the field labels are sorted by series_count in descending order and by label_name in ascending order. The items in the field cardinality are sorted by series_count in DESC order and by label_value in ASC order. The count of cardinality items is limited by request parameter limit.

This endpoint is disabled by default; you can enable it via the -querier.cardinality-analysis-enabled CLI flag (or its respective YAML configuration option).

Requires authentication.

Count series by `inmemory` or `active`

Two methods of counting are available: inmemory and active. To choose one, use the count_method parameter.

The inmemory method counts the number of series in currently opened TSDBs in Mimir’s ingesters. Two subsequent calls might return completely different results if an ingester cut a block between calls. This method of counting is most useful for understanding ingester memory usage.

The active method also counts series in currently opened TSDBs in Mimir’s ingesters, but filters out series that have not received a sample within a configurable duration of time. To configure this duration, use the -ingester.active-series-metrics-idle-timeout parameter. This method of counting is most useful for understanding what label values are represented in the samples ingested by Mimir in the last -ingester.active-series-metrics-idle-timeout. Two subsequent calls will likely return similar results, because this window of time is not related to the block cutting on ingesters. Values will change only as a result of changes in the data ingested by Mimir.

Caching

Request params

label_names[] - required - specifies labels for which cardinality must be provided.
selector - optional - specifies PromQL selector that will be used to filter series that must be analyzed.
count_method - optional - specifies which series counting method will be used. (default=“inmemory”, available options=[“inmemory”, “active”])
limit - optional - specifies max count of items in field cardinality in response (default=20, min=0, max=500).

Response schema

{
  "series_count_total": <number>,
  "labels": [
    {
      "label_name": <string>,
      "label_values_count": <number>,
      "series_count": <number>,
      "cardinality": [
        {
          "label_value": <string>,
          "series_count": <number>
        }
      ]
    }
  ]
}

series_count_total - total number of series across opened TSDBs in all ingesters
labels[].label_name - label name requested via the request param label_names[]
labels[].label_values_count - total number of label values for the label name (note that dependent on the limit request param it is possible that not all label values are present in cardinality)
labels[].series_count - total number of series having labels[].label_name
labels[].cardinality[].label_value - label value associated to labels[].label_name
labels[].cardinality[].series_count - total number of series having label_value for label_name

Querier

Get tenant ingestion stats

GET /api/v1/user_stats

Returns realtime ingestion rate, for the authenticated tenant, in JSON format.

Requires authentication.

Query-scheduler

Query-scheduler ring status

GET /query-scheduler/ring

Displays a web page with the query-scheduler hash ring status, including the state, healthy and last heartbeat time of each query-scheduler. The query-scheduler ring is available only when -query-scheduler.service-discovery-mode is set to ring.

Ruler

The ruler API endpoints require to configure a backend object storage to store the recording rules and alerts. The ruler API uses the concept of a “namespace” when creating rule groups. This is a stand in for the name of the rule file in Prometheus and rule groups must be named uniquely within a namespace.

Ruler ring status

GET /ruler/ring

Displays a web page with the ruler hash ring status, including the state, healthy and last heartbeat time of each ruler.

Ruler rules

GET /ruler/rule_groups

List all tenant rules. This endpoint is not part of ruler-API and is always available regardless of whether ruler-API is enabled or not. It should not be exposed to end users. This endpoint returns a YAML dictionary with all the rule groups for each tenant and 200 status code on success.

List Prometheus rules

GET <prometheus-http-prefix>/api/v1/rules?type={alert|record}&file={}&rule_group={}&rule_name={}&exclude_alerts={true|false}

Prometheus-compatible rules endpoint to list alerting and recording rules that are currently loaded.

The type parameter is optional. If set, only the specified type of rule is returned.

The file, rule_group and rule_name parameters are optional, and can accept multiple values. If set, the response content is filtered accordingly. The parameters can also be provided as file[], rule_group[] and rule_name[] - if both are provided e.g file and file[] , file[] will take precdent.

The exclude_alerts parameter is optional. If set, it only returns rules and excludes active alerts.

The group_limit and group_next_token parameters are optional. If group_limit is set, it will limit the number of rule groups returned in a single response. If the total number of rule groups exceeds this value, the response will contain a groupNextToken. This can be passed into subsequent requests via group_next_token to paginate over the remaining groups. The final response will not contain a token. For more information, refer to Prometheus rules.

Requires authentication.

List Prometheus alerts

GET <prometheus-http-prefix>/api/v1/alerts

Prometheus-compatible rules endpoint to list of all active alerts.

For more information, refer to Prometheus alerts documentation.

Requires authentication.

List rule groups

GET <prometheus-http-prefix>/config/v1/rules

List all rules configured for the authenticated tenant. This endpoint returns a YAML dictionary with all the rule groups for each namespace and 200 status code on success.

This endpoint can be disabled via the -ruler.enable-api CLI flag (or its respective YAML config option).

Requires authentication.

Note
To list all rule groups from Mimir, use mimirtool rules list command.

Example response

---
<namespace1>:
- name: <string>
  interval: <duration;optional>
  source_tenants:
    - <string>
  rules:
  - record: <string>
      expr: <string>
  - alert: <string>
      expr: <string>
      for: <duration>
      annotations:
        <annotation_name>: <string>
      labels:
        <label_name>: <string>
- name: <string>
  interval: <duration;optional>
  source_tenants:
    - <string>
  rules:
  - record: <string>
      expr: <string>
  - alert: <string>
      expr: <string>
      for: <duration>
      annotations:
        <annotation_name>: <string>
      labels:
        <label_name>: <string>
<namespace2>:
- name: <string>
  interval: <duration;optional>
  source_tenants:
    - <string>
  rules:
  - record: <string>
      expr: <string>
  - alert: <string>
      expr: <string>
      for: <duration>
      annotations:
        <annotation_name>: <string>
      labels:
        <label_name>: <string>

Get rule groups by namespace

GET <prometheus-http-prefix>/config/v1/rules/{namespace}

Returns the rule groups defined for a given namespace. Escape the {namespace} path segment using percent-encoding, as defined by RFC 3986. For example, escape / to %2F.

This endpoint can be disabled via the -ruler.enable-api CLI flag (or its respective YAML config option).

Requires authentication.

Example response

name: <string>
interval: <duration;optional>
source_tenants:
  - <string>
rules:
  - record: <string>
    expr: <string>
  - alert: <string>
    expr: <string>
    for: <duration>
    annotations:
      <annotation_name>: <string>
    labels:
      <label_name>: <string>

Get rule group

GET <prometheus-http-prefix>/config/v1/rules/{namespace}/{groupName}

Returns the rule group matching the request namespace and group name. Escape the {namespace} and {groupName} path segments using percent-encoding, as defined by RFC 3986. For example, escape / to %2F.

This endpoint can be disabled via the -ruler.enable-api CLI flag (or its respective YAML config option).

Requires authentication.

Note
To retrieve a single rule group from Mimir, use mimirtool rules get command .

Set rule group

POST /<prometheus-http-prefix>/config/v1/rules/{namespace}

Creates or updates a rule group. This endpoint expects a request with Content-Type: application/yaml header and the rules group YAML definition in the request body, and returns 202 on success. The request body must contain the definition of one and only one rule group. Escape the {namespace} path segment using percent-encoding, as defined by RFC 3986. For example, escape / to %2F.

This endpoint can be disabled via the -ruler.enable-api CLI flag (or its respective YAML config option).

Requires authentication.

Note
To load one or more rule groups into Mimir, use mimirtool rules load command .

Note
When using curl to send the request body from a file, ensure that you use the --data-binary flag instead of -d, --data, or --data-ascii.

The latter options don’t preserve carriage returns and newlines.

Example request body

name: MyGroupName
rules:
  - alert: MyAlertName
    expr: up == 0
    labels:
      severity: warning

Delete rule group

DELETE /<prometheus-http-prefix>/config/v1/rules/{namespace}/{groupName}

Deletes a rule group by namespace and group name. This endpoints returns 202 on success. Escape the {namespace} and {groupName} path segments using percent-encoding, as defined by RFC 3986. For example, escape / to %2F.

This endpoint can be disabled via the -ruler.enable-api CLI flag (or its respective YAML config option).

Requires authentication.

Note
To delete a rule group from Mimir, use mimirtool rules delete command.

Delete namespace

DELETE /<prometheus-http-prefix>/config/v1/rules/{namespace}

Deletes all the rule groups in a namespace (including the namespace itself). This endpoint returns 202 on success. Escape the {namespace} path segment using percent-encoding, as defined by RFC 3986. For example, escape / to %2F.

This endpoint can be disabled via the -ruler.enable-api CLI flag (or its respective YAML config option).

Requires authentication.

Delete tenant configuration

POST /ruler/delete_tenant_config

This deletes all rule groups for a tenant, and returns 200 on success. Calling this endpoint when no rule groups exist for a tenant returns 200. Authentication is only to identify the tenant.

This is intended as internal API, and not to be exposed to users. This endpoint is enabled regardless of whether -ruler.enable-api is enabled or not.

Requires authentication.

Alertmanager

Alertmanager status

GET /multitenant_alertmanager/status

Displays a web page with the current status of the Alertmanager, including the Alertmanager cluster members.

Alertmanager configs

GET /multitenant_alertmanager/configs

List all Alertmanager configurations. This endpoint is not part of Alertmanager API and is always available regardless of whether Alertmanager API is enabled or not. It should not be exposed to end users. This endpoint returns a YAML dictionary with all the Alertmanager configurations and 200 status code on success.

Alertmanager ring status

GET /multitenant_alertmanager/ring

Displays a web page with the Alertmanager hash ring status, including the state, healthy and last heartbeat time of each Alertmanager instance.

Alertmanager UI

GET /<alertmanager-http-prefix>

Displays the Alertmanager UI.

Requires authentication.

Alertmanager Delete Tenant Configuration

POST /multitenant_alertmanager/delete_tenant_config

This endpoint deletes configuration for a tenant identified by X-Scope-OrgID header. It is internal, available even if Alertmanager API is disabled. The endpoint returns a status code of 200 if the user’s configuration has been deleted, or it didn’t exist in the first place.

Requires authentication.

Get Alertmanager configuration

GET /api/v1/alerts

Get the current Alertmanager configuration for the authenticated tenant, reading it from the configured object storage.

This endpoint doesn’t accept any URL query parameter and returns 200 on success.

This endpoint can be enabled and disabled via the -alertmanager.enable-api CLI flag (or its respective YAML config option).

Requires authentication.

Note
To retrieve a tenant’s Alertmanager configuration from Mimir, use mimirtool alertmanager get command.

Set Alertmanager configuration

POST /api/v1/alerts

Stores or updates the Alertmanager configuration for the authenticated tenant. The Alertmanager configuration is stored in the configured backend object storage.

This endpoint expects the Alertmanager YAML configuration in the request body and returns 201 on success.

The names of the templates in template_files must be valid file names and not contain any path separators. For example, both /templates/my-template.tpl and ./my-template.tpl are invalid, whereas my-template.tpl is valid.

This endpoint can be enabled and disabled via the -alertmanager.enable-api CLI flag (or its respective YAML config option).

Requires authentication.

Note
To load a tenant’s Alertmanager configuration to Mimir, use mimirtool alertmanager load command.

Note
When using curl to send the request body from a file, ensure that you use the --data-binary flag instead of -d, --data, or --data-ascii.

The latter options don’t preserve carriage returns and newlines.

Example request body

template_files:
  default_template: |
    {{ define "__alertmanager" }}AlertManager{{ end }}
    {{ define "__alertmanagerURL" }}{{ .ExternalURL }}/#/alerts?receiver={{ .Receiver | urlquery }}{{ end }}
alertmanager_config: |
  global:
    smtp_smarthost: 'localhost:25'
    smtp_from: 'youraddress@example.org'
  templates:
    - 'default_template'
  route:
    receiver: example-email
  receivers:
    - name: example-email
      email_configs:
      - to: 'youraddress@example.org'

Delete Alertmanager configuration

DELETE /api/v1/alerts

Deletes the Alertmanager configuration for the authenticated tenant.

This endpoint doesn’t accept any URL query parameter and returns 200 on success.

This endpoint can be enabled and disabled via the -alertmanager.enable-api CLI flag (or its respective YAML config option).

Requires authentication.

Note
To delete a tenant’s Alertmanager configuration from Mimir, use mimirtool alertmanager delete command.

Store-gateway

Store-gateway ring status

GET /store-gateway/ring

Displays a web page with the store-gateway hash ring status, including the state, healthy and last heartbeat time of each store-gateway.

Store-gateway tenants

GET /store-gateway/tenants

Displays a web page with the list of tenants with blocks in the storage configured for store-gateway.

Store-gateway tenant blocks

GET /store-gateway/tenant/{tenant}/blocks

Displays a web page listing the blocks for a given tenant.

Prepare for Shutdown

GET,POST,DELETE /store-gateway/prepare-shutdown

This endpoint changes in-memory store-gateway configuration to prepare for permanently stopping a store-gateway instance but does not actually stop any part of the latter.

After a POST to the prepare-shutdown endpoint returns, when the store-gateway process is stopped with SIGINT / SIGTERM, the store-gateway will be unregistered from the ring.

A GET to the prepare-shutdown endpoint returns the status of this configuration, either set or unset.

A DELETE to the prepare-shutdown endpoint reverts the configuration of the store-gateway to its previous state (with respect to unregistering).

This API endpoint is usually used by Kubernetes-specific scale down automations such as the rollout-operator.

Compactor

Compactor ring status

GET /compactor/ring

Displays a web page with the compactor hash ring status, including the state, healthy and last heartbeat time of each compactor.

Start block upload

POST /api/v1/upload/block/{block}/start

Starts the uploading of a TSDB block with a given ID to object storage. The client should send the block’s meta.json file as the request body. If the complete block already exists in object storage, a 409 (Conflict) status code gets returned. If the provided meta.json file is invalid, a 400 (Bad Request) status code gets returned. If the block’s max time is before the tenant’s retention period, a 422 (Unprocessable Entity) status code gets returned.

The provided meta.json file must have a thanos.files section with the list of the block’s files, otherwise the request will be rejected.

If the API request succeeds, a sanitized version of the block’s meta.json file gets uploaded to object storage as uploading-meta.json, and a 200 status code gets returned. Then you can start uploading files, and once done, you can request completion of the block upload.

Requires authentication.

Upload block file

POST /api/v1/upload/block/{block}/files?path={path}

Uploads a file with a given path, for a block with a given ID. The file path has to be one of the following, otherwise a 400 (Bad Request) status code gets returned:

index
chunks/<6-digit number>

The client must send the content of the file as the body of the request; if the body is empty, a 400 (Bad Request) status code gets returned. If the complete block already exists in object storage, a 409 (Conflict) status code gets returned. If an in-flight meta file (uploading-meta.json) doesn’t exist in object storage for the block in question, a 404 (Not Found) status code gets returned.

If the API request succeeds, the file gets uploaded with the given path to the block’s directory in object storage, and a 200 status code gets returned.

Requires authentication.

Complete block upload

POST /api/v1/upload/block/{block}/finish

Initiates the completion of a TSDB block with a given ID to object storage. If the complete block already exists in object storage, a 409 (Conflict) status code gets returned. If an in-flight meta file (uploading-meta.json) doesn’t exist in object storage for the block in question, a 404 (Not Found) status code gets returned. If the compactor has reached its limit for the maximum number of concurrent block upload validations, which is configured with -compactor.max-block-upload-validation-concurrency, a 429 (Too Many Requests) will be returned.

If the API request succeeds, compactor will start the block validation in the background. If the background validation passes block upload is finished by renaming in-flight meta file to meta.json in the block’s directory.

This API endpoint returns 200 (OK) at the beginning of the validation. To further check state of the block upload, use Check block upload API endpoint.

Requires authentication.

This API endpoint is experimental and subject to change.

Check block upload

GET /api/v1/upload/block/{block}/check

Returns state of the block upload. State is returned as JSON object with field result, with following possible values:

complete – block validation is complete, and block upload is now finished.
uploading – block is still being uploaded, and Complete block upload has not yet been called on the block.
validating – block is being validated. Validation was started by call to Complete block upload API.
failed – block validation has failed. Error message is available from error field of the returned JSON object.

Example response

{ "result": "uploading" }

Example response

{ "result": "failed", "error": "missing index file" }

Requires authentication.

This API endpoint is experimental and subject to change.

Tenant Delete Request

POST /compactor/delete_tenant

Request deletion of ALL tenant data for the tenant specified in the X-Scope-OrgID header. If authentication is disabled, then the default anonymous tenant is deleted (configurable by -auth.no-auth-tenant).

Requires authentication.

Tenant Delete Status

GET /compactor/delete_tenant_status

Returns status of tenant deletion.

Response schema

{
  "tenant_id": "<id>",
  "blocks_deleted": true
}

The blocks_deleted field will be set to true if all the tenant’s blocks have been deleted.

Requires authentication.

Compactor tenants

GET /compactor/tenants

Displays a web page with the list of tenants that have blocks in the storage configured for the compactor.

Compactor tenant planned jobs

GET /compactor/tenant/{tenant}/planned_jobs

Displays a web page listing planned compaction jobs computed from the bucket index for the given tenant.

Overrides-exporter

Overrides-exporter ring status

GET /overrides-exporter/ring

Displays a web page with the overrides-exporter hash ring status, including the state, healthy and last heartbeat time of each overrides-exporter. The overrides-exporter ring is available only when -overrides-exporter.ring.enabled is set to true.

GEM Admin API

Thu, 12 Jun 2025 13:03:13 +0000

GEM Admin API

NOTE: Admin API version v1 /admin/api/v1/ and v2 /admin/api/v2/ endpoints are deprecated and will be removed in a future release of GEM. Use the new Admin API version v3 endpoints /admin/api/v3/ instead.

In addition to the standard Grafana Mimir API endpoints, Grafana Enterprise Metrics supports an admin HTTP API for managing cluster resources such as tenants and tokens.

API operations

The API supports standard CRUD (create, read, update, and delete) operations for most resources. Creates are issued via POST requests, reads via GET requests, updates via PUT requests, and deletes via DELETE requests. When running multiple instances of the Admin API, we recommend that you enable leader election so that all mutations occur on a single, leader-elected instance.

Write consistency (If-Match and ETag headers)

To ensure concurrent changes are propagated in a way that avoids unintentional overwrites, an If-Match header is required for all requests that mutate data. The current version of a resource to be sent in the If-Match header can be found in the ETag header returned on all GET and PUT requests that return a single resource in the response body.

Here are a few examples of how to get the value of the ETag header using curl. The first example uses the argument -D - to write all headers to standard out, and then parses for the ETag header.

curl -u :$API_TOKEN -sD - -o /dev/null http://localhost:8080/admin/api/v3/accesspolicies/{name} | grep -i ETag | cut -d " " -f 2

This second example uses the -i option to print the headers along with the response payload.

curl -u :$API_TOKEN -i http://localhost:8080/admin/api/v3/accesspolicies/{name}

When making a request to mutate data, make sure to include the double quotes (") around the value in the If-Match header. So, for example, the following is a request to update the access policy named the-access-policy, currently at version 1:

curl -u :$API_TOKEN -X PUT -H 'If-Match: "1"' http://localhost:8080/admin/api/v3/accesspolicies/the-access-policy --data '{"status": "active", "realms": [{"tenant": "enterprise-metrics-dev", "cluster": "enterprise-metrics-dev"}], "scopes": ["metrics:write"]}'

Optionally, a wildcard "*" may be passed as the If-Match version which effectively disables the write consistency protections and allows any current version to be updated.

Cluster API

List clusters

GET /admin/api/v1/clusters GET /admin/api/v2/clusters

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v3/clusters instead.

GET /admin/api/v3/clusters

Response:

{
  "items": [
    {
      "name":  "<string>",              // cluster name
      "display_name": "<string>",       // cluster display name
      "created_at": "<rfc3339 string>", // date created
      "kind": "<string>",               // cluster type, currently only cortex is supported
      "base_url": "<string>"            // Base URL of the cluster, optional
    }
    ...
  ],
  "type": "cluster"
}

Example:

$ curl -u :$API_TOKEN http://localhost:8080/admin/api/v3/clusters | jq
{
  "items": [
    {
      "name": "enterprise-metrics-dev",
      "display_name": "enterprise-metrics-dev",
      "created_at": "2020-07-13T16:50:41.953793Z",
      "kind": "metrics",
      "base_url": ""
    }
  ],
  "type": "cluster"
}

Get cluster

GET /admin/api/v1/clusters/{name} GET /admin/api/v2/clusters/{name}

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v3/clusters/{name} instead.

GET /admin/api/v3/clusters/{name}

NOTES:

There is no ETag header on a cluster response because clusters are immutable.

Response:

{
  "name": "<string>", // cluster name
  "display_name": "<string>", // cluster display name
  "created_at": "<rfc3339 string>", // date created
  "kind": "<string>", // cluster type, currently only cortex is supported
  "base_url": "<string>" // Base URL of the cluster, optional
}

Example:

$ curl -u :$API_TOKEN http://localhost:8080/admin/api/v3/clusters/enterprise-metrics-dev | jq
{
  "name": "enterprise-metrics-dev",
  "display_name": "enterprise-metrics-dev",
  "created_at": "2020-07-13T16:50:41.953793Z",
  "kind": "metrics",
  "base_url": ""
}

Tenant API

Model

The tenant model is made up of the following fields:

name: The machine-readable name of a tenant. It must be between 3 and 64 characters and only include the following characters, [a-z0-9-_]. Once set, this field is immutable.
display_name: The human-readable name of a tenant. It can contain any set of characters and can be changed.
status: The current status of this tenant. It can have the following values:
- active
- inactive
- unknown
cluster: The name of the Grafana Enterprise Metrics cluster that this tenant is scoped to. You will only be able to write/query this tenant from this cluster.
limits: Tenant-specific resource usage limits. Limits are key-value pairs (JSON object) that correspond to Grafana Mimir limits configuration.

Deprecated: The limits field is deprecated and set to be removed in a future release of GEM. Set this field to limits: null or remove it from your configuration. Instead, define limits using runtime configuration in Grafana Mimir. For information on how to migrate limit settings to runtime configuration, refer to Migrate your limits configuration.

List tenants

GET /admin/api/v1/instances GET /admin/api/v2/tenants

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v3/tenants instead.

GET /admin/api/v3/tenants

NOTES:

This endpoint only returns tenants in an active status. To get all tenants including inactive ones use the query parameter include-non-active=true

Response:

{
  "items": [
    {
      "name": "<string>",
      "display_name": "<string>",
      "created_at": "<rfc3339 string>",
      "status": "<string>",
      "cluster": "<string>"
    },
    ...
  ],
  "type": "tenant"
}

Example:

$ curl -u :$API_TOKEN localhost:8080/admin/api/v3/tenants | jq
{
  "items": [
    {
      "name": "enterprise-metrics-dev",
      "display_name": "Grafana Enterprise Metrics dev tenant",
      "created_at": "2020-07-13T17:37:59.341728283Z",
      "status": "active",
      "cluster": "enterprise-metrics-dev"
    }
  ],
  "type": "tenant"
}

Create tenant

POST /admin/api/v1/instances POST /admin/api/v2/tenants

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint POST /admin/api/v3/tenants instead.

POST /admin/api/v3/tenants

NOTES:

Because tenant names are unique, you cannot create a new tenant with the same name as a tenant that already exists but is in the inactive state. Instead, to use the same tenant name again you must re-enable the inactive tenant by updating its state to active. To make this change, use the Update Tenant endpoint. When creating a tenant via this API call, it is not required to specify the status field because the status is always overwritten by the API to active.

Example:

$ curl -u :$API_TOKEN -X POST localhost:8080/admin/api/v3/tenants --data-raw '
{
  "name": "enterprise-metrics-dev",
  "display_name": "Grafana Enterprise Metrics dev tenant",
  "created_at": "2020-07-13T17:37:59.341728283Z",
  "status": "active",
  "cluster": "enterprise-metrics-dev"
}'

Response:

{
  "name": "enterprise-metrics-dev",
  "display_name": "Grafana Enterprise Metrics dev tenant",
  "created_at": "2020-07-13T17:37:59.341728283Z",
  "status": "active",
  "cluster": "enterprise-metrics-dev"
}

Update tenant

PUT /admin/api/v1/instances/{name} PUT /admin/api/v2/tenants/{name}

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint PUT /admin/api/v3/tenants/{name} instead.

PUT /admin/api/v3/tenants/{name}

NOTES:

The name field of the object in the request body is ignored during updates.
The created_at field may not be modified during updates.
If-Match header matching the current version is required.

Response:

{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "status": "<string>",
  "cluster": "<string>"
}

Example:

curl -u :$API_TOKEN localhost:8080/admin/api/v3/tenants/enterprise-metrics-dev \
-X PUT -H 'If-Match: "123"` \
--data '{"display_name":"Grafana Enterprise Metrics dev tenant", "status": "active", "cluster": "enterprise-metrics-dev"}' | jq
{
  "name": "enterprise-metrics-dev",
  "display_name": "Grafana Enterprise Metrics dev tenant",
  "created_at": "2020-07-13T17:37:59.341728283Z",
  "status": "active",
  "cluster": "enterprise-metrics-dev"
}

Get tenant

GET /admin/api/v1/instances/{name} GET /admin/api/v2/tenants/{name}

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v3/tenants/{name} instead.

GET /admin/api/v3/tenants/{name}

NOTES:

The current version of the tenant will be returned in an ETag header on the response.

Response:

{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "status": "<string>",
  "cluster": "<string>"
}

Example:

$ curl -u :$API_TOKEN localhost:8080/admin/api/v3/tenants/enterprise-metrics-dev | jq
{
  "name": "enterprise-metrics-dev",
  "display_name": "Grafana Enterprise Metrics dev tenant",
  "created_at": "2020-07-13T17:37:59.341728283Z",
  "status": "active",
  "cluster": "enterprise-metrics-dev"
}

Delete tenant

DELETE /admin/api/v1/instances/{name} DELETE /admin/api/v2/instances/{name}

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. In Admin API version v3, delete operations are no longer supported and have been replaced by soft deletes. Use the endpoint PUT /admin/api/v3/tenants/{name} with "status": "inactive" instead. This operation will inactivate a tenant and subsequent HTTP requests for the tenant will fail with a 401 Unauthorized HTTP status code. To re-enable a tenant, use the endpoint PUT /admin/api/v3/tenants/{name} with "status": "active".

PUT /admin/api/v3/tenants/{name}

{
  "status": "inactive",
  "cluster": "<string>"
}

NOTES:

If-Match header matching the current version is required.

Example:

curl -u :$API_TOKEN localhost:8080/admin/api/v3/tenants/enterprise-metrics-dev \
-X PUT -H 'If-Match: "123"` \
--data '{"status": "inactive", "cluster": "enterprise-metrics-dev"}' | jq
{
  "name": "enterprise-metrics-dev",
  "display_name": "Grafana Enterprise Metrics dev tenant",
  "created_at": "2020-07-13T17:37:59.341728283Z",
  "status": "inactive",
  "cluster": "enterprise-metrics-dev"
}

Access policy API

List access policies

GET /admin/api/v1/accesspolicies GET /admin/api/v2/accesspolicies

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v3/accesspolicies instead.

GET /admin/api/v3/accesspolicies

NOTES:

This endpoint only returns access policies in an active status. To get all access policies including inactive ones use the query parameter include-non-active=true

Response:

{
  "items": [
    {
      "name": "<string>",
      "display_name": "<string>",
      "created_at": "<rfc3339 string>",
      "status": "<string>",
      "realms": [
        {
          "tenant": "<string>",
          "cluster": "<string>"
        }
      ],
      "scopes": [
        "<string>",
        ...
      ]
    },
    ...
  ],
  "type": "access_policy"
}

Create access policy

POST /admin/api/v1/accesspolicies POST /admin/api/v2/accesspolicies

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint POST /admin/api/v3/accesspolicies instead.

POST /admin/api/v3/accesspolicies

Payload:

{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "realms": [
    {
      "tenant": "<string>",
      "cluster": "<string>"
    }
  ],
  "scopes": [
    "<string>",
    ...
  ]
}

Because access policy names are unique, you cannot create a new access policy with the same name as an access policy that already exists but is in the inactive state. Instead, to use the same access policy name again you must re-enable the inactive access policy by updating its state to active. To make this change, use the Update Access Policy endpoint. When creating an access policy via this API call, it is not required to specify the status field because the status is always overwritten by the API to active.

Realms

Realms designate the tenant/cluster pairs that the access policy allows requests for:

tenant: This field must be set to an existing tenant or *. * denotes access to all tenants.
cluster: This field must be set to an existing cluster.
label_policies: This is an optional field that can be set to a list of label policies. An access policy gets access to metrics that match at least one of the label policies. If label policies are not set, all metrics will match.

Label policy, selector, and label matchers

A label policy contains a selector that’s a PromQL metric selector string:

{
  "selector": "<PromQL metric selector>"
}

The PromQL metric selector {job="grafana-labs", role!="software-engineer"} translates into this label policy:

{
  "selector": "{job=\"grafana-labs\", role!=\"software-engineer\"}"
}

Scopes

Scopes designate what operations tokens assigned to this access policy will be able to do when calling the GEM API.

admin: Permission to perform admin operations
admin:read: Permission to view admin resources (access policies and tokens, tenants, and clusters)
alerts:read: Permission to view the Alertmanager UI and get the Alertmanager configuration for a tenant
alerts:write: Permission to create silences in the Alertmanager UI and create and delete the Alertmanager configuration for a tenant
metrics:delete: Permission to delete data from a tenant
metrics:read: Permission to view data from a tenant
metrics:write: Permission to write data to a tenant
rules:read: Permission to read ruler rules
rules:write: Permission to write ruler rules

Update access policy

PUT /admin/api/v1/accesspolicies/{name} PUT /admin/api/v2/accesspolicies/{name}

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint PUT /admin/api/v3/accesspolicies/{name} instead.

PUT /admin/api/v3/accesspolicies/{name}

NOTES:

If-Match header matching the current version is required.
Update differs from create in that only the Display Name, Status, Realms, and Scopes are updatable.

Payload:

{
  "display_name": "<string>",
  "status": "<string>",
  "realms": [
    {
      "tenant": "<string>",
      "cluster": "<string>"
    }
  ],
  "scopes": [
    "<string>",
    ...
  ]
}

Get access policy

GET /admin/api/v1/accesspolicies/{name} GET /admin/api/v2/accesspolicies/{name}

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v3/accesspolicies/{name} instead.

GET /admin/api/v3/accesspolicies/{name}

NOTES:

The current version of the access policy will be returned in an ETag header on the response.

Response:

{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "status": "<string>",
  "realms": [
    {
      "tenant": "<string>",
      "cluster": "<string>"
    }
  ],
  "scopes": [
    "<string>",
    ...
  ]
}

Delete access policy

DELETE /admin/api/v1/accesspolicies/{name} DELETE /admin/api/v2/accesspolicies/{name}

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. In Admin API version v3, delete operations are no longer supported and have been replaced by soft deletes. Use the endpoint PUT /admin/api/v3/accesspolicies/{name} with "status": "inactive" instead. This operation will inactivate an access policy and subsequent HTTP requests with an associated token will fail with a 401 Unauthorized HTTP status code. To re-enable an access policy, use the endpoint PUT /admin/api/v3/accesspolicies/{name} with "status": "active".

PUT /admin/api/v3/accesspolicies/{name}

NOTES:

If-Match header matching the current version is required.

Example:

curl -u :$API_TOKEN localhost:8080/admin/api/v3/tokens/accesspolicies/enterprise-metrics-ap \
-X PUT -H 'If-Match: "123"` \
--data '{"status": "inactive", "realms": [{"tenant": "enterprise-metrics-dev", "cluster": "enterprise-metrics-dev"}], "scopes": ["metrics:write"]}' | jq
{
  "name": "enterprise-metrics-ap",
  "created_at": "2022-02-11T16:55:43.598543386Z",
  "status": "inactive",
  "realms": [
      {
          "tenant": "enterprise-metrics-dev",
          "cluster": "enterprise-metrics-dev"
      }
  ],
  "scopes": [
      "metrics:write"
  ]
}

Token API

List tokens

GET /admin/api/v1/tokens GET /admin/api/v2/tokens

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v3/tokens instead.

GET /admin/api/v3/tokens

NOTES:

This endpoint only returns tokens in an active status. To get all tokens including inactive ones use the query parameter include-non-active=true

Response:

{
  "items": [
    {
      "name": "<string>",
      "display_name": "<string>",
      "created_by": "<string>",
      "created_at": "<rfc3339 string>",
      "status": "<string>",
      "access_policy": "<string>",
      "expiration": "<rfc3339 string>"
    },
    ...
  ],
  "type": "token"
}

Create token

POST /admin/api/v1/tokens POST /admin/api/v2/tokens

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint POST /admin/api/v3/tokens instead.

POST /admin/api/v3/tokens

Payload:

{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "access_policy": "<string>",
  "expiration": "<rfc3339 string>"
}

Because token names are unique, you cannot create a new token with the same name as a token that already exists but is in the inactive state. Instead, to use the same token name again you must re-enable the inactive token by updating its state to active. To make this change, use the PUT /admin/api/v3/tokens/{name} endpoint. When creating a token via this API call, it is not required to specify the status field because the status is always overwritten by the API to active.

Get token

GET /admin/api/v1/tokens/{name} GET /admin/api/v2/tokens/{name}

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v3/tokens/{name} instead.

GET /admin/api/v3/tokens/{name}

NOTES:

The current version of the token will be returned in an ETag header on the response.

Response:

{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "status": "<string>",
  "access_policy": "<string>",
  "expiration": "<rfc3339 string>"
}

Delete token

DELETE /admin/api/v1/tokens/{name} DELETE /admin/api/v2/tokens/{name}

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. In Admin API version v3, delete operations are no longer supported and have been replaced by soft deletes. Use the endpoint PUT /admin/api/v3/tokens/{name} with "status": "inactive" instead. This operation will inactivate a token and subsequent HTTP requests with the token will fail with a 401 Unauthorized HTTP status code. To re-enable a token, use the endpoint PUT /admin/api/v3/tokens/{name} with "status": "active".

PUT /admin/api/v3/tokens/{name}

{
  "status": "inactive"
}

NOTES:

If-Match header matching the current version is required.

Example:

curl -u :$API_TOKEN localhost:8080/admin/api/v3/tokens/enterprise-metrics-token \
-X PUT -H 'If-Match: "123"` \
--data '{"status": "inactive"}' | jq
{
  "name": "enterprise-metrics-token",
  "display_name": "Grafana Enterprise Metrics Dev Token",
  "created_at": "2022-02-11T16:59:36.275826382Z",
  "status": "inactive",
  "access_policy": "enterprise-metrics-ap",
  "expiration": "2050-01-01T00:00:00Z"
}

License API

List licenses

GET /admin/api/v1/licenses GET /admin/api/v2/licenses

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v3/licenses instead.

GET /admin/api/v3/licenses

Response:

{
  "items": [
    {
      "name": "<string>",
      "display_name": "<string>",
      "created_by": "<string>",
      "created_at": "<rfc3339 string>",
      "token": {
        "jti": "<string>",
        "iss": "<string>",
        "sub": "<string>",
        "iat": "<int64>",
        "exp": "<int64>",
        "nbf": "<int64>",
        "lexp": "<int64>",
        "lid": "<string>",
        "max_users": "<int64>",
        "included_admins": "<int64>",
        "included_viewers": "<int64>",
        "lic_exp_warn_days": "<int64>",
        "prod": [
          "<string>",
          ...
        ],
        "company": "<string>",
        "slug": "<string>",
      }
    },
    ...
  ],
  "type": "license"
}

Feature detection API

Get product and feature information

GET /admin/api/v1/features GET /admin/api/v2/features

Deprecated: these endpoints are deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v3/features instead.

GET /admin/api/v3/features

Response:

{
  "name": "<string>",
  "version": "<string>",
  "features": {
      "<string>": "<string>",
      ...
  }
}

Example response:

{
  "name": "GEM",
  "version": "1.1",
  "features": {
    "debug_export": "v1",
    "editable_access_policies": "v1",
    "editable_tenants": "v1",
    "lbac": "v1",
    "self_monitoring": "v1",
    "federated_rules": "v1"
  }
}

GEM Node API

Tue, 22 Apr 2025 16:43:47 +0000

GEM Node API

An HTTP API called Node API is available within each GEM binary, and each binary runs either as a microservice (such as an ingester) or in single-process mode where target=all. The Node API fetches basic information about that microservice or single process.

Get version information

GET /node/api/v1/version

Response:

{
  "go_version": "<string>", // The go version
  "product": "<string>", // The name of the product, in this case "GEM"
  "revision": "<string>", // The git revision of GEM
  "version": "<string>" // The GEM version
}

Example:

$ curl -u :$API_TOKEN http://localhost:8080/node/api/v1/version | jq
{
  "go_version": "go1.16.6",
  "product": "GEM",
  "revision": "1rr03cg6e",
  "version": "v1.3.0"
}

Get debug information

This endpoint exports debug information from the GEM target.

GET /node/api/v1/debug-export

The response is a ZIP file with the following structure:

debug/
├─ tsdb-metas/
│  ├─ team-a/
│  │  ├─ 01FCZEADAH07FB9VER85YGVMBP.json
│  │  ├─ 01FCZEAD3ZN9Z9Q0YMR234VHXY.json
│  ├─ team-b/
│  ├─ __system__/
├─ config.default.yaml
├─ config.actual.yaml
├─ process-mappings.txt
├─ version.json

tsdb_metas/
The TSDB block meta data for the available tenants.

config.default.yaml
The default configuration file for the requested service.

config.actual.yaml
The actual runtime configuration file for the requested service.

process-mappings.txt
This file shows how memory is being used by the process from the OS perspective.

version.json
The version information about the requested service in the same format as is returned for the node/api/v1/version endpoint.

APIs on Grafana Labs

Grafana Mimir HTTP API

Grafana Mimir HTTP API

Endpoints

Path prefixes

Authentication

All services

Index page

Configuration

Different modes

Status Configuration

Status Flags

Runtime Configuration

Different modes

Services’ status

Readiness probe

Metrics

Pprof

Fgprof

Build information

Format query

Memberlist cluster

Get tenant limits

Distributor

Remote write

OTLP

Distributor ring status

Tenants stats

HA tracker status

Ingester

Flush chunks / blocks

Prepare for Shutdown

Shutdown

Prepare partition downscale

Prepare instance ring downscale

Prepare for unregister

TSDB Metrics

Ingesters ring status

Ingester tenants

Ingester tenant TSDB

Querier / Query-frontend

Instant query

Range query

Exemplar query

Get series by label matchers

Get active series by selector

Query parameters

Headers

Response format

Caching

Get label names

Caching

Get label values

Caching

Get metric metadata

Remote read

Label names cardinality

Count series by inmemory or active

Caching

Request params

Response schema

Label values cardinality

Count series by inmemory or active

Caching

Request params

Response schema

Querier

Get tenant ingestion stats

Query-scheduler

Query-scheduler ring status

Ruler

Ruler ring status

Ruler rules

List Prometheus rules

List Prometheus alerts

List rule groups

Get rule groups by namespace

Get rule group

Set rule group

Example request body

Count series by `inmemory` or `active`

Count series by `inmemory` or `active`