Get started with Grafana Mimir on Grafana Labs

Grafana Mimir architecture

Thu, 14 Aug 2025 18:42:07 +0000

Grafana Mimir architecture

Grafana Mimir has a microservices-based architecture. The system has multiple horizontally scalable microservices that can run separately and in parallel. Grafana Mimir microservices are called components.

Grafana Mimir’s design compiles the code for all components into a single binary. The -target parameter controls which component(s) that single binary will behave as.

To get started easily, run Grafana Mimir in monolithic mode with all components running simultaneously in one process, or in read-write mode, which groups components into read, write, and backend paths.

For more information, refer to Deployment modes.

Grafana Mimir components

Most components are stateless and do not require any data persisted between process restarts. Some components are stateful and rely on non-volatile storage to prevent data loss between process restarts. For details about each component, see its page in Components.

The write path

Ingesters receive incoming samples from the distributors. Each push request belongs to a tenant, and the ingester appends the received samples to the specific per-tenant TSDB that is stored on the local disk. The samples that are received are both kept in-memory and written to a write-ahead log (WAL). If the ingester abruptly terminates, the WAL can help to recover the in-memory series. The per-tenant TSDB is lazily created in each ingester as soon as the first samples are received for that tenant.

The in-memory samples are periodically flushed to disk, and the WAL is truncated, when a new TSDB block is created. By default, this occurs every two hours. Each newly created block is uploaded to long-term storage and kept in the ingester until the configured -blocks-storage.tsdb.retention-period expires. This gives queriers and store-gateways enough time to discover the new block on the storage and download its index-header.

To effectively use the WAL, and to be able to recover the in-memory series if an ingester abruptly terminates, store the WAL to a persistent disk that can survive an ingester failure. For example, when running in the cloud, include an AWS EBS volume or a GCP Persistent Disk. If you are running the Grafana Mimir cluster in Kubernetes, you can use a StatefulSet with a persistent volume claim for the ingesters. The location on the filesystem where the WAL is stored is the same location where local TSDB blocks (compacted from head) are stored. The locations of the WAL and the local TSDB blocks cannot be decoupled.

For more information, refer to timeline of block uploads and Ingester.

Series sharding and replication

By default, each time series is replicated to three ingesters, and each ingester writes its own block to the long-term storage. The Compactor merges blocks from multiple ingesters into a single block, and removes duplicate samples. Blocks compaction significantly reduces storage utilization. For more information, refer to Compactor and Production tips.

The read path

Queries coming into Grafana Mimir arrive at the query-frontend. The query-frontend then splits queries over longer time ranges into multiple, smaller queries.

The query-frontend next checks the results cache. If the result of a query has been cached, the query-frontend returns the cached results. Queries that cannot be answered from the results cache are put into an in-memory queue within the query-frontend.

Note
If you run the optional query-scheduler component, the query-schedule maintains the queue instead of the query-frontend.

The queriers act as workers, pulling queries from the queue.

The queriers connect to the store-gateways and the ingesters to fetch all the data needed to execute a query. For more information about how the query is executed, refer to querier.

After the querier executes the query, it returns the results to the query-frontend for aggregation. The query-frontend then returns the aggregated results to the client.

The role of Prometheus

Prometheus instances scrape samples from various targets and push them to Grafana Mimir by using Prometheus’ remote write API. The remote write API emits batched Snappy-compressed Protocol Buffer messages inside the body of an HTTP PUT request.

Mimir requires that each HTTP request has a header that specifies a tenant ID for the request. Request authentication and authorization are handled by an external reverse proxy.

Incoming samples (writes from Prometheus) are handled by the distributor, and incoming reads (PromQL queries) are handled by the query frontend.

Long-term storage

The Grafana Mimir storage format is based on Prometheus TSDB storage. The Grafana Mimir storage format stores each tenant’s time series into their own TSDB, which persists series to an on-disk block. By default, each block has a two-hour range. Each on-disk block directory contains an index file, a file containing metadata, and the time series chunks.

The TSDB block files contain samples for multiple series. The series inside the blocks are indexed by a per-block index, which indexes both metric names and labels to time series in the block files. Each series has its samples organized in chunks, which represent a specific time range of stored samples. Chunks may vary in length depending on specific configuration options and ingestion rate, usually storing around 120 samples per chunk.

Grafana Mimir requires any of the following object stores for the block files:

For more information, refer to configure object storage and configure metrics storage retention.

Download GEM

Fri, 16 Oct 2026 18:17:52 +0000

Download Grafana Enterprise Metrics

Releases

v2.17.21 – October 16th 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: aaeb0822872ff761bec8889fb9bf3ec6d3b8e8d673b914e068ecfde768bb66fc
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.21
  - Manifest digest: sha256:732b3f652a908dc72b98e485c8ad4484f68658a5c9db29191662d840f118932c
  - linux/amd64 digest: sha256:30cc97bf1af7c580da2060538f6d9cfc1fe7bc9906d98e4d451a4effd6e42db8
  - linux/arm64 digest: sha256:98dc4392aeee5f29405d2b7639c479ed8141cbd284a8276f194682d19241769f
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.21
  - Manifest digest: sha256:3386d5e6e2e8aeeb2d7c3cbb9cf6f1981660eae73d45303b5f332ab98259c89d
  - linux/amd64 digest: sha256:f9ed58e06e3831ae9003398993e89cff1701f5c2fddd29b250cd2ce016cab843
  - linux/arm64 digest: sha256:02690beea5a0ae04bdb1dcd351148e473382311f73c9a2d3a2edeac5d75e56e7
License: Grafana Labs license

Changelog

[ENHANCEMENT] Use Go 1.26.9 for build to address GO-2026-6599, GO-2026-6600, GO-2026-6603, GO-2026-6605, GO-2026-6607, GO-2026-6608, GO-2026-6609, GO-2026-6610, GO-2026-6611, GO-2026-6612, GO-2026-6613, and GO-2026-6617.
[ENHANCEMENT] Upgrade Mimir to address CVE-2026-81872, CVE-2026-81871, and CVE-2026-81870.
[BUGFIX] Fix an issue where configuring s3 with signature_version2: true would still result in using AWS Signature Version 4.

Upstream Grafana Mimir details

Version: N/A (after mimir-2.17.11)
Hash: 01abc08e95cee4a9f61868853f119b026ed797f7
Changelog: CHANGELOG.md

v2.17.20 – October 6th 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: 357ed4dcefdf20c90041da2101cfd1a8f1775afd7c4722af9457546bbb92b52a
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.20
  - Manifest digest: sha256:598b35f5c2c55cbbcf0b1b05cb7fd70af5a649344880b9d353898990ba802795
  - linux/amd64 digest: sha256:abc4c24cecda590d7e2540e9cbe04ed3cd0af42e632c8163154768a09332536c
  - linux/arm64 digest: sha256:b3ea44a11ac798a853a224a26c07861e37b955909378aeadd50afed4fe3ff338
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.20
  - Manifest digest: sha256:c6681dadc9a815b20d51f73def491265e51419e2d2c377de213e5eb9dedc734c
  - linux/amd64 digest: sha256:9ff654f39cbad54e8e989d02efc196d862aa2b3953b955a3374a36643e43c0da
  - linux/arm64 digest: sha256:b14e0ebfa14b82416ca9df5bbe5fe83597c91571860b0418d5488467fe72cfb1
License: Grafana Labs license

Changelog

[BUGFIX] Alertmanager: Fix an issue where configuration that referenced files could load local files. Resolves CVE-2026-81938.

Upstream Grafana Mimir details

Version: N/A (after mimir-2.17.11)
Hash: 7c6d75c894689d4d93a351c27e355cc43f241250
Changelog: CHANGELOG.md

v2.17.19 – September 1st 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: 96f31a327b628da35a3974e52845c8bf3f54f293113f44698c83e7ac703f02fa
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.19
  - Manifest digest: sha256:6cb85af6c66c82e1828e4451b0eae2d9ad5d45f64e9ca0d63138546a48ed0dc5
  - linux/amd64 digest: sha256:9f21cae4ff6d7054ffd4f5558f9dc881fba8df9c7852494b004c57b709b7f6e0
  - linux/arm64 digest: sha256:3d5305d8b2b51ef849bac01a6166918e2cc02ccceb8ef751751c01ea9bec3fcb
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.19
  - Manifest digest: sha256:2a5c01412fa9ac0b95ce20b7eeb35272f6c524c808147443fb247ad052c3e508
  - linux/amd64 digest: sha256:a258cf80b4601b69e437f2e1aa430803babf5573fc38e53c74ad364ce067005c
  - linux/arm64 digest: sha256:4b1fe6c36832ee67511f55b7ee24a039fbebb42ceba1cbec392cc06d14af4918
License: Grafana Labs license

Changelog

[ENHANCEMENT] Use Go 1.26.7 for build to address GO-2026-5026, GO-2026-5942, GO-2026-5972, GO-2026-6088, GO-2026-6089, GO-2026-6090, GO-2026-6091, and GO-2026-6218.
[ENHANCEMENT] Upgrade Mimir to address CVE-2026-56865, GHSA-259r-337f-4rfw, and CVE-2026-73500.

Upstream Grafana Mimir details

Version: N/A (after mimir-2.17.11)
Hash: 9771799b71120a5f2a47e86b0810246a8dd602f2
Changelog: CHANGELOG.md

v2.17.18 – August 13th 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: ea24c2b66eef68e3553077cbac5cb6a52eca1bdb13d182e7ebf5385ec3b3f937
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.18
  - Manifest digest: sha256:be901f0c5ffe3caa2ef42dbfbe2532878942fa419bc2e284dabbf0ad4594a48a
  - linux/amd64 digest: sha256:6b473f20d8356d47886be2697ff594f68dc0bbd531cb800f486080d39eb9eb28
  - linux/arm64 digest: sha256:cffe473dbf099831fdcf2642f82042bb5b4c3633b651f039dfb8bee4af2af042
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.18
  - Manifest digest: sha256:b03a2150aa043b42c4d1ff006b531c0c92f6ca70cd609c6680badd8f46d43c5b
  - linux/amd64 digest: sha256:cb75857dcac97a142920e3dfc99a70a104173e506a12fa527a52a89f15d14b56
  - linux/arm64 digest: sha256:0c6c0399ce7fc3a391d49495479a983acfc142bca7228d9c0921a3af5a76989f
License: Grafana Labs license

Changelog

[ENHANCEMENT] Ingester: Optimize TSDB head-chunk access during compaction to avoid O(n²) memChunk.len cost in PopulateBlock.

Upstream Grafana Mimir details

Version: N/A (after mimir-2.17.11)
Hash: 62a504125caf8bfc519b8272b2043ad2463105a3
Changelog: CHANGELOG.md

v2.17.17 – August 12th 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: 32af0125dd5896adf04556fea3af487e0f2a4493baaa31c77899ae41bbccc024
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.17
  - Manifest digest: sha256:733f43193e39024f98ef4319b354b56851e6aae6920324254c1e5ce2dc4d6e22
  - linux/amd64 digest: sha256:abd7e3628811f83bf310133758e57dc7e5676934f8356f1aaabd7943aea4d6d6
  - linux/arm64 digest: sha256:97e51ba9d5a480319cd837c65c851c911834e88a91a37fdb91cd3e36f213e2e7
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.17
  - Manifest digest: sha256:29e475c3f6d7e3baceb445a471e073c256787bfb17b5c44c3dde1d7d63480261
  - linux/amd64 digest: sha256:35fedaf30b1e591f2a19e7056504e3e685ec85d3dd52af418bd91b881895d076
  - linux/arm64 digest: sha256:d14e3a701cec11b095740b87f80ac616feabb62541972f2831f061643a9bf098
License: Grafana Labs license

Changelog

[ENHANCEMENT] Upgrade go.opentelemetry.io/otel to v1.44.0 to address CVE-2026-41178.
[BUGFIX] Ingester: Fix out-of-order queries that span the in-order head mint blocking TSDB head truncation and compaction.

Upstream Grafana Mimir details

Version: N/A (after mimir-2.17.11)
Hash: 075444a9f6d2f9ecabe6d5b476b4f4d4615f595c
Changelog: CHANGELOG.md

v2.17.16 – July 23rd 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: 81b4d752fd6c5e74140c22ed8775ac21c0af2a5eb5683ab73c87b8a1a647d7b3
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.16
  - Manifest digest: sha256:655b18bcc4fb45dd09a847a5d0a92a86368c552a6a450640000fa2ce54ccd68e
  - linux/amd64 digest: sha256:2f50b4a88d16931be7c5e849166a45840b594ddebaed2ae72f5f705af648be4f
  - linux/arm64 digest: sha256:d1eaa52b8af7e9ceaa4e56fbc95a33be61adeae95820e4c6941e5708bddebeb5
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.16
  - Manifest digest: sha256:97eee49b7b398f329c4d97aa3ae62d640042de4a868314c5cba836ffef23a4ca
  - linux/amd64 digest: sha256:427b38d89e1af4f44adfdaad21c754d2f9ff84391cc85a2254155e330bdfc9ed
  - linux/arm64 digest: sha256:a0a5d8a4dc59852b0137ac7d2de40a82351ea69405f79cd94008cfc174ddfd3d
License: Grafana Labs license

Changelog

[ENHANCEMENT] Use Go 1.26.5 for build to address GO-2026-4970 and GO-2026-5856.
[ENHANCEMENT] Upgrade go.mongodb.org/mongo-driver to v1.17.7 to address CVE-2026-2303 and GHSA-cp6g-7hqx-qxhp.
[ENHANCEMENT] Upgrade golang.org/x/net to v0.56.0 and golang.org/x/text to v0.39.0 to address CVE-2026-46600, GO-2026-5942, CVE-2026-56852, and GO-2026-5970.
[ENHANCEMENT] Upgrade google.golang.org/grpc to v1.82.1 to address GHSA-hrxh-6v49-42gf.

Upstream Grafana Mimir details

Version: N/A (after mimir-2.17.11)
Hash: 3659891bf2be526f6582579b5e1e4ea6702f94ca
Changelog: CHANGELOG.md

v2.17.15 – June 17th 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: f8b78700c5c986d21807872cf7129bf66332686d929d4094a7722ac8d62904c7
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.15
  - Manifest digest: sha256:6b880882bc05bb2269495df146b3fc3201f89e2e6dc10211fc5ac563ac78ae9a
  - linux/amd64 digest: sha256:4c5436f20d755a33a45db14c0c5f12b4b5a5485d0a2558ea79f420725b2d70cb
  - linux/arm64 digest: sha256:df75e01bac76fd112b05a3b97d76e95357107bad04553c0a768a306e06177c28
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.15
  - Manifest digest: sha256:4920e428eba6091506f748f4bc68f8861448a2bf3fc24ee3015b1f001f4529b1
  - linux/amd64 digest: sha256:f4000aa365d1afe7e55fa4a652dd753ea7535517cd321e050900ace127f6557d
  - linux/arm64 digest: sha256:b8f798176bd53ac3ef68e42b2b42e0c396c6823e96357911f6ae88311c7d9d3f
License: Grafana Labs license

Changelog

[ENHANCEMENT] Use Go 1.26.4 for build to address CVE-2026-42504, CVE-2026-27145 and CVE-2026-42507.
[ENHANCEMENT] Upgrade base image used for boringcrypto image to address CVE-2026-0915, CVE-2026-4046, CVE-2026-4437, CVE-2025-15281, CVE-2026-0861 and CVE-2026-4438.

Upstream Grafana Mimir details

Version: N/A (after mimir-2.17.11)
Hash: b7213cda07504626d5db2561a06eeecd846f4276
Changelog: CHANGELOG.md

v2.17.14 – June 4th 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: 0354e4ddb224d8cb1af1a0a04110a919ced414932897f865388be4b93cf77262
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.14
  - Manifest digest: sha256:b26e116d0ac46bf5032b9ebd7b64dc1da973b9c715a1e1b74be561dfeb258296
  - linux/amd64 digest: sha256:26e8066a6899ddad270a841220048191ebad7e577f88350eb74aa8d5377df0f8
  - linux/arm64 digest: sha256:596ec043c2051fe65589854af86ded034211d85c525fb42abace730fd7914cc4
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.14
  - Manifest digest: sha256:8fd301177050138ee6daf2cf5ccad7168dee47a31da0f6126b9ec1d6dfd496cf
  - linux/amd64 digest: sha256:ad0d698d5d5cd8fbc58faaf1fc1d8a5d19ca9fd7a534814742301f940d0b6258
  - linux/arm64 digest: sha256:2486e177923998474419705e26639805dcfea3b0c0a273cd1b0f990bcb501b87
License: Grafana Labs license

Changelog

[ENHANCEMENT] Upgrade Mimir to 2.17.11 to address CVE-2026-27141, CVE-2026-33814, CVE-2026-39882.
[ENHANCEMENT] Use Go 1.25.10 for build to address CVE-2026-33814, CVE-2026-33811, CVE-2026-39836, CVE-2026-42499, CVE-2026-39820, CVE-2026-42501, CVE-2026-39826, CVE-2026-39823, CVE-2026-39817, CVE-2026-39825, CVE-2026-39819.
[ENHANCEMENT] Use Go 1.26.3 to align with upstream Mimir.
[ENHANCEMENT] Upgrade Mimir to address CVE-2026-42506, CVE-2026-39833.

Upstream Grafana Mimir details

Version: N/A (after mimir-2.17.11)
Hash: ee8ff899659e458dd24288b5a05e3bb2a662ae2e
Changelog: CHANGELOG.md

v2.17.13 – April 24th 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: 46d0d8b9537c622c29926c85c87f2bb6931a1ee6f7cce21a54abfd351f17289b
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.13
  - Manifest digest: sha256:4197c077504bdcf2e65491b35d6e8cc490eced7ff6854cbdcee6585f3c1f93f4
  - linux/amd64 digest: sha256:d0522683171fc3a0ee9232227628ee9264e875dc29ba8d20c5a48891326c201c
  - linux/arm64 digest: sha256:1b15478f18a7a65c8183fda59b9dafb1a963ff648ba95988121c76bd012962e4
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.13
  - Manifest digest: sha256:21d03ab84e9ba2e39c72b13f48413c38aebb49b8a6a5e56f9e7eeb9d396edded
  - linux/amd64 digest: sha256:b8f5f11c86c64e71afc8c08d56642de85cf3def05a29e88370b116827dbdae9e
  - linux/arm64 digest: sha256:e901935a12fde777dae6aa9d74ec12f2a79443585602415e24de65718b807269
License: Grafana Labs license

Changelog

[ENHANCEMENT] Upgrade github.com/go-jose/go-jose/v3 to v3.0.5 to address CVE-2026-34986.

Upstream Grafana Mimir details

Version: 2.17.10
Hash: 417bc2000117df6f9fb39b1a2a5645e4ecd1af59
Changelog: CHANGELOG.md

v2.17.12 – April 20th 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: a27533633136726ba2a6c371f7ec2cb27a983424b6bb49a4fde9ec2716ec530c
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.12
  - Manifest digest: sha256:160a74a472fb880e5725e7fe896e46bf68e28efe8fd50b3eff6dead2bda880a1
  - linux/amd64 digest: sha256:0a306e52d64bb7d1718b98eb4610c5e35a6c233517f90cc6220388c5aabb3ed6
  - linux/arm64 digest: sha256:2790da59d036b09bd0099f3b876a124f6d1da9e4358ce51e8cbdde8846534a48
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.12
  - Manifest digest: sha256:425b5d1b840eca4c13248c22aa467fa841d8f64be67230c1e25cc0eee5b3092c
  - linux/amd64 digest: sha256:9ddf96dc592ee8a51afea3ee63db92122fce7ea4e8f49c12058b0350abf5670a
  - linux/arm64 digest: sha256:a83e757b72075a57743604b653402f739d6cca6c79eca96a01ff8bd8320723ec
License: Grafana Labs license

Changelog

[ENHANCEMENT] Upgrade Mimir to 2.17.10 to address CVE-2026-39883, GHSA-xmrv-pmrh-hhx2.
[ENHANCEMENT] Use Go 1.25.9 for build to address CVE-2026-32283, CVE-2026-32282, CVE-2026-27144, CVE-2026-27143, CVE-2026-27140, CVE-2026-32289, CVE-2026-32288, CVE-2026-32280, CVE-2026-32281.

Upstream Grafana Mimir details

Version: 2.17.10
Hash: 417bc2000117df6f9fb39b1a2a5645e4ecd1af59
Changelog: CHANGELOG.md

v2.17.11 – April 17th 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: 9786a2853bf171379e548898df50a25b929d1409394a77d606de802a8b04905f
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.11
  - Manifest digest: sha256:f4f573d93323182b1b043f257802ab52668cd66a0bbd8bf6e35fc6ad582e4d39
  - linux/amd64 digest: sha256:371478ae0f178fa46f6732c7182c56a201358f065f34842d04faf19f679c46ed
  - linux/arm64 digest: sha256:4fadd9d5dc9602c962c31edf7800485ff4f5ec65fbf8e404583785e253e8cacb
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.11
  - Manifest digest: sha256:fe42a6218cb3b0fb50ec19dd2d62da89d61a07bec56f5c6353b8585fe15d760c
  - linux/amd64 digest: sha256:5a96515020fd531590a5731e4cea2c872efc7a7a053041f205438beabedbc22e
  - linux/arm64 digest: sha256:aea4aaf37b5d636478007f3d092a0c758ca5148fb5f0041c50ee7abd24d7c976
License: Grafana Labs license

Changelog

[BUGFIX] Admin API: Fix follower-to-leader proxy failing on IPv6-only clusters due to unbracketed IPv6 address in ring registration.

Upstream Grafana Mimir details

Version: N/A (after mimir-2.17.9)
Hash: fc5f2b24db629bfcddc5198e56ff0fead45f6c1d
Changelog: CHANGELOG.md

v2.17.10 – April 17th 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: 615bc49cb4cf464a816439b91c324e50eaf30d0c039112356601b6e88a6e0397
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.10
  - Manifest digest: sha256:a33de9b5c4313b586628e966981d0aebb5cb435a6aa848dde9857806614c1573
  - linux/amd64 digest: sha256:2d29f32b63199d34240bdf75653e94a4cfb28fce06b1329197824fe90c541b49
  - linux/arm64 digest: sha256:05243477ff1de0b17d7f66f8ccdde664d0eab268656cd884f7d76ae77f5e80b4
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.10
  - Manifest digest: sha256:5484375f82d989cd7571eb4e9bfcf189813cee2a98a5a32564010133f29bbbef
  - linux/amd64 digest: sha256:a788d9ca01dbd4c84ca832614d5cfe8e2a115e19911dcb7ab4dd3f3e84b2cd14
  - linux/arm64 digest: sha256:2ca905ec5dd002bc0f5e25c871e22035540e46d8e064c3b47aa506ba31d2920a
License: Grafana Labs license

Changelog

[BUGFIX] Ruler: Fix parsing of rule expressions with leading newlines.

Upstream Grafana Mimir details

Version: N/A (after mimir-2.17.9)
Hash: fc5f2b24db629bfcddc5198e56ff0fead45f6c1d
Changelog: CHANGELOG.md

v2.17.9 – April 8th 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: f4a04384c08d99af52e02740cd3ffa85482aabad25c2225fcd200b9971129671
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.9
  - Manifest digest: sha256:4cf676108ff6d021e2a028c51c49a6e67af38723e8cbb5ab54173a2f866958b9
  - linux/amd64 digest: sha256:118039e62a3e3ec92faae01270773a0331182af4b28a8cd8d935e7704780fe9a
  - linux/arm64 digest: sha256:4e08722333549ceb7d329ed6bf6327a2c5ba601ed6004ea836ffe0678254d836
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.9
  - Manifest digest: sha256:ff23caa3cd4268a0d03450bdcc1b8a671d55f7095657b1036495f6b3e6d8fae1
  - linux/amd64 digest: sha256:47edc4a2ca37eafa15be6fbea747674b19f6e65f1725e90a1e9dad02bf9b49cc
  - linux/arm64 digest: sha256:f846c9dcfe6083a58eceea0965ea53a0b3e2cd02b64e2b1034d107b5389b9207
License: Grafana Labs license

Changelog

[FEATURE] Distributor: Add OTLP usage reporter that tracks OTel scope occurrences and periodically exports them to the stats server. Enabled via -otlp-usage.enabled.
[ENHANCEMENT] Ruler: Propagate GCS object mutation rate limit for rule group uploads.
[ENHANCEMENT] Update all build images to recent Go version.
[ENHANCEMENT] Use Go 1.25.8 for build.
[BUGFIX] Fix not_aggregated_wrapper incorrectly return errors for metric with no sample when MQE is in use.

Upstream Grafana Mimir details

Version: N/A (after mimir-2.17.9)
Hash: d5055263bd51b593baead0b6ff0697ccb3fb56fa
Changelog: CHANGELOG.md

v2.17.8 – Skipped

v2.17.7 – February 27th 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: 053136a46b8ada0eb7f4c7d7e4a78b380e009fc226ccfa91cc67c44bc1dce535
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.7
  - Manifest digest: sha256:8f904c2c126e6d5e2860ee4d966269a6229c4241913e5c7147d11e32861daf50
  - linux/amd64 digest: sha256:9652f49bc3b27381a093879cb6938fa5a8cefdfe68447aeeaf6d444f5373ae35
  - linux/arm64 digest: sha256:b4aad83483d98496d64c5d4acc0d1b1a6487a755cbf2e068431525d478bfd6b9
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.7
  - Manifest digest: sha256:1abf303bd441d39c8c9a278f35f91a316ed2679f67433d3c954c509a96ff513d
  - linux/amd64 digest: sha256:6f2fb24a6bdef06f22362b83a008a85d8cf5e978ad83eea6062cfe88a7c07773
  - linux/arm64 digest: sha256:26763cf63905ab6f559cf55fcc3bae3cc9523f47b428e4f08bbafcd099de9001
License: Grafana Labs license

Changelog

[BUGFIX] Federation: Fix nil pointer dereference when querying a non-existent cluster with partial queries enabled.
[BUGFIX] Federation: Stop emitting cortex_federation_frontend_partial_results metric when partial queries are disabled. Previously when partial queries were disabled, the metric would always report a 0% failure rate, even when cluster requests failed.
[BUGFIX] security: update module golang.org/x/net to v0.45. Fixing CVE-2025-58190 and CVE-2025-47911.

Upstream Grafana Mimir details

Version: 2.17.7
Hash: 7e482a1ea7d45ea178ed4b6a151dec7b6fc632b7
Changelog: CHANGELOG.md

v2.17.6 – February 12th 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: bf78a11bcedd101f3edac73a206797cf85b8e0833518a6e24b29a4859efaa5a6
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.6
  - Manifest digest: sha256:6c2e30d674c9d6b3e93dadaaaa2d37e73712115f7804aee45160ca03cd130e9a
  - linux/amd64 digest: sha256:a503d4f45c86c9ef3ef9575c1ce531b6a76bf795ef8b9af3dde59b8bc70ad2d2
  - linux/arm64 digest: sha256:069fe9a84124a630c1f366348b321b0363e565c8844fce5b494eb2f5507b288c
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.6
  - Manifest digest: sha256:7bf94bd1f9d3aefef2ef7266962cb07cb370aa11a72b25fdc159b6e12e1c5554
  - linux/amd64 digest: sha256:096def779daecbf3827f4213ca8fd04adaf5c2d187ac3971ff26e67450eb1b66
  - linux/arm64 digest: sha256:73edb69250e5aa301ae98e9d0fda8b02e52637011b3d1cac888f2e9a1b8dbd82
License: Grafana Labs license

Changelog

[ENHANCEMENT] Use Go 1.25.7 for build.
[BUGFIX] Ruler: Add path traversal checks when parsing namespaces and groups, which prevents namespace and group name from containing non-local paths.

Upstream Grafana Mimir details

Version: 2.17.6
Hash: 15bd1ec65bec30816aecd15418c3961a315180a0
Changelog: CHANGELOG.md

v2.17.5 – January 29th 2026

Links

Binary (Linux AMD64)
- Download
- SHA256: 182525aab193d98bf471a970597c73934cb1e6ed1a4840c1be7a6f8ac4bc8de7
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.5
  - Manifest digest: sha256:e00034fd8f2c0e578309acdbdb8ccb44cd6880be534c476c988eb168fc55b0b7
  - linux/amd64 digest: sha256:3d94900429ad59b90847e0a95fc3add7d1030ae0f9d15653f69986d2befb9ef4
  - linux/arm64 digest: sha256:ae70d66bbd886047ab940515de5643b8818607a500c162ba80a7ccf8535c3325
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.5
  - Manifest digest: sha256:24676c3d623129409cae79f97061882dded754784c74186cc83a87f992fa9fa4
  - linux/amd64 digest: sha256:bb4ffbb7931932449fce5e7e2878f19cbd78facb8e4321b19cf65bba40e0a53b
  - linux/arm64 digest: sha256:de44d4dffcab48420ef6f07e0e6b3635250d1a4cacbbd9620cc58943e4f16d0c
License: Grafana Labs license

Changelog

[ENHANCEMENT] Update Mimir to 2.17.5.
[ENHANCEMENT] Use Go 1.25.5 for build.

Upstream Grafana Mimir details

Version: 2.17.5
Hash: eeb851d540ab3c0f74e966ae37814540b48441c2
Changelog: CHANGELOG.md

v2.17.4 – December 30th 2025

Links

Binary (Linux AMD64)
- Download
- SHA256: 3d51ab6dc3ab6e101c4e5dcf99eaf0ed16c1c5ed0cab1f6d2a46042c845297ab
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.4
  - Manifest digest: sha256:f4b317412d92c4218ffc4dee78f037632df32bcd3c609af6011649466d442f94
  - linux/amd64 digest: sha256:4a63ed65cbdc1a0bd43e1c2768e15b30cf83aa6f828bed55fdb703e3666a68a3
  - linux/arm64 digest: sha256:d8b207e7aeb1645a39025c377f430934674594e8ce30480ea9d27fd32bcb50ec
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.4
  - Manifest digest: sha256:6362c06d0acb085dc7d38950a682c24f68655133ff0965e46469d63f1a51bc3a
  - linux/amd64 digest: sha256:f6771d19fefa3e0b12e237e603c042a6822ae445231b6910c48bef5e2824343d
  - linux/arm64 digest: sha256:cce2f2e6befb9335872e93be1b93dccced1a695151a294f33e08181612280020
License: Grafana Labs license

Changelog

[BUGFIX] Update to Go 1.24.11 to address CVE-2025-61729 and CVE-2025-61727.
[BUGFIX] Update Go crypto dependency to v0.45.0 to address CVE-2025-47914 and CVE-2025-58181.

Upstream Grafana Mimir details

This is a patch release to resolve newly discovered CVE’s in GEM dependencies. There is no corresponding upstream release.

Version: See changes since: 2.17.2
Hash: 70320fb8478be4f22f13a2ffdd82bb857932c8a5
Changelog: See changes since: 2.17.2

v2.17.3 – November 24th 2025

Links

Binary (Linux AMD64)
- Download
- SHA256: a7872936d5bb7a399d5f2e803a8d680b7f05863121e12ea41a593b06fd3b4d53
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.3
  - Manifest digest: sha256:f9552176e7f9b6bb8f75b5391ab6eaf2e3f1597594f244e8b2d5edea6d6bfbfd
  - linux/amd64 digest: sha256:1b522c8c087fc17ef714a53cd2a6c228c1c3f2ff1734a6a39f0495d4f5aa21a6
  - linux/arm64 digest: sha256:f6db8364789b7c39d1eb1bd83f6bf6d16eec62660bac7a6c154f6e19a541d105
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.3
  - Manifest digest: sha256:dbd40e5135da7ce6fd174a4248905484adf6f488a108912f7ed79e52518bc5d3
  - linux/amd64 digest: sha256:f62e0eddae0c1c42610c0307c936ea1d515176015a37231e547b83ffc212c022
  - linux/arm64 digest: sha256:bc3d637379affa1dce1a16929fdc0a7da5afc4103243d395328ed02f047c0623
License: Grafana Labs license

Changelog

[BUGFIX] Update opencontainers/selinux dependency to v1.13.0 to address CVE-2025-52881.
[BUGFIX] Update Go crypto dependency to v0.43.0 to address CVE-2025-47913.

Upstream Grafana Mimir details

This is a patch release to resolve newly discovered CVE’s in GEM dependencies. There is no corresponding upstream release.

Version: See changes since: 2.17.2
Hash: 70320fb8478be4f22f13a2ffdd82bb857932c8a5
Changelog: See changes since: 2.17.2

v2.17.2 – November 7th 2025

Links

Binary (Linux AMD64)
- Download
- SHA256: 5084228395c59eab0ec379165468a91b6cbd5d93d9cb9e6467ee00d18e671168
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.2
  - Manifest digest: sha256:f5d0bc5e6a20028bba31fe5f1897470706e1779eff749dfcc479ae60e13631d5
  - linux/amd64 digest: sha256:100a87c175306a24a2c0413ba776726c9ddfe7e2edf018852cfa4584d7329def
  - linux/arm64 digest: sha256:6ad03d5d1456d7109d91b4877a981ec3be01a31b12aa73425006c3c5f1038e67
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.2
  - Manifest digest: sha256:08fafd0543bdb3e6eea36165098fc38bdc9b571d00498dc6b74181dc3582cc7d
  - linux/amd64 digest: sha256:ec5931837aa87da718385e0e2940b874ff1420b0c57bfce6eb7e4ffd7bbd4de1
  - linux/arm64 digest: sha256:9ee0abfa07201e57cc482861b33452ec646f0e5dcf6ca7abbf27636c237a6d6f
License: Grafana Labs license

Changelog

[BUGFIX] Update to Go v1.24.9 to address CVE’s.
[BUGFIX] Update distroless images to latest digest to address CVE’s.
[BUGFIX] Distributor: Fix a bug where no NHCB passes validation when native histogram bucket limit -validation.max-native-histogram-buckets is set.
[BUGFIX] Ingest: Fix a data corruption issue caused by memory pooling when a newer version is rolled out and the experimental -distributor.otel-created-timestamp-zero-ingestion-enabled feature is enabled.
[BUGFIX] Memcached: Ignore invalid responses when discovering cache servers using dnssrv+ or dnssrvnoa+ service discovery prefixes.

Upstream Grafana Mimir details

Version: 2.17.2
Hash: bce93699bd2d5b8fc530034073be7d2da4365751
Changelog: CHANGELOG.md

v2.17.1 – October 17th 2025

Links

Binary (Linux AMD64)
- Download
- SHA256: 101cb720b79a803665424391ce97a0a6a3a79981021d44a51de0800395d8954c
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.1
  - Manifest digest: sha256:2155d22979187d0be22308afcf8645c23b7f3dc4fa46a872d108795e81a29318
  - linux/amd64 digest: sha256:d1eb62a23721117a7b49c718a9b57dd2c478f7074de8714b3a163bdc9db7a746
  - linux/arm64 digest: sha256:02f6a721bad44934aecad75d681d773ec93b4bc48e6d9873071945a6977c59df
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.1
  - Manifest digest: sha256:aa74981dc12fd3ab134cd3ddda6d290b06ebcd3dfe836d7a209c2c1bb46a7ccf
  - linux/amd64 digest: sha256:a9870df188a2ab85ffd5c65bf4f470c5a3c8b31af746a0de7c6ca0ba0946ab3c
  - linux/arm64 digest: sha256:9d94ffbf518c13b45f602811a2f4fcf8bb473664f8fa0d5c454731d2a4f42ea6
License: Grafana Labs license

Changelog

[CHANGE] Update root CA certificates in Docker images.
[BUGFIX] Update to Go v1.24.6 to address CVE-2025-4674, CVE-2025-47907.
[BUGFIX] Ingester: Fix a bug ingesters would get stuck in read-only mode after compactions.

Upstream Grafana Mimir details

Version: 2.17.1
Hash: 81df5c3a9b4c4e8477da4a3767784d4dd93c3616
Changelog: CHANGELOG.md

v2.17.0 – August 18th 2025

Links

Binary (Linux AMD64)
- Download
- SHA256: 79b13c8d3063ce26eb00d97968429e5377e0cd17a9b58b410683ae16485b4474
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.17.0
  - Manifest digest: sha256:b8ca992c0d6ea90d8d22a5372db91d0437334ba96c9d582a3fff49f3810547ee
  - linux/amd64 digest: sha256:083afd9ebfab7f2bad4756215103b51b8665713b529ff16e43d6742cff187c61
  - linux/arm64 digest: sha256:180d9aa72af2f6f2ed85f231e85015660708575de61fe14be81d33ff3a95f051
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.17.0
  - Manifest digest: sha256:0024fa10cb9b5212e2313972c1676a12d23546d2b40494cdeba0e7e00245335c
  - linux/amd64 digest: sha256:39e4971f79623aa575bd5160c3c357543e7a23d77b91191bdac30c4479d486bd
  - linux/arm64 digest: sha256:ae605ba45facfa1f5bd54c871c1e9b17f81943c59210012e1ea616f9571a9b00
License: Grafana Labs license

Changelog

[CHANGE] Remove support for using Redis as a cache from the store-gateway or query-frontend.
[ENHANCEMENT] Cross-cluster query federation: allow automatic retries of individual requests that return retryable errors in queries, with the total number of attempts allowed configured with federation.max-attempts-per-request.
[ENHANCEMENT] Cross-cluster query federation: allow configuring extra headers to be forwarded to target clusters with frontend.extra_propagate_headers.
[ENHANCEMENT] Federation: Add HTTP client cluster validation support, via the federation.proxy_targets.cluster_validation_label configuration parameter.
[ENHANCEMENT] Gateway: Add optional HTTP load balancing to the cortex.query and cortex.distributor proxies via the [proxy].load-balancing and [proxy].load-balancing-policy configuration parameters.
[BUGFIX] Cross-cluster query federation: Fix unreported partial results when remote-read stream breaks after the first response.
[BUGFIX] Graphite querier: improve error messages on render errors.
[BUGFIX] Ruler: Allow wal_watcher metrics to be re-registered by different remotes without panicking.

Upstream Grafana Mimir details

Version: 2.17.0
Hash: e150acbc8f6061b1995bff5105f786843c6a5277
Changelog: CHANGELOG.md

v2.16.2 – October 27th 2025

Links

Binary (Linux AMD64)
- Download
- SHA256: 01ea3bae28441d0f140dddb370ea5bac62a1021850bc94ed25aa71b39041b2e6
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.16.2
  - Manifest digest: sha256:770d1b0d22ecbb481483c15f2a714e8602fb59ee59cfe942d03f2a4c8a076f35
  - linux/amd64 digest: sha256:9a2fe85cfcdff56cb8a8c224416eb1aa6425a56a80e1dbce4e6ca77b230d0fb9
  - linux/arm64 digest: sha256:a5bdb6a491ed4afa7354fb87f17b422c56addd6c87c6eb3aa4be041e01bfefa5
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.16.2
  - Manifest digest: sha256:f802410467b409e9b72ec2edc7906477599796e8b63afe6fc45c9415bac0e152
  - linux/amd64 digest: sha256:63ec821814d88f4bcb5c74aa9eaf3ce1136b4c6883dae8d3cc3fd63a3e31e5b1
  - linux/arm64 digest: sha256:9ee92d6adc4c5b521889e557f7387a1c21e2d094732d2c469b6285efea4b0b57
License: Grafana Labs license

Changelog

[BUGFIX] Update Go from v1.23.9 to v1.23.12 and Alpine from v3.21.3 to v3.21.5 to address CVEs.

Upstream Grafana Mimir details

Version: 2.16.2
Hash: 12bba4018a5abcb8362a8c09f23c7c4282d08dc1
Changelog: CHANGELOG.md

v2.16.1 – July 30th 2025

Links

Binary (Linux AMD64)
- Download
- SHA256: 8c310344d96b494c6045a8af0cacfdb182b509531830073f381f2c255354b916
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.16.1
  - Manifest digest: sha256:cf9c0fa12352d75d089a9817cec6e59c045f7269f6f7ab0bce4862bff14da99a
  - linux/amd64 digest: sha256:84a3094128bcff6f3387cfd440385d94680a7e535b931f73022407091beda4b0
  - linux/arm64 digest: sha256:8e52f72b7d174811a41e9679c63291581e17722207da94c01ee100c9763b2de7
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.16.1
  - Manifest digest: sha256:24caa7e453f80f51c5ff8885550ea71f04834521b315e9f8a831aad78c6c9d39
  - linux/amd64 digest: sha256:0187d26a912dcb4e6289ce0249536c2cac2a8d7b6e1391ce5a256b060aa2356d
  - linux/arm64 digest: sha256:2cf73c469f753148f6ff3d6a71f0d97aad1c2b003c76b566f2a2fe3e1726dcf6
License: Grafana Labs license

Changelog

[BUGFIX] Update to Go v1.23.9 to address CVE-2025-22871.
[BUGFIX] Update golang.org/x/net to v0.38.0 to address CVE-2025-22872.
[BUGFIX] Query-frontend: Fix a panic in monolithic mode caused by a clash in labels of the cortex_client_invalid_cluster_validation_label_requests_total metric definition.

Upstream Grafana Mimir details

Version: 2.16.1
Hash: 876d470fb15f9504a4016a0b60aa2ad4d1ba9a0a
Changelog: CHANGELOG.md

v2.16.0 – April 9th 2025

Links

Binary (Linux AMD64)
- Download
- SHA256: 25f54952c474488c628c290b98c5ee21fb07e062be36db073eec618be2cae24f
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.16.0
  - Manifest digest: sha256:668d8416d605fd87b8a1868d6e0a11101b725570c3b2fcdad3a28ef504a59d62
  - linux/amd64 digest: sha256:37d32ea98b37fb2227062e289d7596119fc37e9c9a0415b441e1f988ef075416
  - linux/arm64 digest: sha256:33d553349243f45ec26f5d036087c6eda0a6e19f4b91f390f26d0e8e7c2dd9f0
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.16.0
  - Manifest digest: sha256:175b25134dad2c31e1a75653158d91d400fe31412a8d71400bb1106ac1adfeda
  - linux/amd64 digest: sha256:044ecf53274cdde54a67a7bdbd31cc89f9e0d6e8ea0df24ac99029a765623b0f
  - linux/arm64 digest: sha256:af3f97e13e3708630d785bc4b86d13d46700288ecf2de64c7d20fe147c2e9a89
License: Grafana Labs license

Changelog

[CHANGE] Remove the images grafana/enterprise-metrics-alpine, grafana/enterprise-metrics-boringcrypto-alpine, and related CI tests.
[CHANGE] Remove experimental Datadog proxy.
[CHANGE] Cross-cluster query federation: increase default proxy target timeout to two minutes. This brings it into line with other similar timeouts such as -querier.timeout.
[CHANGE] Breaking: Admin API: Deprecate -admin-api.limits.enabled and default to false in preparation for removal
[ENHANCEMENT] Cross-cluster query federation: Add support for partial queries.
[BUGFIX] Cross-cluster query federation: querying exemplars would fail with a misleading no cluster name provided error. It will now return cross-cluster query federation does not support querying exemplars.

Upstream Grafana Mimir details

Version: 2.16.0
Hash: b4f36dac3af7046a0a7a287bb88b503737e07c48
Changelog: CHANGELOG.md

v2.15.2 – April 15th 2025

Links

Binary (Linux AMD64)
- Download
- SHA256: 42bedce5b98539596080f0c3702b52db6899219381fbfe8f8d6b8cf29145cfd9
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.15.2
  - Manifest digest: sha256:8bcaeb11b61f48295fc0c583dde661d912b568d1effd22ea0b0de98dd23f540f
  - linux/amd64 digest: sha256:5fc33903575061ac2c589ef5d80716bbc438ede40f76471761202cc389f35293
  - linux/arm64 digest: sha256:a738fb90b33f8fb35820d395636ed509b3bc4ebcd723ee8ac9e376a335475ee4
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.15.2
  - Manifest digest: sha256:52972d86da3c86831c46052dfa7620269797a714aefb7e4dafa7103da5523f12
  - linux/amd64 digest: sha256:921ac93f541a74faf36c832f30fd28ccb847ba9fe567121088727eb7f55163ec
  - linux/arm64 digest: sha256:ddd1fc54bf01c638bba429f2636d1c9513424b4e66f24bc600405a06c9655ac5
License: Grafana Labs license

Changelog

[BUGFIX] Update module golang.org/x/net to v0.36.0 to address CVE-2025-22870.
[BUGFIX] Update module github.com/golang-jwt/jwt/v5 to v5.2.2 to address CVE-2025-30204.
[BUGFIX] Update module github.com/golang-jwt/jwt/v4 to v4.2.2 to address CVE-2025-30204.
[BUGFIX] Update module github.com/golang-jwt/jwt/v3 to v3.0.4 to address CVE-2025-30204.
[BUGFIX] Update module golang.org/x/oauth2 to v0.27.0 to address CVE-2025-22868.
[BUGFIX] Update module golang.org/x/crypto to v0.35.0 to address CVE-2025-22869.
[BUGFIX] Update module github.com/go-jose/go-jose/v4 to v4.0.5 to address CVE-2025-27144.
[BUGFIX] Upgrade Go base image to 1.23.7.

Upstream Grafana Mimir details

Version: 2.15.0
Hash: 56defe75b90f0f150d489f735708928fcd8a5b8f
Changelog: CHANGELOG.md

v2.15.1 – February 27th 2025

Links

Binary (Linux AMD64)
- Download
- SHA256: efd4b40239e0a1c78db4cf2ff9dd8eb9149ac2b6a2348af41059ec5ac8d27a53
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.15.1
  - Manifest digest: sha256:302b880f06588a7d88ea24f2f7cfa2a824c2521f7ee29a8450b7d2355d4bba21
  - linux/amd64 digest: sha256:b3fa45de49a01b9b01831a6c387196679cf8a147edb460df2476de71f0a4ff20
  - linux/arm64 digest: sha256:8c91c2f031a87e69c14a0e03b788a7a839a560d8dd766a0f204a07a8c03ad4dc
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.15.1
  - Manifest digest: sha256:c17bc785a63dd089613f7567457538a18b549a16f82d6dee1dcdcb5ac7898616
  - linux/amd64 digest: sha256:2e89c5842c5adcfc62a99b0a7c3bfa37534289f8dac02f9c41f677e91c2f62a5
  - linux/arm64 digest: sha256:25af597769085a6a84de3cf5a7c1e046f583ffc6ebf64c82a8ed27e104e8d0a9
License: Grafana Labs license

Changelog

[ENHANCEMENT] Upgrade Go base image to 1.23.6.
[ENHANCEMENT] Update base image to alpine:3.20.5.
[BUGFIX] Upgrade to github.com/golang/glog to v1.2.4 to resolve CVE-2024-45339

Upstream Grafana Mimir details

Version: 2.15.0
Hash: 56defe75b90f0f150d489f735708928fcd8a5b8f
Changelog: CHANGELOG.md

v2.15.0 – January 9th 2025

Links

Binary (Linux AMD64)
- Download
- SHA256: f99ec51ca9ecafa9812b2bd74229ce32cdaa65ae68264ae851f583e8aca2e50d
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.15.0
  - Manifest digest: sha256:86bf3b2784881c5b825684f8bc17804e742da7205a1f939c4ddf966977593dfd
  - linux/amd64 digest: sha256:0af81db9d3765f8ae002834d52da929ebf4e458fcea138b3a20db43d54a6ed58
  - linux/arm64 digest: sha256:a029b36db23378dd8e2bee78a368073d4cec5b282f921f470b4a13333f4176c8
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.15.0
  - Manifest digest: sha256:1ae1e901405aee0d19bcfb6ac7aecd0000574af43009b9d4609447b9b137fc9e
  - linux/amd64 digest: sha256:250a8001dcdb3f6bbf1f5aa17fba795e4bd526bcb63069b59246eab862b395ff
  - linux/arm64 digest: sha256:05855fbec9718f2b518db75662ebb2fcdc7711511aaa1b4d4affae5b424ce1b0
License: Grafana Labs license

Changelog

[CHANGE] Federation-frontend: proxy targets must be GEM clusters. The federation-frontend can no longer point at any Prometheus-compatible endpoint. To avoid downtime, first upgrade the proxy GEM clusters to GEM 2.15 or later. Then, upgrade the federation-frontend to the same version or later.
[CHANGE] Ruler: the /prometheus/config/v1/rules endpoint no longer returns an error if a rule group is missing in the object storage after being successfully returned by listing the storage, because it could have been deleted in the meantime.
[BUGFIX] Admin API: Include initial ETag version in token creation response.
[BUGFIX] Cross-cluster query federation: series with the label name __cluster__ were incorrectly handled in shardable (can be divided up and sent to different clusters, then combined later) queries. This fixes that issue and allows users to use original___cluster__ as a label matcher to match that label and __cluster__ to match the cluster they defined (not an actual label in the data), making it consistent with the behavior for other kinds of queries. This applies to aggregation grouping labels as well.
[BUGFIX] Upgrade go-jose to address CVE-2024-28180.
[BUGFIX] Upgrade go-oidc to address CVE-2024-28180.

Upstream Grafana Mimir details

Version: 2.15.0
Hash: 24e4281c138d873772076c5ac276a0f20f633d0d
Changelog: CHANGELOG.md

v2.14.2 – January 8th 2025

Links

Binary (Linux AMD64)
- Download
- SHA256: 30132e9a8e10671794c85b5dd11eef75adb21a79de6cf53cef79e7e15c858ea2
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.14.2
  - Manifest digest: sha256:246f8d17d4f374e95536655bfab6e56ddd8afc4d023bd7970f01ab42c13e09c9
  - linux/amd64 digest: sha256:179d512f41fe89c3c734e36c0f43636c74b1b356b80b7bcd7ec441aa018458f0
  - linux/arm64 digest: sha256:6b59da3e9fd324601be2f53b5929d567bc435465791dd6553cdc5b2749a248f9
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.14.2
  - Manifest digest: sha256:55f6d7c9f4b3151127b3f98a4efa8bb50a37f3bb21c2af0562261a754633fac0
  - linux/amd64 digest: sha256:30d85adb242179608840468441635b661ee3e9af76a0ecd0a688871d02b23a55
  - linux/arm64 digest: sha256:1b096b7629269dd4b438f4f401f2e2dd53ffaf522b3e1486d81321394effbe70
License: Grafana Labs license

Changelog

[ENHANCEMENT] Use Go 1.22.10 for build. [ENHANCEMENT] Update Mimir to 2.14.3 to get golang.org/x/crypto 0.31.0 and golang.org/x/net 0.33.0 update which includes a fix for CVE-2024-45337 and CVE-2024-45338. [ENHANCEMENT] Update github.com/golang-jwt/jwt/v4 to 4.5.1 to fix clear CVE-2024-51744.

Upstream Grafana Mimir details

Version: 2.14.3
Hash: b7d9445b8230f76e11dc1da8e1d1a0292fe9912e
Changelog: CHANGELOG.md

v2.14.1 – November 18th 2024

Links

Binary (Linux AMD64)
- Download
- SHA256: bc6167d4f5296e35317c31932e24b6f7b2ebec88f6361d39e2fd91a4fb368164
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.14.1
  - Manifest digest: sha256:cc257cd31b30745a0bd8a20c8eae44a612f1b80d1bf7bef1d2aa6ee830a48db3
  - linux/amd64 digest: sha256:0de84a698983bd2cd502c392ae715cf08b511ef1528dbe59ef5580a1236bbc41
  - linux/arm64 digest: sha256:4f3e8408db1c4d246ea81a9ed4b77b724b8b906dd1a1d6c7c82df0dbf23a9596
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.14.1
  - Manifest digest: sha256:5a1142699308357cd9a75ede289defbe3dc177f2da41fe1f0165a896cee13311
  - linux/amd64 digest: sha256:b42467dc82b8fbfd3abe1cbe9e8f81113a887cc45ee9462ecc5db22d14c9a5ab
  - linux/arm64 digest: sha256:6d3296d5500388abafc48a9c2585e9de0f3956e1d2e75863744595809c23a2b4
License: Grafana Labs license

Changelog

[ENHANCEMENT] Use Go 1.22.8 for build. [BUGFIX] Update Mimir to 2.14.2.

Upstream Grafana Mimir details

Version: 2.14.2
Hash: 2db2c4d9de8565c52fcef3050b95b8a2808b407e
Changelog: CHANGELOG.md

v2.14.0 – October 14th 2024

Links

Binary (Linux AMD64)
- Download
- SHA256: 4aa30c143389adc5ad2a42fd3fb7396a33ecdfe4f1a9ded3fbcc1395aed4b802
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.14.0
  - Manifest digest: sha256:9b781a4f91f36a39e1a1022a0229e17a466bba6bd5d710571a4caf5713f00f5c
  - linux/amd64 digest: sha256:d6d6c92b60349cd7ece1fdd7c328a44abdd98e42758004ff90018f15f3b8e1ca
  - linux/arm64 digest: sha256:ec5ab3b76ae4ef4b21a537e38d04428e3c4171a4bdc211d4c3b6c3d6b485ca88
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.14.0
  - Manifest digest: sha256:3459020dbe511f4b1762e9ed9a2669fc60fe30a22ec2613d16f091349ebabdec
  - linux/amd64 digest: sha256:66c92f32844a2bdd7862b4a806610f05a61f9fbf31a18c92a803cfb866c40991
  - linux/arm64 digest: sha256:e98d3df1caa521c0f2fff50c7691b2e4faa60bb984200195c5856e84641caeec
License: Grafana Labs license

Changelog

[ENHANCEMENT] Cross-cluster query federation: make timeout for queries to remote targets configurable.
[ENHANCEMENT] Cross-cluster query federation: apply timeout for label names and label values requests to remote targets.
[ENHANCEMENT] Pass access-policy and token names from gateway to backend services as headers to use log them in query statistics.
[ENHANCEMENT] LBAC: Enforce label policies on LabelNames endpoint.
[ENHANCEMENT] Internal gateway gRPC proxy clients can now have their maximum send and receive size limits configured via proxy.grpc-max-recv-msg-size and proxy.grpc-max-send-msg-size config flags.
[BUGFIX] Graphite querier: fix "name" tag setting in various functions, including aliasSub, aliasByNode among others.
[BUGFIX] Fetching unknown tokens via admin API now returns status code 404, and not 500.

Upstream Grafana Mimir details

Version: 2.14.0
Hash: bc5a6ce2f67de0f5fbade9b000e4bf46d6d59518
Changelog: CHANGELOG.md

v2.13.2 – October 2nd 2024

Links

Binary (Linux AMD64)
- Download
- SHA256: ed09a3c43f1535eec3338a21b66713b1116352a37e1a375be36e1c9c89ac4888
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.13.2
  - Manifest digest: sha256:23d335223e8e43f43e5a0f4e0235f2d2739af057ff3282b8e694c1fffda3c695
  - linux/amd64 digest: sha256:bdae9403d0e19bdd180842c425cb6eeb781bcbcffd4f5167ebe8f3f42d852949
  - linux/arm64 digest: sha256:3d2e8e0b21d1acffb756694af6d40142a89896e80895a30996a9cca6ea8ed3cb
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.13.2
  - Manifest digest: sha256:3d1abc079d3e7288975f87dc89f85e65b1a8bd57ef34605092c90aa2dd6726fe
  - linux/amd64 digest: sha256:7f7d55564a7dd506f3a90ee72d3c6d344f8919bc378e39e89aa0872dabdbaddd
  - linux/arm64 digest: sha256:dd8f23fcc64fc6fbc4f091cfe741a29f032d1843a62067c3f6c67f372b39d8c9
License: Grafana Labs license

Changelog

[CHANGE] Update base image for grafana/enterprise-metrics-alpine image to alpine:3.20.3.

Upstream Grafana Mimir details

Version: 2.13.0
Hash: 4775ec156855ba1eb498e564f09abad35053a44f
Changelog: CHANGELOG.md

v2.13.1 – August 15th 2024

Links

Binary (Linux AMD64)
- Download
- SHA256: e75440830bf73a76acbc4606a62abe85c4ae96173693b27b50bb8cb802d4e5f8
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.13.1
  - Manifest digest: sha256:bef3d3390d9b2e2a2208552f3ab98a23a7afa90d98fda8f950d1b23bae99575f
  - linux/amd64 digest: sha256:8b208b8ecefc7905608da445f128e2aa571f9e1eec7c1427ac637e41a5922be8
  - linux/arm64 digest: sha256:f1113c92e13987a7377aa8617a4b361914fde77d3f38e902d79ed6c9333d5445
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.13.1
  - Manifest digest: sha256:a3c4d70ccae452db70f6ad0e3a1d9aec29a29734761781de35afe57020100b92
  - linux/amd64 digest: sha256:7fb83ba3377afe63808ee146e27ef334dd1eaf71557d4bd4fb5873a03b42015c
  - linux/arm64 digest: sha256:484321e918b5af25159f863d492b88697986938841ed6e53850bf9fda68ba499
License: Grafana Labs license

Changelog

[CHANGE] Cross-cluster query federation: log failed requests to remote targets.
[CHANGE] Active series: the default value for active series idle timeout (-ingester.active-series-metrics-idle-timeout) increased from 10m to 20m.
[CHANGE] Active series: active series metrics must now be enabled and their update interval must be set to 20m; otherwise, GEM fails to start.
[BUGFIX] Cross-cluster query federation: fix issue where time range for label names and label values requests was not propagated to remote targets.

Upstream Grafana Mimir details

Version: 2.13.0
Hash: 4775ec156855ba1eb498e564f09abad35053a44f
Changelog: CHANGELOG.md

v2.13.0 – July 11th 2024

Links

Binary (Linux AMD64)
- Download
- SHA256: 6b929e58f746f029d8e4e5760fad7bfacae344436b74c986c305701d1d16e4fc
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.13.0
  - Manifest digest: sha256:3a7b42c70e1f9d67b0a3da8bd1540e172708929f98e73a60aa00f0ddd45db3a1
  - linux/amd64 digest: sha256:2dc0140c4adb1c06d7e050ad200ef156701bb3bb99c1b6f7496ce09ba8a52560
  - linux/arm64 digest: sha256:63f07bd37fff257a54111e6dde2e0acd565f52731201bda7cf848f8cf4a2c882
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.13.0
  - Manifest digest: sha256:b21e3489fad6e4334b3f7645b1c219873187c1d4de88fbfed67376d61c458bc0
  - linux/amd64 digest: sha256:0a81b72217b27b09672980ea3e9f8f87545f5eda273189897dbd36648652f041
  - linux/arm64 digest: sha256:9e07d6094c272426325685d7ae7a7047eec97429839283d271518bf6f7b8788a
License: Grafana Labs license

Changelog

[CHANGE] Implemented enhanced usage statistics for more accurate reporting.
[CHANGE] Build: grafana/enterprise-metrics docker image is now based on gcr.io/distroless/static-debian12 image. Alpine-based docker image is still available as grafana/enterprise-metrics-alpine, until 2.15.
[CHANGE] Clamp GOMAXPROCS to runtime.NumCPU.

Upstream Grafana Mimir details

Version: 2.13.0
Hash: 4775ec156855ba1eb498e564f09abad35053a44f
Changelog: CHANGELOG.md

v2.12.1 – August 14th 2024

Links

Binary (Linux AMD64)
- Download
- SHA256: 18862c2278c3e7a4b3db1eba0af5d93b7d639acb8b18189d989adaf1f73ab555
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.12.1
  - Manifest digest: sha256:027aff48b2ee252e8af21165d0ade2cfba2523846f490b001ff44102d6be4fa5
  - linux/amd64 digest: sha256:e22ca392807a8986c6aabaaab1242162fc74f011c49fd4808af87d1bd0a1d390
  - linux/arm64 digest: sha256:8d50c66d0892783387746b710a30b156f24711449fbd5bd6f3d3b33633fdddfa
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.12.1
  - Manifest digest: sha256:09ba5d7d316576fcdec5a9357721b543247b4acb2c95b882417faf3535154466
  - linux/amd64 digest: sha256:f42e8db3268a75b3c08b6e00deda037d80c1eda09bd6d5d44c6934772b128bdb
  - linux/arm64 digest: sha256:f2ab9cd77ef42116caa8f897f5be20a559c211871ded61172132c894ab1858bc
License: Grafana Labs license

Changelog

[ENHANCEMENT] Update base image to alpine:3.18.8

Upstream Grafana Mimir details

Version: 2.12.0
Hash: c7aab9e039d63397d2293114ad063b03626e247b
Changelog: CHANGELOG.md

v2.12.0 – April 5th 2024

Links

Binary (Linux AMD64)
- Download
- SHA256: 72b821d40f5e7548c5f30fe4a4246058c0fe278abb46deb9f77ef1c6dc655186
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.12.0
  - Manifest digest: sha256:fb8514119242911882899145bb97339d2a3e281dbc75d9eac974d57d4244d5e4
  - linux/amd64 digest: sha256:3b11a70277cf79422f776da04c0cea365a604227375ed2dc693a1fdc705b74ce
  - linux/arm64 digest: sha256:4d977a53e7213d2854c2576d1820422891646f750101f7dbb0742b83ada9a525
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.12.0
  - Manifest digest: sha256:730142084b03f9f87d6ad7913c20038b170b26a42fe30d61ffec50660a223a1c
  - linux/amd64 digest: sha256:8d2dca5749eddfaaa1bf9a7b5eaf87e5fa12c26a0fc45e7a46bc237c0dc72238
  - linux/arm64 digest: sha256:ef7b1f111d66160f2c67ecf56f928b3efc307630f2b7299e9ac5f56a3bd3ff45
- Standard distroless image: docker pull grafana/enterprise-metrics-distroless:v2.12.0
  - Manifest digest: sha256:ae917c1032688f262f0a78e0592b06194e70865c0848c22bb17eadc27f1f93c4
  - linux/amd64 digest: sha256:15767ad1d23eb3f08d23a8b0bec4b0902f90fc05c43ab62e1e084b0dc8745eff
  - linux/arm64 digest: sha256:1159ff518547f0234cb746fdb676393bb3f2639de46db85ad7bb059a29785699
- BoringCrypto distroless image: docker pull grafana/enterprise-metrics-boringcrypto-distroless:v2.12.0
  - Manifest digest: sha256:dc336656922d99f5e5b7556768b55aa2c97254220b3f98da52e7ac5e3ce0bca4
  - linux/amd64 digest: sha256:5192a4d2e9db563450e36b5546ded7ec389df2336c9812ae5040141b336796b8
  - linux/arm64 digest: sha256:8193426a4373d956c4436ad249549d604f17a0931bb19ca4bcc6395b594cdd8d
License: Grafana Labs license

Changelog

[CHANGE] Server: responses with HTTP 4xx status codes are now treated as errors and used in status_code label of request duration metric.
[CHANGE] Disable SSL certificate expiration collector when using certificates from Vault.
[ENHANCEMENT] Build GEM based on a distroless image.
[ENHANCEMENT] Update base image to alpine:3.18.5
[ENHANCEMENT] Add experimental flag -graphite.querier.deduplicate-samples-in-raw-interval to test a fix for duplicate points appearing in Graphite data. This flag will be removed when the correct behavior is determined.
[ENHANCEMENT] Add support for setting graphite querier rate limits to <= 0 to disable individual limits.
[ENHANCEMENT] Set tracer max tag value length to 16e3 instead of the default 256.
[BUGFIX] LBAC: caching for /prometheus/api/v1/label/* endpoints was not respecting LBAC policies leading to sharing cached responses for different LBAC policies within the same tenant.

Upstream Grafana Mimir details

Version: 2.12.0
Hash: c7aab9e039d63397d2293114ad063b03626e247b
Changelog: CHANGELOG.md

v2.11.2 – March 6th 2024

Links

Binary (Linux AMD64)
- Download
- SHA256: b76a453d8b865621c95c80f7a4dc4b601e4c69281f2a0ab0da2b09d6b932e9d5
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.11.2
  - Manifest digest: sha256:40b0c6a82a98f0ff90c0183089dacbebef56413830e557f7c313ee2d3a6fc67c
  - linux/amd64 digest: sha256:b338df2b83b1b3d4061b2ecba54486359b7f01ed28097cd8b3b8284018c0b86e
  - linux/arm64 digest: sha256:ad021cf4d79cd519d62722f53200598444c3273724c38d9f842fa0374587c87d
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.11.2
  - Manifest digest: sha256:c58bdd0db618b4a1f54eb42f8b66767f5adf5b52cc70092ed1225f2bffc066e6
  - linux/amd64 digest: sha256:a101377e4a35c91cce0bd8cc050223bef608a75dd69b3803b4b777414d2e38fa
  - linux/arm64 digest: sha256:3d5857ea72d71acbfc6fcc00f6b4d40f14a4c21c0c3d328689b749fc450c4921
License: Grafana Labs license

Changelog

[BUGFIX] Use Go 1.21.8.
[BUGFIX] Update base image to alpine:3.18.5

Upstream Grafana Mimir details

Version: 2.11.0
Hash: c8939ea55f0818337e3428924f25dafc6ea256d1
Changelog: CHANGELOG.md

v2.11.1 – January 9th 2024

Links

Binary (Linux AMD64)
- Download
- SHA256: 3270a41f04156561bad0ab6394c7c015513fe3732b26cf04380e09f2ba1537f1
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.11.1
  - Manifest digest: sha256:42a73166fbfa331e6124c55db642c1d0609a902be631d1a2b04814632b49c9f7
  - linux/amd64 digest: sha256:8a4264f4edf7703a16919c8e0e0ff51f62c026f15fa059e38beba693200361b9
  - linux/arm64 digest: sha256:96d8789cb22b33b650796554be8dd17548dea57ee526fdadf8386fc4381939b4
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.11.1
  - Manifest digest: sha256:e74f4e64af43d0c06e644feb47823b9aa27ba355a76b4c11489439327592afe8
  - linux/amd64 digest: sha256:bb7dcfd7ae176afb252ab90392b2e4b0ac9c0522ea0983b8042150ea226b24eb
  - linux/arm64 digest: sha256:ed6d25e7593c1106c20d617dbd8ac1c934581bc68dda6e7b7b5a10d98d8e05a2
License: Grafana Labs license

Changelog

[CHANGE] Remove support for using AWS Marketplace licenses (-license.type=aws-marketplace).
[CHANGE] Network errors are now responded with 598 status code instead of 504 (originally 408).
[ENHANCEMENT] Graphite querier: Add support for maxDataPoints=-1 in /render endpoint.
[ENHANCEMENT] Update base image to alpine:3.18.4
[ENHANCEMENT] Gateway audit logging: support decimal values for sampling rate. For example, setting admin-api.auditlogging.sample-rate to 0.1 means sampling of 0.1%.
[ENHANCEMENT] Graphite querier: use input time range for querying tag values.
[ENHANCEMENT] Graphite querier: added new -graphite.querier.max-query-length flag to limit the largest time range allowed for queries.
[ENHANCEMENT] Add -auth.required-for-debug option for disabling auth on /debug endpoints. Auth is required by default.
[BUGFIX] Fix federation-frontend panic because of nil ExemplarQueryable
[BUGFIX] Fix execution: attempted to read series at index X from ingester chunks stream, but the stream has series with index Y errors when multiple LBAC label selectors apply to a request and ingester to querier or store-gateway to querier chunks streaming is enabled.

Upstream Grafana Mimir details

Version: 2.11.0
Hash: c8939ea55f0818337e3428924f25dafc6ea256d1
Changelog: CHANGELOG.md

v2.10.6 – March 6th 2024

Links

Binary (Linux AMD64)
- Download
- SHA256: f8bb921cce9b5bbcb6b3b2897a8ff6be0a0f53e5f735f2ce6520fedc874e2c33
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.10.6
  - Manifest digest: sha256:9df8bbe69487e1975831f5c3f9a8479494d902b5cf9e4bbddf73217edb907567
  - linux/amd64 digest: sha256:1cf8c2db208104bfa7ae10c881c22ff40bcd88392bb874cb6baa841a9ee73c0b
  - linux/arm64 digest: sha256:1bc90da7fbf4b0dc998d8c0f2950b3bc5a713997ef6a83f18b7aee4fbfa91c55
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.10.6
  - Manifest digest: sha256:2d6658d1c9e300f138b7cfa17ef790fae1cb1cae9c0ac538c16353a615314fb0
  - linux/amd64 digest: sha256:e3cc9e2d4d339a84a18ef207df179ceaf69da61c214699cf4e2faadf52239bda
  - linux/arm64 digest: sha256:40fd2f3634f86b1d9ee139646f3696963c0b8d82c6305d55d332a1bf0acc8f30
License: Grafana Labs license

Changelog

[BUGFIX] Use Go 1.21.8.

Upstream Grafana Mimir details

Version: 2.10.5
Hash: f0830941c68406123603422701b9b2e5bc24b87a
Changelog: CHANGELOG.md

v2.10.5 – December 13th 2023

Links

Binary (Linux AMD64)
- Download
- SHA256: f4620b4ce10b477bc47d4e2532e85173778e160e7d979509fa23c49f79615d84
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.10.5
  - Manifest digest: sha256:2a2b93c5d3dfe177e2e6bcbf496eb92224f46c55e0163fded23cbe83c6f173d1
  - linux/amd64 digest: sha256:d38023d560bc6ba623532e55f5021cfe9da71b34372256bf9cf6300bcc12f95e
  - linux/arm64 digest: sha256:cb2bc8348c3423cbdc5aad80b5d4fc3d352f1246f42562313e500fe1295788ea
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.10.5
  - Manifest digest: sha256:f4476e021c3472bd675e872e3e06c046374db3c2c9c65d5378858b05b593178c
  - linux/amd64 digest: sha256:f71c68cb8f4edf47b7e460b929f7c949cc8f092f819e985687f256b9700b67cb
  - linux/arm64 digest: sha256:7a419080ff7ce974bcc0801aef6048dce4cd0c4494eab378402f92c649031b10
License: Grafana Labs license

Changelog

[BUGFIX] Update base image to alpine:3.18.5
[BUGFIX] Use Go 1.21.5.

Upstream Grafana Mimir details

Version: 2.10.5
Hash: f0830941c68406123603422701b9b2e5bc24b87a
Changelog: CHANGELOG.md

v2.10.4 – November 14th 2023

Links

Binary (Linux AMD64)
- Download
- SHA256: 4f928f5a7a69eb49925b9d7386aed1211ddd98d76f5701c3d5af301d5f6cc759
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.10.4
  - Manifest digest: sha256:a59219679aa471817a39a34e48bfb0ac371960aa8f53b516f6a03bdf5fc47f5c
  - linux/amd64 digest: sha256:d061004df0d39a5c7c3ce3aea1a4a2b67b8d1f6223ebf2ae0e85d0424e676d62
  - linux/arm64 digest: sha256:fcb88978bb8bde446c06914f92db44190a7501ba8df47cb03d01c7ae59737a47
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.10.4
  - Manifest digest: sha256:e40b88405c0cc35523ff52e75896c9ffc68b78d8e64231bec0e2c68020751a3d
  - linux/amd64 digest: sha256:b873b49989bfce04b2a4f8697136b3560c48233181ce2ebc62507e9a8f35731e
  - linux/arm64 digest: sha256:9ad3f7dc90bcd6186971fc88fc81476ca636dec4a5e09acfa18937e1437c7e68
License: Grafana Labs license

Changelog

[BUGFIX] Update otelhttp library to v0.44.0 as a mitigation for CVE-2023-45142.
[BUGFIX] Admin client: fix memory leak.

Upstream Grafana Mimir details

Version: 2.10.4
Hash: d1f4f1291001db7384ba66ca90b4f753309f262d
Changelog: CHANGELOG.md

v2.10.3 – October 18th 2023

Links

Binary (Linux AMD64)
- Download
- SHA256: 8dc77b1bd0a4707573d8cc1fa9d3b1361a7e6942719720734793e110b2e7c638
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.10.3
  - Manifest digest: sha256:da9c1ef4fdc779532f9d2d81cc97f3d909b55b19ccc6536aa6a91eb0cc971fdc
  - linux/amd64 digest: sha256:5def0a00df7655dd1127bc9ac8aedb684fb0a45f7a62c27541b0a2e4dc5a42e9
  - linux/arm64 digest: sha256:2b13a29d643a2d987c6a5a0ba693264c233ef387dd3e4db85adefc330b4c9c42
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.10.3
  - Manifest digest: sha256:80a728cbadab32673062e06b3bdd43883f96c638b8a2d95508d82847eb5da858
  - linux/amd64 digest: sha256:89704845628a413eca8e5b773d207321c3be394fe2acdaa62f3ba8851306abe3
  - linux/arm64 digest: sha256:23d8c368a8842c7c781ee8cbc96773f4dfbbe2975b5703c38e460cf6e0aa6053
License: Grafana Labs license

Changelog

[BUGFIX] Update grpc-go library to 1.57.2-dev to fix a concurrency issue introduced in 1.57.1.

Upstream Grafana Mimir details

Version: 2.10.3
Hash: eb097eae69b4a324cc67e083f9d99489e36d2ab9
Changelog: CHANGELOG.md

v2.10.2 – October 13th 2023

Important note

Warning
This release contains a known bug in the grpc-go library that can drastically affect the performance of grpc servers. We recommend updating to the newer v2.10.3 that fixes this issues.

Links

Binary (Linux AMD64)
- Download
- SHA256: acde7e6080f05072bd42e29f9257198f88fd2da7b5414de8f9968ac8f5948f54
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.10.2
  - Manifest digest: sha256:fa86c72666322a0663d12778aee1db1147f09514a3b30ac00a2948ba66ecbc5a
  - linux/amd64 digest: sha256:a898416b8aa0967ebdde8308434897841bf4b97e7a1eb5873e376b7d13963da1
  - linux/arm64 digest: sha256:2bc766cf3e49b40398219f514bf6a7f401b3b7e3851bb9c88933c0690ea432ae
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.10.2
  - Manifest digest: sha256:7ca15841d01f83379cafd7a1f4aec8849632c0f25d8b19e4abaf11cb20cd441c
  - linux/amd64 digest: sha256:655776c78d52ebd89d342979ee56d156d5606bd0abdf325fe8f56cd78c30f1b8
  - linux/arm64 digest: sha256:c165171f2858eb45bbb7d6ce27dceb2b815732d387b3a2d5e3b4681c0ca18f2e
License: Grafana Labs license

Changelog

[ENHANCEMENT] Update Go to version 1.21.3, grpc-go to version 1.57.1 and golang.org/x/net to version 0.17 to fix CVE-2023-44487.

Upstream Grafana Mimir details

Version: 2.10.2
Hash: e0ed0f5986389fe5357bc6cba8589b5a1fc5ff3a
Changelog: CHANGELOG.md

v2.10.1 – September 19th 2023

We have encountered problems during publishing artifacts for Grafana Enterprise Metrics v2.10.0, so we have skipped that version.

Links

Binary (Linux AMD64)
- Download
- SHA256: 9f08d5caadf2931142c3f7b0b291ccb14debc43d2fea3cf3cb59af5c6b1a1740
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.10.1
  - Manifest digest: sha256:bc3636d33e1564cf5db9fc3d631eb235bcdab832541bbbf4fb8a78e776e60f6d
  - linux/amd64 digest: sha256:9632070af8db99f455380dd0ed45828a2557085c738dcbcf53196c2d87ad2068
  - linux/arm64 digest: sha256:e9721f93c6cf506361184dc6cb93061805fb5fff34f1f026872a2913445ace48
- BoringCrypto image: docker pull grafana/enterprise-metrics-boringcrypto:v2.10.1
  - Manifest digest: sha256:59def74ac3c5900d205f673cf3448ee15f50d5c3f35b06220a71ac53b69eddd0
  - linux/amd64 digest: sha256:c58764fd32b575d01dbde201389e633b0f51913309055a27b065b9ce84f6f1f7
  - linux/arm64 digest: sha256:a69af77cd0bf74f45947e72f104395d9d233569600b2adc1218072d9c99141e5
License: Grafana Labs license

Changelog

[ENHANCEMENT] Graphite querier: CarbonAPI adds support for inf literal for keepLastValue and interpolate functions.
[ENHANCEMENT] Graphite querier: skip all whitespace during query parsing.
[ENHANCEMENT] Enterprise Metrics: use new data structure for labels to reduce memory consumption.
[ENHANCEMENT] Graphite querier: /functions api now natively supported instead of being proxied to stock Graphite Web.
[ENHANCEMENT] Enterprise Metrics Ruler: added insight=true field to ruler’s evaluation logs.
[ENHANCEMENT] Update base image to alpine:3.18.3.
[ENHANCEMENT] Added support to rate limit application logs:
- -log.rate-limit-enabled
- -log.rate-limit-logs-per-second
- -log.rate-limit-logs-per-second-burst
[ENHANCEMENT] Use Go 1.21.1.
[BUGFIX] Graphite querier: return 499s instead of 500s when clients cancel requests in certain scenarios.
[BUGFIX] Graphite querier: correctly support setting SplitQueriesByInterval to 0.
[BUGFIX] Graphite querier: fixes a JSON serialization bug when a returned value is infinity.
[BUGFIX] Graphite querier: fixes handling of holtWinters functions, that are now proxied to Graphite Web.
[BUGFIX] Graphite querier: Updated rollup aggregation to aggregate NaN or empty sample lists to NaN instead of zero.
[BUGFIX] Graphite querier: wildcard matchers in patterns are now correctly honored, and series with less nodes than queried are no longer returned as part of the result.
[BUGFIX] Graphite querier: CarbonAPI’s groupByNode function implementation now returns the expected number of series.
[BUGFIX] Graphite querier: CarbonAPI’s holtWintersConfidenceBands and holtWintersForecast functions implementation now return correct results.
[BUGFIX] Graphite querier: CarbonAPI’s moving* functions implementation now returns correct results.
[BUGFIX] Graphite querier: CarbonAPI’s exponentialMovingAverage functions implementation now returns correct results.
[BUGFIX] Graphite querier: filter out series with fewer nodes than queried when path expressions end with glob patterns.
[BUGFIX] Graphite querier: handle remote querier disconnections more gracefully, avoiding internal errors in certain scenarios.
[BUGFIX] Admin client: Fix not invalidating in-memory cache sooner. If admin client cache is in-memory, then revert to the original maximum 1 minute cache expiration before -admin.client.cache.expiration was introduced.
[BUGFIX] Admin client: Fix ignoring -admin-api.leader-election.client configuration parameters.

Upstream Grafana Mimir details

Version: 2.10.0
Hash: 77906f76055188998b0f8f28b89aaeb68ab08feb
Changelog: CHANGELOG.md

v2.9.4 – December 11th 2023

Links

Binary (Linux AMD64)
- Download
- SHA256: 9b92071a7a4041bd97f753e093d4d1897022216b286dac146899eff85426ab05
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.9.4
  - Manifest digest: sha256:9bd1106439216fac3d1e99414fcded85b39b59e00539a60f6a7966f1ecd5e575
  - linux/amd64 digest: sha256:f773e16b65856e754122c3fb5f909a8cd1a68ca94e2239a96c74efb6939a6895
  - linux/arm64 digest: sha256:151ad40bccfe112c6f007b37ca2281d118bb1dc927379943b0bed5c50b65db64
License: Grafana Labs license

Changelog

[BUGFIX] Update Alpine to 3.18.5 to address CVE-2023-5363.
[BUGFIX] Update Golang to 1.20.12 to address CVE-2023-39326.

Upstream Grafana Mimir details

Version: 2.9.3
Hash: 97e892ac6f17a682e831b744ad96cbc09a3b0d44
Changelog: CHANGELOG.md

v2.9.3 – November 14th 2023

Links

Binary (Linux AMD64)
- Download
- SHA256: 5817d4a576eccb08e5b99a7d86aa6493838d83a9c19c9e1a62a791e2476b4ba6
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.9.3
  - Manifest digest: sha256:e92e1fff77ef0a9d3a11a43d00ac5d9cf8b178b872f4e0e748480cb180138c45
  - linux/amd64 digest: sha256:ff6aa6376432a944739d0f1d42ee05519bb9f91e8f88b23963a2fe3cc919ed3c
  - linux/arm64 digest: sha256:6cda00e4a839c38a33e6e6c9acbeaa0e2fca564692486deb2698f86ee736867b
License: Grafana Labs license

Changelog

[BUGFIX] Update otelhttp library to v0.44.0 as a mitigation for CVE-2023-45142.
[BUGFIX] Admin client: fix memory leak.

Upstream Grafana Mimir details

Version: 2.9.3
Hash: 97e892ac6f17a682e831b744ad96cbc09a3b0d44
Changelog: CHANGELOG.md

v2.9.2 – October 19th 2023

Links

Binary (Linux AMD64)
- Download
- SHA256: a0e5610c52010c9238143cfbeb5c31d8c8b1c0471aed6a3a5bd9068cfabf2e40
Docker images:
- Standard image: docker pull grafana/enterprise-metrics:v2.9.2
  - Manifest digest: sha256:1a63c2ded4bd50aec629e86ce64def21ee6610b810f5ba7264a80f088b2aad80
  - linux/amd64 digest: sha256:3857c1d2705c02ff653608c317494416d3c1bb414ee2c81cedf608ad18ac464b
  - linux/arm64 digest: sha256:820766d44a554e8b6cf96ed2b27e6ef1d808df6ac45670354b0f676e6df2eaf0
License: Grafana Labs license

Changelog

[BUGFIX] Update grpc-go library to 1.57.2-dev to fix a concurrency issue introduced in 1.57.1.

Upstream Grafana Mimir details

This is correct, Mimir version was not changed for this release.

Version: 2.9.2
Hash: 74bc18c39f0c81cd50b979103fd8ade62703f198
Changelog: CHANGELOG.md

v2.9.1 – October 17th 2023

Changelog

[ENHANCEMENT] Update Go to version 1.20.10, grpc-go to version 1.57.1 and golang.org/x/net to version 0.17 to fix CVE-2023-44487.

Upstream Grafana Mimir details

Version: 2.9.2
Hash: 74bc18c39f0c81cd50b979103fd8ade62703f198
Changelog: CHANGELOG.md

v2.9.0 – June 22nd 2023

Changelog

[FEATURE] Admin API: Add experimental audit logging for requests on the admin-api path. Enabled with admin-api.auditlogging.enabled.
[ENHANCEMENT] GRPC client config for components can now set three new advanced flags: connect_timeout, connect_backoff_base_delay, and connect_backoff_max_delay.
[ENHANCEMENT] Graphite querier: applyByNode and useSeriesAbove functions are now correctly processed by CarbonAPI and not proxied to stock Graphite Web.
[ENHANCEMENT] Publish a multi-architecture Docker image with x86 and ARM images.
[ENHANCEMENT] Graphite querier: aliasQuery is now correctly processed by CarbonAPI and not proxied to stock Graphite Web.
[ENHANCEMENT] Update base image to alpine:3.18.2.
[BUGFIX] Graphite querier: count function in summarize now correctly only counts non-NaN values.
[BUGFIX] Graphite querier: smartSummarize is now correctly processed by CarbonAPI and not proxied to stock Graphite Web.
[BUGFIX] Graphite querier: hitcount is now correctly processed by CarbonAPI and not proxied to stock Graphite Web.
[BUGFIX] Graphite querier: certain bad requests no longer return 500 errors.
[BUGFIX] Make gRPC proxy fail write request immediately if receiving request body times out, not updating cortex_request_downstream_duration_seconds metric.
[BUGFIX] Ruler: fix performance of max rule group enforcement when per-tenant max rule group limits are disabled.

Upstream Grafana Mimir details

Version: 2.9.0
Hash: 761114d8b026dac77ea94517fd9773632c32b42b
Changelog: CHANGELOG.md

v2.8.1 – May 8th 2023

Changelog

[BUGFIX] Ruler: fix performance of max rule group enforcement when per-tenant max rule group limits are disabled.

Upstream Grafana Mimir details

Version: 2.8.0
Hash: f917e084d4a5596565b5debd659db0d4a5f9da6d
Changelog: CHANGELOG.md

v2.8.0 – May 3rd 2023

Changelog

[CHANGE] Graphite querier: New default retry behavior for the querier whenever proxied requests to graphite web fail with a 502 error.
[CHANGE] Graphite querier: the heavy rate limiting now tries to delay requests instead of dropping them directly and returning an error.
[CHANGE] Removed raw_increase PromQL function.
[FEATURE] Experimental PromQL mapping for pre aggregated metrics: removed -query-aggregations-enabled and -query-aggregations-optonal experimental flags, instead of that
- Added -query-aggregations-middleware-enabled which can be used to add the middleware to the query path.
- Added -query-aggregations-mapping-enabled-default which can be used to enable mapping by default.
- Per-request X-Query-Aggregations header can be used with true/false values to force mapping regardless of the configuration.
- Runtime limits query_aggregations: mapping_enabled: true/false configuration can be used to enable mapping on per-tenant basis.
[ENHANCEMENT] Include timezone data in Docker image. In particular, this enables the use of timezones in alertmanager silencing rules.
[ENHANCEMENT] Update all images to use Alpine 3.17.3.
[BUGFIX] Ruler: fix parsing of Mimir-specific YAML fields under ruler.query_frontend.

Upstream Grafana Mimir details

Version: 2.8.0
Hash: f917e084d4a5596565b5debd659db0d4a5f9da6d
Changelog: CHANGELOG.md

v2.7.2 – May 4th 2023

Changelog

[BUGFIX] Updated Go to version 1.20.4 to fix CVE-2023-24539, CVE-2023-24540, CVE-2023-29400.

Upstream Grafana Mimir details

Version: 2.7.1
Hash: dbe4ccd39cb58391e6174441ae11ae25bfebd395
Changelog: CHANGELOG.md

v2.7.1 – April 20th 2023

Changelog

[BUGFIX] Updated Go to version 1.20.3 to fix CVE-2023-24538.

Upstream Grafana Mimir details

Version: 2.7.1
Hash: dbe4ccd39cb58391e6174441ae11ae25bfebd395
Changelog: CHANGELOG.md

v2.7.0 – March 17th 2023

Changelog

[FEATURE] Adding new PromQL function raw_increase which will be used to perform aggregations.
[FEATURE] Added experimental PromQL mapping middleware that adds support for pre-aggregated time series.
- Added -query-aggregations-enabled which can be used to add the middleware to the query path.
- Added -query-aggregations-optonal which can be used to make the query mapping optional unless X-Query-Aggregations: true header is passed (default true, set to false to enforce).
[ENHANCEMENT] Added remote cache support for object storage client.
- The following CLI flags (and their respective YAML config options) have been added:
  - -admin.client.cache.expiration
  - -admin.client.cache.type
  - -admin.client.cache.memcached.addresses
  - -admin.client.cache.memcached.max-async-buffer-size
  - -admin.client.cache.memcached.max-async-concurrency
  - -admin.client.cache.memcached.max-get-multi-batch-size
  - -admin.client.cache.memcached.max-get-multi-concurrency
  - -admin.client.cache.memcached.max-idle-connections
  - -admin.client.cache.memcached.max-item-size
  - -admin.client.cache.memcached.timeout
  - -admin.client.cache.redis.endpoint
  - -admin.client.cache.redis.username
  - -admin.client.cache.redis.password
  - -admin.client.cache.redis.db
  - -admin.client.cache.redis.master-name
  - -admin.client.cache.redis.connection-pool-size
  - -admin.client.cache.redis.dial-timeout
  - -admin.client.cache.redis.idle-timeout
  - -admin.client.cache.redis.read-timeout
  - -admin.client.cache.redis.write-timeout
  - -admin.client.cache.redis.min-idle-connections
  - -admin.client.cache.redis.max-connection-age
  - -admin.client.cache.redis.max-async-buffer-size
  - -admin.client.cache.redis.max-async-concurrency
  - -admin.client.cache.redis.max-get-multi-batch-size
  - -admin.client.cache.redis.max-get-multi-concurrency
  - -admin.client.cache.redis.max-item-size
  - -admin.client.cache.redis.tls-enabled
  - -admin.client.cache.redis.tls-ca-path
  - -admin.client.cache.redis.tls-cert-path
  - -admin.client.cache.redis.tls-key-path
  - -admin.client.cache.redis.tls-server-name
  - -admin.client.cache.redis.tls-insecure-skip-verify
  - -admin.client.cache.redis.tls-min-version
  - -admin.client.cache.redis.tls-cipher-suites
  - -admin.client.cache.refresh-interval
[ENHANCEMENT] Update all base images from alpine:3.17.1 to alpine:3.17.2.

Upstream Grafana Mimir details

Version: 2.7.1
Hash: dbe4ccd39cb58391e6174441ae11ae25bfebd395
Changelog: CHANGELOG.md

v2.6.1 – April 21st 2023

Changelog

[ENHANCEMENT] Update all base images from alpine:3.17.1 to alpine:3.17.3.
[BUGFIX] Updated Go to version 1.19.8 to fix CVE-2023-24538.

Upstream Grafana Mimir details

Version: 2.6.0
Hash: 27698f399fc9e13c6fe0a8c79f882993814fda4a
Changelog: CHANGELOG.md

v2.6.0 – February 22nd 2023

Changelog

[CHANGE] Graphite querier: add experimental support for optional rate limitting at the subqueries level, using the .max-concurrent-sub-queries-per-request flag.
[CHANGE] Graphite querier: the storage aggregation method set in storage-aggregation.conf can no longer be overridden during runtime using consolidateBy when metrictank is used as a render engine for remote queries.
[CHANGE] Graphite querier: /tags/autoComplete/values now takes input time range into account. Previously only tag values in the last hour were returned.
[CHANGE] Graphite querier: The to/until parameter for /find and /tags endpoints are now respected by the querier, though if to/until is greater than the current time, it’s adjusted to the current time. Previously this value was always overwritten by the current time.
[FEATURE] Graphite querier: Add support for /metrics/expand endpoint.
[ENHANCEMENT] Refactor caching logic in the versioned bucket client to reduce the number of requests to object storage.
[ENHANCEMENT] Update all base images from alpine:3.16.2 to alpine:3.17.1.
[BUGFIX] Graphite querier: flush metric name cache for remote queries.
[BUGFIX] Graphite querier: fix panic when running certain combinations of functions.
[BUGFIX] Fix issue where authentication caches were not sized correctly resulting in poor performance.

Upstream Grafana Mimir details

Version: 2.6.0
Hash: 27698f399fc9e13c6fe0a8c79f882993814fda4a
Changelog: CHANGELOG.md

v2.5.3 – April 21st 2023

Changelog

[BUGFIX] Fix CVE-2023-24538 by upgrading to Go 1.19.8.

Upstream Grafana Mimir details

Version: 2.5.0
Hash: 25533fdfcf5d8e26ee8f49bcaf13e30bd678d4b3
Changelog: CHANGELOG.md

v2.5.2 – February 17th 2023

Changelog

[BUGFIX] Fix issue where authentication caches were not sized correctly resulting in poor performance

Upstream Grafana Mimir details

Version: 2.5.0
Hash: 25533fdfcf5d8e26ee8f49bcaf13e30bd678d4b3
Changelog: CHANGELOG.md

v2.5.1 – January 6th 2023

Changelog

[BUGFIX] Fix empty buildinfo in GEM binary

Upstream Grafana Mimir details

Version: 2.5.0
Hash: 25533fdfcf5d8e26ee8f49bcaf13e30bd678d4b3
Changelog: CHANGELOG.md

v2.5.0 – December 15th 2022

Changelog

[CHANGE] Flag -*.azure.msi-resource is now ignored, and will be removed in Mimir 2.7. This setting is now made automatically by Azure.
[CHANGE] Graphite querier: the storage aggregation method set in storage-aggregation.conf can no longer be overridden during runtime using consolidateBy when metrictank is used as a render engine. This matches Graphite’s behavior.
[CHANGE] Graphite querier: caches default TTL is now lowered to 10 minutes. This is done to keep consistency in the event that out-of-order ingestion is enabled in mimir so that graphite queries answer with latest available data instead of caching responses for days.
[ENHANCEMENT] Added .tls-min-version and .tls-cipher-suites flags to configure cipher suites and min TLS version supported by servers.
[ENHANCEMENT] All: Add clustername label to cpu usage metrics (cortex_quota_cpu_count, cortex_quota_gomaxprocs, cortex_quota_cgroup_cpu_max, cortex_quota_cgroup_cpu_period). The value is the cluster name in the GEM license.
[ENHANCEMENT] Add recording rules to fulfill requirements for all Mimir mixin dashboards. The following recording rules have been added to GEM Self Monitoring to better align with the Mimir mixin: target:cortex_ingester_queried_exemplars:99quantile target:cortex_ingester_queried_exemplars:50quantile target:cortex_ingester_queried_exemplars:avg target:cortex_ingester_queried_exemplars_bucket:sum_rate target:cortex_ingester_queried_exemplars_sum:sum_rate target:cortex_ingester_queried_exemplars_count:sum_rate target_instance:cortex_alertmanager_alerts:sum target_instance:cortex_alertmanager_silences:sum target:cortex_alertmanager_state_replication_total:rate5m target:cortex_alertmanager_state_replication_failed_total:rate5m cortex_alertmanager_alerts_invalid_total:rate5m target:cortex_alertmanager_alerts_received_total:rate5m target:cortex_alertmanager_partial_state_merges_total:rate5m target:cortex_alertmanager_partial_state_merges_failed_total:rate5m target_integration:cortex_alertmanager_notifications_total:rate5m target_integration:cortex_alertmanager_notifications_failed_total:rate5m
[ENHANCEMENT] Optimise the latest version lookup mechanism for versioned bucket client to reduce count of requests to object storage. Now it uses binary search instead of using sequential search.
[BUGFIX] Fixed a bug in the Graphite querier where render requests that failed to be processed by the native engine were not being proxied to Graphite web.

Upstream Grafana Mimir details

Version: 2.5.0
Hash: 25533fdfcf5d8e26ee8f49bcaf13e30bd678d4b3
Changelog: CHANGELOG.md

v2.4.2 – April 21st 2023

Changelog

[BUGFIX] Fix CVE-2023-24538 by upgrading to Go 1.19.8.

Upstream Grafana Mimir details

Version: 2.4.0
Hash: 32137ee2c4c41fa649abfb9582e1f33a9e13363b
Changelog: CHANGELOG.md

v2.4.1 – February 17th 2023

Changelog

[BUGFIX] Fix issue where authentication caches were not sized correctly resulting in poor performance

Upstream Grafana Mimir details

Version: 2.4.0
Hash: 32137ee2c4c41fa649abfb9582e1f33a9e13363b
Changelog: CHANGELOG.md

v2.4.0 – November 14th 2022

Changelog

[CHANGE] CarbonAPI is now being used instead of MetricTank as the default native query engine for the Graphite querier.
[CHANGE] Enterprise metrics docker image no longer requests the CAP_NET_BIND_SERVICE capability as the default HTTP port was changed from 80 to 8080.
- If you set -server.http-listen-port or -server.grpc-listen-port to a value lower than 1024, then you need to modify your configuration
  - When using Docker provide the flag --cap-add net_bind_service.
  - When using the mimir-distributed Helm chart, make sure that all the GEM components have the following additional securityContext setting in their respective values file sections:
    YAML
    securityContext: sysctls: - name: net.ipv4.ip_unprivileged_port_start value: "0" # might be set to the lowest listen port number as well
[FEATURE] Added a new flag -graphite.querier.cache-ttl to the Graphite querier to configure the TTL of cached metric names and aggregation configs.
[FEATURE] Added optional rate limiting capabilities to the Graphite querier.
- This can be configured using the following flags:
  - -graphite.querier.rate-limit-enabled
  - -graphite.querier.rate-limit-qps
  - -graphite.querier.tenant-rate-limit-qps
  - -graphite.querier.heavy-rate-limit-qps
[ENHANCEMENT] Ruler: Add <prometheus-http-prefix>/api/v1/status/buildinfo endpoint.
[ENHANCEMENT] Update all build images to use Go 1.19.2.
[BUGFIX] Fix CVE-2022-44643

Upstream Grafana Mimir details

Version: 2.4.0
Hash: 32137ee2c4c41fa649abfb9582e1f33a9e13363b
Changelog: CHANGELOG.md

v2.3.3 – April 21st 2023

Changelog

[BUGFIX] Fix CVE-2023-24538 by upgrading to Go 1.19.8.

Upstream Grafana Mimir details

Version: v2.3.1
Hash: e18bcff2f8648c4edb8446bff0256a070717684c
Changelog: CHANGELOG.md

v2.3.2 – February 17th 2023

Changelog

[BUGFIX] Fix issue where authentication caches were not sized correctly resulting in poor performance

Upstream Grafana Mimir details

Version: v2.3.1
Hash: e18bcff2f8648c4edb8446bff0256a070717684c
Changelog: CHANGELOG.md

v2.3.1 – November 14th 2022

Changelog

[BUGFIX] Fix CVE-2022-44643

Upstream Grafana Mimir details

Version: v2.3.1
Hash: e18bcff2f8648c4edb8446bff0256a070717684c
Changelog: CHANGELOG.md

v2.3.0 – September 28th 2022

Changelog

[CHANGE] Gateway: Dial timeout now defaults to 5s instead of 30s.
[CHANGE] Gateway: Dialing gRPC proxy backends during startup now blocks until the connection is established.
[FEATURE] Gateway: Add support for TSDB block upload routes.
[FEATURE] Admin client: common config block introduced in Mimir now configures Admin Client in GEM too.
[ENHANCEMENT] Gateway: the CLI flag -gateway.request.limit has been added for configuring request limiter middleware.
[ENHANCEMENT] Update all build images to use Go 1.18.6.
[ENHANCEMENT] Update all images to use Alpine 3.16.2.
[ENHANCEMENT] Gateway: Dial timeout is now configurable via -gateway.proxy.*.dial-timeout.
[BUGFIX] Gateway: Expose /distributor/ring endpoint on the distributors.
[BUGFIX] LBAC: some query limits would not be applied for requests that use LBAC.

Upstream Grafana Mimir details

Version: v2.3.1
Hash: e18bcff2f8648c4edb8446bff0256a070717684c
Changelog: CHANGELOG.md

v2.2.2 – April 24th 2023

Changelog

[BUGFIX] Fix CVE-2023-24538 by upgrading to Go 1.19.8.

Upstream Grafana Mimir details

Version: v2.2.0
Hash: 65344e2ed2cf305b50de805824026f5c5a6fadcf
Changelog: CHANGELOG.md

v2.2.0 – July 21st 2022

Changelog

[CHANGE] Ruler: /api/v1/rules* and /prometheus/rules* configuration endpoints are removed in favour of /prometheus/config/v1/rules*. Requests through the gateway are unaffected.
[CHANGE] The remote subquerier for the Graphite query proxy is no longer optional
- The following CLI flags (and their respective YAML config options) have been removed:
  - -graphite.querier.enable-remote-subquerier
  - -graphite.querier.use-remote-results
[CHANGE] The YAML config options for the datadog.api have been broken out into datadog.read_api and datadog.write_api
[ENHANCEMENT] Admin-client: added experimental support for refreshing authentication cache entries before they expire. When enabled, a cache entry is refreshed and its time to live is extended if it is retrieved and has less than or equal to -auth.cache.refresh.refresh-ttl time left to live in the cache.
- The following CLI flags (and their respective YAML config options) have been added:
  - -auth.cache.refresh.buffer
  - -auth.cache.refresh.concurrency
  - -auth.cache.refresh.enabled
  - -auth.cache.refresh.refresh-ttl
  - -auth.cache.refresh.retry-interval
[ENHANCEMENT] Gateway: Rewrite requests to deleted ruler configuration endpoints to use supported endpoints.
[BUGFIX] Docs: Make config category labels consistent across command-line help and generated documentation.

Upstream Grafana Mimir details

Version: v2.2.0
Hash: 65344e2ed2cf305b50de805824026f5c5a6fadcf
Changelog: CHANGELOG.md

v2.1.2 – April 24th 2023

Changelog

[BUGFIX] Fix CVE-2023-24538 by upgrading to Go 1.19.8.

Upstream Grafana Mimir details

Version: v2.1.0
Hash: 3cff860d16e08d14e8aaa10649053a9c0f0f15a7
Changelog: CHANGELOG.md

v2.1.0 – June 2nd 2022

Changelog

[FEATURE] Ruler: Added support for expression remote evaluation.
- The following CLI flags (and their respective YAML config options) have been added:
  - -ruler.query-frontend.address
  - -ruler.query-frontend.auth-token
  - -ruler.query-frontend.tls-enabled
  - -ruler.query-frontend.tls-ca-path
  - -ruler.query-frontend.tls-cert-path
  - -ruler.query-frontend.tls-key-path
  - -ruler.query-frontend.tls-server-name
  - -ruler.query-frontend.tls-insecure-skip-verify
[ENHANCEMENT] Self-monitoring: Emit OOM kill and page fault metrics as part of self-monitoring.
[BUGFIX] Ruler API: Ruler Limits are now enforced during rule group creation.
[BUGFIX] Authentication: Expose internal errors during authentication only in logs, not to clients.

Upstream Grafana Mimir details

Version: v2.1.0
Hash: 3cff860d16e08d14e8aaa10649053a9c0f0f15a7
Changelog: CHANGELOG.md

v2.0.1 – April 14th 2022

Changelog

[BUGFIX] Authentication: Only include active tenants when resolving the wildcard tenant (*).

Upstream Grafana Mimir details

No changes since GEM v2.0.0:

Version: v2.0.0
Hash: 9fd2da5d3dc764fc00e4396a5c0ddd12ccebb00d
Changelog: CHANGELOG.md

v2.0.0 – April 13th 2022

Changelog

[CHANGE] Admin-API: enable leader election by default
[CHANGE] Change default value of instrumentation.enabled to true
[CHANGE] Graphite Querier: The GRPC server is now registered to enable subquerier requests. This requires using the flag EnableRemoteSubquerier.
[CHANGE] Graphite Querier: The remote read query is now the default behavior. Also, the previous implementation has been removed.
[CHANGE] Admin-API: Change auth.type default from trust to enterprise
[CHANGE] Limits: The max_series_per_query limit has been removed from the Admin API and runtime configuration and is no longer enforced by GEM during queries.
[CHANGE] Graphite: The Graphite Querier and Graphite Write Proxy have been removed from single binary mode (the all target). They can still be run using the graphite-querier and graphite-write-proxy targets, respectively.
[CHANGE] Query-frontend and Graphite Querier: migrated memcached backend client to the same one used in other components (memcached config and metrics are now consistent across all services).
- The following CLI flags (and their respective YAML config options) have been added:
  - -graphite.querier.metric-name-cache.backend (set it to memcached)
  - -graphite.querier.aggregation-cache.backend (set it to memcached)
- The following CLI flags (and their respective YAML config options) have been changed:
  - -graphite.querier.metric-name-cache.memcached.hostname and -graphite.querier.metric-name-cache.memcached.service: use -graphite.querier.metric-name-cache.memcached.addresses instead
  - -graphite.querier.aggregation-cache.memcached.hostname and -graphite.querier.aggregation-cache.memcached.service: use -graphite.querier.aggregation-cache.memcached.addresses instead
- The following CLI flags (and their respective YAML config options) have been renamed:
  - -graphite.querier.metric-name-cache.background.write-back-concurrency renamed to -graphite.querier.metric-name-cache.memcached.max-async-concurrency
  - -graphite.querier.metric-name-cache.background.write-back-buffer renamed to -graphite.querier.metric-name-cache.memcached.max-async-buffer-size
  - -graphite.querier.metric-name-cache.memcached.batchsize renamed to -graphite.querier.metric-name-cache.memcached.max-get-multi-batch-size
  - -graphite.querier.metric-name-cache.memcached.parallelism renamed to -graphite.querier.metric-name-cache.memcached.max-get-multi-concurrency
  - -graphite.querier.metric-name-cache.memcached.timeout renamed to -graphite.querier.metric-name-cache.memcached.timeout
  - -graphite.querier.metric-name-cache.memcached.max-item-size renamed to -graphite.querier.metric-name-cache.memcached.max-item-size
  - -graphite.querier.metric-name-cache.memcached.max-idle-conns renamed to -graphite.querier.metric-name-cache.memcached.max-idle-connections
  - -graphite.querier.aggregation-cache.background.write-back-concurrency renamed to -graphite.querier.aggregation-cache.memcached.max-async-concurrency
  - -graphite.querier.aggregation-cache.background.write-back-buffer renamed to -graphite.querier.aggregation-cache.memcached.max-async-buffer-size
  - -graphite.querier.aggregation-cache.memcached.batchsize renamed to -graphite.querier.aggregation-cache.memcached.max-get-multi-batch-size
  - -graphite.querier.aggregation-cache.memcached.parallelism renamed to -graphite.querier.aggregation-cache.memcached.max-get-multi-concurrency
  - -graphite.querier.aggregation-cache.memcached.timeout renamed to -graphite.querier.aggregation-cache.memcached.timeout
  - -graphite.querier.aggregation-cache.memcached.max-item-size renamed to -graphite.querier.aggregation-cache.memcached.max-item-size
  - -graphite.querier.aggregation-cache.memcached.max-idle-conns renamed to -graphite.querier.aggregation-cache.memcached.max-idle-connections
- The following CLI flags (and their respective YAML config options) have been removed:
  - -graphite.querier.aggregation-cache.default-validity: new setting is hardcoded to 7 days
  - -graphite.querier.aggregation-cache.memcached.circuit-breaker-consecutive-failures: feature removed
  - -graphite.querier.aggregation-cache.memcached.circuit-breaker-interval: feature removed
  - -graphite.querier.aggregation-cache.memcached.circuit-breaker-timeout: feature removed
  - -graphite.querier.aggregation-cache.memcached.consistent-hash: new setting is always enabled
  - -graphite.querier.aggregation-cache.memcached.update-interval: new setting is hardcoded to 30s
  - -graphite.querier.metric-name-cache.default-validity and -frontend.memcached.expiration: new setting is hardcoded to 7 days
  - -graphite.querier.metric-name-cache.memcached.circuit-breaker-consecutive-failures: feature removed
  - -graphite.querier.metric-name-cache.memcached.circuit-breaker-interval: feature removed
  - -graphite.querier.metric-name-cache.memcached.circuit-breaker-timeout: feature removed
  - -graphite.querier.metric-name-cache.memcached.consistent-hash: new setting is always enabled
  - -graphite.querier.metric-name-cache.memcached.update-interval: new setting is hardcoded to 30s
- The following metrics have been changed:
  - cortex_cache_dropped_background_writes_total{name} changed to thanos_memcached_operation_skipped_total{name, operation, reason}
  - cortex_cache_value_size_bytes{name, method} changed to thanos_memcached_operation_data_size_bytes{name}
  - cortex_cache_request_duration_seconds{name, method, status_code} changed to thanos_memcached_operation_duration_seconds{name, operation}
  - cortex_cache_fetched_keys{name} changed to thanos_cache_memcached_requests_total{name}
  - cortex_cache_hits{name} changed to thanos_cache_memcached_hits_total{name}
  - cortex_memcache_request_duration_seconds{name, method, status_code} changed to thanos_memcached_operation_duration_seconds{name, operation}
  - cortex_memcache_client_servers{name} changed to thanos_memcached_dns_provider_results{name, addr}
  - cortex_memcache_client_set_skip_total{name} changed to thanos_memcached_operation_skipped_total{name, operation, reason}
  - cortex_dns_lookups_total changed to thanos_memcached_dns_lookups_total
  - For all metrics the value of the “name” label has changed from frontend.memcached to frontend-cache.
  - Above mentioned metrics are now also available with name=metric-name and name=aggregations for caches used by Graphite Querier.
- The following metrics have been removed:
  - cortex_cache_background_queue_length{name}
[CHANGE] Compactor: -compactor.compaction-strategy option removed. The only compactor that can be now used is “split and merge” compactor.
[CHANGE] Graphite: Enabled distributed subqueries by default and renamed remote_write YAML flags.
- -graphite.querier.use-remote-results and -graphite.querier.enable-remote-subquerier now default to true. This means by default subqueries will be distributed across queriers.
- remote_write YAML flags have been renamed:
  - keepalive has been renamed to keep_alive
  - maxidleconns has been renamed to max_idle_conns
  - maxconns has been renamed to max_conns
  - skiplabelvalidation has been renamed to skip_label_validation.
[FEATURE] Admin-API Deletion Markers:
- Update status field in tenants
- Add status field to access policies and tokens
- Add new Admin API v3 endpoints with soft-deletion of entities
  - /admin/api/v3/accesspolicies
  - /admin/api/v3/clusters
  - /admin/api/v3/features
  - /admin/api/v3/licenses
  - /admin/api/v3/tenants
  - /admin/api/v3/tokens
- List endpoints only return entities in active status
- Update HTTP authentication layer to only authorize requests of active entities
- Update storage cache logic to only store the object’s latest version
- Add v3 endpoints to gateway routes
[FEATURE] Graphite Write Proxy: Added -graphite.remote-write-proxy.enabled, -graphite.remote-write-proxy.write-endpoint and -graphite.write-proxy.skip-label-validation to enhance the internal series write performance of the graphite writer. It’s recommended to enable this flag on every installation as soon as possible because it will become a default configuration in future releases.
[FEATURE] Ruler: Added federated rule groups support.
- Exposed cortex_ruler_sync_unauthorized_groups metric to track the number of skipped rule groups during storage synchronizations.
[FEATURE] Divide configuration parameters into categories “basic”, “advanced”, and “experimental”. Only flags in the basic category are shown when invoking -help, whereas -help-all will include flags in all categories (basic, advanced, experimental).
[FEATURE] Datadog: Added experimental support for ingesting and querying Datadog metrics by adding a Datadog translation layer on top of GEM.
[FEATURE] Gateway: Forward requests to deprecated and removed endpoints in Mimir 2.0 (grafana/mimir#763) to their non-legacy equivalents.
[ENHANCEMENT] Update all build images to use Go 1.17.8.
[ENHANCEMENT] Admin-API: Allow the max_global_exemplars_per_user limit to be set via the Admin API.
[ENHANCEMENT] Admin-API: Enable compactor_blocks_retention_period to be set on a per-tenant basis via the Admin API.
[ENHANCEMENT] Querier: Apply Label Based Access Policy (LBAC) rules to exemplar endpoints.
[ENHANCEMENT] Federation frontend: Add bearer_token configuration for proxy targets.
[ENHANCEMENT] Self-monitoring: Add support for emitting exemplars as part of self-monitoring metrics.
[ENHANCEMENT] Federation frontend: Return richer error when downstream data source is failed.
[BUGFIX] Graphite: no need to configure Mimir’s queryable when starting only -target=graphite-querier.
[BUGFIX] Graphite: When configured with enterprise authentication, requests sent to cortex remote read api now forward authorization headers if present.
[BUGFIX] LBAC: Filter label values using LBAC policies correctly.
[BUGFIX] Authentication: HTTP 500 errors are now returned for transient errors while attempting to authenticate user requests.
[BUGFIX] Authentication: Do not cache transient errors while attempting to authenticate user requests.
[BUGFIX] Config: Enterprise configuration extensions now appear in the /config endpoint
[BUGFIX] Admin: Validate the access policy name used for token generation.
[BUGFIX] Admin: Fixed a cosmetic issue that could report an incorrect license expiration timestamp in the metric grafana_labs_license_expiry_timestamp if multiple valid licenses exist in local storage and object storage.
[BUGFIX] Gateway: All Alertmanager endpoints are correctly proxied to the alertmanager backend proxy. Previously, only the /alertmanager endpoint was proxied. Users were able to authenticate but not access the alerts UI page at /alertmanager/#/alerts.

Upstream Grafana Mimir details

Version: v2.0.0
Hash: 9fd2da5d3dc764fc00e4396a5c0ddd12ccebb00d
Changelog: CHANGELOG.md

v1.7.1 – November 14th 2022

Changelog

[BUGFIX] Fix CVE-2022-44643

v1.7.0 – January 6th 2022

Changelog

[FEATURE] Admin-API: Added support for Azure Storage
[ENHANCEMENT] Federation Frontend: Propagate requests’ bearer token when it is present.
[ENHANCEMENT] Federation Frontend: Support TLS configuration for targets.

v1.6.2 – January 6th 2022

Changelog

[BUGFIX] GEM update from v1.5.0 (or older) to v.1.6+ will not invalidate tenant limits set via API anymore.

v1.6.1 – November 18th 2021

Changelog

[BUGFIX] Admin-API: Make sure that read-path limits inherit defaults from global limits.

v1.6.0 – November 15th 2021

Changelog

[CHANGE] Admin-client: Rename the “default” auth method to “trust”.
[CHANGE] License: Deprecated flag -bootstrap.license.path has been removed. The new flag to use for specifying a license is -license.path.
[CHANGE] Ruler: endpoints for listing rules (/api/v1/rules, /api/v1/rules/{namespace}) now return HTTP status code 200 and an empty map when there are no rules instead of an HTTP 404 and plain text error message.
[CHANGE] Query-frontend: added sharded label to cortex_query_seconds_total metric.
[CHANGE] Query-frontend: changed the flag name for controlling query sharding total shards from -querier.total-shards to -frontend.query-sharding-total-shards.
[CHANGE] Flag -querier.parallelise-shardable-queries has been renamed to -query-frontend.parallelize-shardable-queries
[CHANGE] Querier/ruler: Option -querier.ingester-streaming has been removed. Querier/ruler now always use streaming method to query ingesters.
[CHANGE] Limits: Option -ingester.max-samples-per-query is now deprecated. YAML field max_samples_per_query is no longer supported. It required -querier.ingester-streaming option to be set to false, but since -querier.ingester-streaming is removed (always defaulting to true), the limit using it was removed as well.
[CHANGE] Limits: Set the default max number of inflight ingester push requests (-ingester.instance-limits.max-inflight-push-requests) to 30000 in order to prevent clusters from being overwhelmed by request volume or temporary slow-downs.
[CHANGE] Update Go version to 1.16.9.
[CHANGE] Admin-API: Require that tenant updates include the status field.
[FEATURE] Querier: Added label names cardinality endpoint <prefix>/api/v1/cardinality/label_names that is disabled by default. Can be enabled/disabled via the CLI flag -querier.cardinality-analysis-enabled or its respective YAML config option. Configurable on a per-tenant basis.
[FEATURE] Querier: Added label values cardinality endpoint <prefix>/api/v1/cardinality/label_values that is disabled by default. Can be enabled/disabled via the CLI flag -querier.cardinality-analysis-enabled or its respective YAML config option. Configurable on a per-tenant basis.
[FEATURE] Compactor: added support for a new compaction strategy -compactor.compaction-strategy=split-and-merge. When the split-and-merge compactor is used, source blocks for a given tenant are grouped into -compactor.split-groups number of groups. Each group of blocks is then compacted separately, and is split into -compactor.split-and-merge-shards shards (configurable on a per-tenant basis). Compaction of each tenant shards can be horizontally scaled. Number of compactors that work on jobs for single tenant can be limited by using -compactor.compactor-tenant-shard-size parameter, or per-tenant compactor_tenant_shard_size override.
[FEATURE] Query Frontend: Updated experimental querysharding for the blocks storage. You can now enabled querysharding for blocks storage (-store.engine=blocks) by setting -query-frontend.parallelize-shardable-queries to true. The following additional config and exported metrics have been added.
- New config options:
  - -frontend.query-sharding-total-shards: The amount of shards to use when doing parallelisation via query sharding.
  - -frontend.query-sharding-max-sharded-queries: The max number of sharded queries that can be run for a given received query. 0 to disable limit.
  - -blocks-storage.bucket-store.series-hash-cache-max-size-bytes: Max size - in bytes - of the in-memory series hash cache in the store-gateway.
  - -blocks-storage.tsdb.series-hash-cache-max-size-bytes: Max size - in bytes - of the in-memory series hash cache in the ingester.
- New exported metrics:
  - cortex_bucket_store_series_hash_cache_requests_total
  - cortex_bucket_store_series_hash_cache_hits_total
  - cortex_frontend_query_sharding_rewrites_succeeded_total
  - cortex_frontend_sharded_queries_per_query
- Renamed metrics:
  - cortex_frontend_mapped_asts_total to cortex_frontend_query_sharding_rewrites_attempted_total
- Modified metrics:
  - added sharded label to cortex_query_seconds_total
- When query sharding is enabled, the following querier config must be set on query-frontend too:
  - -querier.max-concurrent
  - -querier.timeout
  - -querier.max-samples
  - -querier.at-modifier-enabled
  - -querier.default-evaluation-interval
  - -querier.active-query-tracker-dir
  - -querier.lookback-delta
- Sharding can be dynamically controlled per request using the Sharding-Control: 64 header. (0 to disable)
- Sharding can be dynamically controlled per tenant using the limit query_sharding_total_shards. (0 to disable)
- Added sharded_queries count to the “query stats” log.
- Number of shards is adjusted to be compatible with number of compactor shards used by split-and-merge compactor. Querier can use this to avoid querying blocks that cannot have series in given query shard. This only works when using split-and-merge compactor.
[FEATURE] Graphite: Added -graphite.querier.remote-read-enabled and -graphite.querier.query-address to enhance the internal query performance of the graphite querier. It’s recommended to enable this flag on every installation as soon as possible because it will become a default configuration in future releases.
[FEATURE] Ingester: Enable snapshotting of in-memory TSDB on disk during shutdown via -blocks-storage.tsdb.memory-snapshot-on-shutdown.
[FEATURE] Query-Frontend: Added -query-frontend.cache-unaligned-requests option to cache responses for requests that do not have step-aligned start and end times. This can improve speed of repeated queries, but can also pollute cache with results that are never reused.
[ENHANCEMENT] Admin-client: Make the cluster_name configuration optional.
[ENHANCEMENT] Admin-API: Add new Admin API v2 endpoints that replace the term ‘instance’ used in version v1 with the term ’tenant’
- /admin/api/v2/accesspolicies
- /admin/api/v2/clusters
- /admin/api/v2/features
- /admin/api/v2/licenses
- /admin/api/v2/tenants
- /admin/api/v2/tokens
[ENHANCEMENT] LBAC: Optimize filtering when using single selector in LBAC policy by passing matchers to downstream querier.
[ENHANCEMENT] Distributor: reduce latency when HA-Tracking by doing KVStore updates in the background.
[ENHANCEMENT] Compactor: when sharding is enabled, skip already planned compaction jobs if the tenant doesn’t belong to the compactor instance anymore.
[ENHANCEMENT] Compactor: Blocks cleaner will ignore users that it no longer “owns” when sharding is enabled, and user ownership has changed since last scan.
[ENHANCEMENT] Query federation: improve performance in MergeQueryable by memoizing labels.
[ENHANCEMENT] Querier / store-gateway: optimized regex matchers.
[ENHANCEMENT] Query-frontend: added cortex_query_frontend_non_step_aligned_queries_total to track the total number of range queries with start/end not aligned to step.
[ENHANCEMENT] Compactor: added -compactor.compaction-jobs-order support to configure which compaction jobs should run first for a given tenant (in case there are multiple ones). Supported values are: smallest-range-oldest-blocks-first (default), newest-blocks-first (not supported by default compaction strategy).
[ENHANCEMENT] Add option (-querier.label-values-max-cardinality-label-names-per-request) to configure the maximum number of label names allowed to be queried in a single <prefix>/api/v1/cardinality/label_values API call.
[ENHANCEMENT] Make distributor inflight push requests count include background calls to ingester.
[ENHANCEMENT] Store-gateway: added an in-memory LRU cache for chunks attributes. Can be enabled setting -blocks-storage.bucket-store.chunks-cache.attributes-in-memory-max-items=X where X is the max number of items to keep in the in-memory cache. The following new metrics are exposed:
- cortex_cache_memory_requests_total
- cortex_cache_memory_hits_total
- cortex_cache_memory_items_count
[ENHANCEMENT] Store-gateway: log index cache requests to tracing spans.
[ENHANCEMENT] Ingester: reduce CPU and memory utilization if remote write requests contains a large amount of “out of bounds” samples.
[ENHANCEMENT] Ingester: reduce CPU and memory utilization when querying chunks from ingesters.
[ENHANCEMENT] Querier: when fetching data for specific query-shard, we can ignore some blocks based on compactor-shard ID, since sharding of series by query sharding and compactor is the same. Added metrics:
- cortex_querier_blocks_found_total
- cortex_querier_blocks_queried_total
- cortex_querier_blocks_with_compactor_shard_but_incompatible_query_shard_total
[ENHANCEMENT] Querier&Ruler: reduce cpu usage, latency and peak memory consumption.
[ENHANCEMENT] Overrides Exporter: Add max_fetched_chunks_per_query limit to the default and per-tenant limits exported as metrics.
[BUGFIX] License: Fixed initialization of AWS subscription manager so it creates a cluster object if not present when running GEM as AWS Marketplace product.
[BUGFIX] Admin-API: Change the way per-instance limits are stored to avoid breaking changes between versions.
[BUGFIX] Self-monitoring: Ensure system rules adhere to the sharding configuration of the rulers.
[BUGFIX] Graphite: fixed invalid label error when querying metrics with dashes in the tags.
[BUGFIX] Authentication: Fix caching behavior to ensure tokens are eventually removed from the cache.
[BUGFIX] Authentication: Enforce that instances must exist even when using wildcard access policies.
[BUGFIX] Admin-API: Expose metrics cortex_admin_api_clients and cortex_admin_client_is_leader for leader election correctly.
[BUGFIX] Limits: Fix the way cortex_limits_admin_store_last_update_timestamp_seconds is set to emit a correct UNIX timestamp.
[BUGFIX] Alertmanager: don’t replace user configurations with blank fallback configurations (when enabled), particularly during scaling up/down instances when sharding is enabled.
[BUGFIX] Query-frontend: Ensure query_range requests handled by the query-frontend return JSON formatted errors.
[BUGFIX] Query-frontend: don’t reuse cached results for queries that are not step-aligned.
[BUGFIX] Querier: fixed UserStats endpoint. When zone-aware replication is enabled, MaxUnavailableZones param is used instead of MaxErrors, so setting MaxErrors = 0 doesn’t make the Querier wait for all Ingesters responses.

v1.5.1 – September 21st 2021

Changelog

[ENHANCEMENT] Add ADFS compatibility to our OIDC auth.
[BUGFIX] Ruler: Use predictable names for Ruler WALs ensuring they are used after crashes and cleaned up.

v1.5.0 – August 24th 2021

Changelog

[CHANGE] Alertmanager: allowed to configure the experimental receivers firewall on a per-tenant basis. The following CLI flags (and their respective YAML config options) have been changed and moved to the limits config section:
- -alertmanager.receivers-firewall.block.cidr-networks renamed to -alertmanager.receivers-firewall-block-cidr-networks
- -alertmanager.receivers-firewall.block.private-addresses renamed to -alertmanager.receivers-firewall-block-private-addresses
[CHANGE] Memberlist: Expose default configuration values to the command line options. Note that setting these explicitly to zero will no longer cause the default to be used. If the default is desired, then do set the option. The following are affected:
- -memberlist.stream-timeout
- -memberlist.retransmit-factor
- -memberlist.pull-push-interval
- -memberlist.gossip-interval
- -memberlist.gossip-nodes
- -memberlist.gossip-to-dead-nodes-time
- -memberlist.dead-node-reclaim-time
[CHANGE] Authentication: Access Policy names passed via a JWT token in the OIDC auth flow will be downcased before being matched against Access Policies in GEM. This improves interoperability between GEM and other systems since GEM only allows lowercase characters in Access Policy names
[CHANGE] Change default value of -server.grpc.keepalive.min-time-between-pings from 5m to 10s and -server.grpc.keepalive.ping-without-stream-allowed to true.
[CHANGE] Changed -alertmanager.storage.type default value from configdb to local.
[CHANGE] Changed -ruler.storage.type default value from configdb to local.
[CHANGE] Cortex chunks storage has been deprecated and it’s now in maintenance mode: all Cortex users are encouraged to migrate to the blocks storage. No new features will be added to the chunks storage. The default Cortex configuration still runs the chunks engine; please check out the blocks storage doc on how to configure Cortex to run with the blocks storage.
[CHANGE] Dependency: update go-redis from v8.2.3 to v8.9.0.
[CHANGE] Deprecated the bootstrap target in favor of the tokengen target.
[CHANGE] Enable strict JSON unmarshal for pkg/util/validation.Limits struct. The custom UnmarshalJSON() will now fail if the input has unknown fields.
[CHANGE] Graphite: proxy no longer generates generic metrics metadata. This helps to reduce ingestion rate as counted by Cortex and used for limits.
[CHANGE] Ingester: Change default value of -ingester.active-series-metrics-enabled to true. This incurs a small increase in memory usage, between 1.2% and 1.6% as measured on ingesters with 1.3M active series.
[CHANGE] License: Flag -bootstrap.license.path has been deprecated in favor of -license.path.
[CHANGE] Memberlist: the memberlist_kv_store_value_bytes has been removed due to values no longer being stored in-memory as encoded bytes.
[CHANGE] Querier / ruler: Change -querier.max-fetched-chunks-per-query configuration to limit to maximum number of chunks that can be fetched in a single query. The number of chunks fetched by ingesters AND long-term storare combined should not exceed the value configured on -querier.max-fetched-chunks-per-query.
[CHANGE] Querier / ruler: deprecated -store.query-chunk-limit CLI flag (and its respective YAML config option max_chunks_per_query) in favour of -querier.max-fetched-chunks-per-query (and its respective YAML config option max_fetched_chunks_per_query). The new limit specifies the maximum number of chunks that can be fetched in a single query from ingesters and long-term storage: the total number of actual fetched chunks could be 2x the limit, being independently applied when querying ingesters and long-term storage.
[CHANGE] Query-frontend: Enable query stats by default, they can still be disabled with -frontend.query-stats-enabled=false.
[CHANGE] Removed configdb support from Ruler and Alertmanager backend storages.
[CHANGE] Removed log_messages_total metric.
[CHANGE] Removed query sharding for the chunks storage. Query sharding is now only supported for blocks storage.
[CHANGE] Renamed metric deprecated_flags_inuse_total as deprecated_flags_used_total.
[CHANGE] Renamed metric experimental_features_in_use_total as experimental_features_used_total.
[CHANGE] Some files and directories on local disk now have stricter permissions, and are only readable by owner, but not group or others.
[CHANGE] The example Kubernetes manifests (stored at k8s/) have been removed due to a lack of proper support and maintenance.
[CHANGE] Update Go version to 1.16.6.
[FEATURE] Added flag -debug.block-profile-rate to enable goroutine blocking events profiling.
[FEATURE] Alertmanager: Added -alertmanager.max-config-size-bytes limit to control size of configuration files that Cortex users can upload to Alertmanager via API. This limit is configurable per-tenant.
[FEATURE] Alertmanager: Added -alertmanager.max-templates-count and -alertmanager.max-template-size-bytes options to control number and size of templates uploaded to Alertmanager via API. These limits are configurable per-tenant.
[FEATURE] Alertmanager: Added rate-limits to notifiers. Rate limits used by all integrations can be configured using -alertmanager.notification-rate-limit, while per-integration rate limits can be specified via -alertmanager.notification-rate-limit-per-integration parameter. Both shared and per-integration limits can be overwritten using overrides mechanism. These limits are applied on individual (per-tenant) alertmanagers. Rate-limited notifications are failed notifications. It is possible to monitor rate-limited notifications via new cortex_alertmanager_notification_rate_limited_total metric.
[FEATURE] Alertmanager: support negative matchers, time-based muting - upstream release notes.
[FEATURE] Allow for reporting CPU time usage to AWS Marketplace metering service in case GEM is running as AWS Marketplace container product.
[FEATURE] Collect and store CPU time usage reports in Admin store, which can later be used to submit to metering services, such as the AWS Marketplace API
[FEATURE] Querier/Ruler: Added new -querier.max-fetched-chunk-bytes-per-query flag. When Cortex is running with blocks storage, the max chunk bytes limit is enforced in the querier and ruler and limits the size of all aggregated chunks returned from ingesters and storage as bytes for a query.
[FEATURE] Querier: Added new -querier.max-fetched-series-per-query flag. When Cortex is running with blocks storage, the max series per query limit is enforced in the querier and applies to unique series received from ingesters and store-gateway (long-term storage).
[FEATURE] Query Frontend: Add cortex_query_fetched_chunks_total per-user counter to expose the number of chunks fetched as part of queries. This metric can be enabled with the -frontend.query-stats-enabled flag (or its respective YAML config option query_stats_enabled).
[FEATURE] Query Frontend: Add cortex_query_fetched_series_total and cortex_query_fetched_chunks_bytes_total per-user counters to expose the number of series and bytes fetched as part of queries. These metrics can be enabled with the -frontend.query-stats-enabled flag (or its respective YAML config option query_stats_enabled).
[FEATURE] Query Frontend: Add experimental querysharding for the block storage. You can now enabled querysharding for block storage (-store.engine) by setting -querier.parallelise-shardable-queries to true.
[FEATURE] Ruler Storage: S3 header extensions were added to the new ruler storage S3 config block.
[FEATURE] Ruler: Add new -ruler.query-stats-enabled which when enabled will report the cortex_ruler_query_seconds_total as a per-user metric that tracks the sum of the wall time of executing queries in the ruler in seconds.
[FEATURE] When running GEM as AWS Marketplace container product then the Go runtime variable GOMAXPROCS is automatically set to match the container CPU quota, in case Kubernetes CPU resource limits are set.
[FEATURE] Alertmanager: The experimental sharding feature is now considered complete. Detailed information about the configuration options can be found here for alertmanager and here for the alertmanager storage. To use the feature:
- Ensure that a remote storage backend is configured for Alertmanager to store state using -alertmanager-storage.backend, and flags related to the backend. Note that the local and configdb storage backends are not supported.
- Ensure that a ring store is configured using -alertmanager.sharding-ring.store, and set the flags relevant to the chosen store type.
- Enable the feature using -alertmanager.sharding-enabled.
- Note the prior addition of a new configuration option -alertmanager.persist-interval. This sets the interval between persisting the current alertmanager state (notification log and silences) to object storage. See the configuration file reference for more information.
[ENHANCEMENT] Add Cassandra support.
[ENHANCEMENT] Add timeout for waiting on compactor to become ACTIVE in the ring.
[ENHANCEMENT] Added tenant_ids tag to tracing spans
[ENHANCEMENT] Added option -distributor.excluded-zones to exclude ingesters running in specific zones both on write and read path.
[ENHANCEMENT] Added zone-awareness support to alertmanager for use when sharding is enabled. When zone-awareness is enabled, alerts will be replicated across availability zones.
[ENHANCEMENT] Admin-API: Add a new endpoint for returning product and feature information at /admin/api/v1/features
[ENHANCEMENT] Admin-API: Allow admin-api to operate for read-only request when no license is present.
[ENHANCEMENT] Alertmanager: Added -alertmanager.max-alerts-count and -alertmanager.max-alerts-size-bytes to control max number of alerts and total size of alerts that a single user can have in Alertmanager’s memory. Adding more alerts will fail with a log message and incrementing cortex_alertmanager_alerts_insert_limited_total metric (per-user). These limits can be overrided by using per-tenant overrides. Current values are tracked in cortex_alertmanager_alerts_limiter_current_alerts and cortex_alertmanager_alerts_limiter_current_alerts_size_bytes metrics.
[ENHANCEMENT] Alertmanager: Added -alertmanager.max-dispatcher-aggregation-groups option to control max number of active dispatcher groups in Alertmanager (per tenant, also overrideable). When the limit is reached, Dispatcher produces log message and increases cortex_alertmanager_dispatcher_aggregation_group_limit_reached_total metric.
[ENHANCEMENT] Alertmanager: Cleanup persisted state objects from remote storage when a tenant configuration is deleted.
[ENHANCEMENT] Authentiation: OIDC integration now supports a JWT with multiple roles. When present, these roles will be rolled up into a “virtual” access policy that provides metrics read access to the union of instances contained in those roles.
[ENHANCEMENT] Blocks storage: support ingesting exemplars and querying of exemplars. Enabled by setting new CLI flag -blocks-storage.tsdb.max-exemplars=<n> or config option blocks_storage.tsdb.max_exemplars to positive value.
[ENHANCEMENT] Distributor: Added distributors ring status section in the admin page.
[ENHANCEMENT] Etcd: Added username and password to etcd config.
[ENHANCEMENT] Expose CPU quota information (number of cores, cgroup quota) as Prometheus metrics.
[ENHANCEMENT] Expose error counters and timestamps of CPU usage reporting as Prometheus metrics when AWS Marketplace meterting is enabled.
[ENHANCEMENT] Expose value of GOMAXPROCS as Prometheus metrics.
[ENHANCEMENT] Facilitate running GEM Docker image as a non-root user. Usage is documented in the Kubernetes deployment documentation.
[ENHANCEMENT] Ingester: Added option -ingester.ignore-series-limit-for-metric-names with comma-separated list of metric names that will be ignored in max series per metric limit.
[ENHANCEMENT] Ingester: added option -ingester.readiness-check-ring-health to disable the ring health check in the readiness endpoint.
[ENHANCEMENT] License: Added flag -license.type that is used to specify that the APP is running through AWS Marketplace.
[ENHANCEMENT] License: Implemented /licenses endpoint that responds with static list of licenses that replaces default implementation if the APP is running through AWS Marketplace.
[ENHANCEMENT] License: Implemented logic to check if AWS Marketplace subscription is active instead of checking license file if the APP is running through AWS Marketplace.
[ENHANCEMENT] Memberlist: expose configuration of memberlist packet compression via -memberlist.compression=enabled.
[ENHANCEMENT] Memberlist: optimized receive path for processing ring state updates, to help reduce CPU utilization in large clusters.
[ENHANCEMENT] Node-API: Added TSDB block metadata to the exportable debug archive.
[ENHANCEMENT] Node-API: Register a new endpoint for fetching a compressed debug file containing config and version information at /node/api/v1/debug-export.
[ENHANCEMENT] Node-API: Register a new endpoint for fetching version information about the nodes at /node/api/v1/version.
[ENHANCEMENT] Querier now can use the LabelNames call with matchers, if matchers are provided in the /labels API call, instead of using the more expensive MetricsForLabelMatchers call as before. This can be enabled by enabling the -querier.query-label-names-with-matchers-enabled flag once the ingesters are updated to this version. In the future this is expected to become the default behavior.
[ENHANCEMENT] Reduce memory used by streaming queries, particularly in ruler.
[ENHANCEMENT] Ring, query-frontend: Avoid using automatic private IPs (APIPA) when discovering IP address from the interface during the registration of the instance in the ring, or by query-frontend when used with query-scheduler. APIPA still used as last resort with logging indicating usage.
[ENHANCEMENT] Ruler: added rule_group label to metrics cortex_prometheus_rule_group_iterations_total and cortex_prometheus_rule_group_iterations_missed_total.
[ENHANCEMENT] Scanner: add support for DynamoDB (v9 schema only).
[ENHANCEMENT] Scanner: retry failed uploads.
[ENHANCEMENT] Storage: Added the ability to disable Open Census within GCS client (e.g -gcs.enable-opencensus=false).
[ENHANCEMENT] Store-gateway: added -store-gateway.sharding-ring.wait-stability-min-duration and -store-gateway.sharding-ring.wait-stability-max-duration support to store-gateway, to wait for ring stability at startup.
[ENHANCEMENT] Wildcard Datasource: Wildcard “*” datasources are now supported in datasource urls for GEM. This allows an action to have access to all instances in all access policies associated with the provided token. If that set of instances includes a wildcard “*”, then access is expanded to all instances in the cluster.
[ENHANCEMENT] Added instrumentation to Redis client, with the following metrics:
- cortex_rediscache_request_duration_seconds
[ENHANCEMENT] Include additional limits in the per-tenant override exporter. The following limits have been added to the cortex_overrides metric:
- max_fetched_series_per_query
- max_fetched_chunk_bytes_per_query
- ruler_max_rules_per_rule_group
- ruler_max_rule_groups_per_tenant
[ENHANCEMENT] License Manager: Added functionality to regularly check the local license file and sync it to the license storage backend.
- Added metrics grafana_labs_license_syncs_total and grafana_labs_license_sync_failures_total.
[ENHANCEMENT] Ring: allow experimental configuration of disabling of heartbeat timeouts by setting the relevant configuration value to zero. Applies to the following:
- -distributor.ring.heartbeat-timeout
- -ring.heartbeat-timeout
- -ruler.ring.heartbeat-timeout
- -alertmanager.sharding-ring.heartbeat-timeout
- -compactor.ring.heartbeat-timeout
- -store-gateway.sharding-ring.heartbeat-timeout
[ENHANCEMENT] Ring: allow heartbeats to be explicitly disabled by setting the interval to zero. This is considered experimental. This applies to the following configuration options:
- -distributor.ring.heartbeat-period
- -ingester.heartbeat-period
- -ruler.ring.heartbeat-period
- -alertmanager.sharding-ring.heartbeat-period
- -compactor.ring.heartbeat-period
- -store-gateway.sharding-ring.heartbeat-period
[ENHANCEMENT] Alertmanager: introduced new metrics to monitor operation when using -alertmanager.sharding-enabled:
- cortex_alertmanager_state_fetch_replica_state_total
- cortex_alertmanager_state_fetch_replica_state_failed_total
- cortex_alertmanager_state_initial_sync_total
- cortex_alertmanager_state_initial_sync_completed_total
- cortex_alertmanager_state_initial_sync_duration_seconds
- cortex_alertmanager_state_persist_total
- cortex_alertmanager_state_persist_failed_total
[ENHANCEMENT] Memberlist: introduced new metrics to aid troubleshooting tombstone convergence:
- memberlist_client_kv_store_value_tombstones
- memberlist_client_kv_store_value_tombstones_removed_total
- memberlist_client_messages_to_broadcast_dropped_total
[ENHANCEMENT] Ruler: added new metrics for tracking total number of queries and push requests sent to ingester, as well as failed queries and push requests. Failures are only counted for internal errors, but not user-errors like limits or invalid query. This is in contrast to existing cortex_prometheus_rule_evaluation_failures_total, which is incremented also when query or samples appending fails due to user-errors.
- cortex_ruler_write_requests_total
- cortex_ruler_write_requests_failed_total
- cortex_ruler_queries_total
- cortex_ruler_queries_failed_total
[BUGFIX] Graphite: Fix handling of consolidateBy and make aggregation method part of aggregation cache key.
[BUGFIX] Alertmanager: fix Alertmanager status page if clustering via gossip is disabled or sharding is enabled.
[BUGFIX] Authentication: fix handling of missing instances, or when instance has no matching access policy, by properly returning a 401 instead of crashing.
[BUGFIX] Compactor: fixed panic while collecting Prometheus metrics.
[BUGFIX] Graphite: Apply the max-points-per-req-hard limit correctly.
[BUGFIX] Graphite: Fix race in index.json API endpoint which lead to incomplete results.
[BUGFIX] HA Tracker: when cleaning up obsolete elected replicas from KV store, tracker didn’t update number of cluster per user correctly.
[BUGFIX] Ingester: fix issue where runtime limits erroneously override default limits.
[BUGFIX] Ingester: fixed infrequent panic caused by a race condition between TSDB mmap-ed head chunks truncation and queries.
[BUGFIX] Ingester: fixed ingester stuck on start up (LEAVING ring state) when -ingester.heartbeat-period=0 and -ingester.unregister-on-shutdown=false.
[BUGFIX] Invalidate cached authentication tokens when they are deleted from object storage.
[BUGFIX] Make multiple Get requests instead of MGet on Redis Cluster.
[BUGFIX] Memberlist: fix to setting the default configuration value for -memberlist.retransmit-factor when not provided. This should improve propagation delay of the ring state (including, but not limited to, tombstones). Note that if the configuration is already explicitly given, this fix has no effect.
[BUGFIX] Purger: fix Invalid null value in condition for column range caused by nil value in range for WriteBatch query.
[BUGFIX] Querier: Fix issue where samples in a chunk might get skipped by batch iterator.
[BUGFIX] Querier: fix queries failing with “at least 1 healthy replica required, could only find 0” error right after scaling up store-gateways until they’re ACTIVE in the ring.
[BUGFIX] Query-frontend: Fix 401s during query_range requests when enterprise authentication is used. The workaround involving disabling enterprise authentication on the querier can now be removed.
[BUGFIX] Ruler: Fix bug in rule forwarding with remote write which could cause filling up the disk because it was not truncated.
- New flags called -ruler.remote-write.wal-truncate-frequency, -ruler.remote-write.min-wal-time and -ruler.remote-write.max-wal-time have been added.
[BUGFIX] Ruler: Honor the evaluation delay for the ALERTS and ALERTS_FOR_STATE series.
[BUGFIX] Ruler: fix /ruler/rule_groups endpoint doesn’t work when used with object store.
[BUGFIX] Ruler: fix startup in single-binary mode when the new ruler_storage is used.
[BUGFIX] Ruler: fixed counting of PromQL evaluation errors as user-errors when updating cortex_ruler_queries_failed_total.
[BUGFIX] Store-gateway: when blocks sharding is enabled, do not load all blocks in each store-gateway in case of a cold startup, but load only blocks owned by the store-gateway replica.
[BUGFIX] Upgrade Prometheus. TSDB now waits for pending readers before truncating Head block, fixing the chunk not found error and preventing wrong query results.

v1.4.2 – July 21st 2021

Changelog

[SECURITY] Prevent path traversal attack from users able to control the HTTP header X-Scope-OrgID. (CVE-2021-36157)
- Users only have control of the HTTP header when GEM is configured with flags -auth.type=default and -tenant-federation.enabled=false
[SECURITY] Update build image to use Go 1.16.6. (CVE-2021-34558) #1874
[BUGFIX] Ruler: Register remote write metrics correctly. #1814

Upstream Cortex details

Cortex Hash: 2210ebb7052a9efb99d0e4dc53043a3f5d806d00

v1.4.1 – June 29th 2021

Changelog

[BUGFIX] Update the GEM build image to use Alpine 3.14, python 3.9 and gsutil 4.52.

Upstream Cortex details

Cortex Hash: 98dd0c4d69576fdfaf2b9bfd7aa475e835e11429

v1.4.0 – June 28th 2021

Changelog

[CHANGE] Breaking: Verify token issuer when using OIDC authentication. Includes a breaking change for users of OIDC authentication. #1571
- Before this change the configuration of OIDC authentication required the OIDC provider’s jwks_uri to be set in the configuration flag auth.admin.oidc.url. This flag has been deprecated.
- A new flag named auth.admin.oidc.issuer-url has been added, and it must be set to the URL of the OIDC provider. For example: -auth.admin.oidc.issuer-url=https://accounts.google.com Note: This is not simply a rename of the old flag; you also need to update the value. The defined issuer is required to provide the OIDC discovery endpoint (/.well-known/openid-configuration)
[CHANGE] Breaking: The GEM/GEL Ruler can now be accessed by access policies with rules read/write permissions, which are no longer metrics/logs specific #1366 & #1403
- Before this change, there were metric rule specific permissions metrics:rules:read and metrics:rules:write.
- The data representation for this change in object storage is backwards compatible, so no change is needed for existing access policies using the new rules.
- The JSON representation for these rules is not backwards compatible, and so any JSON interactions with the API that specified the strings metrics:rules:read or metrics:rules:write must be updated to the strings rules:read and rules:write respectively.
- This breaking change applies to the GEM Plugin as well, so please update to version v3.0.X.
[CHANGE] Remove enterprise_features config block entirely. #1453
[CHANGE] Alertmanager: deprecated -alertmanager.storage.* CLI flags (and their respective YAML config options) in favour of -alertmanager-storage.*. This change doesn’t apply to alertmanager.storage.path and alertmanager.storage.retention.
[CHANGE] Blocks storage: removed the config option -blocks-storage.bucket-store.index-cache.postings-compression-enabled, which was deprecated. Postings compression is always enabled.
[CHANGE] GEM now fails fast on startup if it is unable to connect to the ring backend.
[CHANGE] Querier / ruler: deprecated -store.query-chunk-limit CLI flag (and its respective YAML config option max_chunks_per_query) in favor of -querier.max-fetched-chunks-per-query (and its respective YAML configuration option max_fetched_chunks_per_query). The new limit specifies the maximum number of chunks that can be fetched in a single query from ingesters and long-term storage: the total number of chunks that are actually fetched, in the worst case, can be twice the limit because the limit is applied to ingesters as well as long-term storage.
[CHANGE] Query frontend: removed the configuration option -querier.compress-http-responses, which was deprecated. Instead, use-api.response-compression-enabled.
[CHANGE] Runtime-config / overrides: removed the config options -limits.per-user-override-config (use -runtime-config.file) and -limits.per-user-override-period (use -runtime-config.reload-period), both deprecated.
[FEATURE] Add embedded recording rules to the Enterprise Ruler to support building dashboards and alerts from internal metrics written directly to GEM itself via a distributor. #1459
- To enable or disable the feature, use the -instrumentation.enabled flag or associated enabled setting on the instrumentation configuration block. The feature is disabled by default.
[FEATURE] Add the ability to write internal metrics directly to GEM itself via a distributor. #1281
- To configure, or enabled or disabled the feature, user the -instrumentation.enabled flag and associated other flags or the instrumentation configuration block:
  YAML
```
instrumentation:
  enabled: false
  flush_period: 15s
  write_timeout: 10s
  distributor_client:
    address: dns:///:9095
    connect_timeout: 5s
    tls_enabled: false
    tls_cert_path:
    tls_key_path:
    tls_ca_path:
    tls_server_name:
    tls_insecure_skip_verify:
```
  The feature is disabled by default.
[FEATURE] Self-monitoring: expose filesystem usage metrics to source the disk utilization panel in the self-monitoring resource dashboards #1618
[FEATURE] Add an experimental GEM component federation-frontend, which can be used to federate queries between multiple GEM clusters. #1274
[FEATURE] Querier: Added new -querier.max-fetched-series-per-query flag. When GEM is running with blocks storage, the max series per query limit is enforced in the querier and applies to unique series received from ingesters and store-gateway (long-term storage).
[FEATURE] Querier/Ruler: Added new -querier.max-fetched-chunk-bytes-per-query flag. When GEM is running with blocks storage, the max chunk bytes limit is enforced in the querier and ruler and limits the size of all aggregated chunks returned from ingesters and storage as bytes for a query.
[ENHANCEMENT] Introduce configuration parameter to limit how many points we process per query. #1292
[ENHANCEMENT] Adding API endpoints via which a user can post / get their storage schemas / aggregations. #1389
[ENHANCEMENT] Admin-API: Listing mutable resources now includes a comma separated list of versions for those resources in the ETag header #1419
[ENHANCEMENT] Admin-API: Updating a mutable resources now allows a wildcard value ("*") to be passed as the If-Match header, which allows the updating of any current version #1449
[ENHANCEMENT] The /config HTTP endpoint now also returns GEM specific options alongside regular Cortex configuration. #1380
[BUGFIX] Fix LBAC regular expression matchers #1305
[BUGFIX] Validate all fields of JWT tokens used for auth, except the issuer. #1500
[BUGFIX] Ruler: ensure the S3 rule storage flags properly maps to the upstream flags. #1460
[BUGFIX] Admin-API: rejecting update requests when access policies have empty scopes or realms. #1447
[BUGFIX] Updated licenses are now persisted to object storage, fixing the responses from the license API which would show old license information. #1568
[BUGFIX] Validate all fields of JWT tokens used for auth, except the issuer. #1500
[BUGFIX] OAuth: Don’t use default access policy when an invalid JWT claim is provided. #1635
[BUGFIX] Authentiation: Invalidate cached authentication tokens when they are deleted from object storage. #1703

Upstream Cortex details

Cortex Hash: 98dd0c4d69576fdfaf2b9bfd7aa475e835e11429
Cortex Commits

v1.3.1 – Jul 21st 2021

Changelog

[SECURITY] Prevent path traversal attack from users able to control the HTTP header X-Scope-OrgID. (CVE-2021-36157)
- Users only have control of the HTTP header when GEM is configured with flags -auth.type=default and -tenant-federation.enabled=false
[SECURITY] Update build image to use Go 1.16.6. (CVE-2021-34558) #1874
[BUGFIX] Update the GEM build image to use Alpine 3.14, python 3.9 and gsutil 4.52. #1781
[BUGFIX] Ruler: Register remote write metrics correctly. #1814

Upstream Cortex details

Cortex Hash: 64592254fe91c86e903882947a58d572a316884d

v1.3.0 – April 26th 2021

Changelog

[SECURITY] Alertmanager: Fix a local file disclosure vulnerability when -experimental.alertmanager.enable-api is used (CVE-2021-31231):
- The HTTP Basic auth password_file can be used as an attack vector to send any file content via a webhook.
- The Alertmanager templates can be used as an attack vector to send any file content because the Alertmanager can load any text file specified in the templates list.
[CHANGE] Admin API: Concurrent requests to the same resource are no longer allowed. If two requests are issued to create, update, or delete the same resource, then the first one to achieve a lock executes and the second one returns a conflict error. This is handled per process. To enforce this behavior on multiple processes, use leader election. #1186
[CHANGE] Admin API: all errors encountered during the processing of HTTP requests are converted to GRPC errors in order to determine the correct HTTP status to return. This enforces consistency for leader election, because some requests are handled internally, and others are forwarded to other instances. #1217
[CHANGE] Admin API: all mutation operations (PUT/DELETE) now require an If-Match header to be set (an integer between "" such as "27") to verify that the correct version of the resource is being modified and prevent against race conditions. You can find the current version of a resource in the ETag header that is returned when that resource is read (via GET) or updated (via PUT).
[FEATURE] Admin API: you can set per-instance resource limits via the Admin API. This is enabled by default. #1173
- You can enable or disable this feature by using the -admin-api.limits.enabled or -admin-api.limits.refresh-period flags. Also, you can configure this feature by using the admin_api configuration block:
  YAML
```
admin_api:
  limits:
    enabled: true
    refresh_period: 1m
```
[ENHANCEMENT] Upgrade build image to use Go 1.16.3. #1294
[ENHANCEMENT] Admin client: Add cortex_admin_client_is_leader gauge metric to determine when the client considers itself the leader. #1175
[ENHANCEMENT] Admin API: update an access policy via the Admin API using a PUT request. #1139
[ENHANCEMENT] Admin API: Update an instance via the Admin API using a PUT request. #1180
[ENHANCEMENT] Gateway: Forward /multitenant_alertmanager/ring and /ruler/ring routes to the alertmanager and ruler proxy backends. #1144
[BUGFIX] Graphite: Fix aggregation cache to generate cache keys using correct input data. #963
[BUGFIX] Authentication: Fix issue where all requests would trigger a panic if authentication is enabled but no admin client is configured. A error is now printed instead. #1106

Upstream Cortex details

Cortex Hash: 2d8477c4a325ce5071676e906efcee4adb687513
Cortex Commits

v1.2.1 – April 27 2021

Changelog

[SECURITY] Alertmanager: Fix a local file disclosure vulnerability when -experimental.alertmanager.enable-api is used (CVE-2021-31231):
- The HTTP Basic auth password_file can be used as an attack vector to send any file content via a webhook.
- The Alertmanager templates can be used as an attack vector to send any file content because the Alertmanager can load any text file specified in the templates list.

v1.2.0 – March 10 2021

Changelog

[CHANGE] Gateway: Remove purger proxy configuration, which is not a supported target for blocks clusters.
[CHANGE] Auth: Override authentication flags have been renamed:
- The auth.override-admin-token flag has been changed to auth.override.token.
- The auth.override-admin-token-file flag has been changed to auth.override.token-file.

[FEATURE] Gateway: Improve the gateway target to support unique TLS configurations and write timeouts for each backend.

New fields have been added to allow for configuration:

gateway:
  proxy:
    default:
      tls:
        tls_cert_path: <string>
        tls_key_path: <string>
        tls_ca_path: <string>
        tls_insecure_skip_verify: <bool>
    distributor:
      read_timeout: <duration>
      write_timeout: <duration>
      tls:
      ...

[FEATURE] Compactor: Introduced time-sharding compaction strategy.

[ENHANCEMENT] Distributor: Wrap remote writes in distributor to sample and log them as business intelligence events.

[ENHANCEMENT] Metrics emitted for TLS certificate expiration now reflect certificates being reloaded.
[ENHANCEMENT] Remove the Graphite Auto Complete Index and use Cortex index instead.
[ENHANCEMENT] Add Graphite API endpoint /metrics/index.json.
[ENHANCEMENT] Distributor: Wrap remote writes in distributor to sample and log them as business intelligence events.
[ENHANCEMENT] Call Cortex Distributor over gRPC from Graphite Write Proxy (formerly Graphite Distributor)

[ENHANCEMENT] Admin API: Add feature to elect and admin-api leader instance to handle all mutation requests. Requests to non-leader instances are forwarded to the leader instance.

New fields have been added to allow for configuration:

admin_api:
  leader_election:
    enabled: <bool>
    ring:
      kvstore: <kv.Config>
      heartbeat_period: <duration>
      heartbeat_timeout: <duration>
      tokens_observe_period: <duration>
      instance_interface_name: <[]string>
    client_config: <grpcclient.Config>

[BUGFIX] LBAC: Fix issue where debug logs would not print the selector and instead print selector="unsupported value type".
[BUGFIX] Admin-Client: Warning logs are no longer created on resource creation.
[BUGFIX] Ruler: Fix issue where invalid remote-write URLs cause a panic.
[BUGFIX] Querier: Apply label access filters on multi tenant access policies.

Upstream Cortex details

Cortex Hash: 003eb33266ca464d7290a938a9d767c36b9a03a4
Cortex CHANGELOG

v1.1.3 – April 27 2021

Changelog

[SECURITY] Alertmanager: Fix a local file disclosure vulnerability when -experimental.alertmanager.enable-api is used (CVE-2021-31231):
- The HTTP Basic auth password_file can be used as an attack vector to send any file content via a webhook.
- The Alertmanager templates can be used as an attack vector to send any file content because the Alertmanager can load any text file specified in the templates list.

v1.1.2 – January 20 2021

Changelog

[BUGFIX] Querier: fix default value incorrectly overriding -querier.frontend-address in single-binary mode.

v1.1.1 – January 14 2021

Changelog

[BUGFIX] Ruler: Minimize gaps on rule evaluations with stale input and enabled ruler evaluation delay.

v1.1.0 – January 12 2021

Changelog

[CHANGE] Admin-API: Resources must not be both prefixed and suffixed with the __ characters. If any of your existing resources exist with this naming pattern, they must be deleted and recreated with a new name before upgrading.
[CHANGE] Graphite: Allow storage schema and storage aggregation configs to be defined per tenant.
[CHANGE] Admin-Client: Instance management client calls no longer use object storage Iter calls when retrieving the latest version of a resource.
[CHANGE] Graphite: Add API endpoints to explore the available Graphite functions.

[CHANGE] Admin: The selectors for label policies are now provided as PromQL label strings instead of typed objects.

Deprecated:

"label_policies": [
  {
    "selector": [
      {
        "name": "env",
        "value": "dev",
        "type": "EQ"
      }
    ]
  }
]

New:

"label_policies": [
  {
    "selector": "{env=\"dev\"}"
  }
]

[CHANGE] Admin: Operations with an ADMIN scope are no longer restricted to operating on clusters they have as a configured realm.

[CHANGE] Deprecate enterprise_features config section in favor of the Cortex config extension.

Deprecated:

enterprise_features:
  ruler_s3_request_headers:
    file: <string>
    poll_interval: <duration>
  ruler_remote_write:
    enabled: <bool>
    wal_dir: <string>

New:

ruler:
  storage:
    s3:
      header_map_file_path: <string>
      header_map_poll_interval: <duration>
  remote_write:
    enabled: <bool>
    wal_dir: <string>

[FEATURE] Ruler: Alerts can now be correctly forwarded to the Alertmanager with enterprise authentication enabled by setting the basic authentication username to __alertmanager__ and the password to a API token with access to every instance.
[FEATURE] Queries: LBAC enforcement has been added for queries and label value requests.
- When GEM is run using the default authentication mode, LBAC policies are specified using the X-Prom-Label-Policy HTTP header in the format: X-Prom-Label-Policy: <tenant-id>:urlEscaped(<prometheus label selector>). For example, a policy that only allows metrics with the label env equal to dev for tenant test-instance could specified with the following header: X-Prom-Label-Policy: test-instance:%7Benv=%22dev%22%7D. To specify multiple policies either set the header multiple times or set the header with a single string of multiple policies separated by an unescaped comma.

[FEATURE] Admin API: add label_policies field, which contains an array of label matchers to the access policy realm JSON.

{
  "realms": [
    {
      "instance": "<string>",
      "cluster": "<string>",
      "label_policies": [
        {
          "selector": [
            {
              "type": "<enum: EQ | NEQ | RE | NRE>",
              "name": "<string>",
              "value": "<string>"
            }
          ]
        }
      ]
    }
  ]
}

[FEATURE] Admin: Add target tokengen to generate tokens for the default or a custom access policy.
[FEATURE] Admin: Added a default __admin__ access policy that has an ADMIN scope. This policy can be disabled adding the following to the GEM configuration file.
YAML
```
admin_client:
  disable_default_admin_policy: true
```
[FEATURE] Querier: Queries can be federated across multiple tenants. The tenants IDs involved need to be specified separated by a | character in the X-Scope-OrgID request header.
[FEATURE] Add gateway target that can be configured to proxy requests to microservices and can be used to load balance remote_write requests to the distributors.
[ENHANCEMENT] AdminAPI: Add scope for read only admin access, admin:read.
[ENHANCEMENT] AdminAPI: Add separate set of scopes for alerts and rules.
- alerts:read
- alerts:write
- logs:rules:read
- logs:rules:write
- metrics:rules:read
- metrics:rules:write
[ENHANCEMENT] Reduce allocations in Graphite Ingester, when ingesting untagged Graphite metrics.
[ENHANCEMENT] Serve Graphite /metrics/find requests by keeping track of all recent metrics in an in-memory index on the Ingesters to reduce latency.
[ENHANCEMENT] Add auxiliary Graphite API endpoints to explore tags and obtain auto-complete suggestions for the Grafana query editor.
[ENHANCEMENT] Admin API: add ClusterKind support for Logs & Traces.
[ENHANCEMENT] Admin API: add scopes for Logs.
[ENHANCEMENT] Admin: The bootstrap target no longer needs to be run before being able to start GEM with enterprise features. Every target will now try to perform bootstrapping on startup if it has not already been done. Failure to bootstrap will not prevent GEM running, but enterprise features will not be available.
[ENHANCEMENT] Add grafana_labs_license_expiry_timestamp metric to expose GEM license expiration as a UNIX timestamp, in seconds.
[BUGFIX] Graphite: Fixing a bug in the request parsing of GET requests on the auto-complete endpoints.
[BUGFIX] Graphite: When ingesting datapoints resulting in out-of-order/out-of-bounds/duplicate-sample we need to return status 200 to prevent an indefinite loop.
[BUGFIX] Ruler: Fix issue where remote-write rule groups are created then immediately deleted when a rule group name contains the / delimiter character.

Upstream Cortex changes

Upstream Cortex hash: c3b8c46fd8fc9a2aa85accbe54cb00be2552dcd9
Changes since last GEM release

v1.0.2 – October 16 2020

Changelog

[CHANGE] Update vendored Cortex from v1.4.0 to [v1.4.0-21bad5][21bad5]
[BUGFIX] Fix potential panic due to writing into a closed chan in the graphite query executor.
[ENHANCEMENT] Admin: Access policy create operations now enforce valid instance/cluster names for the realms configured on the access policy.
[ENHANCEMENT] Add -version flag to GEM.
[FEATURE] Add config options to rate limit the LIST methods of buckets.
[FEATURE] Adds the Graphite /render API endpoint, which can be used to query metrics with the Graphite query language.

[FEATURE] Add config options to specify and poll files to inject arbitrary HTTP headers in requests to S3 for the admin and blocks client.

blocks_storage:
  s3:
    header_map_file_path: <path to header file>
    header_map_poll_interval: <duration string>
admin_client:
  storage:
    s3:
      header_map_file_path: <path to header file>
      header_map_poll_interval: <duration string>

[FEATURE] Adds the Graphite /metrics/find API endpoint, which can be used to obtain lists of metrics matching a given pattern (Grafana query editor auto-complete, dashboard variable population, etc).
[FEATURE] Add a default access policy option for OpenID Connect tokens.

Upstream Cortex details

Cortex Hash: [21bad57b346c730d684d6d0205efef133422ab28][21bad5]
Cortex CHANGELOG

v1.0.1 – October 06 2020

Upstream Cortex details

Cortex Hash: 23554ce028c090a4a3413ac0e35e5e1dc9fa929f
Cortex Version: 1.4.0

Changelog

[CHANGE] Update vendored Cortex to v1.4.0.

v1.0.0 – September 17 2020

Upstream Cortex details

Cortex Hash: bb5fcc929832f7bd2a6c2df348b387abcb8b961e
Cortex Version: 1.4.0-rc.0

Changelog

[BUGFIX] Make config field names consistent.
[CHANGE] Use Go 1.14.9 to build the project and cut build-image@v0.1.3.

v1.0.0-rc.2 – September 15 2020

Upstream Cortex details

Cortex Hash: c3a344784a0c8ce70ef2521f543033dee3dce6c6
Cortex Version: 1.3.1

Changelog

[BUGFIX] Admin API: Fix panic on start up for admin-api target.

v1.0.0-rc.1 – September 04 2020

Upstream Cortex details

Cortex Hash: 4f6e1e5c48ccad2c1988cf1d36ca522ae0c805ed
Cortex Version: 1.3.1

Changelog

[CHANGE] Admin-Client: The storage backend for the admin client no longer defaults to s3. Instead no default is set and the admin client will not start up unless a default is set.
[CHANGE] The following features will no longer be active unless GEM is started with access to a valid license.
- Admin API
- Ruler S3 auth headers
- Ruler API to configure remote write rule groups

v0.6.3 – August 20 2020

Upstream Cortex details

Cortex Hash: 2bda7b94
Cortex Version: 1.2.1

Changelog

[CHANGE] Auth: removed auth.enable flag and add auth.type flag with default and enterprise options.
[FEATURE] Admin API: Add list endpoint for stored licenses.