This is documentation for the next version of Grafana Alloy Documentation. For the latest stable release, go to the latest version.

General availability (GA) Open source

database_observability.mysql

database_observability.mysql connects to a MySQL database and collects observability data from the performance_schema and information_schema. The component collects query details, schema information, explain plans, query samples, and lock information. It forwards this data as log entries to Loki receivers and exports targets for Prometheus scraping.

Usage

Alloy
database_observability.mysql "<LABEL>" {
  data_source_name = <DATA_SOURCE_NAME>
  forward_to       = [<LOKI_RECEIVERS>]
}

Arguments

You can use the following arguments with database_observability.mysql:

NameTypeDescriptionDefaultRequired
data_source_namesecretData Source Name for the MySQL server to connect to. Required when no database_instance blocks are defined.no
forward_tolist(LogsReceiver)Where to forward log entries after processing.yes
targetslist(map(string))List of external targets to scrape.no
disable_collectorslist(string)A list of collectors to disable from the default set.no
enable_collectorslist(string)A list of collectors to enable on top of the default set.no
exclude_schemaslist(string)A list of schemas to exclude from monitoring.["alloydbadmin", "alloydbmetadata", "azure_maintenance", "azure_sys", "cloudsqladmin", "rdsadmin"]no
allow_update_performance_schema_settingsbooleanWhether to allow updates to performance_schema settings in any collector. Enable this in conjunction with other collector-specific settings where required.falseno

The following collectors are configurable:

NameDescriptionEnabled by default
query_detailsCollect queries information.yes
schema_detailsCollect schemas and tables from information_schema.yes
query_samplesCollect query samples.yes
setup_consumersCollect enabled performance_schema.setup_consumers.yes
setup_actorsCheck and update performance_schema.setup_actors settings.yes
locksCollect queries that are waiting/blocking other queries.no
explain_plansCollect explain plans information.yes
table_statsCollect table-level index usage statistics.no
index_statsCollect per-index usage statistics.no

Blocks

You can use the following blocks with database_observability.mysql:

cloud_provider

The cloud_provider block has no attributes. It contains zero or one of the aws, azure, or gcp blocks. You use the cloud_provider block to provide information related to the cloud provider that hosts the database under observation. This information is appended as labels to the collected metrics. The labels make it easier for you to filter and group your metrics.

aws

The aws block supplies the ARN identifier for the database being monitored.

NameTypeDescriptionDefaultRequired
arnstringThe ARN associated with the database under observation.yes

azure

The azure block supplies the identifying information for the database being monitored.

NameTypeDescriptionDefaultRequired
subscription_idstringThe Subscription ID for your Azure account.yes
resource_groupstringThe Resource Group that holds the database resource.yes
server_namestringThe database server name, for example orders-db for the host orders-db.mysql.database.azure.com.no

gcp

The gcp block supplies the identifying information for the GCP Cloud SQL database being monitored.

NameTypeDescriptionDefaultRequired
connection_namestringThe Cloud SQL instance connection name in the format project:region:instance, for example my-project:us-central1:my-db.yes

database_instance

The database_instance block defines one database server to monitor. Repeat the block to monitor several databases with a single component. The block label must be unique across database_instance blocks and identifies the database in the component’s metrics endpoint path. Each database_instance block must also point to a distinct server: two blocks that resolve to the same host, port, and database name are rejected.

NameTypeDescriptionDefaultRequired
data_source_namesecretData Source Name for the MySQL server to connect to.yes

Each database_instance block can also contain a [cloud_provider][cloud_provider] block that applies to that database only, and [rule][rule] blocks that relabel the metrics and logs of that database only.

The component always embeds a mysqld_exporter for each database_instance block and serves its metrics on the block’s metrics path. Use the [prometheus_exporter][prometheus_exporter] block to configure it. External exporter targets are only supported in the top-level single-DSN form.

When you define database_instance blocks, don’t set the top-level data_source_name, targets, and cloud_provider arguments. They’re mutually exclusive with database_instance blocks. All other arguments and blocks, such as collector settings and prometheus_exporter, apply to every configured database.

The metrics for each database are served on a separate /db/<LABEL>/metrics path under the component’s HTTP endpoint, and the exported targets point to the corresponding path. When you don’t define database_instance blocks, the component serves metrics on its historical /metrics path. The metrics endpoints are served exactly at those paths: requests to any other path under the component’s HTTP endpoint return HTTP 404.

For example:

Alloy
database_observability.mysql "pool" {
  forward_to = [loki.write.logs_service.receiver]

  database_instance "orders" {
    data_source_name = sys.env("ORDERS_DSN")

    cloud_provider {
      aws {
        arn = "orders-rds-db-arn"
      }
    }

    rule {
      target_label = "environment"
      replacement  = "production"
    }
  }

  database_instance "billing" {
    data_source_name = sys.env("BILLING_DSN")

    rule {
      target_label = "environment"
      replacement  = "staging"
    }
  }
}

In this example, the metrics and logs of the orders database carry environment="production", and those of the billing database carry environment="staging".

rule

The rule block relabels the metrics and the logs of the database_instance block that contains it. Use it to tell the data of each monitored database apart, for example by environment, team, or tier.

The rule block defines a relabeling rule applied to the metrics and the logs of a single monitored database. If you define more than one rule block, the rules are applied in top-down order, after the labels the component sets itself, so a rule can read those labels with source_labels.

It takes the same arguments as the rule block used elsewhere in Alloy, in a restricted form: because one rule set applies to two signal paths, it supports the labelling actions only, and it can’t overwrite most of the labels the component sets. Alloy refuses to load a configuration containing a rule that breaks either restriction, so such a rule is never silently ignored.

Two further constraints are specific to a database_instance block:

  • target_label must be a literal label name: Alloy checks it against the reserved labels when it loads the configuration, so it can’t contain a capture group reference.
  • hashmod requires source_labels, and can’t target instance: hashing the empty value gives every database the same result, and hashing into instance buckets several databases onto one value even when the source is present.

Caution

A relabeling rule that computes an empty value removes its target label instead of setting it, and the default replacement of "$1" computes an empty value when a rule has no source_labels. So rule { target_label = "environment" } removes environment rather than setting it, and Alloy rejects it. Always set replacement alongside a target_label.

Alloy can only reject such a rule where the outcome is certain. A rule that reads source_labels depends on the labels present at runtime, so a missing or misspelled source label still removes the target label, except for the reserved labels and instance, which keep their previous value. Only removal is prevented: a rule that computes a non-empty value still overwrites those labels, which is why hashmod can’t target instance at all.

You can use the hashmod, labeldrop, lowercase, replace, and uppercase actions. The rest aren’t supported:

  • drop, dropequal, keep, and keepequal filter rather than label, and filtering means two different things on the two signal paths. Dropping a metrics target stops the database exporting metrics entirely, whereas dropping a log entry discards one line. To filter one database’s logs, use a loki.relabel component on the receiver named in forward_to, with a drop rule that matches that database’s instance or server_id. Every database the component monitors forwards to the same receivers, so the rule has to identify the one it applies to. To filter one database’s metrics, use a discovery.relabel component on the component’s exported targets.
  • labelmap builds label names from replacement at runtime, so Alloy can’t check them against the reserved labels when it loads the configuration.
  • labelkeep removes every label its regex doesn’t match, so a rule that preserves the reserved labels would have to match all of them. Use labeldrop to remove specific labels instead.

Because the rules run after the component’s own labels, and those are protected, a labeldrop rule can only remove a label that an earlier rule in the same block created.

Reserved labels

A rule can’t write to the following labels, and a labeldrop regular expression can’t match them:

  • Both metrics and logs: job and server_id.
  • Logs only: engine and op.
  • Metrics only: provider_name, provider_region, provider_account, __address__, __scheme__, and __metrics_path__.

This applies to rule blocks only. A downstream discovery.relabel or loki.relabel component can still overwrite any of them, but it applies to every database the component monitors rather than one.

The instance label is a special case: a rule can rewrite its value, which is a common reason to use a rule block, because a friendly name such as orders_db then applies to that database’s metrics and logs together. A rule can’t remove it, because metrics and logs are correlated on it, and because an empty instance on a metrics target makes Prometheus fill it in from __address__, which every database in one component shares, collapsing them all onto one series.

For the same reason, two database_instance blocks can’t rewrite instance to the same value. Alloy rejects that when both values are fixed in the configuration, but it can’t when the value comes from a label the databases share: source_labels = ["job"] gives every database the same instance, since they all carry the same job. Derive instance from something that differs per database, or set a literal value in each block.

Labels set on only one signal path

Some of the labels above exist on one signal path and not the other, and a rule that reads one with source_labels behaves differently on each path: where the label is absent the rule computes an empty value, which removes its target_label rather than setting it to an empty string.

A rule reading provider_region, which a database with a cloud_provider block carries on its metrics targets but not on its log entries, sets region on that database’s metrics and removes region from its logs. Reading a logs-only label such as op does the same in reverse. Only job, server_id, and instance are set on both paths, so prefer those as source_labels, or a label an earlier rule in the same block created.

clustering

NameTypeDescriptionDefaultRequired
enabledboolEnables distributing databases with other cluster nodes.falseyes

When Alloy is using clustering, and enabled is set to true, then this database_observability.mysql component instance opts-in to distributing its configured databases between all cluster nodes.

Clustering assumes that all cluster nodes are running with the same configuration file. All component instances opting in to clustering use the instance key of each configured database, <network>(<host>:<port>)/<dbname>, and a consistent hashing algorithm to determine ownership of each database between the cluster peers. Each peer then only collects from the subset of databases it’s responsible for, and only exports the targets of those databases, so prometheus.scrape components on the same node scrape exactly the databases the node owns. When a node joins or leaves the cluster, every peer recalculates ownership: expect a short gap or a brief duplicate collection for a database while its ownership moves. This includes the collectors that update performance_schema settings, such as setup_actors and setup_consumers, which can briefly run from two nodes during the move: their updates are idempotent. While the cluster isn’t yet ready to admit traffic, for example while it’s still forming and waiting for the minimum cluster size, the component doesn’t collect from any database.

Clustering is also useful with a single database: when several cluster nodes run an identical configuration, exactly one node collects from the database at a time, which gives you a highly available setup without duplicate collection.

If Alloy is not running in clustered mode, then the block is a no-op and database_observability.mysql collects from every configured database.

setup_consumers

NameTypeDescriptionDefaultRequired
collect_intervaldurationHow frequently to collect performance_schema.setup_consumers information from the database."1h"no

query_details

NameTypeDescriptionDefaultRequired
collect_intervaldurationHow frequently to collect information from database."1m"no
statements_limitintegerMax number of recent queries to collect details for.250no

schema_details

NameTypeDescriptionDefaultRequired
collect_intervaldurationHow frequently to collect information from database."1m"no
cache_enabledbooleanDeprecated. Whether to enable caching of table definitions.trueno
cache_sizeintegerDeprecated. Cache size.256no
cache_ttldurationDeprecated. Cache TTL."10m"no

The cache_enabled, cache_size, and cache_ttl settings are deprecated: they are accepted for backward compatibility, but ignored.

explain_plans

NameTypeDescriptionDefaultRequired
collect_intervaldurationHow frequently to collect information from database."1m"no
initial_lookbackdurationHow far back to look for explain plan queries on the first collection interval."24h"no
per_collect_ratiofloatRatio of explain plan queries to collect per collect interval.1.0no

locks

NameTypeDescriptionDefaultRequired
collect_intervaldurationHow frequently to collect information from database."30s"no
thresholddurationThreshold for locks to be considered slow. Locks that exceed this duration are logged."1s"no

query_samples

NameTypeDescriptionDefaultRequired
collect_intervaldurationHow frequently to collect information from database."10s"no
disable_query_redactionboolCollect unredacted SQL query text (including query parameters) and query error messages.falseno
auto_enable_setup_consumersbooleanEnables specific performance_schema.setup_consumers options. You must also enable allow_update_performance_schema_settings.falseno
setup_consumers_check_intervaldurationHow frequently to check if setup_consumers are correctly enabled."1h"no
sample_min_durationdurationMinimum duration for query samples to be collected. Set to “0s” to disable filtering and collect all samples regardless of their duration."0s"no
wait_event_min_durationdurationMinimum duration for a wait event to be collected. Set to “0s” to disable filtering and collect all wait events regardless of their duration."1us"no
enable_pre_classified_wait_eventsbooleanWhen true, emits telemetry data with pre-classified wait event information.falseno

setup_actors

NameTypeDescriptionDefaultRequired
auto_update_setup_actorsbooleanEnables updates to performance_schema.setup_actors settings. You must also enable allow_update_performance_schema_settings.falseno
collect_intervaldurationHow frequently to check if setup_actors are configured correctly."1h"no

health_check

NameTypeDescriptionDefaultRequired
collect_intervaldurationHow frequently to run health checks."1h"no

prometheus_exporter

The prometheus_exporter block configures the embedded mysqld_exporter scrapers. The data_source_name is inherited from the parent block.

Refer to prometheus.exporter.mysql docs for the full list of supported arguments and sub-blocks.

Example

Alloy
database_observability.mysql "orders_db" {
  data_source_name = "user:pass@tcp(mysql:3306)/"
  forward_to       = [loki.relabel.orders_db.receiver]
  targets          = prometheus.exporter.mysql.orders_db.targets

  enable_collectors = ["query_samples", "explain_plans"]

  cloud_provider {
    aws {
      arn = "your-rds-db-arn"
    }
  }
}

prometheus.exporter.mysql "orders_db" {
  data_source_name  = "user:pass@tcp(mysql:3306)/"
  enable_collectors = ["perf_schema.eventsstatements"]
}

loki.relabel "orders_db" {
  forward_to = [loki.write.logs_service.receiver]
  rule {
    target_label = "job"
    replacement  = "integrations/db-o11y"
  }
  rule {
    target_label = "instance"
    replacement  = "orders_db"
  }
}

discovery.relabel "orders_db" {
  targets = database_observability.mysql.orders_db.targets

  rule {
    target_label = "job"
    replacement  = "integrations/db-o11y"
  }
  rule {
    target_label = "instance"
    replacement  = "orders_db"
  }
}

prometheus.scrape "orders_db" {
  targets    = discovery.relabel.orders_db.targets
  job_name   = "integrations/db-o11y"
  forward_to = [prometheus.remote_write.metrics_service.receiver]
}

prometheus.remote_write "metrics_service" {
  endpoint {
    url = sys.env("<GRAFANA_CLOUD_HOSTED_METRICS_URL>")
    basic_auth {
      username = sys.env("<GRAFANA_CLOUD_HOSTED_METRICS_ID>")
      password = sys.env("<GRAFANA_CLOUD_RW_API_KEY>")
    }
  }
}

loki.write "logs_service" {
  endpoint {
    url = sys.env("<GRAFANA_CLOUD_HOSTED_LOGS_URL>")
    basic_auth {
      username = sys.env("<GRAFANA_CLOUD_HOSTED_LOGS_ID>")
      password = sys.env("<GRAFANA_CLOUD_RW_API_KEY>")
    }
  }
}

Replace the following:

  • <GRAFANA_CLOUD_HOSTED_METRICS_URL>: The URL for your Grafana Cloud hosted metrics.
  • <GRAFANA_CLOUD_HOSTED_METRICS_ID>: The user ID for your Grafana Cloud hosted metrics.
  • <GRAFANA_CLOUD_RW_API_KEY>: Your Grafana Cloud API key.
  • <GRAFANA_CLOUD_HOSTED_LOGS_URL>: The URL for your Grafana Cloud hosted logs.
  • <GRAFANA_CLOUD_HOSTED_LOGS_ID>: The user ID for your Grafana Cloud hosted logs.

Compatible components

database_observability.mysql can accept arguments from the following components:

database_observability.mysql has exports that can be consumed by the following components:

Note

Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.