Kubecost

Data	Supported
Metrics	Yes, with history
Logs	No
Traces	No
Alerting	Yes

Dashboards for what Kubernetes costs, read from Kubecost: cost by cluster, namespace and team, idle and unallocated cost, the cloud bill inside and outside the cluster, and Kubecost's savings recommendations. The plugin only reads from Kubecost. Nothing is installed in the cluster.

Requirements

• Grafana 12.4 or later.
• Kubecost 3.0.5 or later.
• Kubecost's frontend Service on port 9090, or its ingress, reachable from Grafana.
• A cloud billing integration connected in Kubecost, for the cloud bill, out-of-cluster spend and prices reconciled against the bill.

Getting started

1. Create a credential, if Kubecost asks for one. Credentials, below, says which method fits your Kubecost and has the steps.

2. Add the data source. In Grafana, go to Connections, then Data sources, and add Kubecost. Enter the address of the Kubecost frontend, such as https://kubecost.example.com. Under Authentication, choose the Method and enter the credential. Save & test reports the Kubecost version, whether a cloud bill is connected, and the longest range the Kubecost license allows.

3. Import the dashboards. On the data source's page, open Dashboards and select Import beside each one. Start with Kubecost Cost overview.

If a private certificate authority signed the certificate at the Kubecost URL, turn on Use a certificate authority and paste that authority's certificate. Skip TLS certificate verification connects without verifying the certificate.

To try the plugin without Kubecost, turn on Demo mode in the data source settings. It serves sample data for two clusters and sends nothing over the network.

If Grafana cannot reach Kubecost directly, use Grafana's secure SOCKS proxy or Private Data Source Connect, and turn on Connect through the secure proxy. The proxy or agent needs to reach Kubecost's frontend Service only.

Credentials

Kubecost install	Method
Single sign-on is off	None
Single sign-on is on, with SAML or OIDC	API key
Behind a proxy or ingress that asks for one	Basic, Bearer token or Custom headers

API key. Use a service account key created for the plugin, with read-only access.

1. In Kubecost, open Teams and create a service account key.
2. Give the key the readonly role.
3. Paste the key into the data source's API key field.

Basic, Bearer token and Custom headers. Enter the username and password, token or headers that the proxy or ingress in front of Kubecost accepts.

Data source settings

Setting	Default	Description
Demo mode	Off	Serves the built-in sample data. The connection, authentication and TLS settings are not used
URL		The Kubecost frontend's address
Method	None	See Credentials
Custom headers	None	Sent with every request
Skip TLS certificate verification	Off
Use a certificate authority	Off	Shows the CA certificate field
Connect through the secure proxy	Off	Shown when Grafana has a secure SOCKS proxy or Private Data Source Connect configured
Timeout (seconds)	60	Per request, from 1 to 300
Concurrent requests	4	The most requests sent to Kubecost at once, from 1 to 32

Costs are in the currency Kubecost is configured with.

Dashboards

Dashboard	Shows	Select	Opens on
Kubecost Cost overview	Where the money goes, and whether it is rising	Cluster, Label key	Last 7 days and today
Kubecost Showback	What each team costs, with idle and shared cost spread across the teams	Cluster, Namespace, Label key, Team, Shared namespaces	Last 14 days and today
Kubecost Efficiency and savings	Capacity paid for and not used, and Kubecost's recommendations	One cluster, Namespace, Node	Last 7 days and today
Kubecost Unallocated cost	Cost with no team label, no owning controller or no resource requests	Cluster, Namespace, Label key	Last 7 days and today
Kubecost Cloud and infrastructure	The cloud bill inside and outside the cluster, and nodes, disks and load balancers	Cluster	Last 14 days and today

The dashboards use UTC, and each range starts at midnight.

Each dashboard has a Data source selector. One data source is one Kubecost install, with every cluster that install covers.

Label key is the Kubernetes label that names a workload's owner. It starts as team. On Showback, the cost of the namespaces in Shared namespaces is spread across the teams. It starts as kube-system.

A plugin update can bring a newer version of a dashboard, which Dashboards offers to import over the one you have. When customizing one, save a copy and edit that.

Query editor

Query type	Returns	Limits
Allocation	Cost by cluster, namespace, controller, pod, label or another Kubecost aggregation	10 series by default and 50 at most. 100 rows by default and 1,000 at most
Cost shares	Idle, unallocated and allocated cost for each cluster, with the idle and unallocated shares
Assets	Nodes, disks, load balancers and other infrastructure	100 rows by default and 1,000 at most
Cloud costs	The cloud bill by service, account, provider, category, region or label, split into Kubernetes and out-of-cluster spend	Daily or weekly. 50 rows by default
Billing status	One row for each billing integration, with whether it is connected
Savings	Kubecost's recommendations: a summary by category, container requests, abandoned workloads, unclaimed volumes and node groups	A lookback of 1 to 90 days, 7 by default. Node groups needs one cluster
Entities	Lists of clusters, namespaces, controllers, nodes, label keys and label values, for template variables
Direct API	Kubecost's allocation, asset, cloud cost, savings, efficiency and budget reports, as a table	1,000 rows by default and 10,000 at most

Allocation, Cost shares, Assets and Cloud costs return a time series or a table. Aggregating Allocation by pod or container needs table output, or a namespace or controller filter.

Idle and unallocated cost are rows of their own, marked in the kind column. An Allocation time series that reaches its series limit shows the remainder as one "other" series.

Idle chooses where idle cost goes: its own row, a row for each cluster or node, spread across the other rows, or left out. Cost as gives the total over the range, or an hourly, daily or monthly rate. Reconcile adjusts prices to the connected cloud bill. Shared cost spreads chosen namespaces, labels or a fixed monthly cost across the other rows.

Time range

A query covers the dashboard's time range. Tables and single values are one total over it.

• Resolution sets the bucket size of a time series: hour, day, week, or Auto, which follows the panel's interval. Hourly buckets are for a range that starts within the last 48 hours.
• Exclude incomplete bucket leaves out the last bucket of a time series, which is still filling.
• Exclude today ends a table's range at midnight UTC. A rate is then taken over whole days.
• Previous period returns the same Allocation query for the range just before the dashboard's.
• Savings use the lookback set in the query, not the dashboard's time range.

Template variables

For building your own dashboards. Create each as a Query variable on the Kubecost data source, and set it to refresh on time range change.

List	Narrow by	Lists
Cluster		Clusters
Namespace	$cluster	Namespaces
Controller	$cluster, $namespace	Controllers, as kind:name
Node	$cluster	Nodes
Label key		Label keys
Label value	$cluster, $namespace	Values of one label key, including the unlabeled

On a multi-value variable with Include All, set the custom all value to $__all. All then removes the filter.

Alert rules

The plugin includes the six alert rules below, as a provisioning file at alerting/kubecost.yaml in the plugin's directory. On a Grafana you run yourself, copy it into Grafana's provisioning/alerting directory, set the environment variable KUBECOST_DATASOURCE_UID to the Kubecost data source's uid, and restart Grafana. Grafana Cloud does not read provisioning files. There, create the rules through the alerting provisioning API, Terraform, or by hand from this table.

Rule	Fires when	Checked every	For	Severity
Cluster spend rising	A cluster's hourly spend is more than 30% above its 7-day rate	1h	1h	warning
Idle share high	More than 30% of a cluster's cost is idle	1h	3h	warning
Unallocated share high	More than 25% of a cluster's cost has no team label	1h	3h	warning
Cloud billing not connected	A cloud billing integration in Kubecost is not connected. One alert per integration	1h	6h	warning
Projected monthly cost above budget	A cluster's monthly rate over the last week is above the budget	6h	0s	warning
Cost data stopped arriving	Kubecost reports no cost for a cluster for an hour, or cannot be queried	15m	30m	critical

The monthly budget is 2000, written in the file. Replace it with your own figure in the Projected monthly cost above budget rule, in both its threshold and its summary. Unallocated share high reads the team label. The file's comments say where to change it.

A Grafana alert rule can query Allocation, Cost shares, Assets, Cloud costs, Billing status and Savings. A time series query gives one alert instance for each series, labeled with its cluster, namespace or other dimension.

A Savings rule uses the lookback set in its query, whatever the rule's range.

Support

Email helpdesk@integrationplumbers.io.

Use of the plugin is subject to the Integration Plumbers acceptable use policy.