Profiles Drilldown on Grafana Labs

Access or install Profiles Drilldown

Mon, 28 Sep 2026 08:26:02 +0000

Access Profiles Drilldown

Grafana Profiles Drilldown is a native Grafana application designed to integrate seamlessly with Pyroscope, the open source continuous profiling platform, providing a smooth, queryless experience for browsing and analyzing profiling data.

You can use Profiles Drilldown in Grafana Cloud or in your own Grafana instance.

Before you begin

To use Profiles Drilldown with Grafana Cloud, you need:

A Grafana Cloud account
A Grafana stack in Grafana Cloud with a configured Hosted profiles or Pyroscope data source

To use Profiles Drilldown with Grafana open source or Grafana Enterprise, you need:

Grafana 13.1.0 or later
Your own Grafana instance
A configured Pyroscope data source

Install the Profiles Drilldown plugin

Profiles Drilldown is distributed as a Grafana Plugin. You can find it in the official Grafana Plugin Directory.

Note
Grafana 13.1.0 and later includes all Grafana Drilldown apps by default, so no plugin installation is required.

Concepts

Mon, 21 Sep 2026 17:16:23 +0000

Concepts

To use profiling, you should understand these concepts:

Profile data hierarchy
Profile types
Labels
Flame graphs
Exemplars

Profile data hierarchy

Each profile has data for services. A service can be one of the processes running on your system or application.

Each service has multiple profile types, such as CPU, memory, exceptions, and locks. These profiles provide details about the processes in a service.

Service: This is the main application or system you are monitoring. For example, it could be an online shopping platform, a social media application, or a cloud storage service.
- Profile type: You can monitor each service through different profile types, each focusing on a specific performance aspect. Example profile types:
  - CPU
  - allocated memory
  - in-use memory
  - goroutines
- Labels: These are tags or descriptors that provide additional context to the profile types. Labels help categorize and filter the data for easier analysis. For example:
  - Namespace
  - Pod
  - Instance
  - IP
  - Region

Profile types

Profile types refer to different dimensions of application performance analysis, focusing on specific aspects like CPU usage, memory allocation, or thread synchronization.

For more information, refer to Understand profiling types.

Profile type	What it shows	When to use
CPU profiling	Measures the amount of CPU time consumed by different parts of your application code	To identify and optimize CPU-intensive functions
Memory allocation	Tracks the amount and frequency of memory allocations by the application	For identifying and optimizing memory usage patterns
Goroutine	Measures the usage and performance of lightweight routines in Go	Especially useful in Go applications for concurrency management
Contentions (Mutex)	Mutex profiling involves analyzing mutex (mutual exclusion) locks, used to prevent simultaneous access to shared resources.	To optimize thread synchronization and reduce lock contention
Block	Measures the frequency and duration of blocking operations, where a thread is paused or delayed	To identify and reduce blocking delays

Labels

Labels help identify parts of profiling data. For example, labels identify Kubernetes Pod names, namespaces, repositories, and references. You can also create custom labels.

This feature is crucial for identifying performance anomalies and understanding the behavior of different application segments under various conditions.

Flame graphs

A fundamental aspect of continuous profiling is the flame graph, a convenient way to visualize performance data. These graphs provide a clear, intuitive understanding of resource allocation and bottlenecks in the application.

You can use a flame graph visualization if you need to:

Identify any performance hot spots to find where code optimizations may be needed.
Diagnose the root cause of any performance degradation.
Analyze the behavior of complex systems, including distributed systems or microservices architectures.

To learn more, refer to the Flame graph documentation. To learn more about the flame graph user interface, refer to Flame graph visualization panel.

Exemplars

Exemplars are individual profiles that appear as diamond-shaped markers on timeseries panels. While a flame graph normally shows an aggregate of all profiles in the selected time range, clicking an exemplar lets you inspect one specific individual profile. This makes it easier to understand what was happening at a precise moment, for example, during a CPU spike.

Each exemplar includes a profile ID, a sample value, and associated labels that provide additional context such as pod, namespace, or cluster.

To learn more about using exemplars, refer to Drill down to an individual profile.

Get started with Profiles Drilldown

Tue, 06 Oct 2026 12:20:50 +0000

Get started with Profiles Drilldown

Profiles can help you identify errors in your apps and services. Using this information, you can optimize and streamline your apps.

Your investigation begins with the big picture and then drills down using profile types, labels, and flame graphs to explore your data. To learn more, refer to Concepts.

Note
Expand your observability journey and learn about Traces Drilldown.

Before you begin

To use Profiles Drilldown with Grafana Cloud, you need:

A Grafana Cloud account
A Grafana stack in Grafana Cloud with a configured Hosted profiles or Pyroscope data source

To use Profiles Drilldown with Grafana open source, you need:

Grafana 13.1.0 or later
Your own Grafana instance
Grafana 13.1.0 and later includes all Grafana Drilldown apps by default.
A configured Pyroscope data source

Explore your profile data

Most investigations follow four general steps:

Determine your use case.
Choose a view.
Investigate trends and spikes.
Identify issues as a result of your investigation.

Give it a try using Grafana Play

With Grafana Play, you can explore and see how it works, learning from practical examples to accelerate your development. This feature can be seen on the Grafana Play site.

Try it

Determine your use case

Mon, 21 Sep 2026 17:16:23 +0000

Determine your use case

When you start investigating, you may already know what’s wrong or you may want to identify resource hot spots so you can address them.

This can lead you to two different starting points:

Use case 1 - You want to investigate an issue to determine the root cause.
Use case 2 - You want to research resource and performance hot spots to determine areas that can be optimized.

Your use case determines what’s most important. For example, in use case 1, if a service is misbehaving, then you might want to see the profile types so you can see the CPU and memory profiles alongside each other.

For either use case, the first step is to identify areas of interest by reviewing profiles or a single service. Selecting different profile types lets you focus on memory allocation, CPU processes, allocation sizes, blocks, or lock contention.

The available profile types depend on how you have instrumented your app to generate profiling data. For more information, refer to Profiling types for help selecting a profile type to match your use case. Refer to Understand profile types to learn about profile types and instrumentation methods.

After you identify the problem process or service, you can filter and explore using labels and flame graphs to investigate at lower levels. With capabilities like the Flame graph AI interpreter or the GitHub integration, Profiles Drilldown helps you locate the root cause and how to address it.

Use case 1: Investigate an issue

Profiling data is ideal when you know there is a specific service or area where there is a performance issue. Maybe an alert from spike in CPU led you to profiles or maybe your logs showed OOM issue for a particular service and you need to debug it. Grafana Profiles Drilldown lets you quickly drill into a service and identify the performance issue in these scenarios.

Example: Know the problem service, not the cause

If you know the affected service, your investigation starts by viewing that service and then reviewing the profiles for that service. In this example, you may have identified that the checkoutservice has an issue. You can use Search services to locate checkoutservice and then use the Profile types view to examine all profiles for that service.

Alternatively, you can select the Profile types view and then choose the checkoutservice from the Service drop-down list.

Example: Know there is an issue, need to investigate

If you only know there is an issue and have to investigate, then your investigation starts by using the All services view. Using the Profile type selector, you can check services’ CPU processes, memory allocation, blocks, locks, exceptions, and other available profile types.

After you locate the profile with a spike, select either Profile types view to examine all profile types for that service, or select Labels to view the labels (such as hostname or span_name) for that service.

Use case 2: Research performance and resource bottlenecks

Illuminate performance issues across a number of dimensions when you are doing a proactive analysis. This could be a cost-cutting exercise or attempt to improve latency or memory usage in a non-incident scenario. This exercise could be across multiple services or within a service across profile types, labels, or a combination of the two.

Next step: Investigate and identify issues

Choose a view

Fri, 09 Oct 2026 10:18:13 +0000

Choose a view

The available views in Grafana Profiles Drilldown correspond to how you move through your data: from broader to more specific. For example, you can start with All services and then move to Profile types or Labels for more specific views into your data.

This matches the hierarchy of the profiling data.

After you choose a view, you can Investigate trends and spikes.

Select a view

Select a view by choosing an Exploration tab: All services, Profile types, Labels, Flame graph, Diff flame graph, or Favorites.

You can refine a selected view by selecting a service, profile type, label, or flame graph from one of the charts. Click and drag on an area of interest to see more detail.

Available views

Determining which view to use frames the rest of your investigation.

View	What it shows	Used for
All services	Overview of all services for any given profile metric	Reviewing all the services that are being profiled Finding an anomalous service out of all services (or set of related services) by profile type
Profile types	Overview of all the profile metrics for a single service	Analyzing the performance of a single service across all profile types Finding relations of profile metric spikes across profile types (or discovering an anomalous metric)
Labels	Single service label exploration and filtering	Analyzing profiling metrics of a single service and profiling type across all label dimensions Discovering anomalous profiling metrics by label
Flame graph	Single service flame graph	Analyze the flame graph of a particular service, profile type, and label selection. Use tools like Analyze with Assistant (Grafana Cloud) or Explain Flame Graph (Grafana open source), GitHub code view, and function details to get deeper insights into performance. Enable Exemplars to see individual profile markers on the timeseries and click one to inspect a single individual profile. Use the Span heatmap to investigate resource usage per span and open associated traces.
Diff flame graph	Compare the differences between two flame graphs	Compare queries that may have different total amounts of time spent in each function
Favorites	List of visualizations saved as favorites	Save frequently accessed panels for quick access

Exploration type examples

All services view

The All services view is the default view when you first open Profiles Drilldown.

Use the Filters bar to narrow the services shown in this view. When you open a service in the Flame graph view, the filters you set here are applied there too, so you can keep exploring without losing your place. Your All services filters stay in place when you return. These filters apply only to the All services view, not to the Labels or Diff flame graph views.

Profile types

The Profile types view shows one chart for each profile type available for the selected service. You can select to display the charts in a grid or one chart per row.

Labels

The Labels view lets you navigate and analyze performance data through labels. This feature is crucial for identifying performance anomalies and understanding the behavior of different application segments under various conditions.

Flame graphs

The Flame graph view visualizes profiling data of a single service in a flame graph format, helping you identify resource-intensive functions.

On views with a flame graph, you can use Analyze with Assistant (Grafana Cloud) or Explain Flame Graph (Grafana open source) to get an AI-generated analysis of bottlenecks, likely root causes, and recommended fixes. For more information, refer to Flame graph AI.

You can also use line-level insights from the GitHub integration.

In this view, you can also:

Search for functions in the flame graph.
Change the color scheme.
Expand or collapse all call stacks.
Toggle between the top table, flame graph, and call tree, or show a combination of views.
Export profile data.

Export profile data

To export profile data, click the Export profile data download icon above the flame graph. The options depend on the view:

Flame graph: Download png, json, or pprof, or select gcx command to copy a gcx profiles query command that downloads the profile as a pprof file.
Diff flame graph: Download png or json, or select gcx commands to copy two gcx profiles query commands that download the baseline and comparison profiles as pprof files.

png is only available when the flame graph is showing.

Span heatmap (Profiles to Traces)

Use the span heatmap to spot changes in resource usage across trace spans and find spans with unusually high profile values. Inspect an individual span’s flame graph to identify resource-intensive functions, or open its associated trace to understand request timing and relationships between services.

The span heatmap is a visualization within the Flame graph view, not a separate Exploration tab. It requires span profiles in Pyroscope; viewing associated traces also requires matching trace data in Tempo.

For the complete walkthrough and troubleshooting guidance, refer to Move from profiles to traces.

Diff flame graph

The Diff flame graph view shows the differences between two profiling datasets. It normalizes the data by comparing the percentage of total time spent in each function so that the resulting flame graph is comparing the share of time spent in each function rather than the absolute amount of time spent in each function. This lets you compare two different queries that may have different total amounts of time spent in each function.

Similar to a git diff, it takes the selected flame graphs and highlights the differences between them.

To configure comparisons more quickly, you can use:

Auto-select to choose a comparison range automatically.
Comparison presets to apply saved or predefined comparisons.
Sync time ranges to keep baseline and comparison windows aligned.

To export a diff flame graph, refer to Export profile data.

Favorites

The Favorites view shows all your saved favorite visualizations. Clicking on the star (⭐️) at the top-right corner of a visualization saves it as a favorite.

Using favorites, you can create an overview of what’s important, as well as jump to the Flame graph view or the Labels view.

In this view, you can also use:

Quick filter to find saved items.
Grid or Rows layout options.
Hide panels without data to simplify the display.

Investigate trends and spikes

Fri, 09 Oct 2026 10:18:13 +0000

Investigate trends and spikes

Grafana Profiles Drilldown provides powerful tools that help you identify and analyze problems in your applications and services.

Using these steps, you can use the profile data to investigate issues.

Give it a try using Grafana Play

With Grafana Play, you can explore and see how it works, learning from practical examples to accelerate your development. This feature can be seen on the Grafana Play site.

Try it

Explore your profile data

When you use Profiles Drilldown, your investigations usually follow these steps.

Verify your data source in the Data source drop-down.
Choose an Exploration tab. All services is selected by default. Learn about the available views.
Look for spikes or trends in services to identify where to investigate. Use the Profile type drop-down to change profile metrics.
After you identify the service to explore, you can change views:
- Select Profile types to review profile metrics for a service.
- Select Labels to view labels for a service and refine the scope of your investigation.
- Select Flame graph to view the flame graph for a service.
Optional: Select filters to focus on problem areas. Each filter is added to the filter expression near the top of the page. You can add filters in the following ways:
- Use filter selectors in the filter bar to add labels and operators.
- In Labels view, use Include or Exclude on areas of interest.
If Labels view shows no data, select a different service, profile type, or group-by label.
Optional: Click and drag on a chart to zoom to a smaller time range.
To compare two flame graphs, open Diff flame graph.
- Configure Baseline and Comparison using time range selectors, label filters, and chart range selection.
- Use Auto-select or choose a comparison preset to speed up setup.
Use Diff flame graph to compare where relative time share changes between baseline and comparison, and then drill into functions to identify likely causes.

Drill down to an individual profile

While flame graphs show an aggregate of all profiles in the selected time range, you may want to inspect the exact profile behind a spike. Exemplars are individual profiles shown as markers on the timeseries panel. They let you go from an aggregated view to a specific individual profile.

Navigate to the Flame graph or Labels view for your service. Exemplars are enabled by default in these views. In other views with a timeseries panel, select the Exemplars toggle in the panel header to enable them.
Each diamond marker on the timeseries represents an individual profile.
Click an exemplar marker to view its details, including the profile ID, value, timestamp, and associated labels such as pod, namespace, and cluster.
Select View profile in the exemplar popover. The flame graph updates to show only that single individual profile. A profile id selector tag appears above the flame graph confirming your selection.
To return to the aggregated flame graph, click X on the profile id selector tag.

Move from profiles to traces

Use the span heatmap to investigate how resource usage varies across trace spans. Start with the distribution of span profile values, select an exemplar, and inspect its flame graph or associated trace.

The span heatmap is a visualization within the Flame graph view, not a separate Exploration tab.

Before you begin

You need:

A Pyroscope data source with span profiles for the service, profile type, filters, and time range you want to investigate. Aggregated profile data alone doesn’t guarantee that span profiles are available.
To inspect associated traces, a Tempo data source containing the matching trace data and permission to query it. You can inspect span profiles without selecting Tempo.

Open the span heatmap

Open Profiles Drilldown and select your Pyroscope Data source.
Select the Flame graph view and choose a service and Profile type.
Set the time range and any label filters for your investigation.
In the Profile timeline visualization toggle above the chart, select Span heatmap.

To return to the time series visualization, select Time series in the same toggle.

Read the heatmap

The heatmap groups spans into buckets:

The horizontal axis represents time.
The vertical axis represents the selected profile metric’s value per span.
Cell color represents the count of spans in a bucket. Use the tooltip and color scale to interpret the counts.

Look for changes in the distribution or spans with unusually high profile values. For example, with a CPU profile type, higher-value buckets represent spans with more profiled CPU usage, not necessarily longer elapsed duration.

Exemplar markers identify individual span profiles you can investigate. The Top span exemplars table provides timestamps, span IDs, and profile metric values. Trace details, including duration and trace ID, are loaded from the selected Tempo data source when available.

Inspect an individual span profile

Find an exemplar in the heatmap or the Top span exemplars table.
Use Select exemplar to highlight it in the heatmap and focus the table on that exemplar.
In the table, select Open flame graph to inspect the span’s profile and identify the functions contributing to its resource usage.

Use Clear exemplar selection in the table to remove the exemplar highlight and restore the table’s other rows.

Open the associated trace

In Top span exemplars, select the Tempo data source that contains the service’s traces.
Wait for trace details to load. Use the span name, duration, timestamp, and profile metric value to find a span of interest.
Select Open trace for a span with resolved trace details. The associated trace opens in a drawer.
Inspect the trace to understand the span in the context of the request. Close the drawer to continue investigating the heatmap.

A span’s profile metric value and its trace duration describe different things. Use the flame graph to understand profiled resource usage and the trace to understand request timing and relationships between spans.

Troubleshoot missing data

Span heatmap is unavailable

If Span heatmap is disabled, its tooltip says: No span profiles are available for the current service, filters, and time range.

Check that you selected the intended service and profile type. Expand the time range or remove restrictive filters, then try again. If span profiles are still unavailable, verify that your profiling setup collects span profiles. Seeing data in Time series doesn’t by itself mean span profiles are present.

No span exemplars are shown

If the table displays No span exemplars in the selected time range, try a time range and filters that include span profiles. The exemplar table is not a complete list of every span represented by the heatmap.

Trace details are missing

Select the Tempo data source that contains the matching traces. A missing trace ID or Open trace action means no trace details have been resolved for that exemplar.

If you see Unable to load trace details from Tempo, check your access to the selected Tempo data source and whether it can be queried. Also verify that the matching traces are still available within your trace retention period. You can continue investigating the available profile data without trace details.

Common tools during investigations

In the Profiles toolbar, you can also use these features while investigating:

Upload ad hoc profiles to load profile data for one-off analysis. The ad hoc view provides a Single view to inspect one uploaded profile and a Diff view to compare two uploaded profiles. In the diff view, select Side by side to show each profile in its own flame graph, or Diff flamegraph to show a computed difference. To compute a difference, the two profiles must share a common profile type.
Copy shareable link to capture the current investigation state and share it with teammates.
View/edit tenant settings to adjust settings such as collapsed flame graphs, function details, and maximum node count.

Add a time series panel to a dashboard

You can save a time series visualization from Profiles Drilldown to a Grafana dashboard to monitor it alongside your other observability data.

Hover over a time series panel and open the panel menu.
Select Add to dashboard.
Choose an existing dashboard or create a new one, then save.

Save and reuse searches

When you apply filters, you can save the current search and return to it later. A saved search captures the data source, profile type, and filters.

Apply one or more filters.
Select Save in Saved queries and enter a title and an optional description.
To reuse a saved search, select Load Saved query and choose one from the list.

When the Grafana query library is available, searches are saved to the shared Saved queries library. When it isn’t available, these actions appear as Save search and Load saved search, and searches are stored locally in your browser and only available on that device.