---
title: "Troubleshoot a failing check | Grafana Cloud documentation"
description: "Troubleshoot a Synthetic Monitoring check by finding the failed operation, comparing probe executions, and using execution logs to choose the next investigation."
---

> For a curated documentation index, see [llms.txt](/llms.txt). For the complete documentation index, see [llms-full.txt](/llms-full.txt).

# Troubleshoot a failing check

Use this procedure when a Synthetic Monitoring check reports failures or has missing results. Start with one execution, find the operation that failed, and compare the result across probes before deciding which system or configuration to investigate.

## Before you begin

- Identify the affected check and the approximate time the problem occurred, including the time zone.
- You need access to the check’s dashboard and execution logs. If you can’t open them, ask your Grafana administrator for access.
- To investigate a network or application error, you need access to the relevant application, server, or network logs, or help from the team that manages them.

## Open an affected execution

1. Go to **Testing & synthetics > Synthetics > Checks**.
2. Use **Search checks** to find the check by its job name or the endpoint you’re monitoring, then click **View dashboard**.
3. Use the time picker to select the period when the problem occurred. Check the displayed time zone, and set the **probe** filter to **All** so you can compare locations.
4. Below the summary panels, find the time point explorer and select **Reachability** to view individual probe executions.
5. Click a time point in the graph near the time of the problem. Use the mini map above the graph to move to an earlier part of the selected period if necessary.
6. In the details below the graph, select the affected probe’s tab.

The details show either execution logs or a message about the execution’s state. The left and right arrows above the probe tabs are **Previous timepoint** and **Next timepoint**. Use them to move between intervals. Choose your next step from the state:

Expand table

| State       | Next step                                                                                                                                           |
|-------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|
| **Failure** | The execution reported `probe_success = 0`. Continue to [Find the operation that failed](#find-the-operation-that-failed).                          |
| **Success** | The execution reported `probe_success = 1`. Select a failing probe’s tab, or use **Previous timepoint** to find an affected execution.              |
| **Pending** | The execution is in progress or recently started. Use **Previous timepoint** to inspect an earlier execution while you wait for this one to finish. |
| **Missing** | No execution result exists for this probe under the current configuration. Follow [Investigate missing results](#investigate-missing-results).      |

The newest time point can still be waiting for results. Don’t use that message alone to conclude that the check or probe is broken. For the explorer’s controls and state definitions, refer to [Visualize check execution](/docs/grafana-cloud/observe-and-act/testing/synthetic-monitoring/analyze-results/visualize-check-execution/).

## Find the operation that failed

1. Keep **Event** selected in the execution details to read the parsed log messages.
2. Record the **Execution ID**, probe name, and execution timestamps.
3. Find an error message that describes a specific operation. A final message such as `Check failed` or `script did not execute successfully` summarizes the result; read the preceding messages to find the underlying error.
4. Record the error text and any associated `err`, `error`, `filename`, or `line` fields. For an HTTP request, also record the requested host and resolved IP address if the logs show them.

Use the message to select an investigation:

Expand table

| What the message reports                                 | What to do next                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|----------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `context deadline exceeded` or `i/o timeout`             | Identify the last operation shown before the timeout. If address resolution completed, record the resolved address and check the server’s listening port, firewall rules, and access logs for that time. Use [Compare probe executions](#compare-probe-executions) to find whether other locations report the same failure. A timeout alone doesn’t identify which system caused it.                                                                                         |
| A name-resolution error                                  | Compare the hostname configured in the check with the hostname you intend to monitor. Ask the DNS owner to check the record and resolution from the affected network. For a DNS check, also review its configured server, record type, and expected response in [DNS check options](/docs/grafana-cloud/observe-and-act/testing/synthetic-monitoring/create-checks/checks/dns/#options).                                                                                     |
| A TLS or certificate-validation error                    | Compare the hostname in the request with the certificate served by the endpoint. Have the team that manages the endpoint inspect the certificate validity and chain, and compare them with the check’s [TLS configuration](/docs/grafana-cloud/observe-and-act/testing/synthetic-monitoring/create-checks/checks/http/#tls-config).                                                                                                                                          |
| An unexpected HTTP response or failed response assertion | Record the returned status or response detail and the expected value. Ask the application owner to inspect the matching request in the application or web server logs. Compare the result with the check’s [HTTP validation](/docs/grafana-cloud/observe-and-act/testing/synthetic-monitoring/create-checks/checks/http/#validation) or [MultiHTTP assertions](/docs/grafana-cloud/observe-and-act/testing/synthetic-monitoring/create-checks/checks/multihttp/#assertions). |
| A browser locator or script error                        | Record the script filename, line number, and expected value or element. Ask the check owner to inspect that line and the preceding navigation or request. Determine whether the application changed, the script selected the wrong element, or a prerequisite failed before changing the assertion or increasing its timeout.                                                                                                                                                |

For example, an HTTP execution can contain this sequence, with fields omitted for clarity:

text ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```text
Beginning check                 timeout_seconds=60
Resolved target address         ip=34.133.192.185
Making HTTP request             host=quickpizza.grafana.fun
Error for HTTP request          err=... context deadline exceeded
Check failed                    duration_seconds=60.00089016
```

This execution resolved the hostname, then timed out during the HTTP request. It provides a resolved address, operation, and time window for the application or network team to investigate. It doesn’t show that increasing the check timeout would fix the problem.

A browser failure can instead identify a script line and an unmet expectation:

text ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```text
test aborted: error=expect(locator).toBeVisible()
filename=script.k6 line=48 expected=visible received=hidden
```

Start with the locator and the action leading to line 48. Reaching the application doesn’t establish that the expected page element appeared.

## Compare probe executions

1. Keep the affected time point selected and record the failing probe’s operation and error.
2. Select another probe’s tab in the same time point. Record whether it succeeded, failed, or has no reported result. If it failed, identify its operation and error using the same steps.
3. Repeat for the remaining probes. Compare the resolved addresses as well as the error messages when those details are available.
4. If no probe succeeded, use **Previous timepoint** to look for an earlier successful execution from the same probe. Record both timestamps so you can identify what changed between them. If the selected period has no successful executions, continue with the failure evidence you have.

Use the comparison to decide whom to involve and what evidence to provide:

- **Some locations succeed and others fail:** Ask the team that manages the application’s network or security rules to search firewall, web application firewall (WAF), or content delivery network (CDN) logs for the affected time and public probe source addresses. Use the current [Synthetic Monitoring source IP ranges](/docs/grafana-cloud/platform/security-and-account-management/security-and-access/allow-list/#synthetic-monitoring). Check for a matching denial or challenge before changing an access rule. For private probes, use the source address the application or server sees from that probe’s network. A successful request from your laptop doesn’t test the same network path.
- **Locations fail at the same operation:** Send the operation, resolved addresses, timestamps, and errors to the application or network team. Ask them to check the corresponding listener, request, or access rule. Matching errors narrow the investigation but don’t prove a shared cause.
- **A script or assertion starts failing after a change:** Compare the failing line and expected result with the application change and the check’s configuration. If you see a check-update annotation, record its timestamp and ask the check owner to review what changed. Don’t weaken the assertion solely to make the check succeed.

The dashboard’s **Uptime** and **Reachability** panels help summarize this scope. High uptime with lower reachability can reflect successes from some probes and failures from others. With one probe, the two indicators don’t provide a comparison between locations. For calculations and examples, refer to [Uptime and reachability](/docs/grafana-cloud/observe-and-act/testing/synthetic-monitoring/analyze-results/uptime-and-reachability/).

## Inspect a complete execution in Explore

Use Explore when you need to filter the logs or share a reproducible query:

1. Select the relevant time point and probe in the execution details, then click **View Logs in Explore**, the log icon beside the time point navigation arrows. Explore opens with the log data source, check labels, probe, and time range populated.
2. Compare Explore’s time range with the execution’s start and finish timestamps. If the execution finishes in the next time point, widen the range to include its finish. Use **Zoom out time range**, or set a range with the time picker.
3. In **Code**, keep the generated `job`, `instance`, and `probe` filters. If the logs expose an `execution_id` field, append a filter for the **Execution ID** you recorded:
   
   logql ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy
   
   ```logql
   {job="YOUR_JOB", instance="YOUR_INSTANCE", probe="YOUR_PROBE"}
     | logfmt
     | execution_id="YOUR_EXECUTION_ID"
   ```
4. Copy the `job`, `instance`, and `probe` values from the generated query, replace `YOUR_EXECUTION_ID` with the ID you recorded, and click **Run query**. Keep the generated `instance` value even if it isn’t a URL; scripted and browser checks can use a descriptive value.
5. Confirm that the returned entries include the execution’s beginning and final result. If they don’t, check the time range and filter values before concluding that logs are missing. If `execution_id` isn’t available, retain the generated label filters and use the timestamps to identify the relevant entries.

Executions belong to the time point in which they start, but their final log messages can be timestamped in a later interval. A query restricted to the starting interval can therefore show an incomplete execution or messages from a preceding execution.

## Investigate missing results

A missing result doesn’t report whether the endpoint you’re monitoring was available. Investigate the data gap separately from a reported failure:

1. Use **Previous timepoint** to select an earlier interval that should have finished. If the earlier execution has results, distinguish that completed execution from the recent one still waiting for results.
2. Check the selected period against the check’s creation time and any no-data annotations. Move to a period after creation and within the available retention period if necessary.
3. Select another probe’s tab to find whether the gap affects one probe or several.
4. If logs are available, follow [Inspect a complete execution in Explore](#inspect-a-complete-execution-in-explore) to check the full execution interval. A gap in one view doesn’t establish that both logs and metrics are absent.
5. If other checks use the affected probe, open their dashboards from **Checks**, select the same period, and inspect that probe’s results. Missing results across several checks can indicate a wider probe or reporting issue rather than a gap limited to one check.
6. If completed intervals still have no results, [collect the evidence](#verify-recovery-or-escalate) for your Grafana administrator or Grafana support. Include which probes and data are missing. The execution state alone doesn’t identify the cause, so don’t reset the plugin or change data sources based only on that state.

Uptime excludes time points with no reported results. A high uptime value therefore doesn’t establish availability during a gap. For an example of interrupted result reporting, refer to [A visual example and known limitations](/docs/grafana-cloud/observe-and-act/testing/synthetic-monitoring/analyze-results/uptime-and-reachability/#a-visual-example-and-known-limitations).

## Verify recovery or escalate

After addressing the suspected cause, return to the check dashboard and select a time range after the change. Keep **probe** set to **All**, then inspect new completed executions from each previously affected probe. Confirm that the intended request or journey succeeds under the check’s success criteria.

If you need another team or Grafana support to continue the investigation, provide:

- The check’s job name, monitored endpoint, and check type.
- The affected time range and time zone.
- The failing and successful probes, with **Execution ID** values where available.
- The specific operation, error message, and script line or resolved endpoint IP address where applicable.
- Relevant log entries or an Explore query with the full execution interval.
- Any check or application changes near the start of the problem.
- For a data gap, which probes have missing results, whether logs, metrics, or both are affected, and whether other checks using those probes have the same gap.