---
title: "Run Synthetic tests in your CI/CD pipelines with the Datadog CircleCI orb"
description: "Automate Synthetic testing and monitor the results in your CI/CD pipelines, preempting issues arising from faulty code deployments."
author: "Nicholas Thomson, Beth Glenfield"
date: 2022-05-05
tags: ["synthetic monitoring", "circleci", "testing", "digital experience monitoring", "ci/cd"]
blog_type_id: the-monitor
locale: en
---

[CircleCI](https://www.datadoghq.com/blog/circleci-monitoring-datadog.md) is a CI/CD service that allows organizations to rapidly build, test, and deploy within their pipelines on a single platform. If you are using CircleCI for your CI/CD pipelines, you can now leverage the [Datadog Synthetics CircleCI orb](https://circleci.com/developer/orbs/orb/datadog/synthetics-ci-orb) to implement [Synthetic tests](https://www.datadoghq.com/knowledge-center/synthetic-testing/) as part of [shift-left testing](https://www.datadoghq.com/blog/shift-left-testing-best-practices.md). [CI/CD testing](https://www.datadoghq.com/blog/datadog-synthetic-ci-cd-testing.md) is a widely adopted DevOps standard that helps teams mitigate any potential issues that could arise as a result of faulty code deployments.

[Datadog Synthetic Monitoring](https://docs.datadoghq.com/synthetics.md) enables you to implement a suite of end-to-end tests in your CI/CD pipelines, so you can automatically detect and block the deployment of code changes that would break key workflows or endpoints. The Datadog CircleCI orb enables you to implement these Synthetic tests in your CircleCI pipelines, surfacing issues that you can then [diagnose and debug](https://www.datadoghq.com/blog/datadog-synthetic-ci-cd-testing.md#shift-system-wide-troubleshooting-to-the-left) with Datadog. Adding Synthetic tests to your CircleCI pipelines will help you ensure that new code deployments are reliable, preempting any potential problems for your end users.

In this post, we’ll show you how to [set up the CircleCI orb to run Synthetic tests](#set-up-the-datadog-circleci-orb-to-run-synthetic-tests), [debug failing Synthetic tests](#debug-failing-synthetic-tests), and [monitor CircleCI pipelines together with Datadog’s existing CircleCI integration](#improve-the-reliability-and-efficiency-of-your-circleci-pipelines-with-datadog).

## Set up the Datadog CircleCI orb to run Synthetic tests

A CircleCI [orb](https://circleci.com/orbs/) is an open source, reusable config file for automating processes, expediting project startup, and easing integration between your CircleCI pipelines and services across the stack. To get started, simply add your Datadog [API and application keys](https://docs.datadoghq.com/account_management/api-app-keys.md) as environment variables to your CircleCI project. You can then start [configuring the Datadog CircleCI orb](https://docs.datadoghq.com/synthetics/cicd_integrations/circleci_orb.md) to execute Datadog Synthetic tests in your pipelines.

To make sure that code deployments don’t introduce unexpected issues for your end users, you can implement a suite of Synthetic end-to-end tests to confirm that your application is functioning as expected. For example, the CircleCI pipeline configuration below utilizes the Synthetics CircleCI orb.

```yaml

version: 2.1

orbs:
  synthetics-ci: datadog/synthetics-ci-orb@1.0.1

jobs:
  e2e-tests:
    docker: 
      - image: cimg/base:stable
    steps:
      - synthetics-ci/run-tests:
          public_ids: 'abc-d3f-ghi, jkl-mn0-pqr'

workflows:
  run-tests:
    jobs:
      - e2e-tests


```

The above configuration will automatically trigger the Synthetic tests associated with the specified `public_ids` every time you deploy code changes. You can configure the orb to block the deployment if the tests fail using a [config override](https://docs.datadoghq.com/synthetics/cicd_integrations/configuration.md?tab=npm) or through the [Synthetics UI](https://www.datadoghq.com/blog/datadog-synthetic-ci-cd-testing.md#run-synthetic-tests-in-your-ci-pipeline-for-earlier-issue-detection). The results of your Synthetic tests will appear in your CircleCI pipeline (as shown in the example below).

![View your Synthetic test executions with CircleCI](https://web-assets.dd-static.net/42588/1776358622-run-synthetic-tests-in-circeci-pipelines-with-datadog-circle-ci-workflow.png)

If you want to run tests on applications that are not publicly available (e.g., internal applications), you can also configure local testing by [starting a local server and triggering Synthetic tests using the `datadog-ci` tunnel](https://docs.datadoghq.com/synthetics/testing_tunnel.md) (as shown in the example below).

```yaml
version: 2.1

orbs:
  synthetics-ci: datadog/synthetics-ci-orb@1.0.1

jobs:
  e2e-tests:
    docker:
      - image: your-image
    steps:
      - checkout
      - run:
          name: Running server in background
          command: npm start
          background: true
      - synthetics-ci/run-tests:
          config_path: tests/tunnel-config.json
          files: tests/*.synthetics.json
          test_search_query: 'tag:e2e-tests'
          tunnel: true

workflows:
  test-server:
    jobs:
      - build-image
      - integration-tests:
          requires:
            - build-image

```

By virtue of the orb’s reusability, you can leverage the same [unique test suite](https://www.datadoghq.com/blog/test-maintenance-best-practices.md#leverage-a-unique-test-suite-across-environments) in development, staging, and production to simplify and standardize your testing across different environments. For example, you can configure your tests to automatically run in your `dev` environment after they’ve passed locally to ensure that new features work properly before deployment. Simply [override](https://docs.datadoghq.com/synthetics/cicd_integrations/configuration.md?tab=npm) the `startURL` in your global config file so that it points to your development environment.

## Debug failing Synthetic tests

[Datadog Synthetic Monitoring](https://www.datadoghq.com/product/synthetic-monitoring/) integrates with [APM](https://www.datadoghq.com/product/apm/) to offer invaluable context for troubleshooting failing tests (e.g., duration, device, and browser) alongside metrics, logs, and other data. For example, you can inspect a test's correlated [trace](https://docs.datadoghq.com/tracing.md) to learn more about why it failed. In this case, the trace shows you that the request to your API returned a 404 error. Upon further investigation, you find that this request was being transmitted to a recently deprecated client library, so you may be able to resolve this issue by switching to a different provider.

![Troubleshoot failing tests with Datadog Synthetic Monitoring](https://web-assets.dd-static.net/42588/1776358630-run-synthetic-tests-in-circeci-pipelines-with-datadog-synthetic-monitoring.png)

## Improve the reliability and efficiency of your CircleCI pipelines with Datadog

Using the Datadog CircleCI orb in conjunction with Datadog’s [CircleCI integration](https://www.datadoghq.com/blog/circleci-monitoring-datadog.md) allows you to monitor your CircleCI pipeline performance. In addition to running Synthetic tests in CircleCI, you can also use Datadog [CI Visibility](https://www.datadoghq.com/product/ci-cd-monitoring/) to optimize your pipelines. Once you've [instrumented your CircleCI pipelines](https://app.datadoghq.com/ci/setup/pipeline?provider=circleci) with CI Visibility, health and performance metrics will automatically stream into Datadog. You can view the results of your jobs in the [CI Results Explorer](https://app.datadoghq.com/synthetics/explorer/ci) and investigate problematic jobs to see where errors and bottlenecks might be occurring.

Each job contains a [batch](https://docs.datadoghq.com/synthetics/ci_results_explorer.md#explore-test-batches) of test IDs specified by the orb. Once Datadog has run the tests, you can click on a batch of results—tagged with useful metadata, including status, branch, and duration—to pinpoint the specific test(s) that failed. As in the image below, Datadog itemizes the test results from the orb’s execution, allowing you to compare the results of individual test runs across different browsers, devices, and locations.

![View job results in the CI Results Explorer](https://web-assets.dd-static.net/42588/1776358637-run-synthetic-tests-in-circeci-pipelines-with-datadog-ci-results-explorer.png)

CI Visibility shows you which pipelines are the slowest or most error-prone so you can take action to improve the reliability of your tests and builds. Additionally, CI Visibility automatically detects flaky tests—another common source of failing jobs. As in the example below, you can inspect the erroneous span to get more helpful context for troubleshooting the failure.

![Leverage CI visibility to troubleshoot problematic tests](https://web-assets.dd-static.net/42588/1776358641-run-synthetic-tests-in-circeci-pipelines-with-datadog-ci-visibility.png)

## Test with the CircleCI orb and monitor your pipelines with Datadog

[The Datadog CircleCI orb](https://circleci.com/developer/orbs/orb/datadog/synthetics-ci-orb) enables you to easily incorporate Synthetic tests into your CircleCI pipelines, allowing all of your development teams to quickly detect issues before they degrade your user experience. This integration extends our existing support for Synthetic Monitoring in your CI pipelines, complementing the [Datadog GitHub Action](https://www.datadoghq.com/blog/datadog-github-action-synthetics-ci-visibility.md) and [Datadog plugin for Jenkins](https://www.datadoghq.com/blog/jenkins-testing.md). CI Visibility also provides rich context around the health of your CircleCI pipelines, enabling you to optimize your CI/CD workflows. If you’re new to Datadog, sign up for a 14-day <!-- Sign-up trigger (free trial) omitted -->.

-a