Implement monitoring as code with Datadog and CloudFormation Registry | Datadog
Network Performance Monitoring is now generally available! Network Monitoring is now available!

Implement monitoring as code with Datadog and CloudFormation Registry

Author Mallory Mooney

Published: November 18, 2019

AWS CloudFormation is a service that enables you to build infrastructure as code, similar to Terraform. You can create CloudFormation templates to provision and manage all of the resources for your stacks, such as EC2 instances, load balancers, and security groups. These templates automate the process of building infrastructure, creating repeatable steps that you can easily check into version control. This ensures that your configurations do not drift with each new environment you spin up.

You can already use CloudFormation to automate steps such as installing the Datadog Agent on your instances, but Datadog has partnered with AWS CloudFormation Registry to create even more resources for your templates. This enables you to build comprehensive monitoring as code for the infrastructure you’re deploying, and get real-time visibility into all the applications you’re managing with CloudFormation. You can use Datadog’s resources to automatically:

  • enable Datadog’s AWS integration
  • create, update, and delete monitors for your services
  • schedule downtime for monitors
  • manage users for your Datadog account

Below, we’ll walk through a few examples of how you can start using Datadog’s CloudFormation resources to build a reliable, repeatable process for monitoring your infrastructure in real time.

Get started with Datadog’s CloudFormation resources

You can use the AWS CLI to register Datadog’s resources to your account:

aws cloudformation register-type \
    --region <REGION> \
    --type RESOURCE \
    --type-name "<DATADOG_RESOURCE_NAME>" \
    --schema-handler-package <LINK_TO_S3>

Once you register these resources, you can incorporate them into your new and existing CloudFormation templates, whether you’re building them from scratch or using CloudFormation’s template designer.

Automatically enable Datadog’s AWS integration

To start monitoring the resources in your CloudFormation stacks, you can add Datadog’s AWS Integration resource to your CloudFormation templates to automatically enable Datadog’s AWS integration:

cloudformation-aws-template.yaml

Resources:
  DatadogAWSIntegrationResource:
    Type: 'Datadog::Integrations::AWS'
    Properties:
      AccountID: <AWS_ACCOUNT_ID>
      RoleName: DatadogAWSIntegrationRole
      HostTags: ["env:staging", "team:devops"]
      AccountSpecificNamespaceRules: {"ec2": true, "api_gateway": false}
      DatadogCredentials:
        ApiKey: <DD_API_KEY>
        ApplicationKey: <DD_APP_KEY>

This example assumes that you’ve configured role delegation using AWS IAM, so the Datadog role (e.g., DatadogAWSIntegrationRole) has read-only access to your AWS account. You can use this resource in templates to automatically enable the AWS integration, configure your account ID, tags, new role, and any namespace rules such as enabling metric collection for a specific integration (e.g., EC2). You can check out the resource’s documentation for examples and a list of available properties. CloudFormation templates also support dynamic references so you can store your keys in a service like AWS Secrets Manager.

Create alerts for your resources

With Datadog’s Monitor resource, you can quickly create new alerts for your applications, or update and delete existing alerts. For example, you can create an alert that notifies you when an EC2 instance goes down in the us-east-1 region by adding the following to your CloudFormation template:

cloudformation-monitor-template.yaml

Resources:
  DatadogMonitorResource:
    Type: 'Datadog::Monitors::Monitor'
    Properties:
      Type: service check
      Query: '"aws.ec2.host_status".over("region:us-east-1").by("host").last(4).count_by_status()'
      Name: EC2 Uptime/Availability
      Message: "An EC2 instance in the us-east-1 region is offline."
      DatadogCredentials:
        ApiKey: <DD_API_KEY>
        ApplicationKey: <DD_APP_KEY>

You can use Datadog’s Monitor resource to create alerts for any application metric, not just AWS metrics. This enables you to instantly create alerts for every service in your infrastructure.

Datadog’s Downtime resource allows you to schedule downtime for your monitors if you need to mute alert notifications (e.g., during maintenance windows), as shown in the example below.

cloudformation-downtime.yaml

 
Resources:
  DatadogDowntimeUntilDate:
    Type: 'Datadog::Monitors::Downtime'
    Properties:
      Message: "Instances in the us-east-1 region will be offline for weekend maintenance. Monitoring notifications will be suspended from 10/18/2019 9:00PM to 10/19/2019 12:00PM."
      MonitorId: <DATADOG_MONITOR_ID>
      Scope: ["*"]
      Start: 1571432400
      End: 1571486400
      Timezone: "EST"
      DatadogCredentials:
        ApiKey: <DD_API_KEY>
        ApplicationKey: <DD_APP_KEY>

This configures CloudFormation to automatically create a new downtime schedule in your Datadog account, as seen below. You can find more examples and a list of available properties for the Downtime resource in our documentation.

Create a new downtime schedule

These resources enable you to immediately begin alerting on potential issues in your environment—and give you a way to automatically configure downtime for alerts. This helps limit gaps in coverage in newly deployed infrastructure components and eliminates the need to manually set up alerts.

Monitoring as code with Datadog and CloudFormation

With Datadog and CloudFormation, you can create repeatable steps for provisioning and setting up monitoring for all of your resources. These resources build upon our existing support for CloudFormation so you can automate more of the setup process, including installing the Datadog Agent to collect metrics and logs from your instances. Check out our documentation to learn more about how you can use Datadog and CloudFormation to automatically deploy, manage, and monitor your stacks. If you don’t already have a Datadog account, you can sign up for a .