Introducing a New Buildpack for Monitoring Heroku Applications | Datadog

Introducing a new buildpack for monitoring Heroku applications

Author Jason Yee

Published: June 7, 2018

Heroku is a popular platform-as-a-service that simplifies application deployment by automating and abstracting much of the underlying infrastructure required to run those applications. One of those abstractions is Heroku’s buildpack. Buildpacks are discrete, chainable bundles of code that are often used to modify the runtime environment for the application that will be deployed. Rather than performing complex installation and configuration tasks for a particular application or service, users can simply add the buildpack to their project.

We’re pleased to share a new Heroku buildpack that installs the latest version of the Datadog Agent (version 6), with added support for distributed tracing and APM in Heroku.

Datadog hostmap with Heroku hosts

Building the buildpack

In 2014, Mike Fiedler wrote the first Heroku buildpack to run a Python-based DogStatsD server, enabling applications on Heroku to send custom metrics to Datadog. As we developed version 6 of the Datadog Agent to be a pre-compiled Go binary rather than a package of Python scripts, it became clear that the buildpack would need to be rewritten.

Heroku uses a container model to package, deploy, and scale applications. Unlike some container-based platforms, however, Heroku does not allow root access or the ability to write to standard application and configuration directories such as /usr/bin or /etc. This limitation ensures that a slug (Heroku’s term for a container image) deployed to a dyno (a running container) remains isolated from the host machine and other dyno tenants sharing the host. However, this security isolation comes at the cost of some convenience.

The new buildpack and Mike’s original buildpack take a similar approach to the limited-access file system, by downloading and installing the Datadog Agent to a non-standard but accessible directory.

Installing the buildpack

Installing the buildpack is simple. From your Heroku project, first enable Heroku Labs dyno metadata:

heroku labs:enable runtime-dyno-metadata -a $(heroku apps:info|grep ===|cut -d' ' -f2)

Next add the buildpack and provide your Datadog API key as an environment variable:

heroku buildpacks:add --index 1
heroku config:add DD_API_KEY=<your API key>

Finally push your code to Heroku to generate a new slug with the buildpack.

For more information and configuration options, reference the documentation.

Visualizing your Heroku metrics

The Heroku buildpack automatically adds tags for your application name (appname), the dyno name, and the dyno type. The application name and dyno name are particularly helpful for filtering and aggregating metrics. Summing a particular metric by the provided dyno name tag provides visibility into the dyno’s aggregate activity and performance, similar to graphing metrics on a per-host basis when monitoring physical hosts. This helps provide better monitoring continuity as dynos move to new hosts.

You can also override the hostname to be a combination of the application name and dyno name. If your application cycles dynos frequently, it can lead to a high number of reported hosts in Datadog, which can increase your costs. Setting the DD_DYNO_HOST environment variable to true will cause the buildpack to report your hostname in the more stable form my-app-name.dyno-name rather than relying on the name of the ephemeral host.

What’s next?

By default, the Datadog Agent collects host system metrics including CPU, memory, and disk metrics. System metrics at the level of individual dynos are not available at this time, but we’re investigating the possibility of collecting them using the buildpack.

If you require individual dyno metrics today, you can use the Heroku Datadog Drain, which was created by our friends at Apiary and uses a Log Drain.

We’re also exploring ways to support log collection in the buildpack, so stay tuned!

Give it a try

Heroku is a fantastic service that simplifies deploying applications to a robust and scalable platform. The Datadog buildpack makes it easier to monitor those applications with custom application metrics and distributed tracing. If you’re already a Datadog customer, you can install and deploy the buildpack to monitor your Heroku applications today. If you’re not yet using Datadog, .