Docker-ize Datadog with agent containers
Docker is an exciting technology that offers a different approach to building and running applications thanks to a clever combination of linux containers (good for ops) and a git-like approach to packaging software (good for dev) so that your containers have everything they need to run without dependencies.
Many of you who are using Docker are embracing the Docker way and taking a container-only approach. As we release our new Docker integration, we don’t want to force you to break from a container-only strategy because of the traditional Datadog agent architecture. Instead, we’ve also embraced the Docker way and we’re pleased to announce a Docker-ized Datadog agent deployed in a container.
The Docker philosophy
First, a brief introduction on how infrastructure is set up with Docker. In Docker, each of your applications is isolated in its own container. The blueprint for a container is its DockerFile which is a set of steps to create the container. These steps build the standard binaries and libraries and install your application’s code and its dependencies such as Python, Redis, Postgres, etc.
The Docker engine then creates the actual container to run using namespaces and cgroups. These are two features found in recent versions of the Linux kernel used to isolate system calls and resource usage (CPU, memory, disk I/O, etc.) directly on your server. The end result is multiple containers on the server with each application thinking it is in its own machine by itself, without the overhead associated with fully virtualized machines.
The traditional Datadog set-up
Until Docker arrived, applications were built in virtual servers or directly on raw servers. In this case, you install the agent on your server and decide what applications and services you want to monitor in Datadog. If you want to send custom metrics to Datadog, you instrument your application with our Datadog version of StatsD, called DogStatsD. This set-up is illustrated below.
The traditional Datadog set-up in the Docker environment means the Datadog agent runs next to the Docker engine.
Datadog the Docker Way
Because the Docker philosophy is to isolate applications to a container, we have built a “Docker-ized” installation of the Datadog agent. We have isolated the agent into two kinds of Docker containers. Both of the container installations can be illustrated by the diagram below.
The first container includes the Datadog agent plus DogStatsD. The Datadog agent is responsible for sending us both native host and container-specific metrics, like number of containers, load, memory, disk usage, and latency. DogStatsD will send us custom metrics you have instrumented in containerized applications. Again, you can read more about what exactly Datadog monitors in Docker in our Monitor Docker performance with Datadog post.
If you only want to monitor custom metrics in containerized applications, the other Datadog container isolates DogstatsD so that you can send us custom metrics to monitor.
For detailed documentation on how to install the Docker-ized Datadog containers, please visit our Docker installation guide.
As mentioned in the Monitor Docker with Datadog post, if you would like to alert on and visualize Docker metrics, you can sign-up for a 14-day free trial of Datadog. Docker metrics will be available immediately after installing the Datadog agent in its traditional format or as a container.