Service Discovery: monitoring Docker containers that move

/ / / / /

Docker is being adopted rapidly, and for good reason: it simplifies many aspects of running a service in production. But Docker-powered services typically run many more containers than traditional services run hosts, so monitoring is much more complex. Now with platforms like Kubernetes and ECS orchestrating your containers, you may not even know which host your containers are running on—this makes monitoring your services even more complex.

How can you monitor a service which is running on a shifting set of hosts?

Datadog was built to monitor modern, dynamic infrastructure, and it automatically keeps track of what is running where, thanks to its Service Discovery feature.

Whether you use Kubernetes, Amazon ECS, or Docker Swarm to orchestrate your Docker containers, you can now continuously monitor your Dockerized infrastructure without interruption even as it expands, contracts, and shifts across hosts.

What sorcery is this?

The Service Discovery feature is continuously listening to Docker events. Whenever a container is created or started, the Agent identifies which service is running in the new container, looks for a custom monitoring configuration, and starts collecting and reporting metrics. Whenever a container is stopped or destroyed, the Agent understands that too.

If no configuration is defined for the service, the Agent will attempt auto-configuration. This works for basic checks like Apache or Redis among others, which have a simple configuration and do not require custom credentials.

I want it! What do I need to do?

To use Service Discovery, you only need to do two things. First, define the configuration templates for the images you want to monitor in etcd, Consul, or locally in /etc/dd-agent/conf.d/autoconf/–support for other storage backends coming soon.

Here is the structure of a configuration template:

/datadog/
check_configs/
docker_image_0/
- check_names: ["check_name_0"]
- init_configs: [{init_config}]
- instances: [{instance_config}]
docker_image_1/
- check_names: ["check_name_1"]
- init_configs: [{init_config}]
- instances: [{instance_config}]
...

NOTE: Our documentation provides an example of setting up the configuration templates for NGINX monitoring.

Second, configure your Datadog Agents to run Service Discovery. To do so, simply update the parameters in your datadog.conf file. For example:

service_discovery_backend: docker
sd_config_backend: etcd
sd_backend_host: 127.0.0.1
sd_backend_port: 4001

That’s it! Easy, right?

Use variables and the Agent takes care of the rest

Since your container orchestration service takes care of scheduling, you probably don’t know which host IP address or port number the Agent needs to connect to. That’s why our configuration templates support variables that the Agent will automatically replace with information provided by the Docker API: host IP address (%%host%%) and port number (%%port%%) can be used.

Kubernetes users can also use tags (%%tags%%) collected from its API and which represent metadata like pod_name, node_name, labels

Try it out

It is now easier than ever to monitor highly dynamic infrastructure!

If you are already a Datadog user, you can start using Service Discovery right now. Otherwise you can try it out by signing up for a .

Datadog Agent image on the Docker Store

Starting immediately you can get the Datadog Agent image—which includes the new Service Discovery feature—directly from the brand new Docker Store. In one step you’ll be able to pull the image and immediately begin monitoring your hosts and containerized services, however dynamic or high-scale your infrastructure may be. Docker is a great partner to work with and we’re thrilled to be featured on the store at launch.


Want to write articles like this one? Our team is hiring!
Service Discovery: monitoring Docker containers that move