Deploy Datadog on AWS Elastic Beanstalk
Editor’s note (August 2018): Some of the commands below are specific to Agent 5. See our documentation for more details about this integration. The steps below have been modified for Agent 6 by Datadog user Wim Deblauwe and are available on his blog.
At SolveBio we use AWS Elastic Beanstalk to manage some of our apps on EC2. Our API for biomedical data is critical for running diagnostic tests, so we like to keep an eye on its performance. We started using Datadog a few weeks ago to help us out with monitoring and metrics, but we ran into some issues getting it to play nicely with Elastic Beanstalk. This article describes how we got it all working together.
How SolveBio uses Elastic Beanstalk
Similar to Heroku, Elastic Beanstalk makes it super easy to deploy various web apps to EC2. At SolveBio, we mostly work with Python and Node.js, so the method described here focuses on those stacks. The Datadog agent is a daemon managed by
supervisord, which sends system metrics to Datadog but also works as a
One kink we initially encountered was that the Datadog agent and Elastic Beanstalk rely on separate installations of
supervisord. This confuses the heck out of Elastic Beanstalk. Luckily, getting past this kink is simple. You just have to tell Elastic Beanstalk to stop Datadog before deploying and start it again when you’re done.
Deploy the Datadog agent on Elastic Beanstalk
Once your project is all set up for Elastic Beanstalk, create the
.ebextensions/datadog/hooks directories, and pop in the following files:
# .ebextensions/99datadog.config container_commands: 01chmod: command: "chmod +x .ebextensions/datadog/hooks/*" 02mkdir_appdeploy_post: test: '[ ! -d /opt/elasticbeanstalk/hooks/appdeploy/post ]' command: "mkdir /opt/elasticbeanstalk/hooks/appdeploy/post" 02mkdir_configdeploy_post: test: '[ ! -d /opt/elasticbeanstalk/hooks/configdeploy/post ]' command: "mkdir /opt/elasticbeanstalk/hooks/configdeploy/post" 10appdeploy_pre_stop: command: "cp .ebextensions/datadog/hooks/99stop_datadog.sh /opt/elasticbeanstalk/hooks/appdeploy/pre/" 11appdeploy_post_start: command: "cp .ebextensions/datadog/hooks/99start_datadog.sh /opt/elasticbeanstalk/hooks/appdeploy/post/" 20preinit_stop: command: "cp .ebextensions/datadog/hooks/99stop_datadog.sh /opt/elasticbeanstalk/hooks/preinit" 21postinit_start: command: "cp .ebextensions/datadog/hooks/99start_datadog.sh /opt/elasticbeanstalk/hooks/postinit" 30configdeploy_pre_stop: command: "cp .ebextensions/datadog/hooks/99stop_datadog.sh /opt/elasticbeanstalk/hooks/configdeploy/pre/" 31configdeploy_post_start: command: "cp .ebextensions/datadog/hooks/99start_datadog.sh /opt/elasticbeanstalk/hooks/configdeploy/post/" 90install_datadog: command: "cp .ebextensions/datadog/datadog.repo /etc/yum.repos.d/datadog.repo; yum -y makecache; yum -y install datadog-agent; /etc/init.d/datadog-agent stop" 91setup_datadog: test: '[ ! -e /etc/dd-agent/datadog.conf ]' command: "sh -c \"sed 's/api_key:.*/api_key: YOUR_API_KEY/' /etc/dd-agent/datadog.conf.example > /etc/dd-agent/datadog.conf\""
[datadog] name = Datadog, Inc. baseurl = https://yum.datadoghq.com/rpm/x86_64/ enabled=1 gpgcheck=1 gpgkey=https://yum.datadoghq.com/DATADOG_RPM_KEY.public
#!/bin/bash # .ebextensions/datadog/hooks/99start_datadog.sh if [ -e /etc/init.d/datadog-agent ]; then /etc/init.d/datadog-agent start fi
#!/bin/bash # .ebextensions/datadog/hooks/99stop_datadog.sh if [ -e /etc/init.d/datadog-agent ]; then /etc/init.d/datadog-agent stop fi
Once all the files are in there, commit the changes and then deploy your app using
git aws.push. The agent should be installed once the update finishes, and it should also keep working after further deploys to your Elastic Beanstalk instances. We found it helpful to enable SSH login to our instances for debugging.
Now the Elastic Beanstalk monitoring fun begins
Now that we’re up and running, monitoring AWS Elastic Beanstalk with Datadog has already proven to be very helpful. With that said, we’re still getting acquainted with Datadog so I’ll be back in a few weeks to share my thoughts on key AWS Elastic Beanstalk metrics to monitor and any tips and tricks or best practices I’ve gained. In the meantime, if you’d like to gain visibility into your AWS Elastic Beanstalk performance you can try Datadog for free for 14 days.