Monitor Amazon EKS on AWS Fargate with Datadog

AWS Fargate has steadily gained traction in Amazon Elastic Container Service (ECS) environments because it allows users to run containerized applications without thinking about their underlying infrastructure. Today, AWS announced that support for Amazon Elastic Kubernetes Service (EKS) on AWS Fargate is now generally available, giving Amazon EKS users the option to seamlessly manage their infrastructure with AWS Fargate instead of manually provisioning EC2 worker nodes.

Datadog is pleased to work with AWS for the launch of Amazon EKS on AWS Fargate, so you can automatically collect metrics and get deep visibility into your environment. This integration also includes support for Autodiscovery, so the Datadog Agent can immediately detect applications running in your cluster and collect monitoring data from them. You can also configure Datadog APM to collect distributed traces from applications running on Amazon EKS to monitor their performance in real time.

In this post, we’ll show you an example of how you can deploy the Datadog Agent to get visibility into an application that runs on Amazon EKS using AWS Fargate.

Deploy Datadog on Amazon EKS on AWS Fargate

AWS Fargate abstracts away the underlying infrastructure of Amazon EKS and provides on-demand compute capacity for containers. So instead of deploying the Datadog Agent to your nodes, as you would in a regular Kubernetes cluster, you’ll need to run the Datadog Agent as a sidecar container on each pod to ensure that all your pods are monitored. If you’re running a mixed Amazon EKS cluster, with some pods running on AWS Fargate and some pods running on Amazon EC2 instances, you should still deploy the Agent as a DaemonSet to the EC2 instances.

You will also need to set up role-based access control (RBAC) in Amazon EKS so the Datadog Agent can query the Kubernetes API for monitoring data; see the documentation for full details on setting up the Datadog service account, ClusterRole, and ClusterRoleBinding. If your application also utilizes RBAC, you’ll need to ensure that the Agent container and your application containers have all the permissions they need, for example by creating a ClusterRoleBinding that links your application’s service account with the Agent’s ClusterRole, or by adding the Datadog Agent’s required permissions to the ClusterRole associated with your application’s service account.

To deploy Datadog in Amazon EKS on AWS Fargate, define three environment variables in your deployment manifest:

• DD_API_KEY: your Datadog API key
• DD_EKS_FARGATE: set this to true
• DD_KUBERNETES_KUBELET_NODENAME: set this as specified in the example below.

The following example will deploy the containerized Datadog Agent as a sidecar in the same pod as a Redis container.

apiVersion: apps/v1
kind: Deployment
metadata:
 name: redis
 namespace: default
spec:
 replicas: 1
 template:
   metadata:
     labels:
       app: redis
     name: redis
     annotations:
       ad.datadoghq.com/redis.check_names: '["redisdb"]'
       ad.datadoghq.com/redis.init_configs: '[{}]'
       ad.datadoghq.com/redis.instances: |
         [
           {
             "host": "%%host%%",
             "port": "6379"
           }
         ]         
   spec:
     serviceAccountName: datadog-agent
     containers:
     - name: redis
       image: redis:latest
       args:
         - "redis-server"
       ports:
       - containerPort: 6379
     - image: datadog/agent
       name: datadog-agent
       env:
       - name: DD_API_KEY
         value: "<YOUR_DATADOG_API_KEY>"
       - name: DD_TAGS
         value: "[clustername:my-eks-fargate-cluster]"
       - name: DD_EKS_FARGATE
         value: "true"
       - name: DD_KUBERNETES_KUBELET_NODENAME
         valueFrom:
           fieldRef:
             apiVersion: v1
             fieldPath: spec.nodeName

In this example, we’ve added pod annotations to configure Autodiscovery, which means the Agent will automatically run a check on the Redis container in the same pod.

You can also deploy Datadog’s Cluster Agent on Amazon EKS on AWS Fargate by following the steps in our documentation. The Cluster Agent will run on a single pod and collect events from the API server. It will also complete cluster checks if you’ve configured any (e.g., if you’d like to run an HTTP check to monitor the latency of an NGINX service).

To monitor any of the AWS services you’re running alongside Amazon EKS on AWS Fargate, such as Application Load Balancer, make sure to enable our AWS integration if you haven’t done so already.

Monitor Amazon EKS on AWS Fargate with Datadog

Once you’ve deployed the Agent to your application running on Amazon EKS on AWS Fargate, you should see Kubernetes metrics appear in Datadog, along with data from services detected via Autodiscovery (e.g., Redis in the example above). Although Datadog can collect host-level metrics from any EKS nodes that aren’t managed by Fargate, you won’t see any system-level metrics from any of your Fargate-provisioned “hosts” because AWS manages that infrastructure for you.

This integration includes standard Kubernetes and Docker tags (e.g., docker_image, pod_name, kube_deployment) that you can use to filter for any subset of your container infrastructure. If you included a DD_TAGS environment variable in your manifest, you will also be able to leverage any custom tags specified there.

Deep visibility, right out of the gate

With Amazon EKS on AWS Fargate, teams can spend more time developing their container applications, and less time managing the underlying infrastructure. To learn more about running Amazon EKS pods on AWS Fargate, check out the official documentation. And see our in-depth guides for Amazon EKS monitoring and AWS Fargate monitoring for more information.

We’re pleased to provide real-time visibility into your dynamic container environments, regardless of how they’re managed or where they’re running. If you’re already a Datadog customer, consult our documentation to learn how to get started. Otherwise, sign up for a free trial.