The Service Map for APM is here!
Introducing Datadog for serverless

Introducing Datadog for serverless

/ / / /
Published: November 19, 2018

To make serverless architectures more observable, we’re excited to introduce the new Cloud Functions view in Datadog. You can now search, filter, and explore all your AWS Lambda functions in one central view, and dive straight into detailed performance data that is scoped to a single function. The Cloud Functions view brings together Lambda metrics and logs with distributed request traces from your functions, which are now available in Datadog thanks to our new integration with AWS X-Ray.

With these additions to the Datadog platform, all the data you need for troubleshooting serverless functions is now available in one place. When the error rate or latency for one of your Lambdas starts to climb, you can dive into request traces and logs from that function in a single click.

Serverless meets the 3 pillars of observability

From an operations standpoint, serverless functions are fundamentally different from applications running on servers, VMs, or containers. From a monitoring perspective, however, you still need observability data like work metrics (requests, errors, latency), request traces, and logs to ensure that your serverless functions are performing properly, and to troubleshoot any issues that arise.

The new Cloud Functions view in Datadog provides a searchable, sortable view of all your functions. Faceted search allows you to filter your functions using metadata such as function name, AWS account, region, runtime, and team name, whether that metadata is collected from your cloud provider or applied as a custom tag. In the Cloud Functions view, you can also sort all your functions using high-level statistics such as number of invocations, average duration, error count, and memory usage.

The Cloud Functions view in Datadog

Clicking on any function from the Cloud Functions view takes you straight to detailed, function-specific data from all three pillars of observability: metrics, logs, and distributed request traces.

Lambda monitoring data for a serverless function in Datadog's Cloud Functions view
Metrics, request traces, and logs for a Lambda function in Datadog.

Datadog + X-Ray

AWS X-Ray traces application requests from end-to-end as they traverse your cloud infrastructure. Not only does it track request traffic flowing through Lambda functions, but X-Ray also provides auto-instrumentation to capture all calls utilizing the AWS SDK.

Instrument Lambda functions for tracing

To instrument a Lambda function for tracing with X-Ray, navigate to the Lambda function in the AWS console, scroll to the “Debugging and error handling” section, and check the box to “Enable active tracing.” Then, in your function code, import the X-Ray SDK and patch all supported libraries. For Python applications, instrumentation is as simple as importing the SDK and adding a one-liner to your function to start automatically tracing all calls to AWS services and other X-Ray-supported integrations:

patch_all()

Tracing Lambda functions in Datadog

With our new X-Ray integration, Datadog now visualizes Lambda trace data from X-Ray, so you can zero in on the source of any errors or slowdowns, and see how the performance of your functions impacts the overall performance of your applications.

Tracing is especially valuable for Lambda and other serverless platforms, because it allows you to visualize how requests travel between the numerous components of a serverless architecture. If your application’s end-to-end latency starts increasing, you can see in an instant whether the bottleneck is due to code-level issues in one of your Lambda functions, hitting your limits for Lambda concurrency, or issues in a service dependency like DynamoDB.

A serverless request trace from AWS X-Ray displayed in Datadog's Cloud Functions view

If your request invokes multiple Lambda functions connected by other AWS components such as SNS or SQS, X-Ray will automatically instrument each of these functions and tie them together in a single trace. Within the trace detail view in Datadog, you can easily jump from one function to another via auto-generated links.

Serverless metrics and logs

The addition of Lambda tracing to the Datadog platform complements the insights that we’ve long provided for your functions via metrics and logs. Each of these data types provides valuable insights into the usage and performance of your serverless functions, from overall performance summaries to low-level error reporting.

Metrics: High-level performance statistics

Datadog automatically collects a number of Lambda metrics from AWS CloudWatch, including counts of how many times each function has been invoked, latency percentiles for execution times, metrics for the number of concurrent executions, and counts of how many times a function has been throttled for exceeding concurrency limits.

Metrics from AWS Lambda functions displayed on a Datadog dashboard

Logs: Errors, duration, memory usage

Lambda logs are extremely valuable for debugging, as they provide detailed error reporting and low-level details for each invocation. Logs capture data such as the execution time for a particular invocation, the billed duration, and the actual memory usage as compared to the memory allocated to the Lambda function. Because you pay for allocated resources rather than actual usage, these memory statistics can help you identify overprovisioned functions so you can balance Lambda performance with costs. Similarly, you can identify Lambda functions that are underprovisioned, and assign more memory to those functions to improve their performance. These memory statistics are available at full granularity in the Lambda logs, and are also aggregated automatically for each Lambda function in the Cloud Functions view.

Logs from an AWS Lambda function as displayed by Datadog log management

Get full visibility into your serverless functions

If you’re already using Datadog to monitor your applications, cloud infrastructure, and serverless functions, visit the Cloud Functions view today to start monitoring and exploring all the performance data from your Lambda functions in one place. Then enable the new integration with X-Ray to start tracing requests as they travel through your serverless architecture for even greater visibility.

If you don’t yet have a Datadog account, you can today to get deep visibility into your applications, infrastructure, and serverless functions in one platform.