Metrics and Logs: Datadog + Sumo Logic | Datadog

Metrics and logs: Datadog + Sumo Logic

Author Marshal Ma
@MarshalMa

Published: August 2, 2016

Sumo Logic is a SaaS log management tool, providing log aggregation and analysis services for a wide range of applications and infrastructure components. Sumo Logic ingests large amounts of machine-generated events, parsing and searching structured and unstructured text.

Sending parsed log data to Datadog

With this integration, you can send results from any scheduled search in Sumo Logic directly to Datadog. Datadog will report the results as events in the event stream, which provides a blog-like timeline that lets you pinpoint issues and collaborate with people across your organization. You can review, comment, forward, or modify priority of Sumo Logic events the same way as you would for any other event in the event stream.

Sumo Logic performance - event stream with sumo logic alert
Datadog event stream showing events from Sumo Logic and other applications

Match events with metrics

You can also correlate Sumo Logic events with performance metrics to gain a better understanding of what’s going on throughout your infrastructure. For example, a sudden spike in CPU usage may happen at the same time that a number of runtime errors in Java Virtual Machine are captured in logs. By overlaying events reported by Sumo Logic, you can quickly identify the problematic application or even all the servers in a cluster that the application is running on.

Sumo Logic performance - overlaying sumo logic event with metrics from datadog

Alert failed login attempts

Another good example for using Sumo Logic and Datadog together is to alert on unauthorized login attempts. Sumo Logic gathers log from servers in real time, parsing and aggregating related log entries. After processing, periodic summaries of unauthorized attempts will be sent as events to Datadog. If these attempts are anomalous, Datadog can alert your team.

Sumo Logic performance - sample alert message in datadog event stream from Sumo Logic
Sample Datadog event showing an alert from Sumo Logic

Configure event message

You can also customize the message body with markdown and a few variables to provide more actionable insight:

  • $SearchName: Name of the saved search
  • $SearchDescription: Description of the saved search
  • $SearchQuery: The query for the saved search
  • $SearchQueryUrl: The URL link back to the saved search
  • $TimeRange: The time range over which the search was run
  • $FireTime: The time at which the task ran
  • $AggregateResultsJson: JSON object containing search aggregation results
  • $RawResultsJson: JSON object containing raw messages
  • $NumRawResults: Number of raw messages returned by the search

Besides scheduled search results, you can also send Audit Indexes to Datadog. An Audit Index provides an overview of all Scheduled Searches sent to a Webhook Connection with specific user details. With audit index, you can also monitor whether a connection is working properly or not, and send an alert if expected data does not arrive.

Notifying Sumo Logic from Datadog

In addition to sending events to Datadog, Sumo Logic can also receive events from Datadog. After you set up a connection, you can send data and text messages back to Sumo Logic by @-mentioning @sumologic-[connection-name] in any post or comment in Datadog’s event stream. The event may contain markdown and associated metadata such as tags, metric name, priority level, and links to the event.

Sumo Logic performance - display message from Datadog
Sumo Logic displaying Datadog events

You can then leverage Sumo Logic’s powerful regular expression parsing and searching functionality to quickly trace the root cause and analyze system performance issues reported by Datadog. What’s more, Sumo Logic’s LogReduce can aggregate repetitive messages, clip meaningless noise, and highlight outliers.

Getting Started

To use the integration, first set up a webhook connection in Sumo Logic and enter the connection’s URL in Datadog. Then set up or select a scheduled search in Sumo Logic and start to report results through the configured webhook connection. If you are not a Datadog user yet, sign up for a full-featured trial account to get started.