Melanie Yu
Vera Chan
Mallory Mooney
One of the primary challenges with developing in the cloud is knowing which areas of your environment are vulnerable to risks. In order to efficiently identify and respond to legitimate risks, you need real-time visibility into security events. But traditional security platforms are costly and often standalone, which means they may create gaps in visibility. Datadog Cloud SIEM solves this problem by providing security teams with real-time insights into security events across cloud-native environments, enabling you to protect them from legitimate threats.
Cloud SIEM is deeply integrated with Datadog Log Management and Flex Logs, offering comprehensive and cost-effective security insights. Built on the flexibility provided by Datadog's Logging Without Limits™, Flex Logs enables you to retain security logs in the long term and at different levels of granularity, all without incurring high storage costs.
Together, Cloud SIEM and Flex Logs enable real-time and historical analysis on your logs, powering advanced security monitoring, threat detection, and compliance. In this post, we'll show you how Cloud SIEM and Flex Logs work together to:
- Centralize and unify all security logs and data collection
- Cost-effectively retain logs to support use cases that are critical to maintaining a secure environment
- Detect and respond to legitimate, active threats to your environment with Content Packs
Ingest, centralize, and enrich security logs with Log Management
Security logs hold a wealth of information for identifying and analyzing potentially malicious activity within your cloud environment. Configuring the right level of visibility into these logs is a critical step for effective security monitoring and threat detection. Without it, you risk overlooking legitimate security risks.
There are several ways you can improve log collection, including segmenting logs with indexes, enabling long-term retention with Flex Logs, and setting up archives for long-term storage. These options allow you to fully customize how to manage the wide range of logs you may collect, especially if they require varying retention and storage policies. They also give you better visibility into when logs are generated, so you can monitor any anomalies in usage and better control costs.
You can also transform and enrich incoming logs with additional data to not only enhance Cloud SIEM detections but also strengthen your organization's security posture and adhere to regulatory and compliance requirements. For example, you can use the Sensitive Data Scanner processor to automatically redact or hash sensitive data from logs, as seen in the following screenshot:
These strategies create a centralized place to start collecting and efficiently managing the logs necessary for security monitoring and threat detection.
Facilitate advanced, cost-effective threat detection with Flex Logs
Because of the amount of logs you collect, finding the right balance between cost and visibility for which ones to keep can be challenging. While Datadog enables you to easily centralize all your security logs, you don't want to risk losing valuable insights, such as authentication events, due to log volume or storage limitations. Datadog Log Management offers up to 15 months of retention via Flex Logs, with the option to customize the retention period of each index to fit your organization's needs.
Flex Logs is a cost-effective solution for retaining high-volume sources in a searchable tier, bridging the gap between indexing logs for real-time queries and archiving them in the long term for compliance or audits. This is especially useful for sources that generate the data necessary for conducting security investigations and audits, such as network-related logs or audit and activity logs from managed providers, such as AWS CloudTrail and Azure.
To enable Flex Logs, simply navigate to the Flex Logs Control page and configure the appropriate compute size for your needs (requires the flex_logs_config_write permission). Once enabled, Flex Logs allows you to easily support a wide variety of security monitoring use cases, including:
- Analyzing historical data to uncover trends in attack paths
- Detecting sophisticated attacks with advanced threat intelligence analysis, such as user and entity behavior analytics
- Taking advantage of historical data to efficiently test detection rules and conduct threat hunting
- Strengthening regulatory compliance, security auditing, and reporting
- Quickly troubleshooting and responding to security incidents based on real-time insights
For example, Flex Logs enables you to not only ingest logs from sources such as AWS for active security monitoring but also retain them in the long term for compliance audits, which is necessary to maintain compliance with regulatory frameworks. Flex Logs also helps you identify trends in historical logs that need to be covered by a Cloud SIEM detection rule, which are critical for detecting indicators of compromise (IoCs) and other security events. This visibility is foundational to threat detection, where you need to easily analyze activity logs in order to confirm suspicious behavior. In this scenario, Flex Logs can be used to perform retrospective analysis in order to determine how long a threat was active before being detected. These are a few ways Flex Logs works together with Cloud SIEM to support advanced security monitoring and threat detection.
Detect and respond to embedded threats with Cloud SIEM
Centralizing your security logs via Log Management and Flex Logs is a critical step for getting better visibility into security issues in the cloud, such as active threats or misconfigurations. However, you also need timely insights from your logs to efficiently mitigate those risks, which requires connecting your log management and security platforms.
Built on top of Datadog Log Management, Datadog Cloud SIEM is designed to automatically scan your configured logs for issues and can be easily activated in-app. When you configure log collection for a new source, such as AWS CloudTrail, Cloud SIEM will automatically begin flagging malicious activity captured in those logs via its built-in detection rules.
Cloud SIEM insights are also included in your logs, enabling you to quickly pivot back and forth between them and signals to get a better understanding of the risk. For example, Cloud SIEM's Risk Insights for AWS Entities will automatically include information about an AWS user tied to a particular event in associated logs, as seen in the following screenshot:
Cloud SIEM also offers a historical engine, which enables you to conduct investigations on past logs and understand the context of previous security incidents. These capabilities highlight the breadth and depth of Cloud SIEM's integration with your logs and enable you to efficiently surface threats hidden deep within your environment, including:
- Abnormal user behavior: Users logging in from unusual locations
- Credential stuffing and account takeover: Attackers using stolen credentials to gain unauthorized access
- Lateral movement: Attackers moving across your environment to expand their access
- Insider threats: Malicious or negligent users misusing their access to steal data or escalate privileges
Use Content Packs to rapidly onboard security teams and prioritize high-risk signals
One of the challenges with using a cloud SIEM, even after you've taken the steps to customize retention periods and enrich incoming logs, is knowing how to onboard security teams and where to concentrate their monitoring efforts. This challenge is especially significant for detecting deeply embedded threats, such as those that take advantage of existing accounts to move across your cloud environment. Datadog Cloud SIEM simplifies this process with Content Packs, which is a curated list of Datadog integrations that collect valuable security logs.
Let's look at the AWS CloudTrail Content Pack as an example of how Cloud SIEM brings together security logs, detections, workflow automation blueprints, and relevant guides in one place. The Content Pack provides an overview of all of the available detection rules for AWS CloudTrail as well as a dashboard that allows you to visualize the current state of your AWS environment, such as which AWS services and accounts are generating signals. It also includes an investigator for IAM user activity, which allows you to quickly identify the specific user accounts that are sources of potentially malicious activity.
You can easily pivot from the AWS CloudTrail Content Pack to the Signal Explorer, which will automatically be filtered by the source:cloudtrail facet. The following example signal that Cloud SIEM generated indicates impossible travel, which is activity by a single user at the same time or in close succession in two geographic locations:
Cloud SIEM Investigator also includes valuable context for investigating the activity, such as the user account that initiated it, which IPs they used, and a trail of other events associated with that user, as seen below:
Incorporate Cloud SIEM with security incident management
Cloud SIEM signals play a critical role in triaging security incidents by providing you with timely, comprehensive insights into an issue. When Cloud SIEM flags a security risk, such as impossible travel by an AWS user, Datadog On-Call can instantly notify responders, who can either create a case to investigate further or run a workflow directly from the signal. For example, if they confirm an account takeover, they can run the workflow for disabling an AWS IAM user to remove the account's access to your environment. By incorporating Cloud SIEM into your incident management processes, you have end-to-end coverage for detecting, investigating, and resolving a security issue.
Integrate Cloud SIEM with your logs today
In this post, we looked at how Cloud SIEM and Datadog Log Management work together to enable comprehensive and advanced threat detection, historical and entity analysis, and security monitoring. We also looked at how Flex Logs offers a cost-effective solution for ingesting all the logs that are necessary for conducting thorough analysis, investigations, and audits. To get started, you can check out Cloud SIEM and Flex Logs in-app (requires the flex_logs_config_write permission to access the Control page). If you don't already have a Datadog account, you can sign up for a free 14-day trial.
