Nimisha Saxena
Anes Bendimerad
Security teams often rely on rules that detect spikes in logs or the appearance of new field values to alert on suspicious activity. While these approaches are highly effective in many scenarios, they aren’t always suited for identifying more nuanced changes in logs, like subtle deviations in user behavior or unexpected command patterns.
That’s why we’re introducing Content Anomaly Detection in Datadog Cloud SIEM, a new detection method that analyzes the content of logs to find anomalies. It provides a powerful way to detect unusual activity based on what’s being said in your logs—not just how often or how recently.
In this post, we’ll look at how Content Anomaly Detection works, how it complements existing methods, and how to start using it to uncover threats that might otherwise go unnoticed in Datadog Cloud SIEM.
Analyze what's different, not just what's new or frequent
Datadog Cloud SIEM already provides detection methods for new value detections—a method that surfaces new fields to help catch unexpected users, IPs, or service names—and anomaly detections—for surfacing outliers in volume and spotting spikes in log or event activity.
Content Anomaly Detection takes a different approach: it flags logs that are meaningfully different from what’s been seen historically.
To get started, create a detection rule in Cloud SIEM and select the Content Anomaly detection method.
From there, you can specify which logs to analyze with a search query, select fields to evaluate for content anomalies, set grouping fields like user or region to understand normal behavior in context, and configure thresholds for anomaly detection and alerting.
Once the rule is active, it enters a learning period (default: seven days, up to 10 days) during which it builds a baseline of normal field values. After this period, Datadog Cloud SIEM begins assessing incoming logs using Jaccard similarity, implemented via MinHash and Locality Sensitive Hashing, to quickly identify content that deviates from historical norms.
A signal is triggered only if a configured number of anomalous logs occur within a specific evaluation window (e.g., three anomalies in 15 minutes).
Reduce noise with fine-tuned thresholds
When creating a detection rule, Content Anomaly Detection lets you balance precision and sensitivity through three additional rule parameters:
- Similarity threshold: Defines how dissimilar a field value must be to be considered anomalous (default: 70 percent).
- Minimum similar items: Sets how many similar historical logs must exist for a value to be considered normal (default: one).
- Evaluation window: The time frame during which anomalies are counted toward a signal (e.g., 10 minutes).
For example, setting a similarity threshold of 70 percent and requiring two or more similar past values helps surface field content that is both unusual and rare—filtering out minor or common variations.
Let’s say your environment usually logs messages like: user connected to abc network. Later, a log appears that says: user got unauthorized network access.
While these two logs share some words (e.g., “user” and “network”), their meanings are quite different.
Content Anomaly Detection evaluates these changes and can trigger a signal if the new log is dissimilar enough and sufficiently rare, giving you early insight into potentially risky behavior. You can also group logs by attributes like @user_id, @region, or @host to detect anomalies in specific environments or user scopes.
Spot subtle threats with Content Anomaly Detection
With Content Anomaly Detection, Datadog Cloud SIEM now provides deeper visibility into the content of your logs—not just their structure or volume. Whether you’re monitoring for unauthorized access, unexpected behavior, or subtle changes in configuration, this method helps you detect risks that other approaches might overlook. To learn more, explore our documentation.
If you're new to Datadog, get started with a 14-day free trial
