Jaclyn Verga
Security and engineering teams contend with a constant stream of signals about vulnerabilities, incidents, misconfigurations, identity risks, control gaps, and other findings across their environments. But an individual finding’s severity does not always reflect its potential organizational impact. For example, a critical finding on an isolated resource with limited permissions may represent less risk than a medium-severity finding on an internet-facing production service that can access sensitive data. The challenge is understanding how those signals interact, which combinations of conditions create the greatest exposure, and what findings teams should address first.
Datadog’s Risk Engineering team has been working on this problem by building a Systemic Risk Detection Pipeline and Risk AI Agents that correlate signals across our environment to evaluate risk beyond individual findings. The pipeline detects patterns across multiple conditions to identify potential systemic risks. The Risk AI Agents then help security practitioners investigate the relationships, context, and potential impact associated with those risks.
We think of Risk AI as an approach that combines deterministic risk detection with AI-assisted investigation to identify, contextualize, and prioritize systemic risk. Our approach employs deterministic systems where consistency and repeatability matter, and AI agents where flexible investigation and interpretation add value. Security practitioners remain in the loop to validate agent-generated findings and retain ownership of next steps.
In this post, we’ll explore how we combine deterministic detection, agentic reasoning, and contextual risk scoring to help answer two fundamental questions: Where are we most exposed, and what should we address first?
Moving from individual signals to systemic risk
As the Risk Engineering team, we help identify potential risks and give Datadog engineers the context they need to understand and reduce their exposure. A key part of this work is understanding how findings across different systems relate to one another.
To better understand exposure, we apply the established concept of systemic risk to our operational environment. Systemic risk emerges from the interaction of multiple conditions, dependencies, or findings across an environment. It goes beyond individual findings to reveal how interconnected weaknesses, trust relationships, and control gaps can amplify exposure. When connected conditions form a progression from an initial risk condition toward broader organizational impact, we refer to that as a risk path.
Consider an internet-facing service with a medium-severity risk scoring. Evaluated independently, the threat may not require immediate remediation. But if the service runs in production, uses an identity with excessive permissions, has access to sensitive resources, or is associated with another security finding or incident, its risk profile changes. Together, these conditions can form a risk path that shows how an initial exposure could lead to broader organizational impact.
Correlating signals in a Systemic Risk Detection Pipeline
Our Systemic Risk Detection Pipeline brings together security signals, engineering signals, and assets from across our environment and analyzes the relationships between them. The following diagram shows how the pipeline correlates disparate signals to surface potential systemic risks for investigation:
The pipeline brings together information such as vulnerabilities, incidents, identities and permissions, resource exposure, configuration, business context, and other relevant signals. Instead of evaluating findings in isolation, the pipeline maps relationships between signals and assets, then checks combinations of conditions against defined risk patterns to identify potential systemic risks.
Building operational context through coordinated Risk AI Agents
Identifying systemic risk is only part of the process. We still need to investigate the conditions and relationships behind it, gather evidence that supports or challenges it, assess the context that could increase or reduce its potential impact, and identify actions that could reduce the exposure.
We built a set of Risk AI Agents to accelerate this step of the investigation while making sure to preserve human oversight. The agents gather and synthesize relevant relationships and context; security practitioners validate their findings and retain responsibility for downstream decisions.
Each Risk AI Agent contributes a distinct type of context to help practitioners assess a systemic risk. For example, some agents map risks to relevant security domains and controls, while others evaluate contextual factors to assess their potential impact and severity. Additional agents synthesize these findings to surface actions that could help mitigate the risk.
Separating deterministic detection and agentic reasoning
Our systemic risk architecture separates deterministic detection from agentic investigation. The pipeline identifies potential systemic risks, while the agents investigate and contextualize the evidence behind them. We use deterministic systems where consistency and repeatability matter, such as detecting known conditions and providing structured inputs to the pipeline. We use AI agents where flexible investigation and reasoning are valuable, such as for gathering evidence, interpreting relationships, and assessing potential impact.
This distinction also makes explainability particularly important. We do not want a security practitioner to receive a critical risk rating without clear context about why the AI agent made that determination. Our process surfaces the evidence and context behind agent-generated findings so that practitioners have what they need to understand and validate why a risk was prioritized.
Prioritizing risks based on exposure
Our Risk AI approach helps us surface exposures that individual findings may not reveal. From there, we evaluate each risk in its broader context to determine which ones deserve attention first.
Traditional risk severity scores provide an important signal, but they don’t always provide enough context to reflect organizational exposure. To address this, our pipeline evaluates factors that can increase or decrease the significance of a risk.
These factors can include:
Potential impact and likelihood of exploitation
Internet exposure and external reachability
Identity, privilege, and access to systems or sensitive data
Asset and business criticality
Potential blast radius
Existing controls and mitigations
Related incidents and findings
This helps us distinguish between the technical severity of individual findings and the broader organizational risk that emerges when those findings are evaluated in context. Consider three findings: a public-facing application with a known vulnerability, a workload with excessive privileges, and a production data store that contains sensitive data. Evaluated independently, none may warrant the highest priority rating. But if exploiting the application provides access to the privileged workload, which can then reach the data store, that reveals a risk path with significant potential impact.
The following example screenshot shows demo telemetry data with systemic risks scored by priority level in Datadog Work Management:
We also classify detected risks into the appropriate security domains. This creates a consistent taxonomy that connects each risk to its underlying drivers and the teams best positioned to address them.
From risk detection to informed action
As systems become more interconnected, we believe risk prioritization increasingly needs to account for dependencies across systems, resources, and identities. A finding’s severity matters, but so do the identities that can access the affected resource, the systems it can reach, the controls already in place, and the potential impact if the underlying risk materializes.
The Systemic Risk Detection Pipeline and Risk AI Agents build on systems that our Risk Engineering team already uses. These include centralizing a dynamic risk register in Datadog Work Management and building an AI tool to calculate risk scores, correlate risks with related context, and develop mitigation plans. We also use Workflow Automation to automate operational steps in the risk life cycle. This includes routing risks to the appropriate owners, creating and tracking remediation work, and triggering follow-up actions as risks progress toward resolution. We use Agent Observability to evaluate the quality of our AI workflow outputs and monitor hallucinations and performance over time.
If you’re new to Datadog, you can sign up for a 14-day free trial to explore the security, observability, and automation capabilities that support workflows like the ones described in this post.
