Observability Pipelines

Standardize Security Data With OCSF-Powered Pipelines

Stream, transform, and standardize log data in the OCSF format to improve threat detection, speed investigations, and simplify SIEM integration—without increasing cost or complexity.

Why Datadog?

Intuitive and Easy to Use

Kickstart your projects today with easy-to-use templates for popular use cases such as dual shipping logs, reducing log volume, and archiving data

Vendor Agnostic

Ensure seamless onboarding of new log data sources and destinations at your own pace, without disrupting your existing workflows or sacrificing visibility

Scalable Sensitive Data Protection

Help protect sensitive data and achieve compliance by redacting it before it leaves your environment with Datadog Sensitive Data Scanner

Out-Of-The-Box Dashboards

Gain comprehensive visibility into the health and performance of all pipelines deployed in your infrastructure from one centralized dashboard, with real-time alerts for important issues

Product Benefits

Accelerate Security Investigations and Response with Actionable Insights

Transform your logs into industry-standard OCSF events that work seamlessly with any security tool in your arsenal
Use pre-built parsing and OCSF remapping rules covering AWS, Google, Microsoft, Palo Alto Networks, Okta, GitHub, and more to reduce time to triage and investigation
Add business and geographic context to each OCSF event, eliminating time-consuming manual correlation during critical investigations
Custom-tailor your OCSF data to highlight important signals and filter out the noise that slows investigations and delays response times

products/observability-pipelines/feature-4.png

Optimize Security Spend With Standardized, Streamlined Logs

Reduce data volume by transforming high-cardinality security logs into structured OCSF event classes before they hit your SIEM
Turn high-volume endpoint and identity events into compact, meaningful OCSF fields and metrics—maintaining visibility while dramatically reducing storage requirements and associated costs
Transform logs to OCSF at the edge—before data leaves your environment—reducing egress costs and maintaining data sovereignty while optimizing downstream analysis and budget usage
Validate required OCSF fields before forwarding and gain full control over what's ingested, when, and where—enforcing OCSF schemas to ensure visibility without cost overruns

products/observability-pipelines/feature-1.png

Break Free from Vendor Lock-in Without Disrupting Security Operations

Convert proprietary log formats to OCSF for broad compatibility across security tools, SIEMs, and data lakes
Dual-stream OCSF-normalized data to legacy and modern destinations during migrations—without losing visibility or coverage as your safety net
Route OCSF-standardized logs to Splunk, Datadog Cloud SIEM, Amazon Security Lake with automatic Parquet encoding, Google SecOps (Chronicle), Microsoft Sentinel, and more
Use Datadog Observability Pipelines independently of Log Management or Cloud SIEM subscriptions to standardize data without vendor dependencies

products/observability-pipelines/feature-2.png

Easily Manage OCSF Pipelines Through a Single Control Plane

Use drag-and-drop templates to build pipelines that transform, enrich, and forward logs in OCSF
Create and modify OCSF data pipelines using an intuitive point-and-click interface that reduces engineering overhead and frees up valuable team resources
Configure pipelines with confidence and diagnose production issues easily by viewing your data live as it flows through the pipeline using Live Capture
Import your existing OCSF mappings using "Bring Your Own Mapping" to leverage work you've already invested in, or reuse new mappings across multiple sources and versions

blog/observability-pipelines-stream-logs-in-ocsf-format/ocsf-fluentd-pipeline-new.png

Ensure Compliance and Security Measures Align Perfectly With Your Organization's Standards

Apply uniform OCSF schemas across all logs to simplify audit trails and reporting across regions
Identify and redact PII before it leaves your environment while preserving the analytical value of your OCSF-standardized security data
Send security logs to SIEM in OCSF format while keeping DevOps logs in original format for troubleshooting—maintaining operational flexibility without compromising compliance
Enforce consistent OCSF field naming and types across your security tools for reliable compliance reporting and simpler audits organization-wide

products/observability-pipelines/feature-3.png