Observability Pipelines

Maximize Savings with Cost-Effective Processing and Routing of All Your Logs

Proactively filter your logs before routing, compress their sizes, and ensure critical fields are retained
Effectively control your expenses with rule-based quotas or sampling while still capturing critical KPIs and trends
Customize log destinations based on priority and needs - routing noisy logs to an archive to streamline your workflow

Guarantee business continuity throughout the migration process by dual-shipping logs to multiple vendors
Ensure seamless onboarding of new log data sources and destinations at your own pace, without disrupting your existing workflows or sacrificing visibility
Take advantage of best-of-breed solutions with easy routing for different logging use cases

Automatically identify and convert over 150 common log types into easily queryable, structured format using the Grok parser
Enhance security and improve data analysis using the GeoIP Parser, which helps you identify an IP address’s geographical location
Enrich logs with contextual data using using custom tables to enhance usability for downstream analysis
Simplify debugging, increase searchability, and speed up Root Cause Analysis (RCA) by inserting a hostname within log messages

Stay compliant with data residency laws and support Data Loss Prevention initiatives with full control over data routing, and enable built-in or user-defined rules to stay compliant with PCI, GDPR, HIPAA, CCPA, and more
Protect sensitive data with confidence—credit cards, email addresses, IP addresses—by redacting it before it leaves your environment with Datadog Sensitive Data Scanner
Streamline access control and schema standardization by easily adding, copying, and dropping relevant attributes and tags

Kickstart your projects today with easy-to-use templates for popular use cases such as dual shipping logs, reducing log volume, and archiving data
Simplify the process of creating, deploying, and managing pipeline instances from anywhere with a user-friendly point-and-click interface
Gain comprehensive visibility into the health and performance of all pipelines deployed in your infrastructure from one centralized dashboard, with real-time alerts for important issues