Data Streams Monitoring | Datadog
Data Streams Monitoring

Data Streams Monitoring

Easily track and improve the performance of your streaming data pipelines from end to end

Datadog Data Streams Monitoring (DSM) enables you to easily track and improve the performance of event-driven applications that use Kafka and RabbitMQ. DSM automatically maps dependencies between all services and queues and provides out-of-the-box metrics that help you measure end-to-end latencies, locate faulty queues or services and their respective owners, and remediate floods of backed-up messages so that you can avoid critical downtime and meet strict SLAs.

2023 GARTNER® MAGIC QUADRANT™

Datadog has been recognized as a Leader in the Gartner® Magic Quadrant™ for APM and Observability

Get the full report

Detect slowdowns anywhere in your event-driven applications

  • Measure latency between any two points throughout your pipeline, including queues, producers, and consumers
  • Easily identify delayed messages by viewing actionable lag metrics both in seconds and offset
  • Quickly detect abnormal latency or throughput across your streaming data pipelines by creating threshold, anomaly, and outlier monitors
Detect slowdowns anywhere in your event-driven applications

Localize and resolve pipeline bottlenecks and failures

  • Easily detect failing dependencies between queues, producers, and consumers with automatic, end-to-end pipeline topology mapping
  • Triage whether delays are originating from issues with application services or queueing systems, and identify the owning team
  • Prioritize troubleshooting efforts by impact of delay, with message backup volumes for services and queues
Localize and resolve pipeline bottlenecks and failures

Remediate floods of backed-up messages

  • Quickly detect and resolve downstream bottlenecks to prevent long lags, message downtime, and delayed messages
  • Spot where bursts in message flow may be occurring upstream with automated consumer lag notifications for every service
  • Visually track the flow of messages across all dependent services and queues to ensure you constantly meet your SLAs
Remediate floods of backed-up messages

Correlate streaming data pipeline metrics with all telemetry in context

  • Resolve application issues faster by seamlessly pivoting between DSM metrics, distributed traces, infrastructure metrics, and logs
  • Gain complete pipeline visibility by adding latency and throughput metrics to your Kafka or RabbitMQ integration dashboards
  • Streamline incident response with quick access to service owners, on-call engineers, and critical resources via Service Catalog
Correlate streaming data pipeline metrics with other telemetry in context

Customer Testimonials

Building streaming data pipelines to power our digital trading platform requires visibility into throughput and latency across the entire system, including all queues and services. With Data Streams Monitoring our team is able to proactively discover performance bottlenecks and optimize stream processing to ensure maximum throughput and low latency delivery of data to our customers.

Darren Furr

Darren Furr

Solutions Architect at MarketAxess

Datadog Data Streams Monitoring allows us to easily map the topology of our Kafka pipelines across our globally-spread clusters, monitor end-to-end latency, and identify sources of incidents across our entire pipeline in one place. Having DSM integrate seamlessly with APM helps us ensure that millions of gamers around the world always get the best possible experience.

SeungYong Oh

SeungYong Oh

VP of Engineering at Devsisters

Resources

products/data-streams-monitoring/data-streams-monitoring-product-hero-desktop

official docs

Data Streams Monitoring

/blog/data-streams-monitoring/hero

BLOG

Track and improve the performance of streaming data pipelines with Datadog Data Streams Monitoring

/blog/monitoring-kafka-performance-metrics/151208-kafka-cs6-final-notitle-a1-how-to-monitor

BLOG

Monitoring Kafka performance metrics

/blog/kafka-at-datadog/kafka_lessons_learned_190227_FINAL

BLOG

Lessons learned from running Kafka at Datadog

/blog/service-owner-knowledge-with-datadog-service-catalog/service-catalog_feature-announcement_220718_v1a

BLOG

Simplify microservice governance with the Datadog Service Catalog