Datadog Monitoring | Datadog
Datadog logo home

Monitor Databricks with Datadog

Get real-time visibility into the health and performance of your pipelines. Unify underlying infrastructure metrics, logs, Spark performance metrics, and data quality.

databricksdashboardimage_1

1,000+ Turn-Key Integrations, Including

Product Benefits

Detect, Troubleshoot, and Cost-Optimize Your Databricks Jobs

  • Get real-time alerts on failed, delayed, and long-running jobs with out-of-the-box monitoring from Jobs Monitoring
  • Drill into the full execution flow — job, stages, and tasks — to pinpoint exactly where and why a job failed
  • Reduce platform spend with job and cluster cost insights and savings recommendations for code, queries, and configs
dg/jobsbox1.png

Ensure Reliable Data Across Your Databricks Tables

  • Detect data quality issues before they undermine dashboards, AI models, or business decisions with out-of-the-box anomaly detection for freshness, volume, and schema changes
  • Map end-to-end lineage from your Databricks tables to downstream BI and AI applications to assess impact before you triage
  • Correlate data quality issues with the job and cluster health you're already monitoring, without switching tools
dg/databricksquality.png

Secure Your Cloud Infrastructure

  • Assess and enhance your security posture with continuous configuration checks across your cloud accounts, hosts, and containers
  • Track and fulfill audit requirements with out-of-the-box support for over 15 compliance frameworks including PCI DSS, SOC 2, HIPAA, GDPR, and CIS
  • Automatically detect and prioritize cloud vulnerabilities based on potential business impact, deep observability context, and Datadog’s curated security research
  • Visualize your cloud resources and their associated security risks at a glance with a comprehensive resource inventory
dg/cloud-security.png

Fine-Tune Performance with Real-Time Spark Metrics

  • Avoid performance degradation with alerts on stage and shuffle operations to locate the source of memory exceptions
  • Rapidly collect granular metric insights into job performance, nodes and partitioning
  • Minimize the amount of shuffling required to complete data operations; monitor resource metrics at the stage level to understand how Spark is processing your data
  • Unify real-time infrastructure data, logs, and Spark performance metrics
sparkmetricsimage_1.png

Investigate and Troubleshoot Databricks Issues Faster

  • Discover log patterns and errors rapidly without the need to investigate and read every individual log file
  • Visualize and explore collected logs using an intuitive, facet-driven navigation designed for outage responders
  • See log data in context with auto-tagging and metric correlation

Monitoring That's Simple to Deploy and Effortless to Manage

  • Track tens of thousands of infrastructure metrics and hundreds of drilled down query metrics out-of-the-box
  • Deploy and start monitoring without any need for professional services or extensive training
  • Promote adoption across your organization with our intuitive user interface that requires no query language and can be used by anyone

The Essential Monitoring and Security Platform for the Cloud Age

Datadog brings together end-to-end traces, metrics, and logs to make your applications, infrastructure, and third-party services entirely observable.

Platform Diagram

Thousands of Customers Love & Trust the Datadog Platform

Start your free trial Request a Demo