Azure OpenAI is a service for deploying AI applications on Azure resources. With its easy-to-use REST APIs, you can leverage the service to access OpenAI’s powerful language models, such as ChatGPT, for your applications while taking advantage of the reliability and security of the Azure platform. Datadog already offers an out-of-the-box integration for OpenAI so you can monitor key performance trends, such as API usage patterns, token consumption, and more. We’re excited to announce that we’re providing the same comprehensive visibility for Azure OpenAI, enabling you to monitor costs and performance issues that are specific to your Azure instances.
Complete visibility into Azure OpenAI performance
Powered by Datadog’s existing Azure integration, our Azure OpenAI integration requires no additional setup. Once enabled, Datadog will start collecting metrics from your Azure OpenAI instances. You can use the built-in integration dashboard for an overview of performance across all your instances as well as usage trends for the Azure OpenAI service, such as the total number of executed API calls by Azure resource.
You can also configure Datadog to collect Azure OpenAI logs, which provide you with the necessary context to identify the source of unusual changes in performance. For example, you can review your logs in Datadog Log Management to quickly determine the root cause of a sudden spike in errors for a particular Azure OpenAI instance.
Track usage and costs for your Azure OpenAI instances
In addition to the standard OpenAI metrics, Datadog automatically collects performance data that is unique to Azure OpenAI instances. This ensures that you have the information you need to monitor usage and potential costs for all your AI applications, regardless of how they are deployed. For example, you can get a better picture of the costs for running your customized Azure OpenAI models by tracking the number of processed inference tokens and fine-tuned training hours per model deployment.
You can also visualize future usage demands for your model deployments via Datadog’s forecasting algorithm and create an alert based on this data. This enables you to be notified when you are close to exceeding the capacity for existing resources. With this information, you can efficiently manage your Azure resources based on your applications’ historical usage trends.
Start monitoring your Azure OpenAI applications today
With our out-of-the-box integration for Azure OpenAI, you have instant, comprehensive visibility into all of Azure’s supported language models. Check out our documentation to learn more about how to start monitoring your Azure OpenAI instances. If you don’t already have a Datadog account, you can sign up for a 14-day free trial today.
