State of AI Engineering

What production LLM telemetry from 1,000+ organizations reveals about AI in the real world.

AI engineering has moved well past experimentation. Organizations now manage model fleets, orchestration frameworks and multi-step agentic workflows in production, where a single prompt or model change can silently move latency, cost, and failure rates.

In the State of AI Engineering report, we analyzed LLM telemetry from more than 1,000 Datadog customers to show what’s actually happening at scale. Inside, you’ll find:

• How model provider adoption is shifting and why most organizations are now multi-model by default
• Why LLM tech debt is compounding as teams adopt new releases faster than they retire old ones
• What the doubling of agent framework adoption means for observability
• Where hidden token costs are coming from and why prompt caching remains widely underutilized

Download the report to benchmark your AI stack against what’s happening in production today.