Hands-on AI Observability: Build, Monitor, and Optimize LLM Applications | Datadog

Hands-On AI Observability: Build, Monitor, and Optimize LLM Applications

December 15 at 10:00 AM ET

Hands-On AI Observability: Build, Monitor, and Optimize LLM Applications header sub image

Virtual Session

December 15 at 10:00 a.m.–12:00 p.m. ET

By attending this event I acknowledge that I may be exposed to information which the Exhibitor, Datadog, Inc., considers confidential and wishes to limit the disclosure of. I hereby agree to keep such information confidential and to not disclose such information to any third parties. I further represent that I am not employed by a competitor of the Exhibitor and am not attending this Event with the intent to access any such confidential information or gain a competitive advantage for my employer. I hereby give permission to the Exhibitor to use my image, likeness, appearance, voice, and any written or spoken testimonials given by me in connection with the marketing and promotion of this Event and/or any Datadog products or Events.

In-Person Session

December 15 at 9:30 a.m.–12:00 p.m. ET | AWS HQ2, 1770 Crystal Drive, Arlington, VA 22202

By attending this event I acknowledge that I may be exposed to information which the Exhibitor, Datadog, Inc., considers confidential and wishes to limit the disclosure of. I hereby agree to keep such information confidential and to not disclose such information to any third parties. I further represent that I am not employed by a competitor of the Exhibitor and am not attending this Event with the intent to access any such confidential information or gain a competitive advantage for my employer. I hereby give permission to the Exhibitor to use my image, likeness, appearance, voice, and any written or spoken testimonials given by me in connection with the marketing and promotion of this Event and/or any Datadog products or Events.

Build More Reliable AI Applications

As government agencies accelerate AI adoption, visibility into LLM application performance and behavior is essential for delivering reliable AI-powered services. Datadog Agent Observability helps teams monitor, troubleshoot, and optimize AI workflows within a FedRAMP High–certified environment, with end-to-end tracing and insights into latency, token usage, errors, and model interactions.

In this hands-on workshop, you’ll build and instrument a RAG-based chatbot, investigate performance issues using Datadog, and apply improvements in real time.

Attendees will learn how to:

  • Build and instrument a RAG-based chatbot using the OpenAI Python SDK and local models
  • Trace embedding, retrieval, and generation across LLM workflows
  • Analyze model inputs and outputs, latency, token usage, and errors
  • Identify performance issues and validate improvements to build more reliable AI applications

AWS + Datadog AI Observability Workshop

Sign up