Building a foundation to accelerate agentic development
NTT DATA is helping customers move beyond AI experimentation by enabling them to run agentic systems safely, reliably, and cost-efficiently in production.
Within NTT DATA, the Technology Innovation Headquarters leads the evaluation of emerging technologies and supports technically complex projects across the global organization. The team develops best practices that can be shared across the NTT DATA Group, helping improve technical capabilities, operational quality, and innovation at scale. Areas of focus include generative AI, agentic AI, observability, governance, security, and FinOps. “Many organizations are exploring how to use AI, but concerns around quality, governance, security, and operational monitoring often slow adoption,” says Takashi Okamoto, AI Technology Strategist at NTT DATA.
To support that mission, the team launched an initiative to validate enterprise agent architectures using Amazon Bedrock AgentCore. Rather than evaluating a single application, the team set out to understand how organizations could standardize agent execution, observability, governance, evaluation, and cost management as they scale AI systems. “Our goal is to help customers accelerate from experimentation to production by combining AI evaluation, observability, and governance into a repeatable operational framework,” says Okamoto.
Understanding how agents behave in production
As NTT DATA evaluated agentic workflows, the team quickly discovered that traditional monitoring approaches could not provide the visibility required to operate AI systems with confidence.
Application monitoring, logs, and infrastructure telemetry could indicate whether systems were healthy, but they could not explain how agents reached decisions. Questions such as why a particular answer was generated, which prompt influenced a result, which tool call failed, or why token usage increased were difficult to answer through conventional observability practices alone.
“Our goal is to help customers accelerate from experimentation to production by combining AI evaluation, observability, and governance into a repeatable operational framework.”
The team also needed to evaluate challenges unique to AI systems, including hallucinations, off-track reasoning, prompt injection, quality drift, and rising costs. Because agentic systems are inherently non-deterministic, the same input does not always produce the same behavior or outcome. In many cases, the system looked successful, with 200 responses, successful API calls, and no exceptions, while the quality of the agent’s behavior was quietly deteriorating. “Traditional application performance monitoring is effective for identifying where errors occurred or what is slow,” says Tohn Furutani, SRE Engineer at NTT DATA. “But for agentic AI, the core challenge often shifts from errors to decision quality.”
These limitations created lengthy investigation cycles and made AI quality evaluation difficult to standardize across teams. As NTT DATA worked to establish repeatable operational patterns for customers, the team needed a way to understand not only whether agents were functioning, but also how they reasoned, selected tools, and executed tasks across complex workflows.
Combining AWS agent infrastructure with Datadog Agent Observability
To create a foundation for continuously improving agentic AI systems, NTT DATA combined Amazon Bedrock AgentCore with Datadog Agent Observability.
NTT DATA instrumented its agents with the Strands Agents SDK, which emits OpenTelemetry GenAI semantic conventions that Datadog ingests automatically, then propagated a shared trace context across the orchestrator, AgentCore Gateway, and MCP services.
Amazon Bedrock AgentCore provided a standardized foundation for building and operating AI agents, helping the team focus on agent behavior rather than repeatedly solving execution and infrastructure challenges. Datadog Agent Observability gave the team one place to evaluate, experiment with, and observe agent behavior from development to production. With traces, evaluations, latency, and token usage side by side in Datadog, NTT DATA could assess configurations in pre-production and keep visibility into how agents performed once they were live.
Because agent workflows span multiple layers of infrastructure and services, NTT DATA also used Datadog’s 1000+ total integrations and 100+ for AWS to monitor all relevant AWS telemetry in Datadog. By combining Datadog Agent Observability, Application Performance Monitoring (APM), Log Management, and critical AWS telemetry on a single platform, engineers could evaluate agent behavior alongside application and infrastructure performance without losing operational context.
Scaling production-grade agents faster by closing the development loop
This unified view enabled more than production monitoring. During testing, the team identified instances where third-party APIs used as agent tools were being throttled. Rather than manually correlating information across multiple systems, engineers could follow a single end-to-end trace in Datadog, since a shared trace ID connects every step of the workflow. This allowed the team to quickly identify the source of the problem, understand its impact on agent behavior, and apply what they found to improve the workflow before broader rollout. With unified tracing across agents, tools, applications, and infrastructure, the team reduced mean time to resolution by approximately 70%. “Having visibility into both agent behavior and system performance helped us validate architectures faster and move from experimentation to production with much greater confidence,” says Furutani.
Datadog also enabled the team to compare models on Amazon Bedrock, agents, and workflows using evaluations that measured both quality and efficiency. Rather than making changes and waiting to observe the impact in production, NTT DATA could evaluate alternative models and configurations against defined quality criteria before deployment. This gave the team greater confidence in optimization decisions and helped identify opportunities to use more cost-effective models while maintaining desired outcomes, reducing inference costs by approximately 50%. These evaluations helped the team optimize model selection based on both quality and efficiency requirements before deployment.
The same approach extends into a fuller pre-production workflow in Datadog. Teams turn production traces into versioned datasets, rerun experiments against them, and use the annotation queue to fold human judgment back into the examples that future changes are tested against.
“Having visibility into both agent behavior and system performance helped us validate architectures faster and move from experimentation to production with much greater confidence.”
Security runs through that same workflow. Datadog pairs evaluations with continuous scanning for prompt injection, hallucinations, and PII exposure, a defense-in-depth approach to the most common agent risks. Teams can extend that with Datadog AI Guard, which adds a runtime layer that inspects prompts, responses, and tool calls inline and blocks unsafe actions before they execute. “Datadog Agent Observability helps us understand how AI systems on Amazon Bedrock AgentCore behave, evaluate quality, and build the operational trust required for production,” says Okamoto.
Creating a repeatable framework for enterprise AI operations
For NTT DATA, the project was never solely about validating technology. The broader objective was to establish a repeatable framework that could help customers accelerate the journey from experimentation to production deployment.
By combining Amazon Bedrock AgentCore with Datadog Agent Observability, NTT DATA turned that work into a standardized way to develop and operate agentic AI. The same framework that proved out one architecture can be reused as the team takes on new customer use cases, which is what shortens the path from proof of concept to production. By standardizing evaluation, observability, and governance practices, the team also increased deployment velocity across new AI initiatives.
The initiative gave the team a stronger foundation for governance, evaluation, cost control, and operational reliability. By understanding how agents reason, use tools, and consume resources, NTT DATA can guide customers in deploying AI responsibly while improving performance over time.
The project has also contributed to knowledge sharing across the organization and the broader technology community. The team has published technical content, delivered presentations, collaborated with AWS and Datadog, and shared lessons learned from its validation efforts to help advance operational best practices for agentic AI. “Building AI systems that work is only the first step,” says Okamoto. “The next challenge is building AI systems that organizations can understand, operate, govern, and continuously improve. That’s what will enable AI to deliver long-term value at enterprise scale, and Datadog is helping us build the operational trust to do that.”
“Building AI systems that work is only the first step. The next challenge is building AI systems that organizations can understand, operate, govern, and continuously improve. That's what will enable AI to deliver long-term value at enterprise scale, and Datadog is helping us build the operational trust to do that.”
Looking ahead, NTT DATA expects evaluation, experimentation, and observability to play an increasingly important role as agent workflows grow more complex and AI systems take on greater levels of autonomy. The team is particularly interested in advancing AI SRE practices, automated analysis, and evaluation-driven development to further improve reliability, efficiency, and governance.
