Centralize and govern your OpenTelemetry pipeline with the DDOT gateway

As large organizations adopt OpenTelemetry (OTel) to standardize their observability, maintaining consistency across hundreds of services and teams becomes a challenge. Collecting telemetry data at the node level is only a part of the solution; platform and SRE teams still need an effective way to control how data is processed, governed, and routed before it reaches observability providers. Without a centralized layer, they must manage hundreds of distributed collectors, reconcile inconsistent tagging practices, and manually route telemetry data to multiple destinations, all while keeping costs predictable.

The DDOT gateway, now in public Preview, extends the Datadog Distribution of the OTel (DDOT) Collector with a scalable gateway deployment model that gives organizations centralized management of their OpenTelemetry pipelines. This means platform teams can apply consistent governance, perform advanced data pre-processing at the edge, and route data to multiple destinations from a single enterprise-supported source.

In this post, we’ll explore how the DDOT gateway helps organizations:

• Centralize telemetry processing and governance across teams
• Perform advanced data pre-processing and routing at the edge
• Gain end-to-end visibility, elastic scalability, and centralized fleet management with Datadog’s enterprise OpenTelemetry gateway solution

Centralize telemetry processing and governance across teams

Large enterprises often run hundreds of distributed workloads across multiple groups, each maintaining its own observability configuration. Over time, this leads to inconsistent tagging, incompatible schemas, and data quality issues that slow down investigations and drive up ingestion costs. Unlike DaemonSet deployments where one Collector instance runs on every Kubernetes node, a gateway deployment runs one or more Collectors as a centralized, standalone service, giving platform and SRE teams a unified control point for managing telemetry across the organization.

With the DDOT gateway, teams can standardize processing policies in one place rather than updating every node-level collector individually. This makes it easier to enforce consistent tagging and schema requirements, for example ensuring that all production services include env:prod or automatically adding team:ABC when telemetry data contains service:web-store. Instead of relying on each team to maintain its own pipelines, platform engineers can define governance rules once and have them applied everywhere.

The diagram below shows an end-to-end OTel pipeline where each node runs a DaemonSet-based DDOT Collector that consists of a core Agent and an OTel Agent (as separate containers). Applications send OTLP telemetry data to these node-local DaemonSets and are then passed to a centralized DDOT gateway deployment for processing needs. All telemetry data is then forwarded to Datadog or other backends of choice.

Close dialog

Rather than coordinating rolling updates to hundreds or thousands of DaemonSets, administrators can update a single gateway cluster. This allows teams to ship policy updates faster, maintain stronger data consistency, and reduce the risk of configuration drift across their environments.

Perform advanced data pre-processing and routing at the edge

As organizations scale telemetry data collection, cost and signal quality become major concerns. Node-level collectors often lack the context or compute resources to perform advanced processing such as tag aggregation or tail-based sampling.

The DDOT gateway introduces a consolidation layer in the telemetry pipeline architecture where teams can apply sophisticated transformations and routing logic before data enters their observability backend. Gateway deployments support advanced operations such as tail-based sampling, which evaluates traces before deciding what to keep. By running tail-based sampling at the gateway, platform teams can centralize sampling logic, standardize policies across all services, and ingest only the most meaningful traces without requiring each engineering team to manage custom sampling configurations. In the following values.yaml example, the gateway Collector (otelAgentGateway) is configured with the tail_sampling processor. The gateway waits for a complete trace (decision_wait) and then applies specified sampling policies before exporting sampled traces to Datadog:

1
otelAgentGateway:
2
  enabled: true
3
  replicas: 3
4
  ports:
5
    - containerPort: 4317
6
      name: "otel-grpc"
7
  config: |
8
    receivers:
9
      otlp:
10
        protocols:
11
          grpc:
12
            endpoint: "0.0.0.0:4317"
13
    processors:
14
      tail_sampling:
15
        decision_wait: 10s
16
        policies: <Add your sampling policies here>
17
    connectors:
18
      datadog/connector:
19
    exporters:
20
      datadog:
21
        api:
22
          key: ${env:DD_API_KEY}
23
    service:
24
      pipelines:
25
        traces/sample:
26
          receivers: [otlp]
27
          processors: [tail_sampling]
28
          exporters: [datadog]

Gateways allow teams to aggregate and enrich tags to compute summaries or normalize tagging near the source. They also enable schema transformation to align telemetry sources under shared conventions.

In addition to data processing, the DDOT gateway improves routing flexibility. Teams can centrally route telemetry data to multiple destinations—such as regional backends for compliance or failover, or a data lake for internal analytics—by using OTLP exporters. For example, organizations can dual ship all telemetry data to Datadog and to a secondary storage system or forward business-critical usage metrics from specific teams to an internal data lake for business trend analysis. Instead of managing routing logic across many distributed collector DaemonSets, administrators can maintain the entire topology from the gateway layer.

Gain end-to-end visibility, elastic scalability, and fleet management with Datadog’s enterprise OpenTelemetry solution

Gateway deployments with DDOT also provide full life cycle management, supporting automatic horizontal scaling through Kubernetes’ Horizontal Pod Autoscaler (HPA). When otelAgentGateway.autoscaling is enabled (as is shown in the following code block), the DDOT gateway elastically adjusts to traffic volume, maintaining high throughput and preventing dropped telemetry during surges. Autoscaling can be driven by CPU or memory utilization, custom Kubernetes metrics, or external metrics sourced through the Datadog Cluster Agent, providing the flexibility to scale based on real operational demand.

1
targetSystem: "linux"
2
agents:
3
  enabled: false
4
clusterAgent:
5
  enabled: false
6
otelAgentGateway:
7
  enabled: true
8
  ports:
9
    - containerPort: "4317"
10
      name: "otel-grpc"
11
  config: | <YOUR CONFIG>
12
  replicas: 4  # 4 replicas to begin with and HPA may override it based on the metrics
13
  autoscaling:
14
    enabled: true
15
    minReplicas: 2
16
    maxReplicas: 10
17
    metrics:
18
      # Aim for high CPU utilization for higher throughput
19
      - type: Resource
20
        resource:
21
          name: cpu
22
          target:
23
            type: Utilization
24
            averageUtilization: 80
25
    behavior:
26
      scaleUp:
27
        stabilizationWindowSeconds: 30
28
      scaleDown:
29
        stabilizationWindowSeconds: 60

Teams can observe and monitor their gateway infrastructure health by using the same dashboards, alerts, and telemetry they use for their applications. Platform teams can track gateway performance, resource usage, throughput, and error rates to quickly diagnose bottlenecks or misconfigurations that might impact downstream consumers, giving them confidence that the gateway layer is operating reliably at scale.

These capabilities are further enhanced with Datadog Fleet Automation, which supports gateway deployments alongside the DDOT Collector, other OTel Collectors, and Datadog Agent deployments. Fleet Automation automatically discovers gateway collectors across clusters and environments, surfaces their live configurations, and provides metadata such as version and distribution type. This gives teams a centralized place to validate routing logic, detect configuration drift, and troubleshoot issues without jumping between repos, terminals, and cluster views.

With unified monitoring, autoscaling, and centralized fleet management, the DDOT gateway becomes a resilient, self-managing control plane for enterprise telemetry, providing consistent ingestion, routing, and governance even as workloads fluctuate across clusters and regions.

Build a unified OpenTelemetry pipeline with the DDOT gateway

The DDOT gateway enables organizations to centrally manage, govern, and optimize their OTel telemetry pipelines. With a dedicated gateway layer, platform and SRE teams can standardize governance policies, apply advanced pre-processing to control costs and improve data quality, and gain full visibility into the health of their telemetry infrastructure.

To get started, check out our documentation on the Datadog Distribution of the OpenTelemetry Collector. If you’re new to Datadog, sign up for a 14-day free trial.