Datadog Agent on Kubernetes

Deploy the Datadog Agent on Kubernetes using Helm to collect metrics, logs, events, and traces from your cluster and forward them to Kloudfuse.

Requirements

When repointing an existing Datadog Agent at Kloudfuse instead of Datadog HQ, the agent version must be 7.41 or higher.

Check the Helm chart version — must be 3.1.10 or higher:

helm list -n <namespace-where-agent-is-installed>

Check the agent image version:

kubectl describe pod -n <namespace-where-agent-is-installed> | grep Image

The image tag should be 7.41 or later (for example, gcr.io/datadoghq/agent:7.41.0).

Install the Datadog Agent

Add the Datadog Helm repository:

helm repo add datadog https://helm.datadoghq.com
helm repo update

Create a datadog-values.yaml with your Kloudfuse hostname. This configuration enables logs, traces, metrics, and orchestrator exploration:

datadog:
  apiKey: " "
  dd_url: "https://<kloudfuse-hostname>/ingester"   (1)
  logsEnabled: true
  logs:
    enabled: true
    containerCollectAll: true
    containerCollectUsingFiles: true
    autoMultiLineDetection: true
  kubeStateMetricsEnabled: false
  kubeStateMetricsCore:
    enabled: true
    ignoreLegacyKSMCheck: true
  orchestratorExplorer:
    enabled: true
  apm:
    enabled: true
    portEnabled: true
  dogstatsd:
    port: 8125
    useHostPort: true
  processAgent:
    enabled: true
  prometheusScrape:
    enabled: true
    version: 1

clusterAgent:
  replicaCount: 1
  enabled: true
  datadog_cluster_yaml:
    use_v2_api:
      events: true
      series: true
      service_checks: true
    orchestrator_explorer:
      orchestrator_dd_url: "https://<kloudfuse-hostname>"   (1)
  admissionController:
    enabled: false

agents:
  useConfigMap: true
  customAgentConfig:
    use_v2_api:
      events: true
      series: true
      service_checks: true
    logs_config:
      logs_dd_url: "<kloudfuse-hostname>:443"   (1)
      logs_no_ssl: false
      use_http: true
      auto_multi_line_detection: true
      use_v2_api: false
    apm_config:
      enabled: true
      apm_non_local_traffic: true
      apm_dd_url: "<kloudfuse-hostname>:443/ingester"   (1)
    orchestrator_explorer:
      orchestrator_dd_url: "https://<kloudfuse-hostname>"   (1)
    metadata_providers:
      - name: host
        interval: 300
yaml
1 Replace <kloudfuse-hostname> with your Kloudfuse cluster hostname.

Deploy the agent:

kubectl create namespace datadog-agent
helm upgrade --install datadog-agent -f datadog-values.yaml datadog/datadog \
  -n datadog-agent \
  --version 3.65.0

Dual Shipping to Kloudfuse and Datadog

The Datadog Agent supports dual shipping — sending telemetry to both Datadog and Kloudfuse simultaneously. Add the following to your existing Helm values file instead of replacing dd_url.

Agent version 7.41 or higher and Helm chart version 3.1.10 or higher are required.

Under the agents.customAgentConfig key:

agents:
  customAgentConfig:
    additional_endpoints:
      "https://<kloudfuse-hostname>/ingester":
        - <api_key>
    use_v2_api:
      series: true
    logs_config:
      logs_no_ssl: false
      use_v2_api: false
      additional_endpoints:
        - api_key: <api_key>
          Host: <kloudfuse-hostname>
          Port: 443
          use_compression: true
    apm_config:
      additional_endpoints:
        "https://<kloudfuse-hostname>":
          - <api_key>
    process_config:
      additional_endpoints:
        "https://<kloudfuse-hostname>":
          - <api_key>
    metadata_providers:
      - name: host
        interval: 300
yaml

Under the clusterAgent.datadog_cluster_yaml key:

clusterAgent:
  datadog_cluster_yaml:
    additional_endpoints:
      "https://<kloudfuse-hostname>/ingester":
        - <api_key>
    use_v2_api:
      series: true
    process_config:
      additional_endpoints:
        "https://<kloudfuse-hostname>":
          - <api_key>
    orchestrator_explorer:
      additional_endpoints:
        "https://<kloudfuse-hostname>":
          - <api_key>
yaml

Apply the changes:

helm upgrade datadog-agent -f datadog-values.yaml datadog/datadog -n <namespace>

Configure Logs

Log collection from all containers is enabled by the base values file above. To exclude specific containers from log collection:

datadog:
  env:
    - name: DD_CONTAINER_EXCLUDE_LOGS
      value: "name:<container-name>"
yaml

To apply custom parsing or masking rules before forwarding:

agents:
  customAgentConfig:
    logs_config:
      processing_rules:
        - type: mask_sequences
          name: mask_credit_cards
          replace_placeholder: "[MASKED]"
          pattern: \b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b
yaml

To override log settings for a specific pod, use Autodiscovery annotations:

annotations:
  ad.datadoghq.com/<CONTAINER_NAME>.logs: >
    [{"source": "my-app", "service": "my-service"}]
yaml

Configure Events

The Cluster Agent collects Kubernetes events (pod scheduling, node conditions, deployment rollouts) via the Kubernetes API. To forward process-level and container lifecycle events, add the following:

clusterAgent:
  datadog_cluster_yaml:
    process_config:
      events_dd_url: "https://<kloudfuse-hostname>/ingester"

agents:
  customAgentConfig:
    process_config:
      events_dd_url: "https://<kloudfuse-hostname>/ingester"
      container_collection:
        enabled: false
yaml

Verify Events Are Arriving

Kubernetes events collected by the Datadog Cluster Agent are stored in the Events store. Query them in the Kloudfuse UI or via the /events-query GraphQL API:

{
  events(
    durationSecs: 300,
    filter: { and: [{eq: {name: "@source", value: "kubernetes"}}] },
    timestamp: "<ISO-8601-timestamp>",
    limit: 20
  ) {
    id title text severity source eventType host
    labels { name value }
    timestamp
  }
}
graphql

Events from a specific pod appear with labels kube_namespace and kube_name. The eventType field is kubernetes_apiserver for events collected by the Cluster Agent.

Datadog Agent Kubernetes events appear in the Events store (source="kubernetes"). This differs from the OTel Collector path, where Kubernetes events are stored in Logs (tagged kf_events_agent=otlp).

Configure Metrics

Kube-State-Metrics

Kube-state-metrics is enabled by the base values file above. Set kubeStateMetricsEnabled: false to avoid running the legacy KSM check alongside the core check.

DogStatsD Custom Metrics

Configure application pods to send custom metrics to the agent on the node’s host IP:

env:
  - name: DD_AGENT_HOST
    valueFrom:
      fieldRef:
        fieldPath: status.hostIP
  - name: DD_ENTITY_ID
    valueFrom:
      fieldRef:
        fieldPath: metadata.uid
yaml

Prometheus Scraping

Prometheus autodiscovery is enabled by the base values file above. Add the following annotations to any pod that exposes a Prometheus endpoint:

prometheus.io/path: /metrics
prometheus.io/port: "9090"
prometheus.io/scrape: "true"
yaml

For explicit per-container configuration using Autodiscovery annotations:

ad.datadoghq.com/<CONTAINER_NAME>.check_names: '["openmetrics"]'
ad.datadoghq.com/<CONTAINER_NAME>.init_configs: '[{}]'
ad.datadoghq.com/<CONTAINER_NAME>.instances: >
  [{"openmetrics_endpoint": "http://%%host%%:%%port%%/metrics"}]
yaml

High-Cardinality Tags

The Datadog Agent defaults to low cardinality, tagging metrics only at the host level. Setting cardinality to orchestrator enables granular pod- and container-level tags:

datadog:
  env:
    - name: DD_CHECKS_TAG_CARDINALITY
      value: "orchestrator"
yaml

See Datadog — Tag Cardinality for details.

Custom Tags

Add custom tags to all metrics collected by the agent. Each tag must be in key:value format:

datadog:
  tags:
    - custom_tag_name:custom_tag_value
yaml

External References