ECS Sidecar Collector Integration

Overview

You can run a Datadog Agent as a sidecar container in your ECS task definitions. It can collect metrics, logs, events, and traces from containerized workloads and The agent will forward all telemetry directly to Kloudfuse.

Prerequisites

Before you begin, ensure the following requirements are in place:

Requirement Details

AWS access

Access to the AWS Console with IAM and ECS permissions to create roles and register task definitions.

Datadog Agent image

Access to the Datadog Agent container image (gcr.io/datadoghq/agent).

Create an IAM Role

Create an IAM role for the ECS tasks so the Datadog Agent can collect metrics, logs, and traces and interact with AWS services.

In the AWS Console, navigate to IAM and create a new Role with the following specifications:

Field Value

Trusted entity

ECS (Elastic Container Service)

Policy

Custom, or use AWS-managed policies:
AmazonEC2ContainerServiceforEC2Role (ECS task communication)
CloudWatchFullAccess (metrics)
AWSXRayDaemonWriteAccess (tracing)
CloudWatch Logs policy (log access)

ECS Task Definition

Add the Datadog Agent as a sidecar container in your ECS task definition. The following example task definition configures the agent to forward all signals to Kloudfuse:

{
  "containerDefinitions": [
    {
      "name": "datadog-agent",
      "image": "public.ecr.aws/datadog/agent:latest",
      "cpu": 100,
      "memory": 512,
      "essential": true,
      "environment": [
        { "name": "DD_API_KEY",                             "value": "kloudfuse" },
        { "name": "DD_URL",                                 "value": "https://<kloudfuse-hostname>/ingester" },
        { "name": "DD_LOGS_ENABLED",                        "value": "true" },
        { "name": "DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL",   "value": "true" },
        { "name": "DD_LOGS_CONFIG_FORCE_USE_HTTP",          "value": "true" },
        { "name": "DD_LOGS_CONFIG_LOGS_DD_URL",             "value": "<kloudfuse-hostname>:443" }
      ],
      "mountPoints": [
        { "sourceVolume": "docker_sock",      "containerPath": "/var/run/docker.sock" },
        { "sourceVolume": "cgroup",           "containerPath": "/host/sys/fs/cgroup" },
        { "sourceVolume": "proc",             "containerPath": "/host/proc" },
        { "sourceVolume": "pointdir",         "containerPath": "/opt/datadog-agent/run", "readOnly": false },
        { "sourceVolume": "containers_root",  "containerPath": "/var/lib/docker/containers", "readOnly": true }
      ],
      "linuxParameters": {
        "initProcessEnabled": true
      }
    }
  ],
  "family": "datadog-agent-task",
  "taskRoleArn": "arn:aws:iam::<aws-account-id>:role/<custom-ecs-iam-role>",
  "executionRoleArn": "arn:aws:iam::<aws-account-id>:role/<custom-ecs-iam-role>",
  "compatibilities": ["EXTERNAL", "EC2"]
}
json

The five mount points serve different purposes:

Mount point Purpose Required for logs?

/var/run/docker.sock

Docker socket — container discovery and metrics collection

No

/host/sys/fs/cgroup

cgroup filesystem — CPU and memory metrics

No

/host/proc

proc filesystem — system-level metrics

No

/opt/datadog-agent/run (pointdir)

Stores log read offsets so the agent resumes from where it left off after a restart

Yes

/var/lib/docker/containers (containers_root)

Direct read access to container log files on the host

Yes

If you are not collecting logs, remove pointdir and containers_root from mountPoints.

This task definition uses "compatibilities": ["EXTERNAL", "EC2"]. The mount point approach above requires the EC2 launch type. ECS Fargate does not expose the host Docker socket or container filesystem, so Fargate log collection requires a different approach — see Datadog ECS Log Collection.

Metrics (Automatic)

The Datadog Agent automatically collects ECS-specific metrics — CPU, memory, disk, network, and ECS task-level utilization — with no additional configuration once the sidecar is running.

Enable Logs

Log collection requires DD_LOGS_ENABLED=true and DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL=true in the agent container environment (included in the task definition above), plus the pointdir and containers_root mount points.

The agent reads logs directly from container log files on the host filesystem. It collects only from containers whose log driver is default (the Docker json-file driver) or json-file.

Containers using the awslogs log driver write directly to CloudWatch Logs and bypass the host filesystem entirely. The Datadog Agent sidecar cannot collect logs from those containers. To ingest awslogs output in Kloudfuse, use the CloudWatch Logs integration instead.

Set DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL=false and use Autodiscovery labels on individual containers if you want to collect logs from only specific containers rather than all containers in the task.

Events

The Datadog Agent sidecar collects ECS container lifecycle events automatically — task state changes, container starts and stops, and OOM kills. No additional environment variables or configuration are required beyond having the sidecar running in the task definition.

To send custom application-level events from your code, use the DogStatsD API on port 8125. The agent exposes DogStatsD on that port by default.

Enable Traces

APM tracing requires the Datadog tracing SDK installed in each application container and a network path from the application container to the agent sidecar. Two connectivity methods are supported:

Method How it works When to use

Unix Domain Socket (UDS)

Both containers mount a shared volume at /var/run/datadog; the app writes traces to the socket file at that path

Recommended for Agent v7.18 and later on EC2

TCP/IP

The app sends traces to the agent over TCP on port 8126 using the EC2 instance’s private IP

Required when host volumes cannot be used, or for non-root agent containers

Option A: Unix Domain Socket (Recommended)

Add a shared dd-sockets volume to the task definition and mount it in both the agent container and each application container:

{
  "volumes": [
    {
      "name": "dd-sockets",
      "host": { "sourcePath": "/var/run/datadog" }
    }
  ],
  "containerDefinitions": [
    {
      "name": "datadog-agent",
      "mountPoints": [
        { "sourceVolume": "dd-sockets", "containerPath": "/var/run/datadog", "readOnly": false }
      ]
    },
    {
      "name": "my-app",
      "environment": [
        { "name": "DD_TRACE_AGENT_URL",  "value": "unix:///var/run/datadog/apm.socket" },
        { "name": "DD_DOGSTATSD_URL",    "value": "unix:///var/run/datadog/dsd.socket" }
      ],
      "mountPoints": [
        { "sourceVolume": "dd-sockets", "containerPath": "/var/run/datadog", "readOnly": true }
      ]
    }
  ]
}
json

Option B: TCP/IP

Expose port 8126 on the agent container and set DD_APM_NON_LOCAL_TRAFFIC=true so the agent accepts connections from other containers:

{
  "name": "datadog-agent",
  "portMappings": [
    { "hostPort": 8126, "containerPort": 8126, "protocol": "tcp" }
  ],
  "environment": [
    { "name": "DD_APM_NON_LOCAL_TRAFFIC", "value": "true" }
  ]
}
json

In each application container, set DD_AGENT_HOST to the EC2 instance’s private IP address. The instance IP is available from the ECS container metadata file:

{
  "name": "my-app",
  "environment": [
    { "name": "DD_AGENT_HOST", "value": "169.254.169.254" }
  ],
  "entryPoint": [
    "sh", "-c",
    "export DD_AGENT_HOST=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4) && <your-start-command>"
  ]
}
json
DD_AGENT_HOST=localhost is not correct on the EC2 launch type. On EC2, each container has its own network namespace; localhost inside the app container refers to the app itself, not the agent sidecar. Use the host private IP or UDS instead.

References

  • Enrichment IAM Permissions — IAM policy documents for Kloudfuse AWS integrations, including the ECS task role permissions required by the Datadog Agent sidecar.

  • AWS CloudWatch Logs Integration — Alternative log ingestion path for containers using the awslogs log driver, which bypasses the Datadog Agent sidecar.

  • Kloudfuse Enrichment — Helm configuration for attaching AWS resource metadata (ECS cluster, task, and container tags) to ingested telemetry.

External References