Datadog Agent for AWS ECS Clusters for Logs

The Datadog Agent for AWS ECS Clusters collects logs from your ECS tasks and Datadog containers, allowing you to centralize log data from your containerized applications. To enable log collection, you must configure the appropriate environment variable on the Datadog agent container. This integration provides visibility into application and system logs running on your ECS clusters.

Create an IAM Role for ECS Tasks
Install Datadog Agent on ECS
Collect Logs

Create an IAM Role for ECS Tasks

Create an IAM role for the ECS tasks, so they can send data to Kloudfuse and interact with AWS services.

You can use the following policy to allow Datadog Agent to collect metrics, logs, and traces.

Navigate to IAM in the AWS Console, and create a new Role with the following specifications:

Trusted entity: ECS (Elastic Container Service)

Policy

Custom, or use AWS-managed policies:

AmazonEC2ContainerServiceforEC2Role, for ECS task communication
CloudWatchFullAccess, for metrics
AWSXRayDaemonWriteAccess, for tracing
logs policy, for CloudWatch logs access

Install Datadog Agent on ECS

Datadog provides a Docker image that runs as an agent inside the ECS containers. To use this image, configure ECS Task Definitions.

Create an ECS Task Definition with Datadog Agent.

Modify your ECS service’s task definition to run the Datadog agent as a sidecar container.

{
"containerDefinitions": [
{
    "name": "datadog-agent",
    "image": "public.ecr.aws/datadog/agent:latest",
    "cpu": 100,
    "memory": 512,
    "essential": true,
    "environment": [
        {
            "name": "DD_LOGS_ENABLED",
            "value": "true"
        },
        {
            "name": "DD_API_KEY",
            "value": "kloudfuse"
        },
        {
            "name": "DD_URL",
            "value": "https://<kf-domain-name>/ingester"
        },
        {
            "name": "DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL",
            "value": "true"
        },
        {
            "name": "DD_LOGS_CONFIG_FORCE_USE_HTTP",
            "value": "true"
        },
        {
            "name": "DD_LOGS_CONFIG_LOGS_DD_URL",
            "value": "<kf-domain-name>:443"
        },
    ],
    "mountPoints": [
        {
            "sourceVolume": "docker_sock",
            "containerPath": "/var/run/docker.sock"
        },
        {
            "sourceVolume": "cgroup",
            "containerPath": "/host/sys/fs/cgroup"
        },
        {
            "sourceVolume": "proc",
            "containerPath": "/host/proc"
        },
        {
            "sourceVolume": "pointdir",
            "containerPath": "/opt/datadog-agent/run",
            "readOnly": false
        },
        {
            "sourceVolume": "containers_root",
            "containerPath": "/var/lib/docker/containers",
            "readOnly": true
        }
    ],
    "linuxParameters": {
        "initProcessEnabled": true
    },
}
],
"family": "datadog-agent-task",
"taskRoleArn": "arn:aws:iam::<aws-account-id>:role/<custom-ecs-iam-role>",
"executionRoleArn": "arn:aws:iam::<aws-account-id>:role/<custom-ecs-iam-role>",
"placementConstraints": [],
"compatibilities": [
"EXTERNAL",
"EC2"
],
"tags": []
}

code

Collect Logs

Collect logs from both ECS tasks and Datadog containers. To start sending logs, ensure that the DD_LOGS_ENABLED=true is set in the environment for the Datadog agent container.

{
"environment": [
{
"name": "DD_LOGS_ENABLED",
"value": "true"
}
]
}

json