Fluent Bit for AWS ECS/ECS Fargate Container Logs

An Amazon Elastic Container Service (Amazon ECS) task definition allows you to specify only a single log configuration object for a container. This limit means that you can forward logs to a single destination, only. See AWS documentation for LogConfiguration. To forward logs to multiple destinations in Amazon ECS on Fargate, you can use FireLens.

You can work around this restriction by integrating with Fluent Bit, through these primary tasks:

Create a Custom Fluent Bit docker image
Deploy Fluent Bit Container in ECS

Create a Custom Fluent Bit docker image

Create a custom Fluent Bit configuration file, logDestinations.conf, and specify your choice of [OUTPUT] definitions.

[OUTPUT]
        Name http
        Match *
        Host <Kloudfuse DNS>
        Port 443
        TLS on
        TLS.debug 4
        URI /ingester/v1/fluent_bit
        Format {json|json_lines|json_stream|msgpack}

Create a Docker image using the custom Fluent Bit output configuration file.

Use the Dockerfile latest.
```
FROM amazon/aws-for-fluent-bit:latest

ADD logDestinations.conf /logDestinations.conf
```
Using the Dockerfile from the previous step, run this command:
```
docker build -t custom-fluent-bit:latest .
```

Push your image to the ECR Image repo from where the ECS Containers pull images.

docker push aws_account_id.dkr.ecr.region.amazonaws.com/custom-fluent-bit:latest

Deploy Fluent Bit Container in ECS

In the task definition, update the options for your FireLens configuration:

{
  "containerDefinitions": [
    {
      "essential": true,
      "image": "aws_account_id.dkr.ecr.region.amazonaws.com/custom-fluent-bit:latest",
      "name": "log_router",
      "firelensConfiguration": {
        "type": "fluentbit",
        "options": {
          "config-file-type": "file", (1)
          "config-file-value": "/logDestinations.conf"
        }
      }
    }
  ]
}

1	Add options here.

For details of ECS FluentBit task definition, example see AWS documentation on Example Amazon ECS task definition: Route logs to FireLens