Fluent Bit

Kloudfuse accepts logs from Fluent Bit using the built-in HTTP output plugin. Point the output at the Kloudfuse ingester endpoint and Fluent Bit delivers log records directly — no additional plugins or protocol adapters required.

Overview

Fluent Bit is a lightweight, open-source log processor and forwarder designed for containerised and cloud environments. It uses a pipeline model: input plugins collect data, filter plugins enrich and transform it, and output plugins deliver it to a destination.

Kloudfuse exposes an ingester endpoint compatible with the Fluent Bit HTTP output plugin:

https://<kloudfuse-hostname>/ingester/v1/fluent_bit

Supported payload formats: json, json_lines, json_stream, and msgpack.

Prerequisites

Configure Fluent Bit

HTTP Output Plugin

Add an [OUTPUT] block to your Fluent Bit configuration to forward logs to Kloudfuse:

[OUTPUT]
    Name        http
    Match       *
    Host        <kloudfuse-hostname>
    Port        443
    TLS         On
    URI         /ingester/v1/fluent_bit
    Format      json
ini

If ingestion authentication is enabled, add the API key as a header:

[OUTPUT]
    Name        http
    Match       *
    Host        <kloudfuse-hostname>
    Port        443
    TLS         On
    URI         /ingester/v1/fluent_bit
    Format      json
    Header      Kf-Api-Key <token>
ini

For internal Kloudfuse endpoints (within the same cluster), use port 80 and omit TLS On.

Helm Installation

Fluent Bit is commonly deployed as a Kubernetes DaemonSet using the fluent/fluent-bit Helm chart.

config:
  outputs: |
    [OUTPUT]
        Name        http
        Match       *
        Host        <kloudfuse-hostname>
        Port        443
        TLS         On
        URI         /ingester/v1/fluent_bit
        Format      json
        Header      Kf-Api-Key <token>
yaml

Apply with:

helm repo add fluent https://fluent.github.io/helm-charts
helm repo update
helm upgrade --install fluent-bit fluent/fluent-bit \
  -f custom-values.yaml \
  --namespace logging \
  --create-namespace

Enrich Logs with Filters

Kubernetes Metadata

Use the kubernetes filter to enrich log events with pod name, namespace, labels, and container metadata. Enable this filter in any Kubernetes environment:

[FILTER]
    Name                kubernetes
    Match               *
    Merge_Log           On
    Keep_Log            Off
    K8S-Logging.Parser  On
    K8S-Logging.Exclude On
ini

Kloudfuse automatically uses the kubernetes.container_name field as the log source when this filter is active.

See Fluent Bit — Kubernetes Filter for the full list of options.

AWS Metadata

Add the aws filter to attach availability zone, account ID, and instance type to log records from EC2-hosted workloads:

[FILTER]
    Name            aws
    Match           *
    az              true
    account_id      true
    ec2_instance_type true
ini

To add a cluster name label, chain a modify filter:

[FILTER]
    Name    modify
    Match   *
    Add     cluster_name <CLUSTER_NAME>
ini

If you use a key name other than cluster_name or clusterName, map it to the Kloudfuse cluster label in your Kloudfuse custom-values.yaml:

kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_cloud_cluster_name:
            - "$.<KEY_FOR_CLUSTER_NAME>"   # must be a JSONPath expression
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="
yaml

See Fluent Bit — AWS Metadata Filter for the full list of options.

Static Custom Labels

Use the modify filter to add static labels to every log record:

[FILTER]
    Name    modify
    Match   *
    Add     tenant_name <TENANT_NAME>
ini

By default, static labels appear as log facets in Kloudfuse. To promote them to log labels and tags instead, add the following to the logs-parser section of your Kloudfuse custom-values.yaml:

kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_additional_tags:
            - "$.tenant_name"   # must be a JSONPath expression
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="
yaml

Label and Tag Extraction

Log Source

By default, Kloudfuse uses container_name from the Kubernetes filter as the log source. To use a different field, add the following to the logs-parser section of your Kloudfuse custom-values.yaml:

kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_source:
            - "$.<KEY_FOR_LOG_SOURCE>"   # must be a JSONPath expression
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="
yaml

Log Message

Kloudfuse looks for the log message text under the following keys by default: log, LOG, Log, message, msg, MSG, and Message.

To specify a different key, add the following to the logs-parser section of your Kloudfuse custom-values.yaml:

kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_msg:
            - "$.<MSG_KEY_FROM_AGENT_CONFIG>"   # must be a JSONPath expression
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="
yaml

Structured Key-Value Pairs

Fluent Bit parsers extract key-value pairs from unstructured log lines. Kloudfuse adds all extracted pairs as log facets by default.

To promote a set of key prefixes to log labels and tags instead, add the following to the logs-parser section of your Kloudfuse custom-values.yaml:

kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_additional_tags:      (1)
            - "$.<PREFIX_KEY_FOR_AGENT_KV>"   # must be a JSONPath expression
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="
yaml
1 kf_additional_tags is a list of key prefixes. Any top-level JSON key matching a prefix is stored as a log label/tag rather than a facet.
Keys already used as log source, message, or Kubernetes/AWS metadata are automatically excluded from facets regardless of this setting.

Labels from HTTP Headers

Kloudfuse can extract custom labels from HTTP request headers sent with the log payload. Configure this in the ingester section of your Kloudfuse custom-values.yaml:

ingester:
  config:
    labelsFromHeaders:
      X-KFUSE-ENV: "env"
      X-KFUSE-CUSTOM: "custom_key_1"
yaml

When Kloudfuse finds a matching header (for example, X-KFUSE-ENV: prod), it stores the value as a label with the mapped name (env: prod).

AWS ECS and Fargate with FireLens

ECS task definitions support only a single log configuration per container. To forward logs to Kloudfuse from ECS on Fargate, use AWS FireLens with a custom Fluent Bit image.

Build a Custom Fluent Bit Image

  1. Create a Fluent Bit configuration file named logDestinations.conf with the Kloudfuse HTTP output:

    [OUTPUT]
    Name        http
    Match       *
    Host        <kloudfuse-hostname>
    Port        443
    TLS         On
    TLS.debug   4
    URI         /ingester/v1/fluent_bit
    Format      json
    Header      Kf-Api-Key <token>
    ini

  2. Create a Dockerfile that adds the configuration to the AWS Fluent Bit base image:

    FROM amazon/aws-for-fluent-bit:latest

ADD logDestinations.conf /logDestinations.conf

  3. Build and push the image to ECR:

    docker build -t custom-fluent-bit:latest .
docker push <aws-account-id>.dkr.ecr.<region>.amazonaws.com/custom-fluent-bit:latest

Deploy as a FireLens Log Router

Add a log_router container to your ECS task definition, referencing the custom image:

{
  "containerDefinitions": [
    {
      "essential": true,
      "image": "<aws-account-id>.dkr.ecr.<region>.amazonaws.com/custom-fluent-bit:latest",
      "name": "log_router",
      "firelensConfiguration": {
        "type": "fluentbit",
        "options": {
          "config-file-type": "file",
          "config-file-value": "/logDestinations.conf"
        }
      }
    }
  ]
}
json

See AWS — FireLens Task Definition Examples for further options.

Kinesis Firehose Fan-Out

Kloudfuse includes an embedded Fluent Bit sidecar that can forward logs received from Kinesis Firehose to a secondary backend (such as Elasticsearch) alongside Kloudfuse ingestion. Use this when you need to fan out log data to a second datastore.

The data path is: Kinesis Firehose → Kloudfuse ingester → embedded Fluent Bit → Elasticsearch.

Enable the Fluent Bit Sidecar

Add the following sections to your Kloudfuse custom-values.yaml:

  1. Enable the kfuse-fbit sidecar:

    kfuse-fbit:
  enabled: true
    yaml

  2. Configure the ingester to route logs through Fluent Bit:

    ingester:
  config:
    logs:
      msgFormat: fluent-bit   (1)
    yaml
    1 Accepted values: kfuse (default — Kloudfuse only), fluent-bit (forward to Elasticsearch only), all (both Kloudfuse and Elasticsearch).

  3. Configure the embedded Fluent Bit deployment:

    fluent-bit:
  affinity: {}
  tolerations: []
  config:
    inputs: |-
      [INPUT]
          Name    http
          listen  0.0.0.0
          port    9880
    outputs: |-
      [OUTPUT]
          Name            es
          Match           *
          Host            <ES_HOST>
          Index           <ES_INDEX_NAME>
          HTTP_User       <ES_USER>
          HTTP_Passwd     <ES_PASSWD>
          Retry_Limit     False
          compress        gzip
          Suppress_Type_Name On
  extraPorts:
    - containerPort: 9880
      name: in-http
      port: 9880
      protocol: TCP
      targetPort: 9880
  imagePullSecrets:
    - name: kfuse-image-pull-credentials
  kind: Deployment
    yaml

Routing Rules

The embedded Fluent Bit routes events by tag. To forward logs to different Elasticsearch hosts or indices, use the rewrite_tag filter — see Fluent Bit — Rewrite Tag.

Each rewrite_tag rule re-injects the record from the beginning of the pipeline with the new tag. Ensure your rules do not create a cycle.

Add, Remove, and Modify Fields

Use the record_modifier filter to add, remove, or rename fields before forwarding — see Fluent Bit — Record Modifier.

Monitor Fluent Bit Metrics

To collect Fluent Bit internal metrics and scrape them into Kloudfuse via Prometheus, add the following to your custom-values.yaml:

config:
  inputs: |-
    [INPUT]
        Name            fluentbit_metrics
        Tag             internal_metrics
        scrape_interval 15
  outputs: |-
    [OUTPUT]
        Name    prometheus_exporter
        Match   internal_metrics
        host    0.0.0.0
        port    8080

extraPorts:
  - containerPort: 8080
    name: out-metrics
    port: 8080
    protocol: TCP
    targetPort: 8080

podAnnotations:
  prometheus.io/path: /metrics
  prometheus.io/port: "8080"
  prometheus.io/scrape: "true"
yaml
The inputs and outputs keys in these blocks are merged with the base configuration — ensure there are no conflicting keys.

Verify Logs Are Arriving

After restarting Fluent Bit with the updated configuration, generate some log output, then confirm logs are arriving in the Kloudfuse UI:

  1. Click the Logs tab to open the Logs search view.

  2. In the Filters panel on the left, locate the Source facet and select fluent-bit, or type fluent-bit in the search bar to filter by source.

  3. Set the time range to the last 15 minutes using the interval picker in the top-right corner.

  4. Confirm that log entries appear in the list.

  5. Click any log row to open the Detail view. The Facets section displays parsed fields and any custom labels (such as tenant_name). The Kubernetes Labels section shows enriched metadata such as namespace, pod name, and container name when the Kubernetes filter is enabled.

  6. To narrow results by Kubernetes namespace, container, or a custom label value, click the relevant value in the Kubernetes Labels or Facets section and choose Filter By.

Troubleshooting

No Logs Appearing in Kloudfuse

If source="fluent-bit" returns no results:

  1. Check Fluent Bit logs for output plugin errors:

    # Kubernetes DaemonSet:
kubectl logs -n logging -l app=fluent-bit --tail=50

# Systemd:
journalctl -u fluent-bit --since "5 minutes ago" | grep -iE "error|warn|http"

  2. Confirm the URI is exactly /ingester/v1/fluent_bit. A missing or wrong path returns 404.

  3. Verify the Kloudfuse hostname is reachable from the Fluent Bit host or pod:

    curl -sf https://<kloudfuse-hostname>/health/ && echo "reachable"

  4. Confirm the Match pattern in the [OUTPUT] block covers the tags produced by your inputs — use * to match everything.

401 Unauthorized Errors

If Fluent Bit logs show HTTP 401 responses from the ingester:

  • Ingestion authentication is enabled on your cluster.

  • Add Header Kf-Api-Key <token> to the [OUTPUT] block — see Configure HTTP Output.

  • Confirm the token is active in Admin > Settings > Auth key labels in the Kloudfuse UI.

Missing Kubernetes Labels in Kloudfuse

If pod, namespace, or container fields are absent from log records:

  • Confirm the [FILTER] Name kubernetes block is present and Match * covers your log tags.

  • Ensure the Fluent Bit service account has RBAC permissions to read pod and namespace resources.

  • Set Merge_Log On to parse JSON logs emitted by containers — without it, the raw log string is forwarded unparsed.

ECS FireLens Logs Not Arriving

If logs from ECS are not appearing in Kloudfuse:

  • Confirm the FireLens container is marked essential: true and has started successfully.

  • Check that the config-file-value path in the task definition matches the location where logDestinations.conf was added in the Dockerfile.

  • Verify the Kloudfuse ingester hostname is reachable from the Fargate task’s VPC and security groups — port 443 must be open outbound.

External References