Fluent Bit Integration

Kloudfuse integrates with the Fluent Bit agent, an open-source and multi-platform log processor tool for logs processing and distribution.

See Fluent Bit Documentation.

Configure Fluent Bit

  1. Install Fluent Bit; see Fluent Bit documentation for Installation; choose the appropriate platform for your business scenario.

  2. To send data to the Kloudfuse stack, modify the agent’s config or values.yaml file.

  3. Configure the HTTP output plugin to forward data to Kloudfuse. See HTTP Plugin Integration.

HTTP Plugin Integration

Add the following configuration to Fluent Bit agent’s config for HTTP configuration.

    [OUTPUT]
        Name http
        Match <match_pattern>
        Host <kfuse_ingress_ip> (1)
        Port 443
        TLS on
        URI /ingester/v1/fluent_bit
        Format {json|json_lines|json_stream|msgpack} (2)
1 Depending on the Kloudfuse stack configuration, the Host IP/DNS name can be either internal or external.
External

use port 443

Internal

use port 80
2 Supported formats: msgpack, json, json_lines, and json_stream.

Configure

Kloudfuse UI enables you to filter log events based on log labels and tags.

  1. Find the label selectors and filter on the left navigation bar of the UI.

  2. Apply the configurations in the following sections to your environment.

  3. Restart your Fluent Bit agent for new configurations to work.

When adding more labels, eiter through recommended filters or custom labels, ensure that they don’t conflict with existing filter definitions.

Kubernetes Labels

Use the kubernetes filter to enrich the log events using Kubernetes metadata. See Fluent Bid documentation for Kubernetes.

If your applications deploy in a Kubernetes environment, be sure to enable this filter.

Here’s an example configuration:

Extract Kubernetes Labels
    [FILTER]
        Name kubernetes
        Match *          # Match everything
        Merge_Log On
        Keep_Log Off
        K8S-Logging.Parser On
        K8S-Logging.Exclude On

Cloud Labels

Add and configure the aws filter to get the availability-zone and instance-type values. Consult Fluent Bit documentation for AWS Metadata. Minimally, use the filter in Extract Basic Cloud Labels.

Extract Basic Cloud Labels
    [FILTER]
        Name aws
        Match *      # Match everything
        az true (1)
        account_id true (2)
        ec2_instance_type true (3)
1 Extract availability-zone
2 Extract account-id
3 Extract ec2-instance-type

Example Extract Basic Cloud Labels does not include the cluster-name You can add a modify filter that includes the cluster-name definition a separate tag, as demonstrated in Extract and Modify Cloud Labels.

Extract and Modify Cloud Labels
    [FILTER]
        Name modify (1)
        Match *     # Match everything
        Add cluster_name <CLUSTER_NAME> (2)
1 Modify instruction
2 Extract cluster-name

You can further configure the cluster_name key from example Extract and Modify Cloud Labels. When using a key name other than cluster_name or clusterName, add the code in example Modify Cloud Labels During Kloudfuse Ingestion under the logs-parser section in the Kloudfuse custom-values.yaml file.

Modify Cloud Labels During Kloudfuse Ingestion
kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_cloud_cluster_name:
            - "$.<KEY_FOR_CLUSTER_NAME>" # must be JSONPath
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="

Static Custom Labels

To add static custom labels, use the modify filter. Example Extract Static Custom Labels as Log Facets adds a static custom label tenant_name.

Extract Static Custom Labels as Log Facets
    [FILTER]
        Name modify
        Match *     # Match everything
        Add tenant_name <TENANT_NAME>

By default, new static labels appear as log facets in the Kloudfuse platform. To generate them as labels and tags, add the code in Example Extract Static Custom Labels as Labels and Tags under the logs-parser section in your custom-values.yaml file.

Extract Static Custom Labels as Labels and Tags
 kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_additional_tags:
            - "$.tenant_name" # must be JSONPath
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="

Log source

By default, Kloudfuse looks for the container_name in the message payload, as the log source. This information is present only if the agent operates with a kubernetes filter. To use a different key as the log source, include the code in Example Extract Log Source Label under the logs-parser section in the Kloudfuse custom-values.yaml file.

Extract Log Source Label
kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_source:
            - "$.<KEY_FOR_LOG_SOURCE>" # must be JSONPath
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="

Log message

The Fluent Bit agent includes the log event message key in the payload. Occasionally, the agent configuration overwrites it.

Kloudfuse looks in the payload for log message with the following key names, by default: log, LOG, Log, message, msg, MSG, and Message.

Customize the key that the Kloudfuse Platform gets to easily tag log event messages by including the code in Example Extract Log Message Label following section under the logs-parser section in the Kloudfuse custom-values.yaml file.

Extract Log Message Label
kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_msg:
            - "$.<MSG_KEY_FROM_AGENT_CONFIG>" # must be JSONPath
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="

Agent-Extracted Key-Value Pairs

Fluent Bit supports a variety of parsers to extract key value pairs from an unstructured log. For a full list of parsers, see Fluent Bit documentation on Parsers. By default, Kloudfuse adds these key-value pairs to log facets, which you can use in the UI for filtering.

While Kloudfuse does not differentiate between these key-value pairs and metadata fields from any filter other than Kubernetes and AWS, you can instruct Kloudfuse to track these tuples as labels instead of log facets. To customize this, use the code in Example Extract Key-Value Labels under the logs-parser section in the Kloudfuse custom-values.yaml file.

Do not include any keys specified in log_source, message, or metadata fields from either Kubernetes or AWS filters. Kloudfuse automatically treats these as metadata and does not generate it as a log facet.
Extract Key-Value Labels
kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_additional_tags: (1)
            - "$.<PREFIX_KEY_FOR_AGENT_KV>" # must be JSONPath
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="
1 kf_additional_tags is a list of key prefixes. Kloudfuse includes any key that matches a prefix at the top-level json as a log label or tag, and not as a log facet.