Quick Guide for OTel Agent on Kubernetes for Events

Use the following sample helm-values.yaml file to configure the OpenTelemetry Collector to collect events in Kloudfuse. Ensure that you correctly set the protocol as either https or http, based on TLS being enabled or disabled.

  1. Add the k8s_events receiver into your configuration, so that OTel receives kubernetes events, and enable the pipeline to handle the receiver events, process them, and export them to a specific log point.

    This is a separate logs pipeline. Kloudfuse does not use the existing logs pipeline to process events.

  2. Define the exporter to send these processed Kubernetes events to the specified cluster.

  3. The changes in the configuration look something like this. Ensure that you correctly set the protocol as either https or http, based on TLS being enabled or disabled.

    exporters:
      <exporter_name>:
        logs_endpoint: https://<REPLACE KFUSE ADDRESS>/ingester/otlp/k8s_events (1)
    
    receivers:
      k8s_events:
    
    service:
      pipelines:
        logs/k8s_events:
          exporters:
          - <exporter_name>
          processors:
          - <processors>
          receivers:
          - k8s_events
    1 Because OTEL wraps the events into logs, you must set a logs endpoint for the events stream. For example, use ingester/otlp/k8s_events. Kloudfuse can handle the data as events.
  4. To add permissions, alter the clusterrole and add events to the resource.

    rules:
      - apiGroups:
        - ""
        resources:
        - events
        - ...
  5. To distinguish events from specific agents, add a processor and a key/value pair to distinguish OTel from other agents.

    ...
    command:
    ...
      processors:
        batch:
          ...
        resource:
          attributes:
          - key: kf_events_agent
            value: "otlp"
            action: upsert
    ...
  6. Restart your OTel collector, so it can begin sending events to your cluster.

Configure the Events Pipeline

You must add a new logs pipeline just for the Kubernetes events. The complete configuration would have 1 pipleline for logs, and another for logs/k8s_events.

The configuration may appear similar to this:

service:
  pipelines:
    logs/k8s_events:
      exporters:
      - <exporter_name>
      processors:
      ...
      - resource/k8s_events
      receivers:
      - k8s_events

processors:
  resource/k8s_events:
    attributes:
    - key: kf_events_agent
      value: "otlp"
      action: upsert