Quick Guide for OTel Agent on Kubernetes for Infrastructure Objects

This guide shows an end-to-end workflow to collect, process, and export Kubernetes infrastructure objects (pods, nodes, deployments, etc.) to Kloudfuse using OpenTelemetry Collector. You will configure receivers, processors, pipelines, exporters, and RBAC, then verify data in the Kloudfuse UI.

Workflow Overview

The data collection flow is:

Kubernetes Objects → k8sobjects Receiver → Processors (k8sattributes, resource, resourcedetection) → Dedicated Logs Pipeline → OTLPHTTP Exporter → Kloudfuse UI

Steps:

  • Install OpenTelemetry Collector in your Kubernetes cluster.

  • Configure k8sobjects receiver to capture objects.

  • Apply processors for enrichment and metadata extraction.

  • Define a dedicated logs pipeline for infrastructure objects.

  • Configure otlphttp exporter to send data to Kloudfuse.

  • Apply RBAC permissions to allow object access.

  • Enable leader election to avoid duplicate collection.

  • Verify ingestion in the Kloudfuse UI.

Key Points

  • Each data type (logs, events, infrastructure objects) requires a dedicated pipeline.

  • Exporter names must be prefixed with otlphttp/ (e.g., otlphttp/k8sobjects).

  • Use the fixed ingestion endpoint: https://<KLOUDFUSE_ADDRESS>/ingester/otlp/k8s_objects.

  • Enrich objects with k8sattributes, resource, and resourcedetection processors.

  • Ensure RBAC permissions cover all object types collected.

  • Enable leader election for multi-Collector deployments.

  • Do not reuse existing pipelines. Each pipeline must handle only one data type.

  • Misconfigured exporter names or endpoints will prevent ingestion.

  • All collected object types must be covered in RBAC rules.

Step 1: Configure Exporter

Use an OTLP HTTP exporter for infrastructure objects.

exporters:
  otlphttp/k8sobjects:
    logs_endpoint: https://<KLOUDFUSE_ADDRESS>/ingester/otl/k8s_objects
yaml

Step 2: Configure Receiver

Add the k8sobjects receiver to collect desired Kubernetes objects.

receivers:
  k8sobjects:
    objects:
      - name: pods
      - name: deployments
      - name: replicasets
      - name: services
      - name: nodes
      - name: jobs
      - name: cronjobs
      - name: daemonsets
      - name: statefulsets
      - name: persistentvolumes
      - name: persistentvolumeclaims
      - name: roles
      - name: rolebindings
      - name: clusterroles
      - name: clusterrolebindings
      - name: serviceaccounts
      - name: ingresses
      - name: namespaces
yaml
Example

For lightweight testing, collect only pods and deployments:

receivers:
  k8sobjects:
    objects:
      - name: pods
      - name: deployments
yaml

Step 3: Configure Processors

Use processors to enrich objects and prepare them for Kloudfuse dashboards.

processors:
  resource:
    attributes:
      - key: kf_infra_agent
        value: "otlp"
        action: upsert   # required for filtering in Kloudfuse UI

  resourcedetection:
    detectors: [env, gcp]  # replace with your environment (eks, aks, ec2, azure)
    timeout: 2s
    override: false

  k8sattributes:
    extract:
      metadata:
        - k8s.namespace.name
        - k8s.deployment.name
        - k8s.statefulset.name
        - k8s.daemonset.name
        - k8s.cronjob.name
        - k8s.job.name
        - k8s.node.name
        - k8s.pod.name
        - k8s.pod.uid
        - k8s.pod.start_time
        - k8s.replicaset.name
yaml
Example

Filter pods and annotate them for dashboards:

processors:
  resource:
    attributes:
      - key: kf_infra_agent
        value: "otlp"
        action: upsert
yaml

Step 4: Configure Pipeline

Define a dedicated logs pipeline for infrastructure objects.

service:
  pipelines:
    logs/k8sobjects:
      receivers: [k8sobjects]
      processors: [k8sattributes, resource, resourcedetection]
      exporters: [otlphttp/k8sobjects]
yaml

Step 5: Configure RBAC Permissions

Grant the Collector access to all object types.

rules:
- apiGroups: [""]
  resources: [namespaces, nodes, pods, services, persistentvolumeclaims, persistentvolumes, serviceaccounts]
  verbs: ["get", "list", "watch"]

- apiGroups: ["apps"]
  resources: [daemonsets, deployments, replicasets, statefulsets]
  verbs: ["get", "list", "watch"]

- apiGroups: ["rbac.authorization.k8s.io"]
  resources: [clusterroles, clusterrolebindings, roles, rolebindings]
  verbs: ["get", "list", "watch"]

- apiGroups: ["networking.k8s.io"]
  resources: [ingresses]
  verbs: ["get", "list", "watch"]

- apiGroups: ["batch"]
  resources: [jobs, cronjobs]
  verbs: ["get", "list", "watch"]
yaml
Missing RBAC rules can prevent object collection without any visible errors.

Step 6: Enable Leader Election

Prevent duplicate collection in multi-Collector deployments.

extensions:
  leader_election: {}

service:
  extensions: [leader_election]
yaml

Step 7: Complete YAML Example

Combines receivers, processors, exporters, and pipelines.

extensions:
  leader_election: {}

receivers:
  k8sobjects:
    objects:
      - name: pods
      - name: deployments
      - name: replicasets
      - name: services
      - name: nodes
      - name: jobs
      - name: cronjobs
      - name: daemonsets
      - name: statefulsets
      - name: persistentvolumes
      - name: persistentvolumeclaims
      - name: roles
      - name: rolebindings
      - name: clusterroles
      - name: clusterrolebindings
      - name: serviceaccounts
      - name: ingresses
      - name: namespaces

processors:
  resource:
    attributes:
      - key: kf_infra_agent
        value: "otlp"
        action: upsert
  resourcedetection:
    detectors: [env, gcp]
    timeout: 2s
    override: false
  k8sattributes:
    extract:
      metadata:
        - k8s.namespace.name
        - k8s.pod.name
        - k8s.pod.uid
        - k8s.node.name
        - k8s.deployment.name
        - k8s.replicaset.name
        - k8s.statefulset.name
        - k8s.daemonset.name
        - k8s.job.name
        - k8s.cronjob.name

exporters:
  otlphttp/k8sobjects:
    logs_endpoint: https://<KLOUDFUSE_ADDRESS>/ingester/otlp/k8s_objects

service:
  extensions: [leader_election]
  pipelines:
    logs/k8sobjects:
      receivers: [k8sobjects]
      processors: [k8sattributes, resource, resourcedetection]
      exporters: [otlphttp/k8sobjects]
yaml

Step 8: Verify Collection

  • Restart the OTel Collector.

  • Open Kloudfuse UI: Infrastructure > Kubernetes.

  • Confirm that pods, nodes, deployments, and namespaces appear.

  • Use filter kf_infra_agent=otlp to view only OTEL-sourced objects.