Enable Observability for the Kloudfuse Cluster

Kloudfuse can monitor its own cluster using two optional components:

  • A Datadog observability agent that collects metrics, logs, and traces from Kloudfuse services and stores them within the platform.

  • A kfuse-profiler that collects continuous profiling data from Kloudfuse services using Pyroscope.

Both are controlled by flags under global.observability in your custom_values.yaml file.

global.observability.enabled

Setting global.observability.enabled: true deploys the Datadog agent into the Kloudfuse cluster so that the platform can monitor itself.

When enabled, the following resources are created:

  • A Datadog agent ConfigMap (kfuse-observability-agent-config)

  • A Datadog cluster agent ConfigMap (kfuse-observability-cluster-agent-config)

  • A PostgreSQL observability schema with the functions required to store collected data

This allows Kloudfuse service telemetry — metrics, logs, and traces — to be ingested and queried within the same Kloudfuse installation.

To enable, add the following to your custom_values.yaml:

global:
  observability:
    enabled: true
yaml

global.observability.profileEnabled

Setting global.observability.profileEnabled: true enables the kfuse-profiler, which deploys a Grafana Agent configured to scrape continuous profiling data from Kloudfuse services using the Pyroscope protocol.

profileEnabled takes effect only when global.observability.enabled is also true, because the kfuse-profiler is a sub-component of the observability stack.

The profiler collects the following profile types from Kloudfuse services by default:

  • Memory

  • CPU (process_cpu)

  • Goroutine

To enable profiling alongside observability, add the following to your custom_values.yaml:

global:
  observability:
    enabled: true
    profileEnabled: true
yaml

Summary

The table below shows the components deployed for each configuration:

enabled profileEnabled Components deployed

false

(ignored)

No observability components deployed.

true

false

Datadog agent ConfigMaps, PostgreSQL observability schema.

true

true

All of the above, plus the kfuse-profiler (Grafana Agent with Pyroscope scraping).
By default, both global.observability.enabled and global.observability.profileEnabled are set to true in the Kloudfuse Helm chart. Explicitly set them to false in your custom_values.yaml to disable either component.