Consumption Tracking

The Consumption Tracking dashboard shows, for each telemetry stream, how much data Kloudfuse received over the selected time range, how much it dropped and why, how much it kept, and the disk space it added. Tracking labels break those totals down by source — for example by cluster, namespace, or team — so you can attribute usage and find the sources to target with rate control.

The figures come from the ingester’s consumption trackers, which count every data point as it arrives, before any drops.

Configure tracking labels

Tracking labels are the labels on incoming data that the consumption trackers count by. Kloudfuse always tracks two labels:

  • target_kfuse — true for Kloudfuse’s own telemetry.

  • target_agent_type — the type of agent that sent the data, for example datadog, prometheus, or otlp-proto.

To track usage by your own labels, add them to global.trackingLabels in your Helm values file. For example, to track by cluster and namespace:

global:
  trackingLabels: [kube_cluster_name, kube_namespace]
yaml

Then apply the change:

helm upgrade --install kfuse oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/kfuse \
  -n kfuse \
  --version <VERSION> \ (1)
  -f custom-values.yaml
bash
1 Replace <VERSION> with a valid Kloudfuse release value. See Release Notes for the latest release.

For metrics, traces, events, and RUM, tracking labels are read after relabel rules run, so a label that a relabel rule adds or rewrites can be tracked.

Each tracking label appears on the consumption metrics with a target_ prefix, for example target_kube_cluster_name. Data that does not carry a tracking label is counted as missing for that label.

Every tracking label multiplies the number of label combinations the trackers keep. Choose labels with a bounded set of values, such as cluster, namespace, team, or environment, rather than per-pod or per-request labels.

Open the dashboard

Open the consumption dashboards in the Kloudfuse UI (the /f interface); they are not designed for the legacy UI.

  1. In the top navigation, click Dashboards.

  2. Search for Consumption Tracking. The consumption dashboards are in the kfuse-ingest-v2 folder.

Dashboard variables

Variable Description

Organization

The organization whose data to show.

Cluster

The Kubernetes cluster that runs Kloudfuse, or All.

Stream

The streams to show: Metrics, Logs, Traces, Events, and RUM. The dashboard repeats one section per selected stream.

Kloudfuse telemetry

Include or Exclude Kloudfuse’s own telemetry (target_kfuse="true") from the totals.

Tracking labels

The tracking labels to break the tables down by. Select several to see each combination of their values.

All stats and tables cover the time range selected in the time picker. The last half hour of data, which is not yet rolled up, may be missing from the totals.

Panels

Each stream section has the following stats:

Stat Description

Received

Data received by the ingester, before any drops, in bytes as received.

Dropped

Data the ingester dropped: rate control, relabel drop rules, bad timestamps, and other rejects.

Drop Ratio

Dropped data as a share of the data received.

Ingested

Data received minus data dropped: what was written for storage.

Ingest Samples

Data points kept: metric samples, log lines, spans, or events.

Ingest Size

Average size of one kept data point as received.

Disk Used

Estimated Pinot disk space added: rows consumed per table × the table’s average record size on disk, including replicas. Always includes Kloudfuse telemetry.

Record Size

Average size of one stored record on disk.

Below the stats, three tables break the stream down by the selected tracking labels, largest first:

  • Kept by Tracking Label — data kept for each label combination.

  • Disk by Tracking Label — estimated disk added for each label combination, based on its share of the samples kept.

  • Dropped by Tracking Label — data dropped for each label combination and drop reason.

Drop reasons

Reason Meaning

ratelimit

Dropped by a rate control policy.

relabel

Dropped by a relabel drop rule.

max_ts, stale_or_future_ts

The timestamp is too old or too far in the future.

missing_name

The data point has no name.

max_size

The data point exceeds the maximum size.

process_error

The ingester could not process the data.

Next steps