Trace Attribute Cardinality

Overview

Cardinality is the number of distinct values an attribute takes on across your span data. span_type might have 7 distinct values across your whole environment; trace_id has one distinct value per request — effectively unbounded. Cardinality determines what an attribute is useful for: a low-cardinality attribute (tens to low hundreds of values) is a good candidate for grouping and filtering, because Kloudfuse only has to track that many distinct time series or buckets. A high-cardinality attribute (thousands to millions of values) is expensive to group by and often not meaningful to filter on, because each value only matches a handful of spans.

Why cardinality matters

Every time you group by an attribute — in a by clause in the query builder, on the Services or Traces facet panel, or as a label on a metric derived from spans — Kloudfuse has to track one series or bucket per distinct value seen. Grouping by a low-cardinality attribute like service_name or span_name produces a handful of series, which is cheap to compute and easy to read. Grouping by a high-cardinality attribute like trace_id, a raw user ID, or a full URL with query parameters produces one series per request — the query becomes slow to run, expensive to store, and useless to look at, since no single series has enough data points to show a trend.

High cardinality is the single most common cause of a facet, filter, or query feeling slow or returning too many rows to be useful. Before you build a dashboard panel or a filter around an attribute you haven’t used before, check its cardinality.

What counts as high cardinality

There’s no fixed threshold, but a useful rule of thumb: compare an attribute’s Value count for the window against the total span count for that same window.

  • An attribute whose value count stays roughly constant regardless of traffic volume (span_type, service_name, http.method) is low-cardinality — safe to group and filter by.

  • An attribute whose value count scales with the number of requests (approaching one distinct value per span) is effectively unbounded — this includes trace and span IDs, raw timestamps, session tokens, and identifiers that are unique per request.

  • Attributes in between — a pod_name that scales with your fleet size, or a customer_id that scales with your customer count — are usually fine to group by in small-to-medium environments, but worth watching as you scale.

Common high-cardinality culprits in trace data: full URLs or endpoints with embedded IDs or query strings (/api/users/8274921/orders?page=3 instead of a normalized /api/users/:id/orders), raw request or session IDs promoted to a label, IP addresses, and timestamps stored as string attributes rather than the span’s own start/end time.

How to reduce cardinality

  • Normalize endpoints before they become attributes. If your instrumentation captures the raw URL, route it through path templating (/api/users/:id instead of /api/users/8274921) at the SDK or collector level so the http.route or endpoint attribute stays low-cardinality. This is usually the single biggest win, since endpoint attributes are grouped by constantly.

  • Don’t promote unique identifiers to indexed attributes. Trace ID, span ID, and request/session IDs are useful to have on the span for lookup, but should not be used as by-clause or filter targets — search for a specific trace by ID directly instead of trying to group by it.

  • Bucket continuous values. If you need to analyze something like payload size or a numeric ID range, bucket it into ranges before attaching it as an attribute, rather than attaching the raw value.

  • Prefer existing low-cardinality attributes. Before adding a new custom attribute, check whether an existing one (service_name, span_name, span_type, kube_pod_name) already captures what you need — see Filtering traces for the full set Kloudfuse already tracks.

  • Scope high-cardinality lookups to a filter, not a group-by. It’s fine to filter on a specific trace_id or customer_id value to find one thing; the problem is asking Kloudfuse to enumerate and group by every distinct value at once.

Click APM in the top navigation, then Cardinality under Traces.

Using the Cardinality page

Show attributes of toggles between Low cardinality and High cardinality attributes. Optionally add a filtered by clause (attribute / operator / value) to scope the report to a subset of spans, and use limit to (top / bottom, with a configurable count) to control how many rows are returned.

The table lists each attribute with:

Value count (1h)

Distinct values seen for this attribute in the last hour.

Value count

Distinct values seen for this attribute in the selected time range.

Use Search attributes to jump to a specific one. An attribute with a Value count in the tens or hundreds (for example, span_name, service_name, pod_name) is a good grouping candidate for the Traces Explorer's query builder; an attribute with a value count near the total span count for the window is too high-cardinality to group by usefully — see How to reduce cardinality above.