Derived Metrics

Table of Contents

When to use derived metrics
Derived metrics vs. metric shaping rules
Access derived metrics
Create a derived metric
Query the result
Edit, pause, or delete a rule
Troubleshooting
See also

Derived metrics let you precompute a PromQL expression on a fixed interval and store the result as a brand-new metric you can query directly. Behind the scenes, Kloudfuse evaluates your expression every interval and writes a new time series so that downstream dashboards and alerts no longer have to re-execute the expensive original query each time they refresh.

If you have a heavy sum by (…) or histogram_quantile(…) query that you keep referencing in dashboards or alerts, turning it into a derived metric makes that query cheap to read.

When to use derived metrics

Use derived metrics when one or more of the following is true:

A PromQL expression appears in multiple dashboards or alerts, and re-evaluating it on every refresh is slow or expensive.
An aggregation collapses thousands of high-cardinality series into a small fixed set — for example, "requests per cluster" computed from per-pod counters.
A histogram percentile (histogram_quantile) is queried frequently, and you want a single precomputed series instead of scanning all bucket series at read time.

Don’t use derived metrics for one-off ad-hoc queries: the cost of running the rule every interval may exceed the savings.

Derived metrics vs. metric shaping rules

Both features produce a new precomputed metric, but they solve different problems.

Aspect	Derived metrics	Metric shaping rules
Goal	Cache an arbitrary PromQL expression so dashboards and alerts read it cheaply.	Reduce a metric’s cardinality by aggregating it before storage, optionally dropping the original series.
Source	Any PromQL — including rate, histogram_quantile, multi-query formulas.	A single source metric, aggregated by a fixed set of labels.
Original series preserved?	Always. Derived metrics are additive — your source metric is untouched.	Optional. Shaping rules can be configured to replace the source metric.
When the result appears	At the next rule evaluation (after the configured interval).	At the next rule evaluation (after the configured interval).
Where to find it	menu:Metrics[Derived metrics]	menu:Admin[Settings > Shaping rules]

Aspect

Derived metrics

Metric shaping rules

Goal

Cache an arbitrary PromQL expression so dashboards and alerts read it cheaply.

Reduce a metric’s cardinality by aggregating it before storage, optionally dropping the original series.

Source

Any PromQL — including rate, histogram_quantile, multi-query formulas.

A single source metric, aggregated by a fixed set of labels.

Original series preserved?

Always. Derived metrics are additive — your source metric is untouched.

Optional. Shaping rules can be configured to replace the source metric.

When the result appears

At the next rule evaluation (after the configured interval).

Where to find it

menu:Metrics[Derived metrics]

menu:Admin[Settings > Shaping rules]

If you want to reduce the cardinality of an existing high-volume metric, see Recording Rules or use the metric shaping rules UI. If you want to precompute and cache an expression result without touching the original metric, you’re in the right place — keep reading.

Access derived metrics

From the left navigation, click Metrics.
Select Derived metrics from the dropdown menu.

The Derived metrics page opens. The first time you visit, the list is empty:

Create derived metric (top right): Opens the create form.
Search box: Filter the list once you have rules.
Output metric / PromQL / Interval / Folder / Updated by columns: Show the rules' configuration at a glance.

Create a derived metric

Click Create derived metric to open the create form. The form has four sections: output metric name, interval, folder, and the query builder.

Fields

Field Description Required

Field	Description	Required
Output metric name	The name of the new metric that the rulemanager will produce. Must be a valid Prometheus metric name (letters, digits, `_`, and `:`). The form auto-derives a sensible default once you pick a metric in the query builder — you can override it.	Yes
Interval	How often the rule re-evaluates. Accepts Prometheus duration strings (`30s`, `1m`, `5m`, `1h`). The allowed range is `1m` to `1h`.	Yes (defaults to `1m`)
Folder	The folder this rule belongs to. Folders control who can edit and view derived metrics. If you don’t already have a suitable folder, create one first in Admin > Folders.	Yes
Query	The PromQL expression the rule evaluates. Built with the structured query builder (see Build the query).	Yes

Output metric name

The name of the new metric that the rulemanager will produce. Must be a valid Prometheus metric name (letters, digits, _, and :). The form auto-derives a sensible default once you pick a metric in the query builder — you can override it.

Yes

Interval

How often the rule re-evaluates. Accepts Prometheus duration strings (30s, 1m, 5m, 1h). The allowed range is 1m to 1h.

Yes (defaults to 1m)

Folder

The folder this rule belongs to. Folders control who can edit and view derived metrics. If you don’t already have a suitable folder, create one first in Admin > Folders.

Yes

Query

The PromQL expression the rule evaluates. Built with the structured query builder (see Build the query).

Yes

Build the query

The query builder is a structured form, not a raw PromQL text box. You pick a source metric, an aggregation, and a set of group-by labels, and the builder shows you the generated PromQL live underneath.

Click into the Select a metric field and start typing the source metric name. Suggestions appear as you type — select one from the dropdown.
Choose an aggregation from the dropdown next to the metric: avg, min, max, or sum. (avg is the default.)
In the Select… multi-select next to "aggregation", pick one or more labels to group by. The generated PromQL becomes <agg> by (<labels>) (<metric>).
For more complex expressions, use Add query to create a second source query (labeled a, b, c …) and Add formula to combine them mathematically (for example, a / b for error rates).

The chart at the bottom of the form previews the result against live data:

The GENERATED PROMQL panel under the chart shows the final expression that the rulemanager will evaluate. Always confirm it looks right before saving.

Example: precompute total request rate per cluster

This example creates a derived metric that sums envoy_http_downstream_rq_total across all pods, grouped by cluster name.

Open Metrics > Derived metrics > Create derived metric.
In the query builder, type envoy_http_downstream_rq_total into Select a metric and pick it from the dropdown.
Change the aggregation from avg to sum.
Add kube_cluster_name to the group-by selector.

The generated PromQL becomes sum by (kube_cluster_name)(envoy_http_downstream_rq_total).

The chart previews real data — typically a slowly-rising counter line per cluster.
Change the Output metric name to something descriptive, such as demo:envoy_rq_total:by_cluster. The hint below the field says "Query the result with $<your name>$ " — that is the exact name dashboards and alerts will use later.
Pick your folder from the Folder dropdown.
Click Create rule.

The rule starts evaluating immediately and produces its first sample at the next interval. The new metric appears in the list:

Query the result

Once the rule has produced at least one sample (after one full interval), the new metric is queryable like any other.

In a dashboard or alert, reference it by the Output metric name you set:
```
demo:envoy_rq_total:by_cluster{kube_cluster_name="pisco"}
```
promql
Compose with other functions just like a regular metric:
```
rate(demo:envoy_rq_total:by_cluster[5m])
```
promql

To see the benefit, place the precomputed metric and the original expression side by side in the same dashboard:

Side-by-side dashboard panels: raw expression vs. derived metric

In the screenshot, the top panel runs the heavy sum by (kube_cluster_name) (envoy_http_downstream_rq_total) live on every refresh; the bottom panel just reads the precomputed demo:envoy_rq_total:by_cluster series. Values agree within one evaluation interval (rules sample at fixed offsets, slightly before live query time).

Edit, pause, or delete a rule

From the derived metrics list, click into a rule’s row to open its details. From there, you can:

Edit — change the interval, folder, or query. Renaming the output metric is supported, but downstream dashboards or alerts referencing the old name need to be updated separately.
Pause — temporarily stop the rule from evaluating without losing its configuration. Existing samples remain queryable; no new samples are produced until you resume.
Resume — re-enable a paused rule. The first sample after resume comes at the next interval.
Delete — permanently remove the rule. The derived metric stops accumulating new samples; historical samples remain queryable until they age out of retention.

Troubleshooting

The Create rule button stays disabled

Every required field must be filled. If the button is still disabled, check that the query builder produced a non-empty GENERATED PROMQL string — sometimes the structured builder needs the metric to be selected from the dropdown rather than typed and left in the input field.

The new metric does not show up in the dashboard panel picker

The picker reads from a periodically-refreshed metric metadata cache. Recording-rule outputs are only picked up after the cache refreshes (typically a few minutes after the first sample). In the meantime, the metric is still queryable — type the full output name in the metric box, or compose the panel URL directly. Once the cache refreshes, autocomplete starts working normally.

The new metric does not appear in the Metrics Cardinality Explorer

The cardinality explorer queries the 5-minute rollup table. Newly-created derived metrics live in the raw table first and only land in the rollup table after the rollup pipeline processes them — often after one or more rollup cycles. Use PromQL directly, or wait for the next rollup cycle.

The chart preview is empty

The preview runs the same expression against live data. If you see no data, double-check that the source metric is actually emitting samples in the current time range (top-right time picker, defaults to the last hour).