Unwrapped range aggregations

Unwrapped range aggregations compute statistics over a numeric label value instead of counting lines. The unwrap expression selects the label to use as the sample value; the surrounding *_over_time function aggregates those samples over the range window. Use the duration() and bytes() conversion functions to unwrap values like 28.8ms or 2KB.

avg_over_time

Computes the arithmetic mean of the unwrapped values within the range window. This is the go-to statistic for typical latency or size per group — pair it with max_over_time or quantile_over_time to see the tail as well.

Syntax

avg_over_time({<selector>} <pipeline> | unwrap <label> [<range>]) [by (<labels>)]
none

Parameters

Parameter Required Description

<label>

Required

The unwrapped label providing the sample values.

<range>

Required

The window to average over.

by (<labels>)

Optional

Grouping labels; without it, every label combination produces its own series.

Example

Average the duration of Grafana datasource requests per endpoint over the last five minutes. The duration() conversion parses values like 28.8ms into seconds.

avg_over_time(
  {source="grafana"} |= "duration="
  | logfmt
  | unwrap duration(duration) [5m]
) by (endpoint)
Expected output
endpoint Value

queryData

0.1312

Averages hide outliers; check quantile_over_time(0.95, …​) or max_over_time when hunting slow requests.

first_over_time

Returns the first (oldest) unwrapped value within the range window. Use it together with last_over_time to measure how a value changed across the window — for example, a gauge reported in logs.

Syntax

first_over_time({<selector>} <pipeline> | unwrap <label> [<range>]) [by (<labels>)]
none

Parameters

Parameter Required Description

<label>

Required

The unwrapped label providing the sample values.

<range>

Required

The window to select from.

by (<labels>)

Optional

Grouping labels.

Example

Read the duration of the oldest Grafana datasource request in the window, per endpoint.

first_over_time(
  {source="grafana"} |= "duration="
  | logfmt
  | unwrap duration(duration) [5m]
) by (endpoint)
Expected output
endpoint Value

queryData

8.5353e-05

first_over_time and last_over_time order samples by timestamp, not by value.

last_over_time

Returns the last (newest) unwrapped value within the range window. This is the right function for gauge-style fields logged periodically — queue length, cache size, connection count — where only the latest reading matters.

Syntax

last_over_time({<selector>} <pipeline> | unwrap <label> [<range>]) [by (<labels>)]
none

Parameters

Parameter Required Description

<label>

Required

The unwrapped label providing the sample values.

<range>

Required

The window to select from.

by (<labels>)

Optional

Grouping labels.

Example

Read the duration of the most recent Grafana datasource request in the window, per endpoint.

last_over_time(
  {source="grafana"} |= "duration="
  | logfmt
  | unwrap duration(duration) [5m]
) by (endpoint)
Expected output
endpoint Value

queryData

7.5672e-05

Make the range window comfortably larger than the logging interval, or sparse streams return no sample at some steps.

max_over_time

Returns the largest unwrapped value within the range window. This is the worst-case detector: slowest request, biggest payload, highest queue depth — often more actionable than the average.

Syntax

max_over_time({<selector>} <pipeline> | unwrap <label> [<range>]) [by (<labels>)]
none

Parameters

Parameter Required Description

<label>

Required

The unwrapped label providing the sample values.

<range>

Required

The window to take the maximum over.

by (<labels>)

Optional

Grouping labels.

Example

Find the slowest Grafana datasource request per endpoint in the last five minutes.

max_over_time(
  {source="grafana"} |= "duration="
  | logfmt
  | unwrap duration(duration) [5m]
) by (endpoint)
Expected output
endpoint Value

queryData

31.88

For a less spiky worst-case signal, quantile_over_time(0.99, …​) ignores the single most extreme sample.

min_over_time

Returns the smallest unwrapped value within the range window. Use it to find the floor of a measurement — the fastest response, the smallest payload — or to verify that a value never drops below an expected baseline.

Syntax

min_over_time({<selector>} <pipeline> | unwrap <label> [<range>]) [by (<labels>)]
none

Parameters

Parameter Required Description

<label>

Required

The unwrapped label providing the sample values.

<range>

Required

The window to take the minimum over.

by (<labels>)

Optional

Grouping labels.

Example

Find the fastest Grafana datasource request per endpoint in the last five minutes.

min_over_time(
  {source="grafana"} |= "duration="
  | logfmt
  | unwrap duration(duration) [5m]
) by (endpoint)
Expected output
endpoint Value

queryData

1.3956e-05

Combine with max_over_time to see the full spread of a value in one dashboard panel.

quantile_over_time

Computes the given quantile (0 to 1) of the unwrapped values within the range window. Percentiles are the standard language of latency objectives — p95 and p99 response times straight from access or application logs, no instrumentation required.

Syntax

quantile_over_time(<q>, {<selector>} <pipeline> | unwrap <label> [<range>]) [by (<labels>)]
none

Parameters

Parameter Required Description

<q>

Required

The quantile as a number between 0 and 1, such as 0.95.

<label>

Required

The unwrapped label providing the sample values.

<range>

Required

The window to compute over.

by (<labels>)

Optional

Grouping labels.

Example

Compute the 95th-percentile duration of Grafana datasource requests per endpoint over the last five minutes.

quantile_over_time(0.95,
  {source="grafana"} |= "duration="
  | logfmt
  | unwrap duration(duration) [5m]
) by (endpoint)
Expected output
endpoint Value

queryData

0.16

Quantiles are computed per group over the raw samples in the window, so they are exact — not an approximation over pre-aggregated buckets.

stddev_over_time

Computes the population standard deviation of the unwrapped values within the range window. A rising standard deviation with a flat average means the value is becoming erratic — an early sign of saturation or contention.

Syntax

stddev_over_time({<selector>} <pipeline> | unwrap <label> [<range>]) [by (<labels>)]
none

Parameters

Parameter Required Description

<label>

Required

The unwrapped label providing the sample values.

<range>

Required

The window to compute over.

by (<labels>)

Optional

Grouping labels.

Example

Measure how much Grafana datasource latency varies per endpoint over the last five minutes.

stddev_over_time(
  {source="grafana"} |= "duration="
  | logfmt
  | unwrap duration(duration) [5m]
) by (endpoint)
Expected output
endpoint Value

queryData

1.113

stdvar_over_time returns the variance — the square of this value.

stdvar_over_time

Computes the population variance of the unwrapped values within the range window — the square of the standard deviation. Variance is additive across independent components, which occasionally makes it the more convenient form in capacity math.

Syntax

stdvar_over_time({<selector>} <pipeline> | unwrap <label> [<range>]) [by (<labels>)]
none

Parameters

Parameter Required Description

<label>

Required

The unwrapped label providing the sample values.

<range>

Required

The window to compute over.

by (<labels>)

Optional

Grouping labels.

Example

Compute the variance of Grafana datasource latency per endpoint over the last five minutes.

stdvar_over_time(
  {source="grafana"} |= "duration="
  | logfmt
  | unwrap duration(duration) [5m]
) by (endpoint)
Expected output
endpoint Value

queryData

1.195

For a value in the same unit as the samples, use stddev_over_time.

sum_over_time

Adds up all unwrapped values within the range window. Use it for quantities that accumulate — total bytes transferred, total items processed, or total time spent — rather than for point-in-time measurements.

Syntax

sum_over_time({<selector>} <pipeline> | unwrap <label> [<range>]) [by (<labels>)]
none

Parameters

Parameter Required Description

<label>

Required

The unwrapped label providing the sample values.

<range>

Required

The window to sum over.

by (<labels>)

Optional

Grouping labels.

Example

Total the time Grafana spent waiting on each datasource endpoint in the last five minutes — request count times average latency, in one number per endpoint.

sum_over_time(
  {source="grafana"} |= "duration="
  | logfmt
  | unwrap duration(duration) [5m]
) by (endpoint)
Expected output
endpoint Value

queryData

560.41

With duration() conversion the sum is in seconds.

unwrap

Selects a label whose value becomes the metric sample for the surrounding range aggregation, instead of counting lines. unwrap is the bridge between parsed log fields and statistics: extract a latency, size, or count with a parser, unwrap it, and aggregate it with any *_over_time function. Values that are not plain numbers need a conversion function: duration(label) parses Go durations such as 28.8ms, and bytes(label) parses sizes such as 2KB.

Syntax

<agg>_over_time({<selector>} <pipeline> | unwrap <label> [<range>]) [by (<labels>)]
| unwrap duration(<label>)   # convert 28.8ms-style values
| unwrap bytes(<label>)      # convert 2KB-style values
none

Parameters

Parameter Required Description

<label>

Required

The label to use as the sample value. Must parse as a number after optional conversion.

duration(<label>)

Optional

Parses the label as a Go duration and yields seconds.

bytes(<label>)

Optional

Parses the label as a byte size and yields bytes.

Example

Extract each nginx request’s upstream response time with a regexp, unwrap it, and average it over one-minute windows. The by (source) grouping collapses the result to a single series.

avg_over_time(
  {source="nginx"}
  | regexp "(?P<req_time>[0-9]+[.][0-9]+) [[]"
  | unwrap req_time [1m]
) by (source)
Expected output
source Value

nginx

0.166

Lines whose label fails to parse are dropped from the aggregation and flagged with __error__.

duration_seconds() is an alias of duration().