Configure metrics rollup

Learn how to enable and configure metrics roll up on your Kloudfuse cluster.

Enabling metrics rollup

Kloudfuse optimizes metrics through roll up in the background. This feature is disabled by default. To turn on this feature, use the following configuration in the custom_values.yaml file.

In the following example, rollupEnabled determines if Kloudfuse creates roll up metrics.

Enable calculation of roll-up metrics at ingestion
global:
  metrics:
    rollupEnabled: true

Rolled up metrics in query service

The Query Service uses rolled up metrics by default, on all queries that span a time interval of 2 days or longer or step size of 5 minutes or larger. For shorter intervals, query service uses raw metrics. The switch is transparent to the user. For example, if you are using a time picker to select the time interval, the query service switches to rolled up metrics for time intervals greater than 2 days, and then back to raw metrics when the interval decreases to less than 2 days. This can be adjusted in the custom_values.yaml file.

In the following example,

  • RollupTimerangeThresholdSecs specifies the 2-day (172800 seconds) interval that triggers the use of rolled up metrics.

  • RollupStepThresholdSecs specifies the 5-minute (300 seconds) roll up step size.

Specify interval when to use rolled up metrics, and the size of the roll up
query-service:
  config:
    Metric:
      RollupTimerangeThresholdSecs: 172800
      RollupStepThresholdSecs: 300

The query service uses roll up metrics whenever it is appropriate. To turn off the roll up option for queries, use the following configuration command in the custom_values.yaml file.

In the following example, AutoUseRollup determines whether to use rolled up metrics in queries.

Turn off use of rolled up metrics in queries
query-service:
  config:
    Metric:
      AutoUseRollup: false

Retention

By default, the roll up metrics have the same retention policy as raw metrics. To change this timing, typically to a longer retention period, add the new retention policy to the global.retentionPolicy section of the custom_values.yaml file.

In the following example, the retentionTimeValue and retentionTimeUnit specify how long to store raw metrics. The retentionTimeRollupValue and retentionTimeRollupUnit specify how long to store rolled up metrics.

Set the retention policies for metrics
metrics:
  default:
    retentionTimeValue: 30
    retentionTimeUnit: DAYS
    retentionTimeRollupValue: 60
    retentionTimeRollupUnit: DAYS

Mixed data

To handle mixed data, where some (likely older) data has no corresponding rolled up data, the query service accesses the kf_metrics_rollup table for the minimum timestamps. If the query time range contains the minimum timestamp, the query service splits the query into multiple component queries. The older data uses the kf_metrics table, and newer data uses the rolled up table kf_metrics_rollup. The query service then merges the responses before returning the result.