Retention Rules

During ingestion, the Kloudfuse platform supports the rewriting the labels for Metrics, Events, Logs, and Traces data. This configuration follows the Prometheus relabel config format. You can use helm chart values in the relevant relabel_configs section of the custom_values.yaml file.

Relabel rules in the Ingester
ingester:
  config:
    aggregator:
      metric:
        # relabelConfigs - Configure relabel rules for metric labels.
        # Format follows Prometheus relabel_config syntax.
        # Ref: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
        relabelConfigs: []
    events:
      # relabelConfigs - Configure relabel rules for events labels.
      # Format follows Prometheus relabel_config syntax.
      # Ref: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
      relabelConfigs: []
    traces:
      # relabelConfigs - Configure relabel rules for traces labels.
      # Format follows Prometheus relabel_config syntax.
      # Ref: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
      relabelConfigs: []

Add New Labels Based on Current Label Value

In a common scenario, you can enrich some metrics with new labels, depending on the existing label values. For example, for the label aws_account with value 123456, you can add aws_account_name label with AWS_account_name_for_123456 value by inserting the following section at the end of the custom_values.yaml file.

ingester:
  config:
    aggregator:
      metric:
        # relabelConfigs - Configure relabel rules for metric labels.
        # Format follows Prometheus relabel_config syntax.
        # Ref: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
        relabelConfigs:
          - ...
          - regex: "123456"
            replacement: AWS_account_name_for_123456
            source_labels:
            - aws_account
            target_label: aws_account_name
          - ...

After saving the configuration, restart the ingester service:

kubectl rollout restart statefulset ingester

Drop or Keep Series with Matching Label Values

In business scenarios, some series have to be dropped based on existing label values. For example, to stop storing data for the label aws_account with value 123456, you must add aws_account_name label with AWS_account_name_for_123456 value in the relabel config with action drop. To keep certain series, use the keep action.

ingester:
  config:
    aggregator:
      metric:
        # relabelConfigs - Configure relabel rules for metric labels.
        # Format follows Prometheus relabel_config syntax.
        # Ref: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
        relabelConfigs:
          - ...
          - regex: "123456"
            action: drop
            source_labels:
            - aws_account
          - ...

After saving the configuration, restart the ingester service:

kubectl rollout restart statefulset ingester