Store Profiles in Azure Blob Storage

By default, the profiler server keeps profiles only on its local volume and deletes the oldest ones as it fills. Configure an Azure Blob Storage container to keep profiles for a fixed retention period instead. The profiler server uploads a block to the container every hour and continues to query it from there after the local copy is gone. For details, see Storage tiers.

Prerequisites

  • Continuous Profiling is enabled. See Enable the profiler server.

  • An Azure storage account, in the region where Kloudfuse runs, with a blob container dedicated to profiling data.

  • The storage account’s access key, from Security + networking › Access keys in the Azure portal. To authenticate without a key on AKS, see Authenticate with a managed identity.

Configure Azure storage

  1. Store the access key in a Kubernetes Secret in the kfuse namespace:

    kubectl create secret generic kfuse-profiler-storage -n kfuse \
      --from-literal=AZURE_STORAGE_KEY=<ACCOUNT_KEY>
  2. Add the following to your custom-values.yaml file:

    pyroscope:
      pyroscope:
        extraArgs:
          config.expand-env: true (1)
        extraEnvFrom:
          - secretRef:
              name: kfuse-profiler-storage (2)
        structuredConfig:
          storage:
            backend: azure
            azure:
              account_name: <STORAGE_ACCOUNT_NAME> (3)
              container_name: <CONTAINER_NAME> (4)
              account_key: ${AZURE_STORAGE_KEY} (5)
          limits:
            compactor_blocks_retention_period: 30d (6)
    yaml
    1 Lets the profiler server replace ${…​} placeholders in its configuration with environment variables.
    2 Loads the key from the Secret you created as an environment variable.
    3 The name of the storage account.
    4 The name of the blob container inside the storage account.
    5 A placeholder filled from the Secret at startup. Keep it exactly as shown.
    6 How long to keep profiles in the container. Blocks older than this are deleted. The default, 0s, keeps data forever.
  3. Apply the change:

    helm upgrade --install kfuse oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/kfuse \
      -n kfuse \
      --version <VERSION> \ (1)
      -f custom-values.yaml
    bash
    1 Replace <VERSION> with a valid Kloudfuse release value. See Release Notes for the latest release.

Optional settings

Add these under pyroscope.pyroscope.structuredConfig.storage.azure:

Setting Description

endpoint_suffix

The storage endpoint suffix, without a scheme. Leave it unset for the Azure public cloud. Set it for sovereign clouds, for example blob.core.usgovcloudapi.net.

connection_string

A full connection string, used instead of account_key. Use it to authenticate with a SAS token. When set, endpoint_suffix is ignored.

user_assigned_id

The client ID of a user-assigned managed identity. Applies only when both account_key and connection_string are unset; see Authenticate with a managed identity.

max_retries

Number of retries for recoverable errors. The default is 3.

To share one container with other data, set storage.storage_prefix to a prefix of letters and digits only, such as profiles.

Authenticate with a managed identity

If you leave account_key and connection_string unset, the profiler server authenticates with an Azure managed identity instead of a key. It uses the system-assigned identity, or the user-assigned identity whose client ID you set in user_assigned_id. Grant that identity the Storage Blob Data Contributor role on the container. In this case, omit config.expand-env and extraEnvFrom.

Verify

  1. Check that the profiler server started without storage errors:

    kubectl logs -n kfuse kfuse-profiler-server-0 | grep -i "bucket\|level=error"

    Authentication errors point to the account key or the identity’s role assignment. ContainerNotFound points to the container name.

  2. Within about an hour, the first block is uploaded. List the container to confirm:

    az storage blob list --account-name <STORAGE_ACCOUNT_NAME> \
      --container-name <CONTAINER_NAME> --auth-mode key \
      --account-key <ACCOUNT_KEY> --output table | head