Azure Cloud Service Metric Integration

Kloudfuse collects Azure metrics through a dedicated cloud exporter that Kloudfuse deploys inside your cluster. The exporter authenticates to your Azure subscriptions with a service principal and pulls metrics directly from the Azure Monitor API.

This integration does not use the Datadog agent, and it does not install anything on your Azure resources. You only need to create a service principal and grant it read access to the subscriptions you want to monitor. Kloudfuse then discovers your resources and scrapes a fixed, predefined set of metrics for the resource types it supports. See Metrics Collected for the exact list.

Complete these tasks to collect Azure metrics:

How Kloudfuse Collects Azure Metrics

When you enable the Azure metrics exporter, Kloudfuse:

  1. Authenticates to Azure using the service principal credentials (AZURE_CLIENT_ID, AZURE_TENANT_ID, AZURE_CLIENT_SECRET) that you provide.

  2. Discovers the resources in each subscription you list.

  3. Polls Azure Monitor every minute for the supported resource types and stores the results as Kloudfuse metrics.

The set of resource types and metrics that the exporter scrapes is predefined and fixed. Resource types that are not in the supported list — for example, virtual machines (Microsoft.Compute/virtualMachines) — are not collected, even though their metrics are available in Azure Monitor. To request coverage for a resource type that is not listed, see Requesting Additional Metrics.

Create an Azure App Registration and Service Principal

The exporter authenticates as an Azure application (service principal). Create one app registration, generate a client secret, and grant the service principal read access to every subscription you want to monitor. This produces the four values you supply to the cloud exporter: the client ID, tenant ID, client secret, and one or more subscription IDs.

Required Azure Roles

Assign both of the following built-in roles to the service principal, at the subscription scope, for every subscription you want to monitor:

Azure Role Purpose Required For

Reader

Read-only access to enumerate resources in the subscription.

Resource discovery (required for all configurations).

Monitoring Reader

Read-only access to Azure Monitor metrics and metric definitions.

Reading metric data (required for all configurations).

Create the Service Principal (Azure CLI)

  1. Create the app registration and service principal, and assign the Reader role on the first subscription:

    az ad sp create-for-rbac \
  --name "kloudfuse-metrics-reader" \
  --role "Reader" \
  --scopes "/subscriptions/<SUBSCRIPTION_ID>"

    The command output contains the values you need:

    Output field Cloud exporter value

    appId

    AZURE_CLIENT_ID

    password

    AZURE_CLIENT_SECRET

    tenant

    AZURE_TENANT_ID
    The password (client secret) is shown only once. Copy it immediately and store it securely.

  2. Grant the Monitoring Reader role to the same service principal:

    az role assignment create \
  --assignee "<AZURE_CLIENT_ID>" \
  --role "Monitoring Reader" \
  --scope "/subscriptions/<SUBSCRIPTION_ID>"

  3. To monitor additional subscriptions, repeat both role assignments for each subscription ID, reusing the same service principal:

    az role assignment create --assignee "<AZURE_CLIENT_ID>" --role "Reader" \
  --scope "/subscriptions/<ANOTHER_SUBSCRIPTION_ID>"
az role assignment create --assignee "<AZURE_CLIENT_ID>" --role "Monitoring Reader" \
  --scope "/subscriptions/<ANOTHER_SUBSCRIPTION_ID>"

Create the Service Principal (Azure Portal)

  1. In the Azure portal, go to Microsoft Entra ID > App registrations > New registration. Enter a name (for example, kloudfuse-metrics-reader) and select Register.

  2. On the app’s Overview page, copy the Application (client) ID (AZURE_CLIENT_ID) and the Directory (tenant) ID (AZURE_TENANT_ID).

  3. Go to Certificates & secrets > New client secret. Copy the secret Value (AZURE_CLIENT_SECRET) immediately — it is shown only once.

  4. For each subscription you want to monitor, go to Subscriptions > <your subscription> > Access control (IAM) > Add > Add role assignment, and assign both the Reader and Monitoring Reader roles to the app registration you created. Copy the Subscription ID for use in the cloud exporter configuration.

For detailed, illustrated steps, see the Microsoft documentation on registering an application with Microsoft Entra ID. Kloudfuse only requires the Reader and Monitoring Reader roles.

Enable the Cloud Exporter

To send Azure metrics to Kloudfuse, enable cloud-exporter in the global section of the custom-values.yaml file.

global:
  cloud-exporter:
    enabled: true
yaml

Specify Secret and Subscription Information in the Cloud Exporter

Under kfuse-cloud-exporter, add the following code, replacing the redacted values with your actual values from the Azure portal.

Because you can fetch metrics from multiple subscriptions in Azure at the same time, list all your subscriptions in the subscriptions section.

kfuse-cloud-exporter:
  azure-metrics-exporter:
    enabled: true
    subscriptions:
      - xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      - xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    secrets:
      AZURE_CLIENT_ID: "xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
      AZURE_TENANT_ID: "xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
      AZURE_CLIENT_SECRET: "xxxx~xxxxxx"
yaml

Alternatively, you can mount authentication information from a secret instead of adding it in plain text in the custom-values.yaml file. In this case, add the following:

kfuse-cloud-exporter:
  azure-metrics-exporter:
    enabled: true
    subscriptions:
      - xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    extraEnv:
      - name: AZURE_CLIENT_ID
        valueFrom:
          secretKeyRef:
            key: AZURE_CLIENT_ID
            name: <Secret-name>
      - name: AZURE_TENANT_ID
        valueFrom:
          secretKeyRef:
            key: AZURE_TENANT_ID
            name: <Secret-name>
      - name: AZURE_CLIENT_SECRET
        valueFrom:
          secretKeyRef:
            key: AZURE_CLIENT_SECRET
            name: <Secret-name>
yaml

The secret must contain these key-value pairs:

data:
 AZURE_CLIENT_ID: "xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
 AZURE_TENANT_ID: "xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
 AZURE_CLIENT_SECRET: "xxxx~xxxxxx"
yaml

Apply the Changes

Apply the updated values to your Kloudfuse deployment:

helm upgrade --install kfuse oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/kfuse \
  -n kfuse \
  --version <VERSION> \ (1)
  -f custom-values.yaml
1 Replace <VERSION> with a valid Kloudfuse release value; use the most recent one.

After the exporter starts, Azure metrics appear in the Kloudfuse Metrics explorer with the azure_ prefix. If no Azure resource metrics appear, confirm that you have resources of a supported resource type in the configured subscriptions, and that the service principal has the Reader and Monitoring Reader roles.

Metrics Collected

When you enable the Azure metrics exporter, Kloudfuse scrapes the following Azure resource types. Each metric is collected at one-minute granularity and stored in Kloudfuse using the naming convention <prefix><azure-metric-name>, where any character that is not a letter or digit (for example, .) is converted to an underscore (_). For example, the AKS metric node_cpu_usage_percentage becomes azure_containerservice_managedclusters_node_cpu_usage_percentage.

Azure Resource Type Kloudfuse Metric Prefix Metrics

Azure Kubernetes Service (AKS)
Microsoft.ContainerService/managedClusters

azure_containerservice_managedclusters_*

18

Azure Event Hubs
Microsoft.EventHub/Namespaces

azure_eventhub_namespaces_*

26

Azure Data Explorer (Kusto)
Microsoft.Kusto/clusters

azure_kusto_clusters_*

32

Azure IoT Hub
Microsoft.Devices/IotHubs

azure_devices_iothubs_*

19

Azure Database for PostgreSQL
Microsoft.DBforPostgreSQL/flexibleServers

azure_dbforpostgresql_flexibleservers_*

18
For authoritative definitions of each Azure metric, see the Azure Monitor supported metrics reference.

Azure Kubernetes Service (AKS)

Resource type Microsoft.ContainerService/managedClusters. Covers node CPU, memory, disk, and network utilization; Kubernetes node and pod status; cluster autoscaler activity; and API server load.

Metric Description

node_cpu_usage_percentage

Node CPU usage as a percentage.

node_cpu_usage_millicores

Node CPU usage in millicores.

node_memory_rss_bytes

Node resident set size (RSS) memory, in bytes.

node_memory_rss_percentage

Node RSS memory as a percentage.

node_memory_working_set_bytes

Node working-set memory, in bytes.

node_memory_working_set_percentage

Node working-set memory as a percentage.

node_disk_usage_bytes

Node disk usage, in bytes.

node_disk_usage_percentage

Node disk usage as a percentage.

node_network_in_bytes

Bytes received by the node over the network.

node_network_out_bytes

Bytes transmitted by the node over the network.

kube_node_status_condition

Node status conditions (for example, Ready).

kube_node_status_allocatable_cpu_cores

Allocatable CPU cores per node.

kube_node_status_allocatable_memory_bytes

Allocatable memory per node, in bytes.

kube_pod_status_ready

Pod readiness status.

kube_pod_status_phase

Pod lifecycle phase.

cluster_autoscaler_unschedulable_pods_count

Pods the cluster autoscaler cannot schedule.

cluster_autoscaler_unneeded_nodes_count

Nodes the cluster autoscaler considers unneeded.

apiserver_current_inflight_requests

In-flight requests to the Kubernetes API server.

Azure Event Hubs

Resource type Microsoft.EventHub/Namespaces. Covers message and byte throughput, requests, connections, capture activity, errors, and namespace size.

Metric Description

IncomingMessages

Messages sent to the namespace.

OutgoingMessages

Messages received from the namespace.

IncomingBytes

Bytes sent to the namespace.

OutgoingBytes

Bytes received from the namespace.

IncomingRequests

Requests made to the namespace.

SuccessfulRequests

Successful requests.

ThrottledRequests

Throttled requests.

UserErrors

User errors.

ServerErrors

Server errors.

QuotaExceededErrors

Quota-exceeded errors.

ActiveConnections

Active connections.

ConnectionsOpened

Connections opened.

ConnectionsClosed

Connections closed.

CaptureBacklog

Event Hubs Capture backlog.

CapturedBytes

Bytes captured by Event Hubs Capture.

CapturedMessages

Messages captured by Event Hubs Capture.

Size

Size of the Event Hub, in bytes.

EHAMSGS, EHINBYTES, EHOUTBYTES, EHAMBS, FAILREQ, INTERR, MISCERR, SVRBSY, SUCCREQ

Legacy throughput, request, and error metric aliases.

Azure Data Explorer (Kusto)

Resource type Microsoft.Kusto/clusters. Covers cluster health and CPU, ingestion latency and results, query performance, cache utilization, batching, continuous export, and materialized views.

Metric Description

KeepAlive

Cluster health sentinel (returns 1 when the cluster is healthy).

CPU

Cluster CPU utilization.

InstanceCount

Number of cluster instances.

CacheUtilization

Cache utilization.

CacheUtilizationFactor

Cache utilization factor.

IngestionUtilization

Ratio of used ingestion capacity.

IngestionLatencyInSeconds

Ingestion latency, in seconds.

IngestionResult

Number of ingestion operations, by result.

QueueOldestMessage

Age of the oldest message in the ingestion queue.

EventsReceived

Events received by data connections.

EventsProcessed

Events processed by data connections.

EventsDropped

Events dropped by data connections.

BlobsReceived

Blobs received by data connections.

BlobsProcessed

Blobs processed by data connections.

BlobsDropped

Blobs dropped by data connections.

BatchBlobCount

Number of blobs in an ingestion batch.

BatchDuration

Duration of an ingestion batch.

BatchSize

Size of an ingestion batch.

BatchesProcessed

Number of ingestion batches processed.

QueryDuration

Query duration.

TotalNumberOfConcurrentQueries

Concurrent queries running on the cluster.

TotalNumberOfThrottledCommands

Throttled commands.

DiscoveryLatency

Data-connection discovery latency.

FollowerLatency

Leader-follower synchronization latency.

WeakConsistencyLatency

Weak-consistency query latency.

ContinuousExportNumOfRecordsExported

Records exported by continuous export.

MaterializedViewAgeMinutes

Materialized view age, in minutes.

MaterializedViewAgeSeconds

Materialized view age, in seconds.

MaterializedViewDataLoss

Materialized view data-loss indicator.

MaterializedViewHealth

Materialized view health (1 when healthy).

MaterializedViewRecordsInDelta

Records in the materialized view delta.

MaterializedViewResult

Materialized view result.

Azure IoT Hub

Resource type Microsoft.Devices/IotHubs. Covers device counts, device-to-cloud (D2C) and cloud-to-device (C2D) telemetry and command flow, and device twin read/update operations.

Metric Description

devices.totalDevices

Total registered devices.

devices.connectedDevices.allProtocol

Connected devices across all protocols.

d2c.telemetry.ingress.success

Device-to-cloud telemetry messages sent successfully.

d2c.telemetry.ingress.allProtocol

Total device-to-cloud telemetry messages attempted.

d2c.telemetry.egress.success

Device-to-cloud telemetry messages routed successfully.

d2c.telemetry.egress.dropped

Device-to-cloud telemetry messages dropped.

d2c.telemetry.egress.invalid

Device-to-cloud telemetry messages incompatible with the endpoint.

d2c.telemetry.egress.fallback

Device-to-cloud telemetry messages written to the fallback route.

c2d.commands.egress.complete.success

Cloud-to-device commands completed.

c2d.commands.egress.abandon.success

Cloud-to-device commands abandoned.

c2d.commands.egress.reject.success

Cloud-to-device commands rejected.

d2c.twin.read.success

Successful device-initiated twin reads.

d2c.twin.read.failure

Failed device-initiated twin reads.

d2c.twin.update.success

Successful device-initiated twin updates.

d2c.twin.update.failure

Failed device-initiated twin updates.

c2d.twin.read.success

Successful back-end-initiated twin reads.

c2d.twin.read.failure

Failed back-end-initiated twin reads.

c2d.twin.update.success

Successful back-end-initiated twin updates.

c2d.twin.update.failure

Failed back-end-initiated twin updates.

Azure Database for PostgreSQL

Resource type Microsoft.DBforPostgreSQL/flexibleServers. Covers CPU, memory, and storage utilization; IOPS and throughput; connections; and database activity.

Metric Description

cpu_percent

CPU utilization percentage.

memory_percent

Memory utilization percentage.

storage_percent

Storage utilization percentage.

storage_used

Storage used, in bytes.

storage_free

Free storage, in bytes.

iops

I/O operations per second.

read_iops

Read I/O operations per second.

write_iops

Write I/O operations per second.

read_throughput

Read throughput, in bytes per second.

write_throughput

Write throughput, in bytes per second.

active_connections

Active connections to the database.

connections_failed

Failed connection attempts.

network_bytes_ingress

Network bytes received.

network_bytes_egress

Network bytes sent.

is_db_alive

Database availability (1 when alive).

tps

Transactions per second.

numbackends

Number of backend connections.

deadlocks

Number of deadlocks detected.

Requesting Additional Metrics

The list of resource types and metrics above is the complete set that the exporter scrapes today. If you need metrics for a resource type that is not listed — for example, Microsoft.Compute/virtualMachines — file a feature request with your Kloudfuse contact, specifying the Azure resource type and the metrics you need. Support for the new resource type must be added to the exporter configuration before those metrics can be collected.