Azure Integration Architecture

Kloudfuse supports two parallel paths for ingesting Azure observability data: a log pipeline built on Azure Event Hubs and Azure Functions, and a metrics pipeline powered by the Kloudfuse Cloud Exporter.

Kloudfuse Azure integration architecture showing log and metrics data flows

Overview

Azure integration uses two independent pipelines that can be deployed together or separately:

  • Log pipeline — Azure resources emit logs to an Event Hub via Diagnostic Settings. An Azure Function App reads from the Event Hub and forwards records to the Kloudfuse HTTP ingestion endpoint.

  • Metrics pipeline — The Kloudfuse Cloud Exporter, running inside your AKS cluster, periodically scrapes the Azure Monitor Metrics API and stores the results in Kloudfuse.

Both pipelines write to the same Kloudfuse Storage and Query Layer, so logs and metrics from the same Azure resource appear together in the Kloudfuse UI.

Log Collection Pipeline

Data flows as follows:

  1. Azure resources (VMs, App Services, databases, and others) emit activity logs and resource logs.

  2. Diagnostic Settings route those logs to an Azure Event Hub Namespace.

  3. An Azure Function App with an Event Hub trigger reads each event and forwards it to the Kloudfuse logs ingestion endpoint over HTTPS.

  4. Kloudfuse stores the logs and makes them searchable in Log Analytics.

This pipeline is event-driven: logs arrive in Kloudfuse within seconds of being emitted by the Azure resource. The Function App handles batching and retries automatically.

Metrics Collection Pipeline

The Kloudfuse Cloud Exporter is a Prometheus-compatible scraper that runs as a pod in your AKS cluster alongside Kloudfuse. It calls the Azure Monitor Metrics API on a configurable interval (default: five minutes) to retrieve metric data points for all resources in the specified subscriptions.

Data flows as follows:

  1. Azure resources publish metrics to Azure Monitor.

  2. The Cloud Exporter polls the Azure Monitor Metrics API and retrieves metric data.

  3. The Cloud Exporter writes the metrics directly into the Kloudfuse storage layer.

Because the Cloud Exporter runs inside your cluster, no inbound firewall rules are required. Only outbound HTTPS access to the Azure Monitor Metrics API endpoint (management.azure.com) is needed.

Authentication

Both pipelines authenticate to Azure using a Service Principal (app registration):

Credential Where it is used

Client ID (AZURE_CLIENT_ID)

Identifies the app registration

Tenant ID (AZURE_TENANT_ID)

Identifies the Azure Active Directory tenant

Client Secret (AZURE_CLIENT_SECRET)

Authenticates the app registration

Subscription ID

Scopes which Azure subscriptions to collect metrics from

The Service Principal must be assigned the Monitoring Reader role on each subscription you want to monitor. For the log pipeline, the Function App authenticates to the Event Hub using a connection string (SAS policy).

Choosing an Integration Method

Requirement Use

Collect Azure activity and resource logs

Azure Log Integration

Collect Azure Monitor metrics

Azure Metrics Integration

Full Azure observability (logs and metrics)

Both pipelines

Next Steps