Integration Overview

Overview

Integration is the process of connecting your infrastructure and applications to Kloudfuse so that telemetry flows in continuously and becomes queryable without manual export or file uploads.

Kloudfuse is a passive receiver — it does not reach out to your infrastructure to pull data. Instead, you deploy a collection component (an agent, a cloud pipeline, or a log shipper) that reads telemetry from your systems and pushes it to the Kloudfuse ingester over HTTPS.

Once data arrives at the ingester, it is parsed, routed to the appropriate signal store, and immediately available for queries, dashboards, and alerts — no additional processing step required.

The right integration approach depends on where your workloads run, what telemetry you need, and what tools you already have in place. Most production environments use more than one approach simultaneously — for example, a Datadog Agent for application workloads combined with cloud pipelines for managed services, or an OTel Collector alongside existing Prometheus infrastructure.

What Kloudfuse Collects

Kloudfuse organizes telemetry into five signal types. Each integration delivers one or more of these signals depending on which collection method you choose.

Signal What it is How you query it

Metrics

Numeric time-series measurements — CPU usage, request latency, error rates, custom counters

PromQL

Logs

Structured or unstructured text records emitted by applications and infrastructure

FuseQL · LogQL

Traces

Distributed request traces that show how a single request propagates across services

TraceQL · APM views

Events

Point-in-time occurrences — deployments, Kubernetes scheduling events, alert firings

Events explorer

Infrastructure objects

Snapshots of live Kubernetes resources — Pods, Deployments, Nodes, Services

Infrastructure views

By Environment

Kubernetes

If your workloads run on Kubernetes, start with an in-cluster agent. An agent deployed as a DaemonSet collects node metrics, container logs, and pod-level data automatically without per-application configuration.

Agent Best when Signal coverage

Datadog Agent

You are already using Datadog SDKs for APM, or you need Kubernetes events in the Events Store

Metrics, logs, traces, events, objects

OTel Collector

You want open-standard collection and OTLP instrumentation, or you need native histogram support

Metrics, logs, traces, events (in Logs Store)

Prometheus / Grafana Agent

You already run Prometheus Operator or kube-prometheus-stack and only need to add a remote write endpoint

Metrics only

VictoriaMetrics vmagent

You want Prometheus-compatible metric collection with the lowest possible memory footprint

Metrics only

See Kubernetes Integration Architecture for a side-by-side comparison and data flow diagram.

Docker (non-Kubernetes)

For Docker Compose environments or standalone Docker hosts, run the agent in a container alongside your application containers.

Standalone VMs and Bare Metal

For non-containerized workloads running directly on virtual machines or physical servers:

Cloud-Managed Services

Managed services (RDS, S3, Azure SQL, Cloud SQL, etc.) have no host you can access to run an agent. Use a cloud pipeline to collect metrics and logs from the cloud control plane.

Cloud Integration What it collects

AWS

AWS Integration

CloudWatch metrics and logs for all AWS services, CloudTrail audit events, EventBridge events

Azure

Azure Integration

Azure Monitor diagnostic logs, Activity Log, and resource metrics for supported Azure services

GCP

GCP Integration

GCP Cloud Logging output, Stackdriver metrics for GCP resources

Cloud pipelines collect infrastructure-level telemetry only. To add APM traces or custom metrics from code running inside managed services, instrument your application and route traces to a collector running in your cluster.

By Signal Type

I Need Metrics

Any integration delivers metrics. Choose based on what else you need:

  • Metrics from Kubernetes workloads → Datadog Agent, OTel Collector, Prometheus, or vmagent (all four).

  • Metrics from VMs or bare metal hosts → Datadog Agent or OTel Collector installed directly on the host — see Hosts and VM Integration.

  • Metrics from Windows hosts → Datadog Agent on Windows Server — see Windows Integration.

  • Metrics from AWS/Azure/GCP managed services → cloud pipeline for that provider.

  • Custom application metrics → DogStatsD (with Datadog Agent), OTLP push (with OTel Collector), or Prometheus /metrics endpoint scraped by any agent.

I Need Logs

  • Container logs from Kubernetes → Datadog Agent or OTel Collector (both tail /var/log/pods automatically).

  • Logs from VMs or bare metal → Fluent Bit, Fluentd, or Filebeat forwarding from log files.

  • Logs from AWS services → CloudWatch Logs subscription to Kinesis Firehose.

  • Logs from Azure services → Diagnostic Settings → Event Hub → Function App.

  • Logs from GCP services → Cloud Logging sink → Pub/Sub.

  • Logs from Heroku → Heroku log drain.

I Need APM Traces

Tracing requires instrumenting your application code. See Tracing Architecture Integration for how APM tracing works end-to-end.

Instrumentation libraries:

  • Datadog SDKs (dd-trace-go, dd-trace-java, dd-trace-py, etc.) — traces forwarded via the Datadog Agent.

  • OTel SDKs — traces forwarded via the OTel Collector using OTLP.

I Need Kubernetes Events

  • Kubernetes cluster events → Datadog Cluster Agent (events appear in the Events Store) or OTel Collector with the k8sobjects receiver (events appear in the Logs Store tagged kf_events_agent=otlp).

  • AWS account events → EventBridge API Destination or CloudTrail.

Integration Models

Integrations fall into three broad models. The right model depends on where your workloads run and what tooling you already have.

In-Cluster Agents

An agent runs inside your infrastructure alongside your workloads. It reads telemetry directly from the local environment — container runtimes, log files, the Kubernetes API, Prometheus endpoints — and forwards it to Kloudfuse.

Agents are the most complete option: a single deployment can deliver all five signal types.

Supported agents:

For standalone hosts (not Kubernetes), see Hosts and VM Integration. For Docker environments, see Docker Integration.

Cloud Pipelines

Cloud providers offer managed data-delivery services that route telemetry from cloud-native sources (CloudWatch, Azure Monitor, GCP Cloud Logging) to an external HTTPS endpoint — without requiring any agent inside your cluster.

Cloud pipelines are the right choice when you need metrics and logs from managed services (RDS, S3, Load Balancers) that have no accessible host or container to run an agent on.

Supported cloud pipelines:

  • AWS — Kinesis Firehose for CloudWatch metrics and logs, EventBridge for events, CloudTrail for audit events

  • Azure — Event Hub and Function App for logs, Azure Monitor Cloud Exporter for metrics

  • GCP — Cloud Logging and Pub/Sub for logs, Stackdriver Exporter for metrics

  • Heroku — Logplex streams all dyno logs to Kloudfuse via an HTTPS log drain; no agent is installed

Direct Push

Some tools support pushing data directly to a standard endpoint without a dedicated Kloudfuse integration. If the tool speaks a protocol Kloudfuse understands, it can send data straight to the ingester.

Common direct-push integrations:

Existing Vendor Agent Repoint

If your applications are already instrumented with a third-party APM agent, you can redirect that agent at Kloudfuse without re-instrumenting your code. The agent continues to run as-is; only its destination endpoint changes.

  • New Relic — set NEW_RELIC_HOST to your Kloudfuse hostname to forward spans and traces from New Relic-instrumented applications

Leverage Existing Tooling

You already have…​ Recommended path

Datadog Agent deployed

Repoint dd_url to Kloudfuse — see Datadog Kubernetes Integration. No change to application code or SDK configuration required.

Prometheus Operator / kube-prometheus-stack

Add a remoteWrite entry pointing to /ingester/write — see Prometheus Remote Write Integration. Your existing scrape targets and recording rules are unaffected.

Grafana Agent

Add a Kloudfuse remote_write block — see integration/misc/prometheus.adoc#grafana-agent.

OTel Collector

Add a Kloudfuse OTLP exporter to your pipeline — see OTel Kubernetes Integration.

Fluent Bit

Add an HTTP output plugin pointing to the Kloudfuse log ingestion endpoint — see Fluent Bit Integration.

Fluentd

Add an HTTP output plugin — see Fluentd Integration.

Filebeat

Add an HTTP output — see Filebeat Integration.

No existing tooling

Deploy the Datadog Agent (for full signal coverage) or OTel Collector (for open-standards collection). Both provide metrics, logs, traces, and events with a single Helm chart.

Combining Approaches

Running multiple collection components in the same environment is supported and common. Each agent forwards independently — there is no coordination required between them.

Common combinations:

  • Datadog Agent + Prometheus remote write — Datadog Agent for full-stack coverage (logs, traces, events) plus Prometheus Operator metrics forwarded over remote write. This avoids duplicating metric collection while retaining the Prometheus ecosystem.

  • OTel Collector + cloud pipeline — OTel Collector for Kubernetes workloads, plus Kinesis Firehose (or equivalent) for CloudWatch metrics from managed AWS services.

  • Datadog Agent + Fluent Bit — Datadog Agent for metrics and traces, Fluent Bit for log sources the agent cannot reach (for example, legacy file-based logs on VMs).

All signals from all sources land in the same Kloudfuse stores and are queryable together. A single PromQL query can combine metrics from a Datadog Agent and a Prometheus remote write source. A single log search can span container logs forwarded by the agent and file logs shipped by Fluent Bit.

Topics