Hosts and VM Integration

Kloudfuse collects telemetry from standalone hosts such as AWS EC2, Azure VMs, and GCP instances using the OpenTelemetry Collector. For the Datadog Agent alternative, see Hosts and VM Integration.

Prerequisites

Before you begin, ensure the following requirements are in place:

Requirement Details

OTel Collector

The OTel Collector installed on your host — see OTel Collector Installation.

Kloudfuse hostname

The external hostname of your Kloudfuse cluster.

API key

If ingestion authentication is enabled, an API key — see Ingestion Authentication with API Key.

Basic Configuration

The following config.yaml configures the OTel Collector to receive OTLP data and export traces, metrics, and logs to Kloudfuse. Set kf_platform to host so that Kloudfuse associates telemetry with host infrastructure:

receivers:
  otlp:
    protocols:
      grpc:
      http:
        cors:
          allowed_origins:
            - "http://*"
            - "https://*"

exporters:
  otlphttp:
    logs_endpoint: https://<kloudfuse-hostname>/ingester/otlp/v1/logs
    metrics_endpoint: https://<kloudfuse-hostname>/ingester/otlp/metrics
    traces_endpoint: https://<kloudfuse-hostname>/ingester/otlp/traces
    headers:
      Kf-Api-Key: <token>   # omit if auth is not enabled

processors:
  batch:
    timeout: 10s
  resource:
    attributes:
      - key: kf_platform    (1)
        value: "host"
        action: upsert
  resourcedetection:
    detectors:
      - env
      - system
      - ec2
      - gcp
      - azure
    override: true
    timeout: 2s

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch, resource, resourcedetection]
      exporters: [otlphttp]
    metrics:
      receivers: [otlp]
      processors: [batch, resource, resourcedetection]
      exporters: [otlphttp]
    logs:
      receivers: [otlp]
      processors: [batch, resource, resourcedetection]
      exporters: [otlphttp]
yaml
1 kf_platform: host is a required resource attribute. Kloudfuse uses it to link APM data to host infrastructure metrics.

Host and System-Level Metrics

To enable this, add the hostmetrics receiver and the kf_metrics_agent resource attribute to your configuration:

receivers:
  otlp:
    protocols:
      grpc:
      http:
        cors:
          allowed_origins:
            - "http://*"
            - "https://*"
  hostmetrics:
    collection_interval: 30s
    scrapers:
      cpu:
        metrics:
          system.cpu.utilization:
            enabled: true
      memory:
        metrics:
          system.memory.utilization:
            enabled: true
      disk:
      filesystem:
        metrics:
          system.filesystem.utilization:
            enabled: true
      network:
      load:

exporters:
  otlphttp:
    logs_endpoint: https://<kloudfuse-hostname>/ingester/otlp/v1/logs
    metrics_endpoint: https://<kloudfuse-hostname>/ingester/otlp/metrics
    traces_endpoint: https://<kloudfuse-hostname>/ingester/otlp/traces
    headers:
      Kf-Api-Key: <token>   # omit if auth is not enabled

processors:
  batch:
    timeout: 10s
  resource:
    attributes:
      - key: kf_platform
        value: "host"
        action: upsert
      - key: kf_metrics_agent   (1)
        value: "otlp"
        action: upsert
  resourcedetection:
    detectors:
      - env
      - system
      - ec2
      - gcp
      - azure
    override: true
    timeout: 2s

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch, resource, resourcedetection]
      exporters: [otlphttp]
    metrics:
      receivers: [otlp, hostmetrics]
      processors: [batch, resource, resourcedetection]
      exporters: [otlphttp]
    logs:
      receivers: [otlp]
      processors: [batch, resource, resourcedetection]
      exporters: [otlphttp]
yaml
1 kf_metrics_agent: otlp tells Kloudfuse to associate these host metrics with the OTel collector, enabling correlation in the APM Services Infra tab.