OpenTelemetry Collector Log Forwarding

Overview

The OpenTelemetry (OTel) Collector can act as a log forwarder: it reads container and file logs with the filelog receiver, enriches each record, and exports the logs to Kloudfuse over OTLP/HTTP. Reach for it when you want open-standard collection or already run a Collector for metrics and traces — a single Collector can carry all three signals.

Logs are exported to the Kloudfuse OTLP logs endpoint:

https://<kloudfuse-hostname>/ingester/otlp/v1/logs
text

For a lightweight, logs-only alternative, use Fluent Bit, Fluentd, or Filebeat. For the full Collector deployment that also carries metrics and traces, see OTel Kubernetes Integration and Hosts and VM Integration.

Kubernetes (DaemonSet)

For container log collection, deploy the OTel Collector Contrib distribution as a DaemonSet — one Collector pod per node reads the local container log files under /var/log/pods/.

Add the Datadog Helm repository and install with the following helm-values.yaml:

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update
image:
  repository: "otel/opentelemetry-collector-contrib"

mode: daemonset

# Enable log collection from container log files on each node
presets:
  logsCollection:
    enabled: true
    includeCollectorLogs: false
  kubernetesAttributes:
    enabled: true

config:
  exporters:
    otlphttp:
      logs_endpoint: https://<kloudfuse-hostname>/ingester/otlp/v1/logs   (1)
      headers:
        Kf-Api-Key: <token>   (2)

  processors:
    batch:
      timeout: 10s
    resourcedetection:
      detectors: [env, gcp, eks, ec2, aks, azure]
      override: false
      timeout: 2s

  service:
    pipelines:
      logs:
        receivers: [filelog]
        processors: [k8sattributes, resourcedetection, batch]
        exporters: [otlphttp]
yaml
1 Replace <kloudfuse-hostname> with your Kloudfuse cluster hostname.
2 Replace <token> with an ingestion API key, or remove the header if ingestion authentication is disabled.

Deploy:

helm upgrade --install otel-logs open-telemetry/opentelemetry-collector \
  -f helm-values.yaml \
  --namespace otel \
  --create-namespace

The logsCollection preset wires up a filelog receiver reading /var/log/pods///*.log, and the kubernetesAttributes preset enriches each record with pod name, namespace, and container labels.

Standalone Hosts

On a non-Kubernetes host (VM, bare metal, cloud instance), run the Collector directly and point the filelog receiver at your log file paths:

receivers:
  filelog:
    include: [ /var/log/app/*.log, /var/log/syslog ]
    start_at: beginning

processors:
  batch:
    timeout: 10s
  resourcedetection:
    detectors: [env, system, ec2, gcp, azure]

exporters:
  otlphttp:
    logs_endpoint: https://<kloudfuse-hostname>/ingester/otlp/v1/logs
    headers:
      Kf-Api-Key: <token>

service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [resourcedetection, batch]
      exporters: [otlphttp]
yaml

See Hosts and VM Integration for installing the Collector as a host service.

Application Logs over OTLP

If your applications already emit logs over OTLP (for example, through an OpenTelemetry SDK or logging bridge), add the otlp receiver instead of — or alongside — filelog, and keep the same otlphttp exporter:

receivers:
  otlp:
    protocols:
      http:
      grpc:

service:
  pipelines:
    logs:
      receivers: [otlp, filelog]
      processors: [resourcedetection, batch]
      exporters: [otlphttp]
yaml

Log Receivers

The OpenTelemetry Collector collects logs through receivers. Beyond the filelog receiver used above, the Collector publishes more than 100 receivers covering databases, cloud services, hosts, and log protocols — you are not limited to a handful of supported log sources. For an overview of how receivers work and the categories of technology they cover, see OTel Collector Receivers; for the full categorized list, see the Supported Receivers reference.

Each receiver produces OTLP log records that the otlphttp exporter forwards to Kloudfuse, where they are parsed into structured attributes and become searchable and available as facets.

Common Log Receivers

The table lists the receivers most often used for log collection. Add the one you need to the receivers: block and wire it into the logs pipeline as shown above.

Receiver Collects Source

filelog

Log files and container stdout under /var/log/pods

filelogreceiver

journald

The systemd journal

journaldreceiver

k8sobjects

Kubernetes objects and events as logs

k8sobjectsreceiver

syslog

Syslog messages over TCP or UDP

syslogreceiver

tcplog / udplog

Raw log lines over a TCP or UDP socket

tcplogreceiver

fluentforward

Logs from Fluent Bit or Fluentd via the Forward protocol

fluentforwardreceiver

windowseventlog

The Windows Event Log

windowseventlogreceiver

otlp

Logs pushed over OTLP by instrumented applications

otlpreceiver

Reference