Datadog Agent Log Forwarding

Overview

If you are currently run the Datadog Agent — or want a single component that collects logs alongside metrics, traces, and events — the Agent can act as your log forwarder for Kloudfuse.

The Agent can tail container logs, journald, and log files, tags streams with a source and service, and forward records to Kloudfuse.

Log collection is disabled by default in the Datadog Agent.

Enable Log Collection

Under the logs_config section of the Agent configuration:

logs_enabled: true

logs_config:
  logs_dd_url: "<kloudfuse-hostname>:443"   (1)
  logs_no_ssl: false
  use_http: true
  use_v2_api: false
yaml
1 Replace <kloudfuse-hostname> with your Kloudfuse cluster hostname.

This is the standalone-host datadog.yaml form. It matches the values file on Hosts and VM Integration; for the Kubernetes equivalent (the same keys under agents.customAgentConfig.logs_config), see Datadog Kubernetes Integration.

With a Kubernetes install your Helm custom-values.yaml would be.

datadog:
  logsEnabled: true
  logs:
    enabled: true
    containerCollectAll: true
    containerCollectUsingFiles: true
    autoMultiLineDetection: true
yaml
The two forms enable logs with different keys. The raw datadog.yaml (host and Docker installs) uses logs_enabled (snake_case); the Helm chart (Kubernetes) uses datadog.logsEnabled (camelCase) together with datadog.logs.enabled. Use the form that matches how you deploy the Agent.

See Datadog Kubernetes Integration for the complete Kubernetes values file, and Hosts and VM Integration for host-based installs.

Tag Log Sources by Integration

The Agent associates each log stream through a source tag.

Log Files on Hosts

Add a log source to the integration’s configuration file under conf.d/<integration>.d/conf.yaml:

logs:
  - type: file
    path: /var/log/myapp/*.log
    source: myapp
    service: my-service
yaml

Docker Containers

Set the log source with a Docker label on the container:

LABEL "com.datadoghq.ad.logs"='[{"source":"postgresql","service":"postgresql"}]'
yaml

Kubernetes Pods

Use Autodiscovery annotations on the pod to tag its logs:

metadata:
  annotations:
    ad.datadoghq.com/<container-name>.logs: '[{"source":"postgresql","service":"postgresql"}]'
yaml

Per-Integration Log Guidance

The Datadog Agent ships with a catalog of more than 400 integrations covering databases, web servers, message brokers, caches, container runtimes, cloud services, and operating-system subsystems. Every one of these works unchanged when the Agent forwards to Kloudfuse — you are not limited to a handful of supported log sources. For an overview of how integrations work and the categories of technology they cover, see Datadog Agent Integrations; for the full categorized list of supported systems, see the Supported Integrations reference.

Each integration defines a canonical source value. Setting that source on a log stream does two things: it tells the Datadog Agent which technology produced the log, and it selects the matching integration pipeline that parses the log into structured attributes (log level, status code, query duration, client IP, and so on) so the records are immediately searchable and available as facets in Kloudfuse.

Common Integration Sources

The table lists the canonical source value for a few widely used integrations. Use it in a file, Docker-label, or Autodiscovery configuration as shown above.

System source Datadog integration docs

PostgreSQL

postgresql

postgres

MySQL

mysql

mysql

NGINX

nginx

nginx

Apache HTTP

apache

apache

Redis

redis

redisdb

MongoDB

mongo

mongo

Apache Kafka

kafka

kafka

RabbitMQ

rabbitmq

rabbitmq

Elasticsearch

elasticsearch

elastic

Application-Side Logging Setup

Most integrations also require a small amount of application-side configuration so the logs they emit are complete and parseable. For example, the PostgreSQL integration recommends enabling the logging collector and setting a detailed log_line_prefix before tailing the log file with source: postgresql:

logging_collector = on
log_directory = 'pg_log'
log_line_prefix = '%m [%p] %d %a %u %h %c '
log_statement = 'all'
ini

Each integration’s Datadog documentation page — reachable from the Datadog integrations catalog — documents its recommended source, the log-collection block to add, and any application-side logging changes. For agent-wide log-collection options, see the Datadog Agent log collection documentation.

Find the integration you need, apply its recommended source and logging setup, then wire it up with one of the tagging methods above.