OpenTelemetry Collector on a Docker Environment

Basic OpenTelemetry Deployment on Docker Environments

When deploying OpenTelemetry Collector on a Docker Environment, refer to Basic OpenTelemetry Integration on Docker.

See OpenTelemetry Runtime Metrics and Container Resource Attributes Collection for Telemetry Data for optional deployment configurations.

Basic OpenTelemetry Integration on Docker

receivers:
  otlp:
    protocols:
      grpc:
      http:
        cors:
          allowed_origins:
            - "http://*"
            - "https://*"

exporters:
  otlphttp:
    logs_endpoint: https://<REPLACE WITH KFUSE ADDRESS>/ingester/otlp/v1/logs
    metrics_endpoint: https://<REPLACE WITH KFUSE ADDRESS>/ingester/otlp/metrics
    traces_endpoint: https://<REPLACE WITH KFUSE ADDRESS>/ingester/otlp/traces

processors:
  batch:
    timeout: 10s
  resource:
    attributes:
    - key: kf_platform (1)
      value: "docker"
      action: upsert
  resourcedetection:
    detectors:
    - env
    - docker
    - ec2
    - gcp
    - azure
    override: true
    timeout: 2s

connectors:
  spanmetrics:

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch, resource, resourcedetection]
      exporters: [otlphttp]
    metrics:
      exporters: [otlphttp]
      processors: [batch, resource, resourcedetection]
      receivers: [otlp]
    logs:
      exporters: [otlphttp]
      processors: [batch, resource, resourcedetection]
      receivers: [otlp]

1	`kf_platform` with `docker` value is a required resource attribute.

Docker Resource Detector

The Docker Resource Detector ensures that the host machine’s hostname is used as the host.name resource attribute. This requires that the open telemetry collector have permissions to read from the docker.sock file. When running the open telemetry collector as part of docker-compose, add the code in Add READ permission for docker.sock.

Add READ permission for docker.sock

    volumes:
      ...
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
        read_only: True
    user: 10001:<REPLACE GROUP> (1)

1	The `group` of the `docker.sock` file. Get by running `stat -c '%g' /var/run/docker.sock` command at the prompt.

Docker and System-Level Metrics

Kloudfuse APM Services integrate with Docker and System Metrics to show related infrastructure metrics. See Service Detail Infrastructure.

You can export these metrics using either Datadog agent, or the OpenTelemetry collector.

When using the OpenTelemetry Collector, you must update it with additional configurations. Specifically, enable dockerstats and hostmetrics receivers, add additional kf_metrics_agent resource attributes. See Enable Additional Attributes for System-Level Metrics.

Enable Additional Attributes for System-Level Metrics

receivers:
 ...
  docker_stats:

  hostmetrics:
    collection_interval: 30s
    scrapers:
      cpu:
        metrics:
          system.cpu.utilization:
            enabled: true
      memory:
        metrics:
          system.memory.utilization:
            enabled: true
      disk:
      filesystem:
        metrics:
          system.filesystem.utilization:
            enabled: true

processors:
  batch:
    timeout: 10s
  resource:
    attributes:
    - key: kf_platform
      value: "docker"
      action: upsert
    - key: kf_metrics_agent
      value: "otlp"
      action: upsert
  resourcedetection:
    detectors:
    - env
    - docker
    - ec2
    - gcp
    - azure
    override: true
    timeout: 2s

service:
  pipelines:
   ...
    metrics:
      exporters: [otlphttp]
      processors: [batch, resource, resourcedetection]
      receivers: [otlp, docker_stats, hostmetrics]