OpenTelemetry Collector for PostgreSQL

Kloudfuse can receive PostgreSQL metrics collected by the OpenTelemetry Collector postgresqlreceiver. The receiver connects directly to PostgreSQL over TCP, queries the pg_stat_* system views, and exports the results to Kloudfuse via the OTLP HTTP exporter. This approach works in any environment — Kubernetes, virtual machines, or bare metal — without requiring the Kloudfuse observability agent.

Overview

The postgresqlreceiver is part of the OpenTelemetry Collector Contrib distribution (otel/opentelemetry-collector-contrib). You configure it with a host, port, and credentials, and it periodically connects to PostgreSQL to query:

  • pg_stat_database — connections, commits, rollbacks, deadlocks, row activity, block I/O

  • pg_stat_bgwriter — checkpoint and background writer activity

  • pg_stat_replication — replication lag (primary only)

  • pg_stat_statements — per-query execution statistics (requires the extension)

The collector emits these as OTLP metrics and forwards them to the Kloudfuse ingestion endpoint, where they are stored as time-series data queryable via PromQL and FuseQL.

Prerequisites

Enable pg_stat_statements

Add pg_stat_statements to shared_preload_libraries in postgresql.conf:

shared_preload_libraries = 'pg_stat_statements'
ini

If you manage PostgreSQL with Helm (for example, Bitnami’s postgresql chart):

postgresql:
  primary:
    postgresqlSharedPreloadLibraries: "pg_stat_statements"
yaml

Restart PostgreSQL, then create the extension in each database to monitor:

CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
sql

Create a Monitoring User

Create a dedicated low-privilege user for the collector:

CREATE USER kfmon WITH PASSWORD '<password>' CONNECTION LIMIT 20;
GRANT pg_monitor TO kfmon;              -- PostgreSQL 10+
GRANT SELECT ON pg_stat_database TO kfmon;
GRANT SELECT ON pg_stat_statements TO kfmon;
sql

For PostgreSQL 9.6 and earlier, replace the pg_monitor role with the explicit grants listed in the Datadog pre-flight steps (the required grants are the same regardless of which collector you use).

On PostgreSQL 14+, also grant access to the statements info view:

GRANT SELECT ON pg_stat_statements_info TO kfmon;
sql
Set CONNECTION LIMIT to at least 20 (not 5). The postgresqlreceiver opens one connection per database per metric query, so monitoring 4 databases requires up to ~24 concurrent connections on each collection cycle. A limit of 5 causes too many connections for role errors on every scrape.

Ensure pg_hba.conf allows connections from the collector host or Kubernetes pod CIDR using md5 or scram-sha-256 authentication.

Label Enrichment

How Labels Are Attached

The postgresqlreceiver connects directly to PostgreSQL over TCP. It emits one metric series per database with a small set of receiver-provided labels:

Label Source

postgresql.database_name

Database name from pg_stat_database

service.name

Set via the resource processor

service.instance.id

Set automatically to <endpoint> (host:port)

Unlike infrastructure agents that run as a DaemonSet on the same node as PostgreSQL, the OTel Collector does not observe the pod directly — it opens a TCP connection to a service DNS name. This means the k8sattributes processor cannot automatically enrich these metrics with Kubernetes pod labels (kube_namespace, kube_service, pod_name, etc.) because the connection source IP belongs to the collector pod, not the PostgreSQL pod.

Adding Kubernetes Labels Manually

Since the PostgreSQL pod’s metadata is stable and known, add it explicitly via the resource processor. This achieves label parity with agent-based collectors (such as the Datadog Agent) that attach Kubernetes metadata automatically:

processors:
  resource:
    attributes:
      - key: service.name
        value: <postgresql-service>      # e.g. kfuse-configdb
        action: upsert
      - key: db.system
        value: postgresql
        action: upsert
      # Kubernetes identity labels — set to match the target StatefulSet/Deployment
      - key: env
        value: <environment>             # e.g. prod
        action: upsert
      - key: kube_namespace
        value: <postgresql-namespace>
        action: upsert
      - key: kube_service
        value: <postgresql-service>
        action: upsert
      - key: kube_stateful_set
        value: <postgresql-statefulset>  # if deployed as a StatefulSet
        action: upsert
      - key: host
        value: <postgresql-service>.<postgresql-namespace>.svc.cluster.local
        action: upsert
      - key: port
        value: "5432"
        action: upsert
      - key: replication_role
        value: master                    # or replica
        action: upsert
      - key: postgresql_version
        value: "<major.minor>"           # e.g. 14.18
        action: upsert
yaml
The k8sattributes processor enriches metrics using the source pod IP of incoming data. Because the postgresqlreceiver makes outbound TCP connections (not inbound scrapes), the processor sees the collector pod’s IP, not the PostgreSQL pod’s IP. Pod label discovery does not apply to TCP-based receivers — only to receivers that accept inbound data from instrumented pods (such as the OTLP receiver or the Prometheus receiver in pull mode).

Configure the Collector

Collector Configuration File

The following config.yaml configures the postgresqlreceiver and the otlphttp exporter pointed at Kloudfuse:

receivers:
  postgresql:
    endpoint: <postgresql-host>:5432
    username: kfmon
    password: <password>
    databases:
      - <database-name>         # list each database to monitor; omit to monitor all
    tls:
      insecure: true            # set to false and add ca_file for TLS connections
    collection_interval: 60s

processors:
  batch:
    timeout: 10s
  resource:
    attributes:
      - key: service.name
        value: postgresql
        action: upsert
      - key: db.system
        value: postgresql
        action: upsert
      # Add any static labels needed for filtering — see the Label Enrichment section
      # for why k8sattributes cannot be used with the postgresqlreceiver.

exporters:
  otlphttp:
    metrics_endpoint: https://<kloudfuse-hostname>/ingester/otlp/metrics
    tls:
      insecure: false

service:
  pipelines:
    metrics:
      receivers: [postgresql]
      processors: [resource, batch]
      exporters: [otlphttp]
yaml

Replace the following placeholders:

Placeholder Value

<postgresql-host>

Hostname or service name of the PostgreSQL instance

<password>

Password for the monitoring user

<database-name>

One or more database names (one entry per database)

<kloudfuse-hostname>

External hostname of your Kloudfuse cluster

Deploy on Kubernetes (Helm)

  1. Add the OpenTelemetry Helm repository:

    helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update

  2. Store the PostgreSQL password in a Kubernetes secret:

    kubectl create secret generic otel-postgres-credentials \
  --from-literal=password='<password>' \
  -n <collector-namespace>

  3. Create a values.yaml for the collector. The password is injected via an environment variable to avoid embedding it in the config:

    image:
  repository: otel/opentelemetry-collector-contrib

mode: deployment

extraEnvs:
  - name: PG_PASSWORD
    valueFrom:
      secretKeyRef:
        name: otel-postgres-credentials
        key: password

config:
  receivers:
    postgresql:
      endpoint: <postgresql-service>.<postgresql-namespace>.svc.cluster.local:5432
      username: kfmon
      password: ${env:PG_PASSWORD}
      databases:
        - <database-name>
      tls:
        insecure: true
      collection_interval: 60s

  processors:
    batch:
      timeout: 10s
    resource:
      attributes:
        - key: service.name
          value: <postgresql-service>
          action: upsert
        - key: db.system
          value: postgresql
          action: upsert
        # Kubernetes identity — set to match the target pod metadata
        - key: env
          value: <environment>
          action: upsert
        - key: kube_namespace
          value: <postgresql-namespace>
          action: upsert
        - key: kube_service
          value: <postgresql-service>
          action: upsert
        - key: host
          value: <postgresql-service>.<postgresql-namespace>.svc.cluster.local
          action: upsert
        - key: port
          value: "5432"
          action: upsert

  exporters:
    otlphttp:
      metrics_endpoint: https://<kloudfuse-hostname>/ingester/otlp/metrics

  service:
    pipelines:
      metrics:
        receivers: [postgresql]
        processors: [resource, batch]
        exporters: [otlphttp]
    yaml

  4. Install the collector:

    helm upgrade --install otel-postgres open-telemetry/opentelemetry-collector \
  -f values.yaml \
  -n <collector-namespace>

  5. Confirm the pod is running:

    kubectl get pods -n <collector-namespace> -l app.kubernetes.io/name=opentelemetry-collector

Log Collection (Optional)

To collect PostgreSQL container logs on Kubernetes, add a filelog receiver. When PostgreSQL writes logs to stdout (the default for most Kubernetes deployments), configure the receiver to read from the pod log path:

receivers:
  filelog:
    include:
      - /var/log/pods/<postgresql-namespace>_<pod-name-prefix>*/**/*.log
    operators:
      - type: regex_parser
        regex: '^(?P<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d+) \w+ (?P<severity>\w+):  (?P<message>.*)'
        timestamp:
          parse_from: attributes.timestamp
          layout: '%Y-%m-%d %H:%M:%S.%f'
        severity:
          parse_from: attributes.severity
yaml

Add logs_endpoint to the exporter and a logs pipeline:

exporters:
  otlphttp:
    metrics_endpoint: https://<kloudfuse-hostname>/ingester/otlp/metrics
    logs_endpoint: https://<kloudfuse-hostname>/ingester/otlp/v1/logs

service:
  pipelines:
    metrics:
      receivers: [postgresql]
      processors: [resource, batch]
      exporters: [otlphttp]
    logs:
      receivers: [filelog]
      processors: [resource, batch]
      exporters: [otlphttp]
yaml

Verify Metrics Are Arriving

After deploying the collector, confirm metrics are arriving in the Kloudfuse UI:

  1. Click the Metrics tab, then select Explorer from the drop-down menu.

  2. Set the time range to the last 15 minutes using the interval picker in the top-right corner.

  3. In the metric selector, type postgresql_backends and select it from the suggestions list. Confirm a chart appears with data points.

  4. Use the label filters to narrow results by database — for example, filter by db="mydb" to isolate metrics for a specific database.

Key metrics emitted by the postgresqlreceiver:

Metric (PromQL name) Description

postgresql_backends

Number of active client connections (backends)

postgresql_commits

Transactions committed per interval

postgresql_rollbacks

Transactions rolled back per interval

postgresql_db_size

Database size in bytes

postgresql_deadlocks

Deadlock count per interval

postgresql_rows

Rows returned, fetched, inserted, updated, deleted (labelled by state)

postgresql_blocks_read

Blocks read from disk vs buffer cache (labelled by source)

postgresql_bgwriter_buffers_allocated

Buffers allocated by the background writer

postgresql_bgwriter_checkpoint_count

Number of checkpoints performed (labelled by type)

postgresql_replication_data_delay

Replication lag in bytes (primary only)

To verify logs are arriving (when log collection is enabled), click the Logs tab, filter by source postgresql in the Filters panel, and confirm log entries appear in the list.

Troubleshooting

No Metrics in Kloudfuse

If postgresql_backends returns no data:

  1. Check the collector pod logs for scrape errors:

    kubectl logs -n <collector-namespace> -l app.kubernetes.io/name=opentelemetry-collector

    Look for "Error scraping postgresql" or "connection refused" messages.

  2. Confirm the collector can reach the PostgreSQL service:

    kubectl exec -n <collector-namespace> <collector-pod-name> -- \
  nc -zv <postgresql-service>.<postgresql-namespace>.svc.cluster.local 5432

  3. Verify the exporter is successfully reaching Kloudfuse by checking for "Exporting failed" in the logs.

Too Many Connections Error

If the collector logs show pq: too many connections for role "kfmon" (53300) on every scrape:

The postgresqlreceiver opens a separate connection for each database per metric query. Monitoring 4 databases requires up to ~24 concurrent connections per scrape cycle.

Increase the connection limit for the monitoring user:

ALTER USER kfmon CONNECTION LIMIT 20;
sql

A limit of 5 is insufficient when monitoring more than one database.

Authentication Errors

If the collector logs show FATAL: password authentication failed for user "kfmon":

  • Confirm the password in the Kubernetes secret matches the PostgreSQL user password.

  • Verify the user exists: SELECT usename FROM pg_user WHERE usename='kfmon';

  • Confirm pg_hba.conf allows connections from the collector pod CIDR.

Statement Metrics Are Missing

  • Confirm pg_stat_statements is in shared_preload_libraries and PostgreSQL was restarted after the change.

  • Run SELECT * FROM pg_stat_statements LIMIT 1; as the monitoring user.

  • On PostgreSQL 14+, also grant SELECT ON pg_stat_statements_info to the monitoring user.

Replica Metrics Are Missing

postgresql_replication_data_delay is only populated on the primary. To collect replica-side metrics, add a second postgresql receiver block pointing to each replica endpoint:

receivers:
  postgresql/primary:
    endpoint: <primary-host>:5432
    username: kfmon
    password: ${env:PG_PASSWORD}
    databases: [<database-name>]
  postgresql/replica:
    endpoint: <replica-host>:5432
    username: kfmon
    password: ${env:PG_PASSWORD}
    databases: [<database-name>]

service:
  pipelines:
    metrics:
      receivers: [postgresql/primary, postgresql/replica]
      processors: [resource, batch]
      exporters: [otlphttp]
yaml

External References