Ingestion Limits and Notes

Overview

Kloudfuse accepts telemetry in multiple wire formats simultaneously. You do not need to normalize data into a single format — each collection component sends data in its native protocol, and Kloudfuse handles the translation internally.

Kloudfuse supports four signal types and accepts them over a range of industry-standard and vendor protocols.

Metric Naming

PromQL metric names may only contain ASCII letters, digits, and underscores ([a-zA-Z0-9_]). When a metric arrives with a name containing any other character — dots, hyphens, slashes, or spaces — Kloudfuse replaces each unsupported character with an underscore before storing the metric.

Protocol Received name Stored as

Datadog Agent

system.cpu.user

system_cpu_user

Datadog Agent

kubernetes.memory.usage

kubernetes_memory_usage

OTLP

system.cpu.time

system_cpu_time

OTLP

container.memory.usage

container_memory_usage

CloudWatch (via Kinesis Firehose)

aws.ec2.cpuutilization

aws_ec2_cpuutilization

Prometheus remote write

node_cpu_seconds_total

node_cpu_seconds_total (no change — already valid)

Metrics that already use underscores exclusively (native Prometheus metrics) are stored unchanged. When querying in PromQL, always use the underscore form of the name regardless of the source protocol.

Logs

Logs are structured or unstructured text records with a timestamp and a set of label fields. All logs are stored in a single store and queryable with FuseQL and LogQL.

Protocol Ingester endpoint Typical source

Datadog Logs v2 API

POST /ingester/api/v2/logs

Datadog Agent (container log tail)

OTLP HTTP (Protobuf)

POST /ingester/v1/logs

OTel Collector (filelog receiver)

Fluent Bit HTTP output

POST /ingester/kloudfuse/v1/logs or compatible

Fluent Bit

Fluentd HTTP output

HTTP output plugin

Fluentd

Filebeat HTTP output

HTTP output plugin

Filebeat · Elastic Agent

AWS Kinesis Firehose (HTTP endpoint)

POST /ingester/cloudwatch/logs

CloudWatch Log Groups · CloudTrail

Azure Event Hub + Function App

HTTP trigger on Function App → ingester

Azure Monitor Diagnostic Logs

GCP Cloud Logging + Pub/Sub

Pub/Sub subscription → ingester

GCP Cloud Logging sinks

Log records are immediately searchable after ingestion. Full-text search, field extraction, and aggregation are available via FuseQL.

Traces

Traces are collections of spans representing the execution of a distributed request across services. Kloudfuse displays traces in the APM views and supports querying with TraceQL.

Protocol Ingester endpoint Typical source

Datadog APM Agent protocol

POST /ingester (APM path)

Datadog Agent trace listener · DD Lambda Extension

OTLP HTTP (Protobuf)

POST /ingester/v1/traces

OTel Collector · instrumented applications (direct push)

Datadog SDKs (direct)

DD_APM_DD_URL or DD_DD_URL environment variables

Applications instrumented with dd-trace-* SDKs

For application-level tracing, instrument your code with a Datadog or OTel SDK and point it at the Datadog Agent or OTel Collector running in your cluster. The SDK does not need to know about Kloudfuse — the agent handles forwarding.

See APM Instrumentation for instrumentation guides.

Events

Events are discrete point-in-time occurrences — not a continuous stream, but individual records of something that happened. Kubernetes scheduling events, deployment completions, and alert firings are common examples.

Protocol Store Typical source

Datadog Cluster Agent (Kubernetes API)

Events Store (source="kubernetes")

Kubernetes events (Pod scheduling, Node conditions)

Datadog Process Agent

Events Store

Container lifecycle events

OTel k8sobjects receiver

Logs Store (kf_events_agent=otlp)

Kubernetes API events (Watch mode)

AWS EventBridge API Destination

Events Store

EventBridge event patterns

AWS CloudTrail (via CloudWatch Logs + Kinesis)

Logs Store

AWS API audit events
OTel Kubernetes events land in the Logs Store, not the Events Store. Filter on kf_events_agent=otlp in the Logs explorer to view them alongside other log data.

Authentication

Ingestion authentication is optional on each Kloudfuse cluster. When enabled, every request to the ingester must include a valid bearer token:

Authorization: Bearer <token>

Obtain tokens from Admin > Settings > Auth key labels. See Ingestion Authentication with API Key for setup details.

When disabled, the ingester accepts unauthenticated requests from any client that can reach the HTTPS endpoint. In production environments, ingestion authentication is recommended.

Compression and Batching

All protocols support payload compression (typically gzip or snappy) and batching — sending multiple samples or records in a single HTTP request. This is handled automatically by the collection component and does not require any Kloudfuse configuration.

Payload size limits:

  • Metrics: up to 5 MB per request (post-decompression)

  • Logs: up to 10 MB per request (post-decompression)

  • Traces: up to 50 MB per request (post-decompression)

Requests that exceed these limits are rejected with HTTP 413. Most agents batch well below these limits by default.