Ingester API Reference

The Kloudfuse ingester exposes HTTP endpoints for all major telemetry formats. You can send data directly from any client that supports OTLP, Prometheus remote write, Datadog-compatible agents, FluentBit, Fluentd, or Filebeat — without deploying an additional collector or agent.

Overview

All ingester endpoints are reachable at your cluster’s external hostname:

https://<kloudfuse-hostname>/ingester/<endpoint-path>

The ingester accepts compressed payloads via the Content-Encoding header. Supported compression formats: gzip, deflate, zstd. Prometheus remote write additionally requires snappy.

Authentication

If ingestion authentication is enabled on your cluster, include an API key on every request. The header name depends on the endpoint family:

Endpoint family Header Example

OTLP, FluentBit, Fluentd, Filebeat, GitHub, EventBridge

Kf-Api-Key

Kf-Api-Key: <token>

Datadog-compatible (metrics, logs, traces)

Dd-Api-Key

Dd-Api-Key: <token>

Prometheus remote write

Authorization

Authorization: Bearer <token>

AWS Kinesis Firehose

X-Amz-Firehose-Access-Key

X-Amz-Firehose-Access-Key: <token>

To generate an API key, see Ingestion Authentication with API Key.

OTLP (OpenTelemetry Protocol)

The ingester implements the full OTLP HTTP specification. Use these endpoints with any OpenTelemetry Collector, SDK, or instrumented application.

Telemetry type Endpoint Accepted Content-Type

Metrics

POST /ingester/otlp/metrics

application/x-protobuf, application/json

Logs

POST /ingester/otlp/v1/logs

application/x-protobuf, application/json

Traces

POST /ingester/otlp/traces

application/x-protobuf, application/json

Kubernetes events

POST /ingester/otlp/k8s_events

application/x-protobuf, application/json

Kubernetes objects

POST /ingester/otlp/k8s_objects

application/x-protobuf, application/json
Both /otlp/metrics and /otlp/v1/metrics (and similarly /otlp/traces and /otlp/v1/traces) resolve to the same handler and are interchangeable.

Example: Send a Metric via OTLP/HTTP JSON

curl -X POST https://<kloudfuse-hostname>/ingester/otlp/metrics \
  -H "Content-Type: application/json" \
  -H "Kf-Api-Key: <token>" \
  -d '{
    "resourceMetrics": [{
      "resource": {
        "attributes": [
          {"key": "host.name", "value": {"stringValue": "prod-server-1"}},
          {"key": "service.name", "value": {"stringValue": "my-app"}}
        ]
      },
      "scopeMetrics": [{
        "metrics": [{
          "name": "app.request.count",
          "description": "Total HTTP requests",
          "unit": "1",
          "sum": {
            "dataPoints": [{
              "asInt": "1042",
              "timeUnixNano": "1700000000000000000",
              "attributes": [
                {"key": "http.method", "value": {"stringValue": "GET"}}
              ]
            }],
            "aggregationTemporality": 2,
            "isMonotonic": true
          }
        }]
      }]
    }]
  }'

Example: Send a Log Record via OTLP/HTTP JSON

curl -X POST https://<kloudfuse-hostname>/ingester/otlp/v1/logs \
  -H "Content-Type: application/json" \
  -H "Kf-Api-Key: <token>" \
  -d '{
    "resourceLogs": [{
      "resource": {
        "attributes": [
          {"key": "service.name", "value": {"stringValue": "my-app"}}
        ]
      },
      "scopeLogs": [{
        "logRecords": [{
          "timeUnixNano": "1700000000000000000",
          "severityText": "ERROR",
          "body": {"stringValue": "Database connection refused"},
          "attributes": [
            {"key": "env", "value": {"stringValue": "production"}}
          ]
        }]
      }]
    }]
  }'

OpenTelemetry Collector Configuration

To export from an OTel Collector to Kloudfuse, configure the otlphttp exporter:

exporters:
  otlphttp:
    metrics_endpoint: https://<kloudfuse-hostname>/ingester/otlp/metrics
    logs_endpoint: https://<kloudfuse-hostname>/ingester/otlp/v1/logs
    traces_endpoint: https://<kloudfuse-hostname>/ingester/otlp/traces
    headers:
      Kf-Api-Key: <token>
yaml

See OTel Collector on Kubernetes and OTel Collector on Standalone Hosts for complete collector configurations.

Prometheus Remote Write

The ingester implements the Prometheus remote write protocol. Any Prometheus instance or compatible agent (Grafana Agent, VictoriaMetrics agent, etc.) can write directly to Kloudfuse.

Telemetry type Endpoint Notes

Metrics

POST /ingester/write

Snappy-compressed protobuf. Standard Prometheus remote write.

GCP metrics

POST /ingester/gcp_metrics

Same format; used for GCP-labelled metric streams.

Payload must be a snappy-compressed Prometheus WriteRequest protobuf. The Content-Encoding: snappy header is required.

Prometheus Configuration

Add a remote_write block to your prometheus.yml:

remote_write:
  - url: https://<kloudfuse-hostname>/ingester/write
    authorization:
      credentials: <token>
    queue_config:
      max_samples_per_send: 10000
      batch_send_deadline: 5s
yaml

Grafana Agent / Alloy Configuration

prometheus.remote_write "kloudfuse" {
  endpoint {
    url = "https://<kloudfuse-hostname>/ingester/write"
    bearer_token = "<token>"
  }
}
yaml

Datadog-Compatible Endpoints

The ingester accepts data from the Datadog Agent and any Datadog-compatible client. Use Dd-Api-Key for authentication.

Telemetry type Endpoint Accepted Content-Type

Logs (v1)

POST /ingester/v1/input

application/json

Logs (v2)

POST /ingester/api/v2/logs

application/json

Metrics (v2)

POST /ingester/api/v2/series

application/x-protobuf

Traces

POST /ingester/api/v0.2/traces

application/x-protobuf

Example: Send Logs (Datadog JSON Format)

curl -X POST https://<kloudfuse-hostname>/ingester/api/v2/logs \
  -H "Content-Type: application/json" \
  -H "Dd-Api-Key: <token>" \
  -d '[{
    "hostname": "web-01",
    "service": "my-app",
    "status": "error",
    "message": "Unhandled exception in request handler",
    "ddtags": "env:production,version:2.1.0"
  }]'

FluentBit

Point a FluentBit output at the Kloudfuse ingester using the HTTP output plugin.

Telemetry type Endpoint Accepted Content-Type

Logs

POST /ingester/v1/fluent_bit

application/json, application/msgpack

FluentBit Output Configuration

[OUTPUT]
    Name              http
    Match             *
    Host              <kloudfuse-hostname>
    Port              443
    TLS               On
    URI               /ingester/v1/fluent_bit
    Format            json
    Header            Kf-Api-Key <token>
    Compress          gzip
ini

Fluentd

Point a Fluentd output at the Kloudfuse ingester using the HTTP output plugin.

Telemetry type Endpoint Accepted Content-Type

Logs

POST /ingester/v1/fluentd

application/x-ndjson (NDJSON)

Fluentd Output Configuration

<match **>
  @type http
  endpoint https://<kloudfuse-hostname>/ingester/v1/fluentd
  headers {"Kf-Api-Key": "<token>"}
  content_type application/x-ndjson
  <format>
    @type json
  </format>
  <buffer>
    flush_interval 5s
  </buffer>
</match>
ruby

Filebeat

Configure Filebeat to ship logs using the Elasticsearch output pointed at the Kloudfuse ingester, which exposes an Elasticsearch-compatible bulk ingestion endpoint.

Telemetry type Endpoint Accepted Content-Type

Logs (bulk)

POST /ingester/api/v1/filebeat/_bulk

application/x-ndjson

Filebeat Output Configuration

output.elasticsearch:
  hosts: ["https://<kloudfuse-hostname>/ingester/api/v1/filebeat"]
  api_key: "ignored:<token>"   # value after ":" is sent as Kf-Api-Key
  ssl.verification_mode: full
yaml

Events

The ingester accepts raw event payloads from webhook sources.

Telemetry type Endpoint Notes

GitHub webhooks

POST /ingester/github/events

JSON payload. Set as the webhook Payload URL in GitHub settings.

AWS EventBridge

POST /ingester/eventbridge

JSON payload. Used by the AWS EventBridge integration.

See GitHub CI/CD Webhook Integration for the full GitHub webhook setup guide.

Response Codes

HTTP status Meaning

200 OK

Payload accepted and queued for ingestion.

202 Accepted

Payload accepted for async processing.

400 Bad Request

Malformed payload, unsupported content type, or missing/invalid API key.

429 Too Many Requests

Rate limit exceeded. Retry after a short backoff.

503 Service Unavailable

Ingester is at maximum concurrent request capacity. Retry with exponential backoff.

Compression Reference

All endpoints support compressed request bodies via the Content-Encoding header:

Value Notes

gzip

Recommended for most clients. Widely supported.

deflate

zlib deflate format.

zstd

Higher compression ratio; use where client support is available.

snappy

Required for Prometheus remote write (/write, /gcp_metrics).