Pushgateway

This feature is available starting with Kloudfuse Release 3.1.

Kloudfuse integrates directly with Pushgateway to ingest metrics. See the code and documentation for the Promethues-Pushgateway.

The Prometheus Pushgateway enables ephemeral and batch jobs to expose their metrics to Prometheus. Through it, they can push their metrics to a Pushgateway, which exposes these metrics to Prometheus.

Enable Pushgateway on Kloudfuse

Update your Kloudfuse custom-values.yaml file so that the helm chart enables Pushgateway.

global:
  orgId: "..."
  # ...
  pushgateway:
    enabled: true
  # Needed to scrape and send metrics to Kfuse
  cloud-exporter:
    enabled: true
  # ...

Install or upgrade your Kloudfuse release with a helm install/upgrade.

helm upgrade --install  kfuse oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/kfuse --version 3.1.0 -f values.yaml

After the upgrade completes, the kfuse-pushgateway appears in your environment.
When the metrics are flowing into Pushgateway, Kloudfuse scrapes it to ingest metrics into the Kloudfuse database; they appear in the Kloudfuse Metrics Summary interface.

Send Metrics to the Pushgateway

Use the Prometheus text protocol to push metrics through a command-line HTTP tool, like curl. Your favorite scripting language has most likely some built-in HTTP capabilities you can leverage here as well.

In the text protocol, each line has to end with a line-feed character, 'LF' or '\n'. A protocol error results if you end a line in other ways: with 'CR' or '\r', 'CRLF' or '\r\n', or just the end of the packet.

The protocol manages the pushed metrics in groups, identified by a grouping key of any number of labels, where the first grouping must be the job label. The groups are easy to inspect using the web interface.

For implications of special characters on label values, see URL documentation.

Explore the following examples of sending metrics with Pushgateway:

Send Single Sample
Send Complex Sample

Send Single Sample

Push a single sample into the group identified by {job="some_job"}. Because we did not provide data type information, some_metric is of type untyped.

echo "some_metric 3.14" | curl --data-binary @- https://<KFUSE-DNS>/metrics/job/some_job

Send Complex Sample

Push something more complex into the group identified by {job="some_job",instance="some_instance"}. The type information and help strings are part of the protocol in this example. While optional, we strongly encourage using this for more complex jobs.

  cat <<EOF | curl --data-binary @- https://<KFUSE-DNS>/metrics/job/some_job/instance/some_instance
  # TYPE some_metric counter
  some_metric{label="val1"} 42
  # TYPE another_metric gauge
  # HELP another_metric Just an example.
  another_metric 2398.283
  EOF

See the code and documentation for Promethues-Pushgateway to get additional information for this methodology.