Prometheus Agent Remote Write Config

Using an Existing Prometheus Server

  1. Determine the ingress IP for the Kloudfuse stack.

    kubectl get svc | grep kfuse-ingress-nginx-controller

    You can use either the EXTERNAL-IP address, or DNS name.

    DNS name does not work if the Kloudfuse stack is on a different VPC or cluster than the target cluster.

    You can use the IP address in all circumstances.

  2. Add a remote_write endpoint to use the Prometheus server. Choose the correct option, depending on your network protocol (http or https):

    http://<REPLACE KFUSE ADDRESS>/write as the remote_write
    
    https://<REPLACE KFUSE ADDRESS>/write as the remote_write
  3. If the Prometheus server is installed using the standalone Prometheus helm chart, add the following helm values and run helm upgrade:

    serverFiles:
      prometheus.yml:
        remote_write:
          - url: http://<REPLACE KFUSE ADDRESS>/write
            queue_config:
              max_samples_per_send: 4000
    yaml

Prometheus Operator

Helm Chart Values

Use these steps if the Prometheus server is installed using Prometheus Operator or kube-prometheus-stack helm chart:

  1. Add the following helm values:

    prometheus:
      remoteWrite:
        - queueConfig:
            maxSamplesPerSend: 4000
          url: http://<REPLACE KFUSE ADDRESS>/write
    yaml
  2. Run helm upgrade.

Prometheus CRD

Alternatively,configure the remote write through Prometheus CRD.

Update the Prometheus CRD (kind: Prometheus) using the following remoteWrite entry:

remoteWrite:
  - queueConfig:
      maxSamplesPerSend: 4000
    url: http://<REPLACE KFUSE ADDRESS>/write
yaml

Replacing the Prometheus Server

You can also replace the Prometheus server with Grafana Agent, which is lighter-weight and can be configured to scrape and emit write metrics.

Use the following helm chart to install the agent. Use the following helm values:

config:
  prometheus:
    remoteWrite:
      url: http://<REPLACE KfUSE ADDRESS>/write
    configs:
      <copy existing scrape configs here>
yaml