GCP Metrics Integration

Kloudfuse collects GCP Cloud Monitoring metrics using the Prometheus Stackdriver Exporter — an open-source scraper that runs inside your GKE cluster and periodically pulls metrics from the Cloud Monitoring (Stackdriver) Metrics API.

For an overview of how this fits into the broader GCP integration architecture, see GCP Integration Architecture.

Overview

The Prometheus Stackdriver Exporter polls the Cloud Monitoring Metrics API on a configurable interval and exposes the results in Prometheus format, which Kloudfuse scrapes and stores. It supports single and multi-project configurations.

Metric type prefixes control which GCP services are collected. Common prefixes include:

Prefix GCP service

compute.googleapis.com

Compute Engine (VMs, disks, networking)

container.googleapis.com

Google Kubernetes Engine

storage.googleapis.com

Cloud Storage

cloudsql.googleapis.com

Cloud SQL

run.googleapis.com

Cloud Run

pubsub.googleapis.com

Cloud Pub/Sub

kubernetes.io/node

GKE node metrics

See the Google Cloud metrics reference for the full list of available metric type prefixes.

Prerequisites

  • A running Kloudfuse deployment on GKE.

  • The gcloud CLI installed and authenticated, or access to the Google Cloud Console.

  • Permissions to create service accounts and assign IAM roles in your GCP project.

Step 1: Create a Service Account

The Stackdriver Exporter authenticates to GCP using a service account JSON key.

Create the Service Account

gcloud iam service-accounts create kloudfuse-gcp \
  --display-name="Kloudfuse GCP Integration" \
  --project=<project-id>
bash

Assign IAM Roles

Role Purpose

roles/monitoring.viewer

Read-only access to Cloud Monitoring metrics (required)

roles/compute.viewer

Read-only access to GCE instance metadata and labels (required for enrichment only)

 
gcloud projects add-iam-policy-binding <project-id> \
  --member="serviceAccount:kloudfuse-gcp@<project-id>.iam.gserviceaccount.com" \
  --role="roles/monitoring.viewer"

# Required only if enabling metrics enrichment:
gcloud projects add-iam-policy-binding <project-id> \
  --member="serviceAccount:kloudfuse-gcp@<project-id>.iam.gserviceaccount.com" \
  --role="roles/compute.viewer"
bash

For multi-project setups, repeat the role assignments for each additional project.

Create the JSON Key

gcloud iam service-accounts keys create kloudfuse-gcp-credentials.json \
  --iam-account=kloudfuse-gcp@<project-id>.iam.gserviceaccount.com
bash

Step 2: Create the Kubernetes Secret

The credentials file must be named credentials.json inside the secret. 
kubectl create secret generic kfuse-gcp-credentials \
  --from-file=credentials.json=kloudfuse-gcp-credentials.json \
  --namespace <kloudfuse-namespace>
bash

Replace <kloudfuse-namespace> with the namespace your Kloudfuse installation is deployed into.

The secret must be created in the same namespace as your Kloudfuse installation — not necessarily kfuse. The Stackdriver Exporter pod runs in your installation namespace and will fail to start with a secret not found error if the secret exists in a different namespace. Verify your installation namespace with kubectl get pods -A | grep stackdriver.
This secret is shared with the log pipeline. If you already created it for GCP Logs, reuse it — just ensure the service account has both roles/pubsub.subscriber and roles/monitoring.viewer.

Step 3: Configure Kloudfuse via Helm

Enable the Cloud Exporter

In your custom-values.yaml, enable the cloud exporter and set the GCP credentials secret:

global:
  cloud-exporter:
    enabled: true
  gcpConfig:
    secretName: "kfuse-gcp-credentials"
yaml

Single Project Configuration

kfuse-cloud-exporter:
  prometheus-stackdriver-exporter:
    enabled: true
    stackdriver:
      httpTimeout: 30s
      maxRetries: 3
      projectId: "<project-id>" (1)
      metrics:
        typePrefixes: "compute.googleapis.com,container.googleapis.com,storage.googleapis.com" (2)
yaml
1 GCP project ID to monitor.
2 Comma-separated list of metric type prefixes to collect.

Multiple Projects Configuration

kfuse-cloud-exporter:
  prometheus-stackdriver-exporter:
    enabled: true
    stackdriver:
      httpTimeout: 30s
      maxRetries: 3
      metrics:
        typePrefixes: "compute.googleapis.com,container.googleapis.com"
    config:
      project_ids: (1)
        - project-id-1
        - project-id-2
        - project-id-3
yaml
1 Array of GCP project IDs to monitor. Use this format for multiple projects.

Apply the Helm Upgrade

helm upgrade --install kfuse oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/kfuse \
  -n kfuse \
  --version <VERSION> \ (1)
  -f custom-values.yaml
1 Replace <VERSION> with a valid Kloudfuse release value; use the most recent one.

Configure GCP Metrics Enrichment

Metrics enrichment attaches GCP resource labels — such as custom labels applied to Compute Engine instances — to the corresponding metrics. This allows you to filter metrics in Kloudfuse by any label applied to your GCP resources.

Enrichment requires the roles/compute.viewer IAM role on the service account (see Assign IAM Roles).

Global Enrichment Configuration

Add the following under the global section:

global:
  enrichmentEnabled:
    - gcp
  gcpConfig:
    secretName: "kfuse-gcp-credentials"
yaml

Ingester Enrichment Configuration

Configure the ingester scraper settings:

ingester:
  config:
    gcpScrapeIntervalMinutes: 30 (1)
    gcpProjectConfigs:
      - projectId: "<project-id>" (2)
        zones: (3)
          - "us-west1-b"
          - "us-central1-a"
        services: (4)
          - "GCE"
    gcpZones: (5)
      - "us-west1-b"
      - "us-central1-a"
    gcpServices:
      - "GCE"
yaml
1 How often to refresh resource labels from GCP (default: 30 minutes; minimum: 5 minutes).
2 GCP project ID to scrape resource labels from.
3 GCP zones where your resources are deployed.
4 GCP services to enrich. Currently GCE (Compute Engine) is supported.
5 Default zones used when not specified per project.

Multi-Project Enrichment

ingester:
  config:
    gcpScrapeIntervalMinutes: 30
    gcpProjectConfigs:
      - projectId: "project-1"
        zones: ["us-west1-b"]
        services: ["GCE"]
      - projectId: "project-2"
        zones: ["europe-west1-b"]
        services: ["GCE"]
yaml

Verify the Integration

Confirm the Stackdriver Exporter is Running

kubectl get pods -n <kloudfuse-namespace> -l app=prometheus-stackdriver-exporter
bash

The pod status must be Running. If it is CrashLoopBackOff, check the logs:

kubectl logs -n <kloudfuse-namespace> -l app=prometheus-stackdriver-exporter --tail=100
bash

Confirm Metrics are Arriving in Kloudfuse

  1. In the Kloudfuse UI, click the Metrics tab and select Explorer from the drop-down.

  2. In the metric search field, type gcp_ to list all ingested GCP metrics. Metrics follow the naming convention gcp_<service>_<metric_name> in lowercase with underscores, for example:

    Cloud Monitoring metric Kloudfuse metric name

    compute.googleapis.com/instance/cpu/utilization

    gcp_compute_instance_cpu_utilization

    container.googleapis.com/container/cpu/usage_time

    gcp_container_container_cpu_usage_time

    storage.googleapis.com/api/request_count

    gcp_storage_api_request_count

    cloudsql.googleapis.com/database/cpu/utilization

    gcp_cloudsql_database_cpu_utilization

  3. Select a metric and set the time range to the last 15 minutes. A non-empty chart confirms end-to-end delivery.

The first metrics appear approximately one scrape interval after the Stackdriver Exporter pod starts.

Troubleshooting

Stackdriver Exporter Pod is in CrashLoopBackOff

kubectl logs -n <kloudfuse-namespace> -l app=prometheus-stackdriver-exporter --tail=50
bash

Common causes:

Error message Resolution

google: could not find default credentials

The Kubernetes secret is missing or the credentials.json key is absent. Run kubectl get secret kfuse-gcp-credentials -n <kloudfuse-namespace> -o jsonpath='{.data}' and confirm credentials.json is present.

googleapi: Error 403: caller does not have permission

The service account is missing the roles/monitoring.viewer role. Run the role assignment command in Assign IAM Roles.

invalid character in JSON

The service account key file was corrupted during base64 encoding or copy-paste. Recreate the secret from the original downloaded JSON file.

rpc error: code = InvalidArgument

The projectId or a value in project_ids does not match an active GCP project. Verify with gcloud projects list.

No GCP Metrics Appear in Kloudfuse

  1. Confirm the Stackdriver Exporter pod is Running (see Confirm the Stackdriver Exporter is Running).

  2. Wait at least one full scrape interval after the pod starts.

  3. Check the exporter logs for scrape errors:

    kubectl logs -n <kloudfuse-namespace> -l app=prometheus-stackdriver-exporter | grep -i error
    bash

  4. Confirm the typePrefixes in custom-values.yaml match metric types that exist in your project. Verify available metrics with:

    gcloud monitoring metrics list \
  --project=<project-id> \
  --filter="metric.type:compute.googleapis.com" \
  --limit=5
    bash

Metrics from One Project are Missing

  1. Confirm the project ID is listed under config.project_ids (or stackdriver.projectId) in custom-values.yaml.

  2. Confirm the service account has roles/monitoring.viewer on that project:

    gcloud projects get-iam-policy <project-id> \
  --flatten="bindings[].members" \
  --filter="bindings.members:kloudfuse-gcp@" \
  --format="table(bindings.role)"
    bash

Enrichment Labels Not Appearing on Metrics

  1. Confirm enrichmentEnabled includes gcp in custom-values.yaml.

  2. Confirm the service account has roles/compute.viewer.

  3. Check the ingester logs for enrichment errors:

    kubectl logs -n <kloudfuse-namespace> -l app=ingester | grep -i "gcp\|enrich\|error"
    bash

  4. New labels may take up to gcpScrapeIntervalMinutes (default: 30 minutes) to appear. Wait one full interval.

External References

Prometheus Stackdriver Exporter

Google Cloud documentation