AWS CloudWatch Metrics Integration

Overview

Kloudfuse ingests AWS CloudWatch metrics by connecting a CloudWatch Metrics Stream to a Kinesis Firehose delivery stream. Metrics are forwarded continuously to Kloudfuse as they are published by AWS services.

For an overview of the AWS integration architecture, see AWS Integration Architecture.

Prerequisites

Before you begin, ensure the following requirements are in place:

Requirement Details

Kinesis Firehose delivery stream

A Kinesis Firehose delivery stream named kloudfuse-metrics pointed at your Kloudfuse metrics endpoint. If you have not created this yet, complete AWS Kinesis Firehose Integration first.

AWS CLI

AWS CLI configured with permissions to create CloudWatch Metrics Streams.

Step 1: Create a CloudWatch Metrics Stream

Create a metrics stream that forwards all namespaces to the Firehose stream:

aws cloudwatch put-metric-stream \
  --name kloudfuse-metrics-stream \
  --firehose-arn "arn:aws:firehose:<region>:<account-id>:deliverystream/kloudfuse-metrics" \
  --role-arn "arn:aws:iam::<account-id>:role/KloudfuseFirehoseRole" \
  --output-format opentelemetry1.0
bash

To forward only specific namespaces, add --include-filters:

aws cloudwatch put-metric-stream \
  --name kloudfuse-metrics-stream \
  --firehose-arn "arn:aws:firehose:<region>:<account-id>:deliverystream/kloudfuse-metrics" \
  --role-arn "arn:aws:iam::<account-id>:role/KloudfuseFirehoseRole" \
  --output-format opentelemetry1.0 \
  --include-filters '[
    {"Namespace": "AWS/EC2"},
    {"Namespace": "AWS/RDS"},
    {"Namespace": "AWS/Lambda"},
    {"Namespace": "AWS/ApplicationELB"}
  ]'
bash

Step 1a: Enable Services to Emit Metrics (Optional)

Several AWS services do not publish CloudWatch metrics by default and require per-resource opt-in before metrics appear in the stream.

See CloudWatch Enablement Reference for enablement instructions for each service.

Step 2: Enable Metrics Enrichment

Metrics enrichment attaches AWS resource metadata — such as instance type, availability zone, environment tags, and custom resource tags — to incoming CloudWatch metrics. Without enrichment, metrics only carry the default CloudWatch dimensions (for example, InstanceId). With enrichment enabled, every metric also carries any tag you have applied to that resource in AWS.

Configure IAM for Enrichment

Create an IAM policy for the enrichment scraper:

The policy below is a minimal example and is not intended to be copied as-is. For ready-to-use policy JSON, see Enrichment IAM Permissions. 
cat > kloudfuse-enrichment-policy.json <<'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeImages",
        "ec2:DescribeRegions",
        ...
        "tag:GetResources",
        "tag:GetTagKeys",
        "tag:GetTagValues"
      ],
      "Resource": "*"
    }
  ]
}
EOF

aws iam create-policy \
  --policy-name KloudfuseEnrichmentPolicy \
  --policy-document file://kloudfuse-enrichment-policy.json
bash

Configure Kloudfuse via Helm

See Kloudfuse Enrichment for the full Helm configuration, including AWS credential options, namespace and region filtering, and the Helm upgrade command.

Enrichment Reference

The enrichment scraper attaches metadata labels to metrics, by namespace. The table below shows a representative sample of common namespaces:

Namespace Labels added

AWS/EC2

instance_id, instance_type, availability_zone, region, image_id, vpc_id, plus all instance tags

AWS/RDS

db_instance_identifier, db_instance_class, engine, availability_zone, plus all RDS tags

AWS/ECS

cluster_name, container_instance_id, availability_zone, plus all ECS tags

AWS/Lambda

function_name, runtime, memory_size, plus all Lambda tags

AWS/DynamoDB

table_name, plus all DynamoDB tags

AWS/ElastiCache

cache_cluster_id, cache_node_type, engine, plus all ElastiCache tags
This is a sample only. For the complete list of enriched labels and IAM permissions for supported AWS services, see AWS Supported Services.

Step 3: Verify the Integration

Confirm the Metrics Stream is Running

aws cloudwatch get-metric-stream \
  --name kloudfuse-metrics-stream \
  --query '{State: State, OutputFormat: OutputFormat, CreationDate: CreationDate}'
bash

The State should be RUNNING.

  • Check the Firehose delivery stream monitoring for delivery errors (see Troubleshooting).

Confirm Metrics are Arriving in Kloudfuse

  1. In the Kloudfuse UI, click the Metrics tab and select Explorer from the drop-down.

  2. In the metric search field, type aws. to list all ingested CloudWatch metrics. Metrics follow the naming convention aws.<namespace>.<metric_name> in lowercase, for example:

    CloudWatch metric Kloudfuse metric name

    AWS/EC2 CPUUtilization

    aws.ec2.cpuutilization

    AWS/RDS DatabaseConnections

    aws.rds.databaseconnections

    AWS/Lambda Duration

    aws.lambda.duration

  3. Select a metric and set the time range to the last 15 minutes. A non-empty chart confirms end-to-end delivery.

  4. To verify enrichment labels are being applied (environment tags, instance types, etc.), click on a metric data point and confirm that resource tags such as environment or team appear as labels.

References

  • CloudWatch Enablement Reference — Enablement instructions for AWS services that do not publish CloudWatch metrics by default (AutoScaling, S3, EC2, RDS, EBS, NAT Gateway, CloudFront, API Gateway).

  • Enrichment IAM Permissions — Ready-to-use IAM policy documents for the Kloudfuse enrichment scraper.

  • AWS Supported Services — Complete list of supported CloudWatch namespaces, enrichment labels, and required IAM actions.

  • AWS Kinesis Firehose Integration — Kinesis Firehose delivery stream setup that this integration depends on.

  • Kloudfuse Enrichment — Full Helm configuration for AWS credential options, namespace filtering, region scope, and scrape intervals.