Heroku Log Ingestion

Kloudfuse integrates with Heroku to collect and ingest logs from Heroku applications using HTTPS log drains. Heroku’s log draining feature allows you to stream application and system logs to Kloudfuse for centralized monitoring and analysis.

For more information on Heroku logging, see Heroku Log Drains.

Prerequisites

  • Heroku application deployed and running

  • Heroku CLI installed and authenticated

  • Kloudfuse instance accessible via HTTPS

  • Kloudfuse ingress endpoint URL

Get Kloudfuse Endpoint

  1. Obtain the Kloudfuse ingress endpoint URL.

    Use the external ingress IP or DNS name.

  2. Note the endpoint URL format for Heroku log drains:

    https://<kfuse-ingress-external-ip>/ingester/heroku/logs
    text

Configure Heroku Log Drain

Using Heroku CLI

  1. Log in to Heroku CLI:

    heroku login

  2. Add the Kloudfuse log drain to your Heroku application:

    heroku drains:add "https://<kfuse-ingress-external-ip>/ingester/heroku/logs" --app <app-name>

    Replace:

    • <kfuse-ingress-external-ip> with your Kloudfuse endpoint

    • <app-name> with your Heroku application name

  3. Verify the log drain was created successfully:

    heroku drains --app <app-name>

    You should see output similar to:

    https://<kfuse-ingress-external-ip>/ingester/heroku/logs (d.12345678-90ab-cdef-1234-567890abcdef)
    text

    The drain ID specified in () will be added as a label in kloudfuse for easier filtering and organization.

Authentication

For information on configuring ingestion authentication, see Ingestion Authentication with API Key.

If authentication is enabled, append the encoded API key or token to the drain URL with the label kf-api-key:

heroku drains:add "https://<kfuse-ingress-external-ip>/ingester/heroku/logs?kf-api-key=YOUR_API_KEY" --app <app-name>

Log Format and Metadata

Heroku sends logs in the Logplex format. Kloudfuse automatically extracts metadata from Heroku logs, including:

Field Description

source

source for all logs will be labeled as Heroku

drain_token

Drain token (e.g., d.12345678-90ab-cdef-1234-567890abcdef)

timestamp

Log timestamp (e.g., 2022-01-01T00:00:00.000Z)

log_level

Log level (e.g., info, error)

log_type

Log type (e.g., heroku, app)

Configuration

Add Custom Labels

To add custom labels for filtering and organization, you can add them as parameters to the drain URL:

heroku drains:add "https://<kfuse-ingress-external-ip>/ingester/heroku/logs?label1=value1&label2=value2" --app <app-name>

You can use both custom labels and Authentication API keys (see Authentication) in the parameters. If the API key parameter is present, it will not be added as a label.

Troubleshooting

Logs Not Appearing

  1. Verify the log drain is active:

    heroku drains --app <app-name>

  2. Auth Config

    If Auth Config is enabled, confirm that ingest secret is base 64 encoded, and that the param label is kf-api-key

  3. Review Heroku logs for drain errors:

    heroku logs --tail --app <app-name>

Authentication Failures

  1. Verify your API key is valid and has ingestion permissions.

  2. Ensure the API key is properly URL-encoded in the drain URL.

  3. Check Kloudfuse authentication configuration.

Remove Log Drain

To remove the Kloudfuse log drain from your Heroku application use the drain URL from the add step:

heroku drains:remove "https://<kfuse-ingress-external-ip>/ingester/heroku/logs" --app <app-name>

You can also view all drain URLs for a Heroku application via the console:

heroku drains --app <app-name>