Manage Ingestion Rate Control

Starting with Kloudfuse Release 3.3.0, you can customize an ingestion rate control mechanism that enables you to limit the amount of data ingested for each data stream. You can set a different limit for each stream: metrics, events, logs, traces (APM), and RUM.

To manage rate control policies, follow these steps:

  1. Select the Admin tab, and then the Rate Control option in the drop-down menu.

    Navigate to the Rate Control Policy Management interface

  2. The User Rate Control Policy Management interface appears.

Using the Rate Control Policy Management UI

This page contains the following information and functionality:

Rate Control Policy Management page

  1. Search bar to locate a specific stream

  2. Button to Add Rate Control for new stream types; see Add Rate Control for a new stream.

  3. A list of individual stream types: Metrics, Events, Logs, Traces (APM), and RUM.

  4. Stream Type column, within the list of streams; can be Metrics, Events, Logs, Traces (APM), and RUM.

  5. Ingestion Rate column, within the list of streams:

    This represents the maximum number of items that Kloudfuse can ingest for the stream, per second.

  6. Burst column, within the list of streams.

    This represents the maximum number of items that Kloudfuse can ingest for a stream in a single burst.

    To enable bursting while maintaining average rates, we recommend that you modify the burst size to match your business needs:

    burst=rate

    Set the burst equal to the rate to ensure that Kloudfuse ingests up to 1 second’s worth of traffic at the same time.

    burst=rate * N

    Set the burst equal to a multiple of the rate to ensure that Kloudfuse ingests N seconds' worth of traffic at the same time.

  7. Drop Percentage (1h)

    This represents the real drop of percentage over the preceding hour.

  8. Current Configured Percentage

    The configured (maximum) percentage, per class.

    This represents the percentage of quota set for each stream, as a whole. You can split this into defined classes.

  9. Edit stream policy

    To change the specification of the ingestion policy for each stream, hover over the target Stream Type, click edit (Edit), and proceed to change the rate control policy. See detailed information for metrics, events, logs, traces, and RUM.

    If you don’t have an initial policy in place for a specific stream, add that stream by following instructions in Add Rate Control for a new stream.

  10. To delete an existing policy set for any of the defined data streams, hover over the target Stream Type, and click trash clear (Delete). See Delete Rate Control for a stream.

Add Rate Control for a new stream

Initially, you will not have rate control enabled on any of your data streams.

To add rate control for a new stream, follow these steps:

  1. Click Add Rate Control (Add Rate Control) at the top right of the page.

    Add Rate Control

  2. On the resulting page, pick the Stream Type, configure the rate control policy, and click Save Rate Control (Save Rate Control) to create a policy for that stream.

    In this example, we are setting the Metrics Rate Control policy.

    adding new stream

    See the detailed information for setting rate control for metrics, events, logs, traces, and RUM.

Delete Rate Control for a stream

When you want to remove a rate control policy, or make changes so significant that it is best to start from the beginning, delete an existing policy.

Follow these steps:

  1. Delete the policy for the target stream by hovering over that line, and clicking the delete (Delete) icon.

    Delete rate control policy for the Events stream
    Delete rate control policy for the Events stream

  2. A system prompt asks you to confirm your plan to delete the rate control policy.

    confirm deleting events rate control policy
    Confirm your plan to delete the policy for

    When you do, a system success message flashes.

Next steps