Data Scrubbing
Overview
The Scrubbing tool in Kloudfuse provides a powerful way to permanently delete MELT (Metrics, Events, Logs, and Traces) data based on specific filters. This feature helps you manage storage costs, comply with data retention policies, and remove sensitive or unwanted data from your observability platform.
Scrubbing removes data that has already been ingested and stored. To keep sensitive values from reaching Kloudfuse in the first place, mask them in the collector — see Keeping Sensitive Data Out of Kloudfuse.
| Data deletion through the Scrubbing tool is permanent and cannot be undone. Always review your filters carefully before confirming a scrubbing job. |
|
Scrubbing only works on the observability data that is ingested and sealed in a Pinot segment. Jobs may remain incomplete or have random status if realtime data is selected for scrubbing. To identify which data can be scrubbed, please check the last sealed segments in pinot deepstore location. Only data before the last segment timestamp can be scrubbed. |