What’s New in Kloudfuse

Support for MSK(Kafka), Mongo Atlas dashboards.

Support to add custom tags for log groups in AWS and GCP.

Performance improvements to reorder filters based on their efficiency, while executing queries in the database. This can have a considerable effect on user experience.

Improved and reduced disk reads in termsExist filter execution.

Runtime metrics (JVM, Go, Python, and so on) for services based on the detected telemetry language.

Numerous fixes and UX improvements

Improvements to match the other features and screens.

Kloudfuse now supports the use of Metrics metadata, including metric type, description, and units.

You can search facet names and select them using typeahead (prefix matching) in logs search and logs analytics.

Alert workflow is simplified to make it easy to configure various kind of alerts, including anomaly, change, and threshold.

Improved alert artifacts for System, Kubernetes, and Logs integrations.

Facet match/search now works across all sources.

Improvements to the charting interface, and ability to jump to corresponding metrics exploration with support for more visualization types and comparison to previous time periods.

Support for analyzing and breaking down cardinality of various attributes, both indexed and non-indexed.

Support for threshold-based and anomaly-based alerts for APM RED metrics.

Support for RBAC through custom_values.yaml settings.

You can now sort Logs fingerprints by ascending or descending log count.

You can now search Logs sources and facets in the sidebar.

APM dashboards now show the breakdown of RED metrics by services.

Spinner is now displayed in the APM page for initial load. We fixed word wrap and UI distortion issues on many charts.

Term search is now the default search type for logs. Users can quote the search string to use the older 'string contains' search. Term search is faster and more efficient, in general.

We simplified and streamlined Logs analytics UI. You also have an option to add the queries to a dashboard.

You can now group fingerprints by multiple attributes. Earlier, the only grouping before this release was by the source attribute.

The external dependencies for APM services appear in the service details page.

When you select a specific span from the trace details, you can now see the related logs based on various attributes, including traceId, pod, and many others. You can see the metrics related to the service or endpoint.

We significantly improved the speed for queries for rate (for counter type) and histogram quantile (for histograms).

Spinner now displays in the APM pages while columns values are not yet available.

We redesigned the Infrastructure tab to have a cleaner and more responsive UI. In the related logs, you can now open logs details for any specific log line.

Kloudfuse now automatically terminates queries that run over the timeout or cause 'out of memory' issues. In addition, it restarts pods after hitting 'Out of Memory' issues.

Users can now perform analytics on Elastic APM errors globally, and also see error types, frequency, and last occurrences for a specified service.

Before indexing, Kloudfuse now sorts each input JSON log line internally by key names. This improves the storage efficiency and search speed, reducing the number of unique patterns detected in the log streams.

Slice and dice on APM errors, and also on error chart/table in the service details page.

We corrected some discrepancies in calculating RED metrics from the incoming spans and transactions for Elastic APM data.

We made several performance improvements for auto-completion of metric names and labels, and for metric summarization.

ASM provides autonomic observability based on eBPF technology. The kfuse-knight agent discovers and tracks all services and their interactions. ASM delivers RED and USE metrics without any extra instrumentation or change to the application code. It also curates advanced alerts to detect anomalous and outlier behavior in the services.

Using TraceQL, you can query spans. You can also view a service map and flame graph through Grafana.

You can now optionally skip auto-extraction from JSON logs by changing logs parser configuration.

We corrected the broken charting for grammar-derived facets.

We optimized JSON message parsing to reduce the CPU cost for logs parsing. This applies to all log lines, including structured JSON logs, embedded JSON, and partial JSON strings.

Span details now show stack-trace, local variables, and context for Elastic APM. Also, flame graph spans list appears under a separate span list tab.

From span details, you can now filter by (include/exclude) custom span attributes, in addition to the standard OTel attributes.

We made a number of corrections in dashboards and alerts for the Control Plane.

We standardized and streamlined filtering and navigation across all streams, including logs, traces, events, and metrics.

Kloudfuse now supports Service Level Objectives (SLOs). Users can set latency and availability SLOs for any service instrumented with distributed tracing.

Kloudfuse now supports Single Sign On and several authorization methods, including Google, Okta, Azure, and others.

Kloudfuse alerting now supports Change, Outliers, Anomaly, and Forecast alert types in addition to the existing Threshold alerts.

Kloudfuse catalog service supports the migration of dashboards and alerts from external Grafana to Kloudfuse.

Logs parsing extracts, detects and color codes the data types of facets automatically to make it easier to work with large amount of logs data.

You can configure Logs parser pipeline stages through remap, relabel, and transform actions/stages. This enables users to configure and process logs data from any agent, including fluent-bit, fluent-d, OTEL collector, DD-agent, and many others.

We optimized JSON message parsing to reduce the CPU cost for logs parsing. These optimizations apply both to structured JSON and to log lines that contain embedded or partial JSON strings.

In addition to the OTEL collector/format, Kloudfuse stack now supports Elastic APM and Datadog APM payload formats. You can configure the pipeline to drop and relabel various attributes, as required.

Kloudfuse stack produces unified span-derived metrics that you can configure to have arbitrary dimensions. To produce additional span-derived metrics, apply any filters and time/space aggregates to incoming data. Retain the metric data independently of trace retention.

Users can download the full span data in two different formats:

Kloudfuse alerting now supports Change, Outliers, Anomaly, and Forecast alert types, in addition to the existing Threshold alerts.

The Infrastructure view enables you explore every Kubernetes resource, including related logs, traces, metrics, and events.

Control plane now contains information on forecasted disk and outliers resource usage capabilities for the Kloudfuse observability stack resources.

Log data is sorted by multiple keys (fingerprints, labels, timestamp); this results in more efficient disk storage, and therefore better query performance.

You can save log queries as views, and reference them later. You can also save them as adhoc dashboards for use by team members.

Download Logs events from the Kloudfuse UI. The logs download is limited to 10MB.

We support three different download formats:

We plan to bring this feature back in a later release, with significant enhancements.

On E2 machines, metric segments used to take 4-10 minutes due to number of docs in each segment (~50M). We moved to columnar seal instead of row-by-row seal, and seal times decreased by 50% or more.

Kloudfuse can now ingest logs from Fluent-D directly. We support JSON and msg-pack formats.

We now sort the log lines on the disk based on their fingerprint. This results in better storage compression and improved search performance for both grep and facets.

We now support flame graph view for incomplete traces. In certain customer environments, we may not get a root-span or parts of traces may always be missing due to environment setup. We improved the flame graph visualization to render such traces correctly.

To improve query speed, we moved to dictionary-based encoding for span durations and bigger segments.

With the right instrumentation in place, Kloudfuse analytics can now do auto alerting and analysis. Using Hawkeye, you can easily enable auto-alerting for automatic monitoring of all Kubernetes services for anomalies on their RED metrics. Auto-analysis capability using Bullseye generates an analysis report with possible reasons for the alert (anomaly).