What’s New in Kloudfuse

3.2.3 (Latest)

March 14, 2025

This is a major release that adds a significant number of improvements to Alerts, RUM, FuseQL, Metrics, APM, Migration, and Backup and Restore.

Additionally, you can review the bugs we fixed in this release.

Alerts

Bulk Actions

We added several bulk actions to the Alert Rules page.

Bulk Action

You can now select all or some of the alerts, and perform these actions in bulk:

Delete

Delete the alert rules.

Pause

Stop the evaluation of the alert conditions; does not send notifications.

Resume

Resume the evaluation of the alert conditions.

Suppress

Stop sending notifications; continues to evaluate the alert conditions.

Unsuppress

Resume sending notifications.

Clear selection

Deselects all boxes.

Alert Suppress

The action formerly known as Mute Alert is now Suppress Alert.

We make a distinction between pausing alerts, and suppressing alert notifications:

Pause

Paused alerts do not get evaluated, so they never issue an alert notification. When a user deliberately resumes the alert, Kloudfuse starts evaluating the rule that can trigger an evaluation.

You can pause alerts as part of a bulk action, or from the Create Alert / Edit Alert interface.

Pause Alert in the Crete/Edit Alert Interface
Suppress

Suppressed alerts still get evaluated, yet do not fire notifications if alert conditions exist.

You can suppress alerts as part of a bulk action, or by hovering over the Alert rule in the list of alerts, clicking the suppress (suppress) icon, and then selecting the defined time interval, from Next 5 minutes to Next 7 days.

Suppress Alert
Suppress Alert Schedule

You can create complex, multi-component schedules for suppressing alerts.

Suppress Alert Schedule

Subsequently, edit a contact point, and attach the correct Suppress Schedule.

Modify suppress schedule
No Data Handling

This release adds the Evaluate as zero” option to account for missing data from alert query conditions.

Handle no data as zero in alert queries

Real User Monitoring (RUM)

Add and Manage Applications

Implements CRUD operations for applications, by name, type, ID, client token, and so on, through the UI instead of the config file custom-values.yaml.

For full information, see RUM Add New Application.

  1. Navigate to the new RUM > Add and Manage Applications section.

    Navigate to add and manage applications interface
  2. Review the list of Your Applications.

  3. Click Add New Applications, and specify the new application in the Add RUM Application interface: Name of the application, Type (Web, Flutter, or React Native), and whether to Collect Client IP.

    Specify a new application to monitor in RUM
  4. Click Add Application.

  5. Delete: Hover your cursor over the relevant line of the list, and click delete (Delete).

  6. Edit/Verify: Click the line with application information. You can only change the IP collection option.

  7. Integrate: Note and copy the Integration Code that Kloudfuse generates, and use it to integrate the application that you plan to monitor with RUM.

    rum add manage applications verify 3.2.3
Overview URL panel

When you click a panel or chart, it opens a new URL on the right side of the screen.

For example, click the Longest INP by URL, and it opens the full report screen that clearly identifies which URL causes the delay.

expanded Longes INP by URL
Overview URL panel

FuseQL

We expanded the offerings within our proprietary query language, FuseQL.

Scheduled Views

Starting with this release, you can create scheduled views, and query directly from that view.

Scheduled views are pre-aggregated datasets that Kloudfuse generates at scheduled intervals to improve query performance and efficiency. Instead of expensive real-time queries on raw data, scheduled views store advance results, enabling faster access to summarized information.

This is how to use a scheduled view:

  1. Define a FuseQL query – specify the filters and aggregation logic.

  2. Create the scheduled view – it updates every minute for near real-time data availability.

    Define Scheduled View
  3. Store precomputed results – the system processes the query and saves the aggregated results, separately from the log data.

  4. Query the view – users can access the scheduled view results through FuseQL, instead of running raw data queries. This approach ensures faster performance with current insights.

  5. Disable or pause the scheduled view – users can disable, pause, or temporarily stop updates on a scheduled view without deleting it, and then resume processing as necessary.

    Pause Scheduled View
New Operators

We added backshift and dedup operators to support querying of Scheduled Views.

Dashboards

Create new dashboards based on results of Advanced Search across logs.

Alerts

Create new alerts based on results of Advanced Search across logs.

Lookup tables

Create and use lookup tables to supplement your data.

  1. Under the Logs tab, navigate to the new section Lookup Tables.

    logs lookup tables go
  2. Review the list of Lookup Tables.

  3. Click Create Lookup Table, upload the source CSV file and name the new table.

    The preview of the data shows the fields. You can change the data type for each column. Be sure to identify at least one primary key, and click Create Lookup Table.

    Add data and specify primary keys

Metrics

Analytical Views

This release significantly expands the analytical features for Metrics by adding Top List, Table, and Pie Chart analytical views to Metrics Explorer.

  • Top List

  • Table

  • Pie Chart

This view provides a quick summary of top N metrics that match the filter and time interval criteria. It helps with high-level analysis of metrics in your system.

Top List analytical view in Metrics Explorer

This view is a quick summary of top N or bottom N metrics that match the filter and time interval criteria. It helps with high-level analysis of metrics in your system. You can sort by any attribute/column.

Table analytical view in Metrics Explorer

This view is a quick summary of metrics that match the filter and time interval criteria. It is a visual representation of proportional weight of a metric across the data from the time series. The pie chart view also includes the table representation of the items on the chart; you can sort by named columns.

Table analytical view in Metrics Explorer
Default Zero

We addedan interpolation function, default_zero, to handle missing data in Metrics Time Series.

Default Zero function

APM

Deployment Version Marker

We added the existing deployment version markers to the graphical APM Trace List information.

APM Trace List][[service-list]

Note that we already use them on the Service Detail page.

Service detail with deployment marker][[service-detail]

Migration

We deprecated the Kloudfuse Catalog Service in a previous release. Start Using Kloudfuse Customer Scripts instead.

Dashboards

To migrate existing dashboards into the Kloudfuse system, use the consolidated script approach. See Using Kloudfuse Customer Scripts > Dashboards.

Alerts

To migrate existing alerts into the Kloudfuse system, use the consolidate script approach. See Using Kloudfuse Customer Scripts > Alerts.

Platform

Multi-Availability Zone

We introduced an AZ-service that enables you to switch Kloudfuse to a different availability zone.

3.2.2

February 13, 2025

This release adds improvements for APM, RUM, Logs, and Dashboards.

Additionally, you can review the bugs we fixed in this release.

APM

Side Bar Filter

For the Service, Traces, Analytics, and Error pages, we restricted the number of displayed results to improve performance. Kloudfuse, by default, sorts labels and attributes in descending order, and displays up to 1,000 items.

Side Bar Search

Search now leverages regex for the contains operation.

Multi-Query Support

We improved the rendering of the multi-query query results for Trace Analytics.

APM Infrastructure Dashboards

We now support Kubernetes and Host metrics exported by the Otel collector when rendering the APM Infrastructure Dashboard. See System-Level Metrics.

We also improved the filtering selection refresh on the Side Bar; see Side Bar Filter.

Inactive Services

We removed the Show Inactive Services option. Instead, the Services interface has two viewing options: the default Active Services, and All Services.

Active Services vs. All Services

RUM

We enhanced the RUM Performance Overview UI.

RUM Performance Overview
Overall Performance Metric

We added a Tree Visualization to track the values of the overall performance metric. Loading Time is the default metric; you can also choose Largest Contentful Paint, Cumulative Layout Shift, and Interaction to Next Paint. This chart reports the health of the metric across the system, color-coding for Good (green), Needs Improvement (amber), and Poor (red) statistical measurements, based on customer specification.

Optimize Vitals

We added three more visualizations: Loading Time, First Contentful Paint, and User-Centric Page Load Times latency reports. The existing latency reports include Large Contentful Paint (LCP), Cumulative Layout Shift (CLS), and Interaction to Next Paint (INP).

Worst Performance Report

We highlighted the URL addresses of the pages with the poorest performance in terms of Longest INP, Largest CLS, and Slowest LCP.

Logs

Side Bar Filter

We restricted the number of displayed results based on Search, to improve performance. Kloudfuse, by default, sorts in descending order, and displays up to 1,000 items on the Log Side Bar Filter. This number is configurable.

Dashboards

Time Range

We fixed an issue where the overall time range now correctly applies to table data in dashboard panels.

3.2.1

January 31, 2025

This release adds improvements for RUM performance.

3.2.0

January 24, 2025

This release focuses on further refinements to our existing features.

Alerts

Threshold Alerts

We added support for warning threshold, alert recovery, and warning recovery thresholds to threshold alerts on metrics, logs, and services. You can also link alerts to dashboards and panels, pause alert evaluation, and link back to the Kloudfuse alert page from alert notifications.

Metric Threshold Alert
Create a Metric Threshold Alert

Log Alerts

We added support for linking in log alerts, and display matching log lines in alert notifications.

Stream Isolation

You can now Configure Stream Isolation on Pinot Tenants and route specific telemetry streams to tagged nodes.

RBAC

APM

Permissions now persist for downstream and upstream dependencies that cross cloud, Kubernetes cluster, and namespace boundaries. When you set policy filters using service id labels, Kloudfuse supports cross-boundary permissions, and applies RBAC policy filters to both active and inactive services.

Stream-Specific RBAC

We now support RBAC policies at the level of the data stream. While default RBAC policies apply to all non-RUM streams, you can pin custom RBAC policy filters to a specific stream, such as logs, metrics, or RUM.

Custom policy for logs stream
Create a Stream-Specific Custom RBAC Policy

Private Folders

You can now create private folders, where only the creator has access to the folder contents.

Create private folder
Create Private Folders

User Interface Improvements

We made changes to warn users with viewer roles about their permission limitations when they attempt to add, edit, and delete dashboards and alerts.

Dashboards

You can now create new folders in the Dashboards main/default page; simply click add new folder at the top right.

Add New Folder
Add New Folder

FuseQL

We made several improvements and a small number of fixes to the FuseQL functionality. Critically, we significantly enhanced the Advanced Search parsing by adding support for regular expression parse variable pattern recognition with and without start and stop anchors, and JSON pattern parsing. See FuseQL Parse Operators.

RUM

In this release, we added a session IP column, made UI improvements, and fixed a few bugs.

Help Menu

We made several improvements to the Help menu in this release:

  • Added Kloudfuse version information.

  • Added a link to our Technical Product Documentation.

  • Added a link to Customer Support; this is configurable, depending on your contract options.

Help menu
Help Menu

Integration

We added an option to import dashboards and alerts directly from the integration page.

We also made overall improvements to the Integration UI.

integration dashboards

3.1.3

January 13, 2025

In this release, we fixed a message conflict in the LogEvent table.

3.1.2

January 8, 2025

In our first release of the year, we chose to implement fixes to various UI pages, and improved our Alerts functionality.

Integrations

We updated content on the Integrations tab, and included icons for easier identification of third-party agents, services, and so on.

Events

Event Alert Fixes

We improved the functionality of Event alerts.

Real User Monitoring (RUM)

RUM Session IP

We improved the handling of IPs recorded in RUM sessions.

RUM Geo Location and Mapping

We added a geographical location extension for RUM, and the ability to handle the user-provided IP:location mapping.

Mobile RUM SDK

We made improvements to the SDK for Mobile RUM.

RUM Alert Fixes

We improved the functionality of RUM alerts.

FuseQL

We added parse and json extension support for our proprietary FuseQL log processing.

Dashboards

We fixed and improved several dashboards.

3.1.1

December 31, 2024

In our last release of the year, we completed some bug fixes and refreshed the UI.

Let’s get ready for 2025!

3.1

December 24, 2024

This release includes further refinements for our existing features.

Integrations

Integration UI

To better assist you in configuring your data streams into the Kloudfuse platform, we added an Integration section to our UI.

You can browse the site to select the appropriate agents, cloud services, storage services, platforms, and so on, to research the best integration solution for your business needs. You can also use the Search function.

Integration Tab
New Relic

The Kloudfuse platform now ingests span and trace data sent by New Relic agents and SDK.

Pushgateway

The Kloudfuse platform now ingests ephemeral and batch metrics data sent by Pushgateway.

Query Languages

FuseQL, our proprietary Query Language for searching logs, now offers Advanced Search (free-form search), and fully supports the Logs Analytics screens.

Advanced Search

Operates like a pipeline, progressively narrowing down results to help you find exactly what you need. Each operator, separated by a pipe (|), builds on the results of the previous one. This enables you to filter and focus your search with precision as you move through the pipeline.

Advanced Search
Search Bar and Query Builder

FusesQL is part of both the Search Bar and the Query Builder for Logs Analytics: Time Series, Top List, Table, and Pie Chart pages of the Logs interface. Starting with this release, FuseQL is the default query language for logs.

FuseQL Search and Query Builder for Logs
Conversion Functions

In addition to toInt, we added two more type conversion functions: toLong and toString.

SLO

We added the metric detection SLO in this release, to supplement our existing support for SLOs based on latency and availability.

Metric detection method for SLOs

Enhanced Analytics

For logs, metrics, events, and APM, you can now create and edit dashboard panels and alerts directly from the Analytics interfaces.

Dashboard Panels

Click the Export icon, and proceed to name the new panel and save it in a new or existing dashboard.

Create Dashboad Panel
Alerts

Click the Alert icon, and create alerts based on the current analytical setting. This feature is part of all Analytical interfaces (Time Series, Top List, Table, and so on) for APM, Metrics, Logs, and Events.

Creating a Metric Alert from Analytics pages

Real User Monitoring (RUM)

In this release, we added these two features to our RUM offering:

Source maps

Specify source maps to better decode stack traces.

Mobile RUM

We added UI changes to make Mobile RUM reporting better.

3.0

November 6, 2024

This release includes two significant innovations in Kloudfuse platform, Real User Monitoring and FuseQL query language. It also adds significant troubleshooting and analytics tools to the APM system by introducing Trace Heatmap and K-Lens.

Real User Monitoring (RUM)

RUM allows you to capture and analyze data from real users as they interact with your web application, providing insights into their experience and identifying performance issues from the user’s perspective.

We encourage you to look at our introductory video on how RUM addresses the many requirements of Digital Experience Monitoring:

Read about our implementation in RUM Setup and Real User Monitoring (RUM).

FuseQL

Kloudfuse developed its proprietary query language, FuseQL, for a range of applications. It has flexible parameters for answering highly complex questions.

To understand the defining characteristics of FuseQL and how to use it, see FuseQL.

APM

This release introduces two powerful new APM observability tools, Trace Heatmap and K-Lens, to assist in service-level issue detection and advanced debugging using APM Trace data.

Trace Heatmap is an interactive trace data visualization chart that helps you to visually detect deviations and outliers in the latencies reported by APM data.

K-Lens is a proprietary analysis tool that helps you to narrow down the cause of appliance performance issues; it compares thousands of attributes across span events to their baseline performance, and displays ranked results based on significance of deviation.

We encourage you to look at our introductory video about this feature:

Enterprise Readiness and Security

We enhanced our SSO support by adding SAML-based authentication for providers such as Google, Okta, and many others. See Configure SSO Authentication with SAML.

2.7.4

September 2024

Metrics Roll Up

Kloudfuse supports roll up and aggregation of metrics data, computed directly from the raw data stream during ingestion. For longer time intervals, our approach significantly improves query performance, and reduces chart loading times and I/O costs. Depending on the time span or step size of the query, Kloudfuse calculates results either from raw data, or from rolled up data. In the shorter time spans, we continue to use raw metrics because the calculation approach could potentially smooth out the data and miss important signals, such as outliers.

Metrics roll up is off by default, and the default aggregation is for 5 minutes. Contact us to turn on metrics roll up, and help you configure your environment.

For more information about this feature, see Metrics Roll Up.

Service Catalog

Until this release, the Kloudfuse APM interface showed only the services that sent data within the selected time block. We now show all services, including the ones that may be stopped or paused.

To see all services, simply toggle the Show Inactive Services selector. You can see that two new services in this example, frontend-web and loadgenerator, appear in the service list. Because the services were inactive during the relevant time interval, they have empty columns.

using the service catalog

Facet Exploration and Analytics

Logs default sidebar Open facets-in-left-bar.png facets-in-left-bar.png Kloudfuse automatically extracts facets attributes from logs during ingestion. Before this release, we surfaced facets in the Logs side bar, organized under Sources.

Because the number of sources and facets can quickly become very large, the troubleshooting tasks become more difficult and the feature is not as useful as we want it to be. In this release, we re-organized the sidebar to include expandable sections as these components: Filters, Labels, and Facets.

We now organize Facets in their own section. Users can create custom groups that include the facets they want to monitor, across any number of sources.

facets in the left bar

You can group facets by source, and use the search bar to find relevant groups.

When you expand a group (here, the logs-parser), you can see some of the facets tracked in the group. We also added a search bar in each group to help you locate a facet by name.

Facet Explorer

Click the icon on the Facets title line, and Kloudfuse opens the new Facet Explorer. When you choose a source (logs-parser to continue this demonstration), you can typically see many more named facets than you see in the Logs sidebar.

Facet explorer

Facet Favorites

This release also introduces Facet favorites that optimize facet design and usability; you can hover over one of the facets, and click on the hollow icon to “favorite” that facet. In the Edit Favorite interface, simply decide if you want to add the facet as a favorite to the existing group, or create a new group. You can even change the display name of the favorite facet.

Because of these changes, and other usability improvements, you can now control the organization and visibility of facets much more effectively:

  1. From the log event detail pane, you can select a facet and add it do the side bar.

  2. You can organize facets by user-defined groups, so the sidebar has a single-level folder structure.

  3. The Logs page now has a Facet Explorer option that enables you to browse across all facets, add facets, and remove them.

Facet favorite

Changes to existing Kloudfuse UI

  • In the Logs interface left sidebar, Facets replace Sources.

  • When creating/changing Favorites, you can “remember” the source by selecting that option.

  • Upgrade note: The log facets that appeared in the sidebar before this release are now hidden. After upgrading to Kloudfuse Release 2.7.4, Use the Facet Explorer to identify the facets you want to see in the sidebar as “favorites”.

Log Archive and Hydration

You may have to save transactional information for compliance, legal, or other regulatory requirements. In addition to processing logs for observability and analytics, this release of Kloudfuse introduces a supplementary mechanism for archiving pre-processed logs (with identified filters, facets, and so on) into longer-term storage, and a separate mechanism to hydrate these logs to examine them for the relevant data.

The benefits of this approach extend beyond basic regulatory compliance:

You store important historical data in a cost-effective compressed format in a location that you own and control.

When uncompressed, the logs are human-readable and highly searchable because of the high level of indexing through labels and other data attributes.

You can configure the archival instructions in a manner that categorizes data consumption by internal cost center.

We currently support log archive and hydration for AWS S3.

To start hydrating previously archived logs, select Logs in the top navigation bar, and then choose Hydration.

start logs hydration

Contact us at support@kloudfuse.com to enable this feature in your Kloudfuse cluster.

For in-depth information on this feature, see Logs Archive and Hydration.

Log label cardinality analysis

Log labels have two main source: many are attached by the agent that delivers the logs to the Kloudfuse platform, and potentially even more are defined by users. Some labels are meaningful at the time they are defined, yet loose their relevance over time. Some are created accidentally, or as a result of not fully understanding the common use cases or the purpose of log tracking. Automatically-generated labels, from agents, are often cryptic and unnecessary.

Seeing cardinality analysis of a log helps users to remove unnecessary labels, improving data accessibility and making the collected information more actionable.

To determine log label cardinality, select the Logs tab, and then the Cardinality Analytics option in the dropdown menu.

start log cardinality analysis

The Logs Cardinality report shows the overall cardinality; in this case, the cardinality is 335. It further breaks out the data by Label, showing the Value Count (1h) for the preceding hour, the Value Count for the specified most recent time range (the last 5 minutes is the maximum time range and the default setting), and the Value Chart, which is a simple bar chart representation of the count of unique values over the selected time range.

logs cardinality report

To find the prevalence of a specific label value, use the filter at the top of the page to select a label, the operator, and the comparison value.

Graviton support

Starting with this release, Kloudfuse platform can run on instances that are based on AWS Graviton processors.
graviton chip

For more information, see AWS Graviton Processors documentation.

RBAC improvements

User Management

This page now displays all groups to which the user belongs, and their role: Admin or Member. You can:

  • Change the name of the user.

  • Change the user’s role (Admin ↔ Editor ↔ Viewer).

  • Re-assign group membership level ( Admin ↔ Member).

  • Remove the user from any assigned group.

change user detail

User Status

user-status.png At the top right, where we show logged-in user’s information, Kloudfuse UI now displays the user’s role.

user status

Error display improvements

Starting with this release, we are replacing error message toasters with more specific error messages that display directly on the components that throw the error.

contextual errors

2.7.3.P2

September 17, 2024

We improved the performance on the Trace Errors interface.

2.7.3.P1

September 6, 2024

OpenTelemetry on Docker deployments

In this release, we added support for data collection through OpenTelemetry on Docker containers. For details, see OpenTelemetry Collector on a Docker Environment.

Open telemetry on docker

Trace (APM)

Prettify JSON

For the Trace > Log details interface, we improved the display of the JSON format of the log line.

Select the Pretfify JSON option to see the more familiar and easy to read rendering of the log line.

Prettified JSON

Trace Span Details

When examining Trace Latency Breakdown detail, under the Logs option, we added a new filter option. Depending on the nature of your technical stack, you can now select to filter on:

  • Kubernetes: pod_name

  • Docker: container_id

  • Host: hostname

To select the relevant filter:

filters for trace latency
  1. On the top navigation bar, select APM > Traces.

  2. [Optional]

    Use Search to find the relevant service.

  3. In the List of services, click the relevant Trace Latency Breakdown diagram.

  4. In the detail, under the Flame Chart, select the Logs tab.

  5. The header of the list starts with the Filter, set to traceId by default. Click, and select the relevant filter from the drop-down.

2.7.3

August 28, 2024

In this release, we made changes to Logs, APM, and RBAC:

Logs

We made significant improvements to the performance of log search with efficient indexes; comparable searches are now an order of magnitude faster.

APM

We implemented the necessary bug fixes.

RBAC

We added default access policies for users without explicit policy assignment. See Default Policy.

2.7.2

August 19, 2024

RBAC(Role Based Access Control)

We are introducing a new UI support for easier user management:

  • Define Roles assigned to users in the UI.

  • Handle Group Management, Policy Management, and Policy Config Management with ease from UI.

  • Enjoy improved user and access controls.

Starting with this release, KloudFuse does not support RBAC and policy configuration by editing raw configuration files.

APM

We made some targeted performance improvements.

Multi-key auth ingestion

We are introducing support for multiple authorization keys.

2.7.1

August 12, 2024

Kloudfuse 2.7.1 release builds on 2.7.0 version and improves analytics experience for traces and logs. We are also introducing new Forecast and Anomaly models for metrics.

This release includes additional features, and bug fixes.

APM

Trace Analytics
  • Support for pushdown queries for better performance & user experience.

  • Improved Analytics formula for correctness with optimized joins for top k and bottom k results.

  • Support for easy query param selection for multiple queries.

Logs

Log Analytics
  • Support for pushdown queries for better performance & user experience.

  • Improved Analytics formula for correctness with optimized joins for top k and bottom k results.

  • Support for easy query param selection for multiple queries.

Advanced Alerts

Forecast and Anomaly Models
  • Introducing new model(Prophet) for better anomaly detection.

  • Introducing new forecast model(Holt winter-seasonal) for better forecasting.

Dashboards

Kafka

Support for MSK(Kafka), Mongo Atlas dashboards.

Labels

Custom Labels

Support to add custom tags for log groups in AWS and GCP.

2.7.0

July 29, 2024

Kloudfuse 2.7 is a major release with many enhancements and critical bug fixes.

This release has a kafka upgrade that requires specific steps.

APM

Traces
  • Support for non-request (background jobs) transactions for elastic APM.

Service Details
  • Support for Apdex charting and alerting.

  • Service Execution Time chart shows breakdown by the downstream external service by their DNS names/IP addresses.

  • Downstream dependency table now also shows external services.

UI

Bookmarking and State Management
  • Support for bookmarking filters and query states

  • State is preserved within the stream pages while navigating around the kfuse UI.

  • Unified rollup period across all charts

  • UTC support

  • Allow sorting for analytics table and pie chart table

Logs

Improvements
  • Unify Logs Analytics and Trace Analytics to allow multiple queries

  • More responsive logs landing page through streaming and faster terms search

  • Support for larger log lines - up to 1MB

Control Plane

Improvements
  • Improve dashboards for Kloudfuse Overview, Systems, and so on.

  • Additional panel to show Agents Overview

  • Support for Outlier analysis

Alerts

Improvements
  • Support for creation of alerts by cloning an existing alert.

  • Support bulk deletion of alerts and contact points.

  • Slack contact point editing support

  • Support Apdex-based alerts

Infrastructure

Improvements
  • Support for pagination in the Kubernetes views

  • Improve sidebar navigation and grouping support in the Events page

  • Group by support for Kubernetes

  • Multi-Query in Event Analytics

Dashboards

Improvements
  • Support for exporting dashboard and copying dashboard json

  • Support for bulk deletion of dashboards

  • Support for multi-series anomaly visualization

  • Increased precision for charting tooltips

  • Support for math expressions in dashboard panels

2.6.7.P1

May 30, 2024

Kloudfuse 2.6.7 is patch release with some critical bug fixes.

Bug Fixes
  • Fix for RBAC exception when navigating to live trace and APM Derived Metrics. Applies only if RBAC is enabled.

  • Fix for Alignment of values between “Total Requests” and sidebar span counts in the Traces screen.

  • Fix for No Data/Exec State alert condition

2.6.7

May 15, 2024

Kloudfuse 2.6.7 release builds on the 2.6.5 version and continues to improve APM, Logs, and UI. We added support for defining Service Level Objectives (SLOs) at service level, and enhanced Logs Search with additional search operators.

This release also includes many performance improvements, bug fixes, and several minor enhancements.

APM

Databases
  • Databases are now uniquely identified by a user-configurable set of attributes with a reasonable default that includes key cloud and kubernetes attributes. This is similar to service identifiers for Web services.

Service Level Objectives (SLOs)
  • Kloudfuse added support for Service Level Objectives (SLOs). Users can set latency and availability SLOs for any service instrumented with distributed tracing.

  • Every SLO breach can send optional alerts.

  • The Service page includes a high-level summary of any configured SLOs.

Service Details
  • Supports runtime metrics (node.js) for services based on detected telemetry language.

  • Service details page provides quick access to the logs through the Logs tab.

Traces and Flame Graph
  • Trace details and flame graph now support granularity measured in nanoseconds.

Logs

Facet terms exists
  • Logs search has two new operators to support term search within a facet, facetTermsExist(==) and notFacetTermsExist(!==).

    This is similar to facetTermsExist for the full log line:

Filter Performance

Performance improvements to reorder filters based on their efficiency, while executing queries in the database. This can have a considerable effect on user experience.

Disk Reads for Filters

Improved and reduced disk reads in termsExist filter execution.

2.6.5

April 10, 2024

Kloudfuse 2.6.5 is a major release that significantly enhances the APM user experience by introducing the concept of service identifiers, which allow APM services to be uniquely identified by a user-configurable set of attributes with a reasonable default, and includes key cloud and kubernetes attributes.

This release also includes various security fixes to address CVEs, many performance improvements, bug fixes, and other minor enhancements.

APM

Services and Databases List
  • We now display services and databases separately in their own tabs.

  • Services are now uniquely identified using a combination of cloud and kubernetes labels. Service identity is carried over to the service, dependency maps, and service details page. Additionally, Kloudfuse leverages service identity to set APM and ASM alerts and to navigate to traces and errors from service details.

Service Details

Runtime metrics (JVM, Go, Python, and so on) for services based on the detected telemetry language.

Service Map
  • Significant enhancements to the Service Map (global view) and Service Dependency Map (in the Services details page) user experience, leveraging service identity to uniquely identify services and databases

  • Ability to navigate from service to service details page

  • Ability to size the node based on any of the RED metrics

Infrastructure

Kubernetes Infrastructure

Numerous fixes and UX improvements

Look and Feel

Improvements to match the other features and screens.

2.6.0

February 16, 2024

Kloudfuse 2.6.0 is a major release that includes significant UI improvements, and many new APM and Dashboard features.

UI Enhancements

Sidebar
  • Independent scrolling of sources and facets

  • Easier selection of facets, and labels using toggle All/only options

  • Easier charting of facets based on data type, directly from the Logs sidebar

Search bar
  • Uniform and easier editing of search filters across Logs and APM search bars

General UI improvements
  • Better color, fonts, and sizes

  • Uniform look and feel across various screens

Dashboards

Dashboard Edit
  • You can create, delete, and edit APM, Logs, and Metrics dashboards

  • Kloudfuse has Dashboard import, export, and copy functionality.

  • Dashboard templates support variables

Metrics

Metrics Metadata

Kloudfuse now supports the use of Metrics metadata, including metric type, description, and units.

APM

Advanced Services Monitoring (ASM)
  • You can enable ASM for individual services from the APM services list.

  • With ASM enabled, you can show anomalies in RED metrics charts.

  • Service details page shows Kubernetes Infrastructure metrics on per host, and per pod basis, with outlier detection.

Deployment Tracking
  • Auto-detection of deployment changes based on service version

  • Service details page shows first seen time for each of the versions.

  • Service RED metrics and Execution Breakdown charts show markers for deployment.

Service Execution Time Breakdown
  • Breakdown of execution time by downstream service and span type

  • Overlay of deployment markers to correlate deployment changes with service changes

Additional Service Reports
  • We added SLA daily, weekly, and monthly report of RED metrics and Apdex.

  • We added a Performance Report that breaks down RED metrics and Apdex by span names for 24 hour period against a 7 day average.

2.5.4

January 6, 2024

Kloudfuse 2.5.4 is a minor release with a few enhancements, performance improvements, and bug fixes.

2.5.3

December 26, 2023

Kloudfuse 2.5.3 is a minor release. It includes Logs facet autocomplete with typeahead, APM trace search enhancements, performance improvements, and bug fixes.

Logs

Typeahead for Logs Facets in Logs Search and Analytics

You can search facet names and select them using typeahead (prefix matching) in logs search and logs analytics.

APM

APM Search Improvements
  • Improved APM search now allows editing of individual =(eq), !=(neq), =~(regex eq), and !~(regex neq) queries for both low and high cardinality attributes.

  • Simplified and improved editing of duration slider ranges

2.5.2

December 13, 2023

Kloudfuse 2.5.2 is a minor release with performance improvements and bug fixes.

2.5.1

December 10, 2023

Kloudfuse 2.5.1 is a minor release with improved alerting capabilities and performance and bug fixes.

Alerts

Simplified Alert Workflow

Alert workflow is simplified to make it easy to configure various kind of alerts, including anomaly, change, and threshold.

Infrastructure Alerts

Improved alert artifacts for System, Kubernetes, and Logs integrations.

2.5.0

November 21, 2023

Kloudfuse 2.5.0 is a major release with many improvements, new features, and bug fixes.

Logs

Additional Visualizations in Logs Analytics UI
  • Logs Analytics now supports Top List, Table, and Pie Chart visualization. This adds to the Time Series visualization of previous releases.

  • You can aggregate facets, or used in a group that is independent of the source in which they appear.

  • Create alerts directly from the logs analytics screen for both queries and formulas.

  • Kloudfuse has new aggregation functions: first, last, quantile, and so on.

Log Facets

Facet match/search now works across all sources.

Logs Search
  • Logs term search and string search are now faster due to numerous improvements in indexing, caching, and streaming evaluation of counts.

  • From the search bar, search for log lines that contain facets.

  • Sort the logs search results table by custom columns.

  • Chart numeric facets from the sidebar.

  • The sidebar shows facet values, sorted by their count of log lines.

APM

Service Detail UI Improvements

Improvements to the charting interface, and ability to jump to corresponding metrics exploration with support for more visualization types and comparison to previous time periods.

APM Analytics UI Improvements
  • Added support for Top List, Table, and Pie Chart visualization, in addition to the existing Time Series visualization.

  • Support for multiple queries and formulas.

  • Simplification of Analytics UI to match logs analytics.

  • Ability to add analytics queries to dashboards

Cardinality Analytics

Support for analyzing and breaking down cardinality of various attributes, both indexed and non-indexed.

Metrics

Cardinality Analytics
  • Kloudfuse supports analyzing and breaking down cardinality of labels.

  • Several performance improvements.

Alerts

APM alerts

Support for threshold-based and anomaly-based alerts for APM RED metrics.

Role Based Access Control (RBAC)

Configurable RBAC

Support for RBAC through custom_values.yaml settings.

2.2.4

This release adds many performance improvements and features for Logs and APM.

Logs

Fingerprint tab improvement

You can now sort Logs fingerprints by ascending or descending log count.

Log UI Improvements

You can now search Logs sources and facets in the sidebar.

APM

APM dashboards

APM dashboards now show the breakdown of RED metrics by services.

Various bug fixes and Performance Improvements
  • Services view sidebar loads faster.

  • Service Detail view breakdown charts by Span Name supports selection using the legend.

  • Links from a specific error group details to the “Errors” page.

UI improvements

Spinner is now displayed in the APM page for initial load. We fixed word wrap and UI distortion issues on many charts.

2.2.3

This release introduces Term search for Logs, and External Dependency Tracking for APM services. We also made several improvements for Logs, APM, Infrastructure, and Platform.

Logs

Term Search for Logs

Term search is now the default search type for logs. Users can quote the search string to use the older 'string contains' search. Term search is faster and more efficient, in general.

Log Analytics UX improvements

We simplified and streamlined Logs analytics UI. You also have an option to add the queries to a dashboard.

Fingerprint Analytics

You can now group fingerprints by multiple attributes. Earlier, the only grouping before this release was by the source attribute.

APM

Dependency Tracking for APM

The external dependencies for APM services appear in the service details page.

Related Logs and Metrics in Span Details

When you select a specific span from the trace details, you can now see the related logs based on various attributes, including traceId, pod, and many others. You can see the metrics related to the service or endpoint.

Performance Improvements

We significantly improved the speed for queries for rate (for counter type) and histogram quantile (for histograms).

UI improvements

Spinner now displays in the APM pages while columns values are not yet available.

Infrastructure

UI improvements

We redesigned the Infrastructure tab to have a cleaner and more responsive UI. In the related logs, you can now open logs details for any specific log line.

Platform

Overall Stability and Resource Management

Kloudfuse now automatically terminates queries that run over the timeout or cause 'out of memory' issues. In addition, it restarts pods after hitting 'Out of Memory' issues.

2.2.2

This release introduces a key new feature, Error Analytics for Elastic APM. We also made improvements to Logs, APM Distributed Tracing, and Metrics.

Error Analytics

Elastic APM Errors

Users can now perform analytics on Elastic APM errors globally, and also see error types, frequency, and last occurrences for a specified service.

Logs

JSON log sorting

Before indexing, Kloudfuse now sorts each input JSON log line internally by key names. This improves the storage efficiency and search speed, reducing the number of unique patterns detected in the log streams.

APM Distributed Tracing

Error Analytics

Slice and dice on APM errors, and also on error chart/table in the service details page.

Span metrics and edge metrics fixes

We corrected some discrepancies in calculating RED metrics from the incoming spans and transactions for Elastic APM data.

Metrics

Metrics Summary and Exploration Performance improvements

We made several performance improvements for auto-completion of metric names and labels, and for metric summarization.

2.1.0

This release introduces two new features, Advanced Service Monitoring and TraceQL support. We also enhanced Logs, APM Distributed Tracing, and the Control Plane.

Advanced Service Monitoring (ASM)

Automatic Observability

ASM provides autonomic observability based on eBPF technology. The kfuse-knight agent discovers and tracks all services and their interactions. ASM delivers RED and USE metrics without any extra instrumentation or change to the application code. It also curates advanced alerts to detect anomalous and outlier behavior in the services.

TraceQL

Query Spans

Using TraceQL, you can query spans. You can also view a service map and flame graph through Grafana.

Logs

Skip auto-facet extraction from JSON logs

You can now optionally skip auto-extraction from JSON logs by changing logs parser configuration.

Fix data type identification

We corrected the broken charting for grammar-derived facets.

Efficient JSON log message parsing

We optimized JSON message parsing to reduce the CPU cost for logs parsing. This applies to all log lines, including structured JSON logs, embedded JSON, and partial JSON strings.

APM Distributed Tracing

Trace detail enhancements

Span details now show stack-trace, local variables, and context for Elastic APM. Also, flame graph spans list appears under a separate span list tab.

Ability to filter by custom span attributes

From span details, you can now filter by (include/exclude) custom span attributes, in addition to the standard OTel attributes.

Control Plane

Dashboards and Alerts fixes

We made a number of corrections in dashboards and alerts for the Control Plane.

2.0.0

This is a major release with significant feature enhancements for our customers.

Streamlined filtering

Streamlined filtering based on labels and facets across various streams

We standardized and streamlined filtering and navigation across all streams, including logs, traces, events, and metrics.

Service Level Objectives (SLO)

Service Level Objective (SLO) Support

Kloudfuse now supports Service Level Objectives (SLOs). Users can set latency and availability SLOs for any service instrumented with distributed tracing.

Single Sign On (SSO)

Single Sign On (SSO) Support

Kloudfuse now supports Single Sign On and several authorization methods, including Google, Okta, Azure, and others.

Alerting

Enhanced Alerting Support

Kloudfuse alerting now supports Change, Outliers, Anomaly, and Forecast alert types in addition to the existing Threshold alerts.

Migration

Simplified migration for grafana dashboards and alerts

Kloudfuse catalog service supports the migration of dashboards and alerts from external Grafana to Kloudfuse.

Logs

Automatic facet datatype detection

Logs parsing extracts, detects and color codes the data types of facets automatically to make it easier to work with large amount of logs data.

Externalized Logs parser configuration

You can configure Logs parser pipeline stages through remap, relabel, and transform actions/stages. This enables users to configure and process logs data from any agent, including fluent-bit, fluent-d, OTEL collector, DD-agent, and many others.

More efficient JSON log message parsing

We optimized JSON message parsing to reduce the CPU cost for logs parsing. These optimizations apply both to structured JSON and to log lines that contain embedded or partial JSON strings.

APM and Distributed Tracing

Support for Datadog, Elastic, and Otel agents

In addition to the OTEL collector/format, Kloudfuse stack now supports Elastic APM and Datadog APM payload formats. You can configure the pipeline to drop and relabel various attributes, as required.

Unified span-derived metrics and user-derived trace metrics

Kloudfuse stack produces unified span-derived metrics that you can configure to have arbitrary dimensions. To produce additional span-derived metrics, apply any filters and time/space aggregates to incoming data. Retain the metric data independently of trace retention.

Span/Trace download support

Users can download the full span data in two different formats:

  • CSV: download only the columns that appear in the UI.

  • JSON: download all the attributes of the incoming span stored by the stack.

Alerts

Enhanced Alerting

Kloudfuse alerting now supports Change, Outliers, Anomaly, and Forecast alert types, in addition to the existing Threshold alerts.

Infrastructure

Related Resources, Metrics, Events, Logs, and Traces

The Infrastructure view enables you explore every Kubernetes resource, including related logs, traces, metrics, and events.

Control Plane

Forecasting

Control plane now contains information on forecasted disk and outliers resource usage capabilities for the Kloudfuse observability stack resources.

1.3.4

This is a minor release update to fix bugs in Elastic APM ingestion.

1.3.3

This is a minor release update with the following feature enhancements for our customers:

Logs

Composite Sorting

Log data is sorted by multiple keys (fingerprints, labels, timestamp); this results in more efficient disk storage, and therefore better query performance.

Saved Query

You can save log queries as views, and reference them later. You can also save them as adhoc dashboards for use by team members.

Log Download

Download Logs events from the Kloudfuse UI. The logs download is limited to 10MB.

We support three different download formats:

  • TXT: Raw log message as emitted by the application.

  • CSV: Comma-separated log message, along with all fields that appear on the Kloudfuse Logs UI screen.

  • JSON: Full detailed log events with all facets and labels associated with the log event.

Control Plane

Remove pinot exception alerts

We plan to bring this feature back in a later release, with significant enhancements.

1.3.2

This is a minor release update, with performance improvements and feature enhancements for our customers.

Metrics

Improved metric segment seal times

On E2 machines, metric segments used to take 4-10 minutes due to number of docs in each segment (~50M). We moved to columnar seal instead of row-by-row seal, and seal times decreased by 50% or more.

Logs

Fluent-D support

Kloudfuse can now ingest logs from Fluent-D directly. We support JSON and msg-pack formats.

Fingerprint sorting-based segment disk layout

We now sort the log lines on the disk based on their fingerprint. This results in better storage compression and improved search performance for both grep and facets.

APM Traces

Support for missing and no-root spans

We now support flame graph view for incomplete traces. In certain customer environments, we may not get a root-span or parts of traces may always be missing due to environment setup. We improved the flame graph visualization to render such traces correctly.

Improved span segment encoding

To improve query speed, we moved to dictionary-based encoding for span durations and bigger segments.

1.3.1

This is a minor release update with support for better analytics.

Analytics

Auto Alerting and Analysis with Hawkeye and Bullseye

With the right instrumentation in place, Kloudfuse analytics can now do auto alerting and analysis. Using Hawkeye, you can easily enable auto-alerting for automatic monitoring of all Kubernetes services for anomalies on their RED metrics. Auto-analysis capability using Bullseye generates an analysis report with possible reasons for the alert (anomaly).

1.3.0

This release brings many improvements to the Kloudfuse stack.