Query Interleaving

Overview

Query interleaving lets Kloudfuse run multiple Pinot queries concurrently and interleave their execution, so a small interactive query is not held up by a large query that arrived just before it.

Before 4.1, Kloudfuse processed Pinot queries first-come-first-served: a query that started running held the cluster until it finished, and any queries that arrived after it had to wait their turn. With query interleaving, queries that arrive at roughly the same time make progress together — short queries finish quickly while long queries continue to run in the background.

Timeline diagram contrasting before and after 4.1: top panel shows three queries running one after another with the medium and small queries waiting for the large query to finish; bottom panel shows the same three queries running concurrently

When query priorities are configured, higher-priority queries also receive a larger share of cluster capacity. See Query Priorities for how priorities are assigned and resolved.

Query interleaving is enabled by default starting in Kloudfuse 4.1. It can be disabled per cluster from custom_values.yaml; see Disable Query Interleaving.

Disable Query Interleaving

To revert to the previous first-come-first-served behavior — for example, while triaging an issue or comparing behavior — set the following in your Helm custom_values.yaml and reapply the chart:

global:
  queryScheduler:
    enabled: false

yaml

With the scheduler disabled, queries fall back to the pre-4.1 sequential execution model.