FuseQL Best Practices

The fastest FuseQL queries on Kloudfuse touch the least data. Every log line has three layers of structure — indexed stream labels, ingest-time facets, and the raw log body — and the right operator for each layer is different. This page explains the layers, how the query engine evaluates them, and the habits that keep queries fast at scale.

Know the three layers of a log line

Layer Examples How to filter Cost

Indexed stream labels

source, level, kube_namespace, kube_service, pod_name, cluster_name, host, availability_zone, project, docker_image, kube_container_name, instance_id, instance_type

Stream selector or pre-pipe condition: source="nginx" level="error"

Cheapest — resolved against the label index before any data is read

Ingest-time facets

Fields Kloudfuse extracted when the log arrived: statusCode, endpoint, duration, userId, and any other key your pipeline parses

Post-pipe or pre-pipe facet filter: @statusCode=200 or | statusCode="200"

Cheap — resolved from stored facet columns; no log-body scan

Raw log body

The original message text

Term search: 'error'; grep: "err*"; regex: =~"pattern"

Most expensive — requires scanning the log_line column; term search is index-backed and much faster than grep or regex

Structure every query in that order: select streams by label, narrow by facet, and only then search the remaining text.

Start with the most selective stream selector

The stream selector is evaluated first and determines how much data the rest of the pipeline reads. Name the source and add every indexed label you know:

source="grafana" level="error"

The full set of indexed labels — resolved as direct column lookups before any row is fetched — is:

source, level, kube_namespace, kube_service, pod_name, cluster_name, host, availability_zone, project, docker_image, kube_container_name, instance_id, instance_type

Two habits keep selectors fast:

  • Prefer = to =~. An equality matcher is a direct columnar lookup; a regex matcher runs RE2 against every label value in the result set.

  • Never search for what a label already states. source="app" |= "ERROR" scans every line; source="app" level="error" reads only the error streams and catches all capitalizations because Kloudfuse normalises level at ingest.

Avoid match-everything patterns such as source=~".+" — they make the stream selector a no-op and turn the entire time range into the query’s working set.

Filter on ingest-time facets, not on strings

Kloudfuse extracts facets from structured content at ingest and stores them alongside the line. Filter on them directly — no parser stage is needed:

source="grafana" | statusCode="200"

Numeric, duration, and bytes comparisons work the same way:

source="grafana" | duration > 100ms

The pre-pipe facet syntax with @ is equivalent and slightly more concise:

source="grafana" @statusCode=200

Compare this with the string-search version, source="grafana" |= "statusCode=200", which scans every line, breaks if the field order or spacing changes, and also matches lines that merely mention that text.

Remember: facets that are in the stream selector position only match indexed stream labels. Write facet conditions after the opening selector or with the @ prefix:

source="grafana" @statusCode=503 @endpoint="/api/query"

When you need to search the raw log body, prefer term search over grep or regex. Term search resolves through the Lucene inverted index that Kloudfuse builds from every log line at ingest — it is an index lookup, not a scan:

'OutOfMemoryError'

Term search is case-insensitive by design and matches whole tokens only. Combine with stream and facet conditions so the index result set is already narrowed:

source="payment" level="error" @endpoint="checkout" 'OutOfMemoryError'

Grep ("pattern") supports wildcards and is a good choice when you need partial-word matching or a prefix:

"conn*"         # matches connection, connect, connected
"*refused"      # matches suffix

Internally, grep with a literal prefix pre-filters candidates through the index before doing per-row work — significantly faster than a pure regex scan. Reserve =~ and !~ for cases where neither term search nor grep expresses the intent.

Reach for a parser only when the facet does not exist

Parsers (json, logfmt, pattern, regexp) re-extract fields from the raw log body on every line, on every query. That work is wasted when the field is already an ingest-time facet. Use the facet directly:

source="nginx" @statusCode=503

Use a parser when a field is genuinely not extracted at ingest — a one-off pattern inside the message body, or a format your pipeline does not parse:

source="nginx" | regexp `(?P<method>[A-Z]+) (?P<path>[^ ]+)` | method="POST"

When a parser is necessary, note that basic json, logfmt, pattern, and regexp parsers without parameters are pushed down to the storage layer. Parsers with expression parameters are evaluated in the query engine, which is slower. Keep parser stages as simple as possible.

Keep metric queries pushdown-friendly

FuseQL pushes metric computation down into the Pinot storage layer when the query shape allows it. Pushed-down queries are dramatically faster than those computed in the engine. The following stages are pushdown-friendly:

  • Stream label filters and indexed facet filters

  • Line filters (|=, |~, !=, !~)

  • Basic parsers (json, logfmt, pattern, regexp) without parameters

  • label_format used only to rename a label (| label_format new=old)

  • Metric aggregations: count, sum, avg, min, max, and the percentile operators (p50, p95, p99, …) when combined with a timeslice grouping

The following stages disable pushdown for the remainder of the pipeline:

  • line_format with template syntax ({{ .field }})

  • label_format used to add or modify labels (not just rename)

  • Any parser invoked with expression parameters

# Pushdown-eligible — timeslice + count pushed to storage
source="grafana" | timeslice 5m | count by (_timeslice, level)

# Not pushdown-eligible — line_format forces engine evaluation
source="grafana" | line_format "{{.msg}}" | timeslice 5m | count by (_timeslice)

If a line_format stage is needed for display, keep it in a separate log-retrieval query and not in a metric query.

Use compose carefully

The compose operator runs a subquery and uses its results as a filter. It has a hard result limit of 10,000 entries (default: 2,500). Exceeding the limit truncates the subquery result silently and may cause the outer query to miss matching rows:

# Safe — subquery returns a small, bounded set
source="payment" | error_code exists
  | compose error_code maxresults=500

Keep compose subqueries as selective as possible. If you need to match a large set, consider writing the condition directly as a facet filter instead of using compose.

Bound the scope and the time range

Query cost scales with the data volume selected. The time range is a multiplier on everything above. While iterating on a query, work against a short window — the last 5 or 15 minutes — and widen it only when the query is final.

For dashboards, size the range window […​] to the panel’s step rather than defaulting to large windows: rate({…​}[1m]) at a 1-minute step reads each log line once. A 1-hour window at the same step reads 60× more data for no extra granularity.

Add limit N to log-retrieval queries to cap the result set. The maximum is 10,000 rows; queries that omit a limit use the default and may return more data than needed.

  • FuseQL Limitations — hard row limits, series caps, timeout constraints, and pushdown restrictions.

  • Text search operators — the full term search and grep reference: tokens, wildcards, field-scoped search.

  • Predicate operators — label and facet filter reference: string, numeric, duration, and regex comparisons.

  • LogQL Best Practices — LogQL-specific guidance for queries using the Loki-compatible syntax.