Log Parsing Best Practices

These practices assume you’ve already read Log Parsing Architecture and Log Parsing Configuration — they’re about how to apply that configuration well, not what it does.

Scope every function with conditions

A parser, relabel, or transform entry with no conditions block runs against every log line, regardless of source. Add a conditions matcher (typically on #source) to avoid these problems:

  • The tokenizer will be applied to lines it wasn’t intended for. A dissect pattern developed for one source’s line shape is capable of matching a differently-shaped line from another source.

  • The drop or keep rule may also be applied to lines you don’t intend. Without a conditions block, these actions apply to the whole pipeline by default.

Relabel, Transform priority order

relabel and transform run the same action engine, but they are not interchangeable by position — each runs in a fixed stage regardless of where you place it in the config list. relabel always runs before Grammar (dissect/grok), JSON auto-parsing, and addFacet. transform always runs after.

Relabel: A relabel rule can only act on labels and facets already present when it starts — never on a facet your own pipeline extracts. Writing a rule that depends on an extracted facet as relabel doesn’t error; it simply never matches, since the facet doesn’t exist yet when relabel runs. Use relabel to act on a pre-existing label such as source or kube_namespace, or on the raw message text (%kf_msg).

Transform: A transform has access to everything the extraction process has produced by the time it runs — including anything added by relabel, which always runs first. This is what makes transform the right choice for a drop, keep, replace, or facet_to_label_map rule that depends on a facet your own config extracts.

Test patterns before deploying

Changes only take effect after logs-parser picks up a Helm upgrade — this pipeline has no live preview and no hot-reload (Limitations). Validate a dissect tokenizer or grok pattern against a real log line first, using a dissect debugger or grok debugger.

Test in a lower environment (development or staging) before pushing to production. A pattern that looks correct by inspection can still fail to match a line with slight spacing differences or extra fields. Another common problem is a delimiter mismatch — wrong punctuation or a missing space — causing a field to capture more or less than intended.

Promote facets sparingly

Promote a facet to a label only when you need to filter or group by it, since every label is indexed. For a value like source or env that you plan to filter or group by in dashboards, promoting it with facet_to_label_map (Transform) is the right move.

Promoting a high-cardinality facet to a label is expensive to index and offers no query benefit — it should remain a facet, or be dropped if it’s not useful. The same logic applies to metrics labels and log labels alike. See Cleaning up a noisy or expensive label.

Dissect for fixed layouts, Grok for variable ones

A dissect tokenizer is a text splitter — fast and easy to use, but it only works when delimiters are exact and always in the same position. A grok pattern is a named regular expression — it can take longer to write and run, but it can handle optional fields, variable whitespace, or alternation that a dissect tokenizer can’t.

  • dissect for a fixed-format line (an access log, a structured prefix)

  • grok when the line’s shape varies, or to use one of its predefined patterns (%{IP:client_ip}, %{TIMESTAMP_ISO8601:ts}) in place of custom regex

JSON auto-parsing facets

By default, JSON auto-parsing extracts at most 50 facets from a single log line. Any JSON that exceeds that will be skipped for facet extraction.

If a source legitimately needs more fields, raise the ceiling with skipAutoFacet’s `maxFacetsCount (JSON logs) so every field is extracted again. Another option is whitelistFacets, which only extracts the facets you name — keeping the line safely under the ceiling regardless of how many fields the source sends.

Drop and keep are destructive actions

A dropped log line is gone — there’s no undo once relabel/transform’s `drop or keep action has discarded it. Before you deploy one:

  • Scope it with conditions so it applies only to the source it’s meant for; see Scope every function with conditions.

  • Test its regex against real sample lines from that source, including lines you want to keep.

  • Remember that keep drops everything that does not match, so a typo in its regex can drop all of a source’s logs.

References