Log Timestamp Handling
Log events can carry multiple timestamps. Kloudfuse selects the most appropriate one during ingestion and validates it against configurable bounds before storing. This page explains the three timestamp sources, how Kloudfuse chooses between them, and how to configure timestamp bounds checking.
Timestamp Sources
Kloudfuse recognizes three timestamp sources for each log event. Each has different reliability and precision characteristics.
Ingest Timestamp
The ingest timestamp is set by the Kloudfuse ingester when the log payload is first received. It represents the wall-clock time on the ingester host.
-
Always available — every log has an ingest timestamp.
-
Trusted — represents when Kloudfuse first saw the log.
-
Used as the fallback when other timestamps are missing or fail bounds checking.
Agent Timestamp
The agent timestamp is carried inside the agent’s payload envelope. It is set by the agent or the originating application before the log reaches Kloudfuse.
Message Timestamp
The message timestamp is extracted from the log line itself by the parsing pipeline — via grok, dissect, logfmt, setTimestamp, or the automatic timestamp extraction stage.
-
Not always available. Depends on whether the log line contains a recognizable timestamp and whether the pipeline is configured to extract it.
-
Most precise when it works — represents the actual moment the application emitted the log.
Timestamp Selection
You configure which timestamp source to use per source or globally:
| Config Key | Description |
|---|---|
|
Per-source timestamp mode. Overrides the global setting for logs from |
|
Global timestamp mode. Applies to all sources unless overridden by a per-source key. |
The value must be one of:
| Mode | Config Value | Behavior |
|---|---|---|
Agent |
|
Use the agent/payload timestamp. Falls back to ingest timestamp if missing or out of bounds. |
Message |
|
Use the timestamp extracted from the log line. Falls back to agent timestamp, then ingest timestamp if missing or out of bounds. |
Ingest |
|
Always use the ingest timestamp. No bounds check is applied. |
logs-parser:
extraApplicationConfigs: |-
timestamp_for_all_sources = "agent"logs-parser:
extraApplicationConfigs: |-
timestamp.myapp = "message"Timestamp Bounds Checking
Timestamps that are too far in the past or future cause problems for downstream query and storage systems. Kloudfuse validates candidate timestamps against configurable bounds before accepting them.
When a timestamp fails bounds checking, Kloudfuse silently replaces it with the ingest timestamp. The ingest timestamp mode skips bounds checking entirely.
Past vs Future Timestamps
Past and future timestamp errors have different causes and risk levels, which is why Kloudfuse allows you to configure them independently.
Past timestamps are common and usually legitimate
-
Agents buffer logs and send them in batches. A few minutes to hours of delay is normal.
-
Network retries, Kafka consumer lag, or slow pipeline processing add latency.
-
Logs from batch jobs or cron tasks may describe events from hours ago.
A generous past bound (hours) avoids dropping valid delayed logs.
Future timestamps are almost always bugs
-
Misconfigured host clocks or NTP drift.
-
Wrong timezone offsets applied during parsing (for example, a UTC log interpreted as local time, shifting it hours into the future).
-
Bad timestamp parsing: extracting the wrong substring, interpreting a numeric field as epoch seconds when it is actually milliseconds, or matching a non-timestamp number.
A tight future bound (minutes) catches these parsing errors early.
Configuration
Bounds are configured in milliseconds. Both default to 1 hour (3600000 ms). Set a value to 0 to disable that bound (allow any timestamp in that direction).
| Config Key | Default | Description |
|---|---|---|
|
|
Maximum age of a timestamp relative to now. Applies to all sources. |
|
|
Per-source override for the past bound. Takes priority over |
|
|
Maximum amount a timestamp can be ahead of now. Applies to all sources. |
|
|
Per-source override for the future bound. Takes priority over |
Per-source keys always take priority over _all_sources keys.
logs-parser:
extraApplicationConfigs: |-
timestampMaxPastDurationMs_all_sources = 21600000
timestampMaxFutureDurationMs_all_sources = 600000logs-parser:
extraApplicationConfigs: |-
timestampMaxPastDurationMs.myapp = 0
timestampMaxFutureDurationMs.myapp = 0Hydration Exception
When hydrating (replaying) historical logs, Kloudfuse skips bounds checking entirely. Hydrated logs always use the original event timestamp, since the purpose of hydration is to reconstruct historical data. See Log Hydration for details.
