APM Traces

The default view: a raw list of individual traces, newest first, above a stacked Traces by service chart. Each row shows timestamp, service, span name, duration, method, status, endpoint, span count, and a latency breakdown bar. Click a row to open its trace detail flyout.

Use Search / the filter bar (for example, service_name=web, span_type=http) to narrow the list — see Filtering traces for the full set of filterable attributes. Use COLUMNS to control which columns are shown, and the download icon to export the current list.

Plots an aggregated metric over time using the query builder described in The query builder and Formulas — Show count of, by, roll up every, Span Filter, and Limit — with support for multiple queries and formulas. Use COMPARE to overlay the same query from an earlier period (for example, one day back).

The same query builder rendered as a table instead of a chart — useful when you want exact numbers per group (for example, per span_name) rather than a visual trend.

Ranks the groups produced by the query builder from highest to lowest value — the fastest way to answer "which N services/endpoints/pods are driving this metric."

Shows the relative share each group contributes to the total — useful for questions like "what fraction of requests are going to each downstream service."

A single large number (with a small sparkline) for the current query — useful for dashboards or a quick at-a-glance total, such as total request count in range.

The flame graph lays every span out on a shared time axis, indented by parent/child relationship, and colored by service (or, via Color by, span count instead of execution time). This is the fastest way to see which service or span dominates the trace’s total duration. A panel alongside the graph totals Exec time or Span count per service across the whole trace. Click any bar in the flame graph to select that span and populate the span detail panel below.

The same spans as the flame graph, but laid out as an indented list (service and operation name) rather than a nested timeline — useful when the call tree is deep and the flame graph’s timeline gets visually cramped. Selecting a span here populates the same detail panel as the flame graph.

Groups every span in the trace by service and span type, reporting Name, Spans, Duration, Exec Time, and Exec % per group. Expand a group to see its individual spans, and use Search spans or the span-type quick filters (custom, db, http, queue, gRPC) above the list to narrow a long trace down to the spans you care about.

The raw JSON for the entire trace — useful for copying exact attribute values or for pasting into a support ticket.

Selecting a span in any of the views above populates the panel at the bottom of the flyout, with five tabs:

For example, using a time interval of 3 hours, grouping by the app_ads_contextKeys value, limiting the query to the top 100, and rolling up every 2 minutes produces a chart of the top 100 app_ads_contextKeys values over 2-minute buckets across the 3-hour window.

Select Add Query to add a second query, labeled B, as a duplicate of the first. This makes it easy to perform calculations across the time series retrieved by both queries — change the filter in query B to compare it against query A.

By default, multiple queries render on the same chart. Deselect Combine all queries into one chart to render each query as its own chart instead — useful when the queries have significantly different value ranges and are hard to compare on a shared Y axis. Select the option again to recombine them.

A query loaded with a relative time interval (the default is Last 15 minutes) reflects the data as of when it last ran. Select Refresh at the top right of the interface to re-fetch the data for the same options.

You cannot remove the only (last) query on the page; instead, select a different query in the first drop-down. To remove one query out of several, hover over its tile and select Remove. To clear the whole query (remove all tiles), select Remove all.

The interface displays graphs for the underlying queries and the new formula.

Select Remove next to the formula you want to delete. The formula is removed from the interface; the underlying queries are unaffected.