OQL Cookbook

These examples stay within the OQL features implemented by the current storage backends.

Discover available telemetry

List metric names:

SHOW metrics

List known attributes for each signal:

SHOW logs attributes
SHOW metrics attributes
SHOW traces attributes
SHOW spans attributes

List observed values for a filterable attribute:

SHOW logs attribute_values('service.name') | RANGE 1h
SHOW traces attribute_values('service.name') | RANGE 1h

Logs

Find the latest errors from one service:

GET logs
| RANGE 30m
| FILTER severity = 'ERROR' AND service.name = 'checkout-api'
| SORT timestamp DESC
| LIMIT 50

Search for timeout-related log messages, case-insensitively:

GET logs
| RANGE 1h
| FILTER message ILIKE '%timeout%'
| SORT timestamp DESC
| LIMIT 100

Count errors by service:

GET logs
| RANGE 6h
| FILTER severity = 'ERROR'
| STATS count() -> error_count
| BY service.name
| SORT error_count DESC
| LIMIT 20

Count noisy HTTP status codes:

GET logs
| RANGE 1h
| FILTER severity IN ('ERROR', 'WARN') AND http.status_code >= 400
| STATS count() -> count
| BY http.status_code
| SORT count DESC

Metrics

Plot metric point volume by metric name:

GET metrics
| RANGE 3h
| STATS count() -> points
| BY name EVERY 5m
| SORT time_bucket ASC

Inspect raw samples for one metric:

GET metrics
| RANGE 1h
| FILTER name = 'system.cpu.time'
| PICK timestamp, name, type, value
| SORT timestamp ASC
| LIMIT 200

Average a metric by host:

GET metrics
| RANGE 6h
| FILTER name = 'system.cpu.usage'
| STATS avg(value) -> avg_cpu
| BY host.name EVERY 5m
| SORT time_bucket ASC

Traces and Spans

Find the slowest trace summaries:

GET traces
| RANGE 1h
| FILTER trace_duration > 500ms
| SORT trace_duration DESC
| LIMIT 20

Count trace roots by name:

GET traces
| RANGE 24h
| STATS count() -> count
| BY name EVERY 10m
| SORT time_bucket ASC

List spans for one trace:

GET spans
| RANGE 1h
| FILTER trace_id = '00112233445566778899aabbccddeeff'
| PICK trace_id, id, parent_span_id, name, start_timestamp, duration
| SORT start_timestamp ASC

Find slow server spans with HTTP failures:

GET spans
| RANGE 1h
| FILTER span_kind = 'SERVER' AND duration > 250ms AND http.status_code >= 500
| PICK trace_id, name, duration, http.route, http.status_code
| SORT duration DESC
| LIMIT 25

Count error spans by route:

GET spans
| RANGE 2h
| FILTER status_code = 'ERROR' AND http.route IS NOT NULL
| STATS count() -> error_spans
| BY http.route
| SORT error_spans DESC
| LIMIT 20