Logs vs metrics vs traces

Published 18 October 2026

When a patient walks into a hospital, three kinds of record help. The vital signs monitor shows a few numbers continuously: pulse, blood pressure, oxygen. It beeps when one goes out of range. That is metrics. The case notes are detailed, timestamped entries written as things happen: "14:02, complained of chest pain, given aspirin". That is logs. And the patient's path through triage, X-ray, cardiology and the pharmacy, with how long each step took, is a trace.

Each answers a different question. Metrics: is something wrong? Traces: where in the system is it wrong? Logs: what exactly happened, and why?

Metrics: cheap numbers over time#

A metric is a number measured repeatedly and aggregated: requests per second, error rate, p99 latency, queue depth, CPU. Three basic types:

• Counter: only goes up (requests served, errors). You graph its rate.
• Gauge: goes up and down (memory in use, connections open, queue length).
• Histogram: counts values into buckets, so you can compute percentiles (latency).

Because a metric is just a number per time interval, storing a year of it is cheap no matter how much traffic you serve, and querying it is fast. That makes metrics the right basis for dashboards and alerts. Prometheus, Datadog, CloudWatch and Grafana Mimir store them.

The trap is cardinality. Each unique combination of label values is a separate time series. Labels like endpoint, status and region are fine. Add user_id and you have created one series per user, and your metrics system falls over or your bill explodes.

Logs: detailed events, expensive at volume#

A log line is a record of one event, with as much detail as you like: the order id, the coupon code, the exception and stack trace. Logs are where the why lives. Write them structured (JSON with consistent field names), not as free text, so they can be filtered and aggregated:

{"ts":"2026-10-18T20:00:01.412Z","level":"warn","service":"inventory","trace_id":"4bf92f35","order_id":8812,"msg":"reserve slow","duration_ms":240,"db_calls":6}

The cost is volume. A service at 5,000 requests a second writing five lines per request is two billion lines a day. Storage, indexing and search at that volume is often the largest line on an observability bill. So teams set levels carefully, sample noisy debug logs, keep hot storage short (days) and archive the rest cheaply. Elasticsearch or OpenSearch, Loki, Datadog, Splunk and CloudWatch Logs are common stores.

Traces: one request's path#

A trace follows a single request through every service it touches. Each unit of work is a span with a start time, a duration, a parent and attributes. The trace id travels between services in a header (the W3C traceparent standard), so spans from thirty services join into one tree. Traces answer the question metrics and logs cannot: where did the time go?

Here is one slow checkout. Click a span to see its attributes and the log lines attached to it, then apply the fixes.

Tracing every request is expensive, so most systems sample: keep 1% of normal traces, but tail sampling keeps every trace that was slow or failed, which are the ones you want. Jaeger, Zipkin, Grafana Tempo, Honeycomb and Datadog APM store traces.

Together#

The pillars work best joined up, and the glue is the trace id. Put it in every log line, attach trace ids to metric samples as exemplars, and an investigation flows:

1. A metric alert fires: checkout p99 is 2 s.
2. Click an exemplar on the latency graph to open a slow trace: the time is in inventory, six sequential database calls.
3. Jump from the span to its logs: the warning names the order and the query.

OpenTelemetry (OTel) is the vendor neutral standard for producing all three: one SDK per language, one wire protocol (OTLP), and a collector that sends data to whichever backend you choose. It is the safe default for new services in 2026.

In the interview#

Checkpoint#