Skip to content

fix: return each node's spans from the distributed trace API - #366

Open
FrameAutomata wants to merge 1 commit into
mainfrom
fix/352-distributed-trace-spans
Open

fix: return each node's spans from the distributed trace API#366
FrameAutomata wants to merge 1 commit into
mainfrom
fix/352-distributed-trace-spans

Conversation

@FrameAutomata

Copy link
Copy Markdown
Collaborator

Closes #352

Problem

GetDistributedTrace initialised every node with Spans: []models.Span{} and never queried the spans table. The dashboard's Distributed Trace card, traceway traces show, and the MCP get_trace tool therefore reported zero spans for every service in a trace, even though the response model, the CLI client type and the docs all describe nodes carrying their spans.

Fix

Spans are stored under their owning endpoint / task / AI-trace id as trace_id (the ingest re-roots them), which is exactly what the detail pages read via SpanRepository.FindByTraceId. The handler now:

  • collects one models.TraceRef{ProjectId, TraceId, RecordedAt} per entity node,
  • loads all of them with a single SpanRepository.FindByTraceIds query, implemented on the SQLite, DuckDB and ClickHouse backends. The query is bounded to the usual ±24h trace window stretched across the earliest and latest node, uses the (project_id, trace_id) index / ordering key, and is filtered back to the exact (project, trace) pairs so the two IN lists' cross product cannot leak a same-id span from a project that did not reference it,
  • attaches the results per node. Exception-only nodes keep an empty list, and spans is never null.

Spans stored under the OTel trace id itself (the orphan path, when no owner was found in the batch) belong to no node and are still not surfaced, same as today.

Also in this PR

  • CLI contract seed: the seeded span was keyed by the distributed trace id (the orphan path). It now belongs to the seeded endpoint, so the endpoint-detail and distributed-trace goldens gain the spans[].* shape lines (regenerated with -update).
  • CLI: traceway traces show prints a SPANS column.
  • Frontend: the Distributed Trace card shows an N spans toggle per node that expands into that node's SpanWaterfall, reusing the endpoint page's component.

Verification

  • New tests: shared helper unit tests, TestSpanRepository_FindByTraceIds (+ empty) on SQLite and DuckDB, and a handler test TestGetDistributedTraceAttachesSpansToOwningNodes that fails on main (endpoint spans = [], want both owned spans).
  • go build under all three tag sets; go test for controllers, repositories/telemetry/..., models; cd cli && just test lint gen-skills-check contract-test all green; npm run check / npm run lint clean.
  • Live run: booted the backend on SQLite, ingested an OTLP/JSON trace with a checkout-api SERVER span (two child spans) and an email-worker CONSUMER span (two child spans) under one trace id, then opened the endpoint page in headless Chromium. The API returned 2 spans on the endpoint node and 2 on the task node, and the card rendered a 2 spans toggle for each that expands into the waterfall with the ingested span names and durations.

🤖 Generated with Claude Code

GetDistributedTrace built every node with an empty spans array and never
queried the spans table, so the distributed view, the CLI's `traces show`
and the MCP get_trace tool all reported zero spans for every service.

Spans are stored under their owning endpoint/task/AI-trace id as trace_id,
which is what the detail pages read. The handler now collects one
(project, entity id, recordedAt) reference per entity node and loads all
of them with a single SpanRepository.FindByTraceIds query on every
telemetry backend, bounded to the trace window around the earliest and
latest node and filtered back to the exact (project, trace) pairs.
Exception-only nodes keep an empty list, and the array is never null.

The CLI contract seed keyed its span by the OTel trace id, which is the
orphan path; it now belongs to the seeded endpoint, so the endpoint-detail
and distributed-trace goldens carry the span shape. `traces show` gains a
SPANS column, and the dashboard's Distributed Trace card shows a span
count per node that expands into that node's waterfall.

Closes #352

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@FrameAutomata FrameAutomata added the ci Run CI on this PR (remove and re-add to re-validate after a push) label Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Run CI on this PR (remove and re-add to re-validate after a push)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Distributed trace API always returns empty spans for every node

1 participant