feat(telemetry): one outcome event per command, with no channel for a path (backend#1907) - #527
Conversation
… path (backend#1907)
The CLI emits nothing today, so the backend#736 class — the binary landing on a
PATH the shell does not read, `cluster info` reading a kubeconfig context nobody
meant — is only ever visible when a customer mentions it. This wires the #1897
helper to a single terminal event per invocation: command, duration, exit code,
OS/arch, version, error class.
The ticket's "no arguments, no paths, no data" is built as a structure rather
than a rule, because a rule is a thing every future call site has to remember:
* the command is a LOOKUP into the set of paths enumerated from the live cobra
tree, so a value that is not a command the CLI dispatches cannot be emitted
at all — it reports `unregistered`, which stays countable;
* the error class is keyed on an INT, the CLI's own frozen exit-code contract.
The classifier is never handed an error message, so there is nothing for a
path or a cell value to travel down;
* everything else is an int.
There is no redaction regex anywhere in the change. A sanitiser has to
anticipate what it strips; a closed set only admits what was enumerated.
os.type / host.arch go in the RESOURCE layer under OpenTelemetry's own names
(§1.1 forbids re-inventing them as tracebloc.os): they are compile-time
constants of the binary, so they describe the process, not the occurrence, and
adding them to resourceScope means a call site still cannot set them.
The guard is derived, not restated. TestEveryEmittedStringComesFromAClosedSet
walks what the code ACTUALLY emits and requires every value to be an int or a
member of a set assembled from the producer's own declarations — so a free-text
channel fails it whether or not anyone thought to forbid the thing travelling
down it. Thirteen mutations were run against it; each reddened, and each anchor
was asserted to have applied. One of them (smuggling the raw command into a
second attribute) was caught only by the telemetry-side test and NOT by the
cli-side one, which was inspecting a single key — that test now sweeps the whole
payload.
WHAT IS NOT CONNECTED. The transport. The ticket says "rides the gateway and
token"; the 17 Aug decision (rfcs#28) replaced the Collector gateway with an
ingest endpoint on the backend, which is backend#1905 and does not exist yet.
pendingSink() returns nil, so every event is validated and dropped. Validation
runs regardless, so a malformed event fails in CI wherever the binary was built,
and connecting #1905 is one function body.
Opt-out (default on) via TRACEBLOC_NO_TELEMETRY or DO_NOT_TRACK, documented in
docs/troubleshooting.md — and the document's claim about which variables work is
itself a test, because a user who exports a stale name believes they have opted
out and nothing else would ever tell them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
bugbot run |
|
Sibling PR for the installer half of backend#1907: tracebloc/client#747. Independent — neither depends on the other, and the transport for both is tracebloc/backend#1905. |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit efff633. Configure here.
saqlainsyed007
left a comment
There was a problem hiding this comment.
Correctness review — approve.
Reviewed the full diff against the surrounding code it leans on (internal/telemetry/telemetry.go, internal/cli/exitcodes.go, internal/api env helpers, internal/config loader, cmd/tracebloc/main.go).
What I checked and why it holds:
- Event grammar & vocabulary —
cli.command.{succeeded,failed,cancelled}all pass the emitter's<domain>.<object>.<outcome>grammar and closed outcome set. - Exit-code → error class —
exitClasses(1–9) matches the frozen meanings inexitcodes.go; the shared codes (2/5/7) collapse to one bucket as intended; unmapped →unclassified(fail-closed). - Cancel path — the interrupted
*exitErroris non-nil-outer / nil-inner, soExitCodeFromErroryields 130,Recordmaps it tocancelled, and noerror.typeis attached (cancel isn't a failure outcome, soEmitdoesn't require one). - Privacy boundary — command value is a set-membership lookup against paths enumerated after
ExecuteContextCreturns (sohelp/completionare enumerated on the same tree that dispatched, andcommandPathOf(executed)can't be spuriouslyunregistered); the class is keyed on an int. No free-text channel, matching the PR's structural claim. - Env resolution — an unknown/
stagingvalue falls back and then fails closed at the emitter'sExports()gate rather than being guessed. - No new side effect — routing telemetry through
config.Load()is read-only (v1 migration is in-memory; only a laterSavewrites), so recording on every invocation doesn't touch disk. - Duration — monotonic
time.Since, clamped ≥0, zero preserved.
Tests are mutation-anchored, all 27 CI checks are green (Test, Lint, golangci-lint, govulncheck, house-rules, Bugbot), and I found no correctness defect. Nice, careful work — the "closed set, not a sanitiser" framing makes the privacy guarantee auditable.
Parent epic: tracebloc/backend#1872 · Ticket: tracebloc/backend#1907 · Contract: the telemetry contract
The CLI emits nothing today. That is why the backend#736 class — the binary
landing on a PATH the shell does not read,
cluster inforeading a kubeconfigcontext nobody meant — is only ever visible when a customer happens to mention
it. This wires the #1897 helper up to one terminal event per invocation.
What is emitted
One of
cli.command.succeeded/.failed/.cancelled, fromcmd/tracebloc/main.go— the single point every command path converges on, so§6.5's "terminal event on every path" is true by construction rather than true
for the handlers somebody remembered.
event.nametracebloc.cli.commanddata ingest— the contract's own name for this field (§7.1)tracebloc.cli.exit_codetracebloc.cli.duration_mserror.typeservice.name/tracebloc.componentcli/cliservice.versionos.type/host.archruntime.GOOS/runtime.GOARCHdeployment.environment$CLIENT_ENV, else prodservice.instance.idCtrl-C(130) iscancelled, notfailed, and carries noerror.type. Acancel counted as a failure moves the rate D9's alerts are written against every
time somebody changes their mind.
"No arguments, no paths, no data" is a structure, not a rule
A rule is a thing every future call site has to remember. There is no
redaction regex anywhere in this PR, because the record has no free-text
channel to redact:
internal/cli/commandPathsenumerates the paths from the live cobra tree; anything not in that set is
reported as
unregistered. A sanitiser has to anticipate what it strips; aclosed set only ever admits what was enumerated. A recorder built with no
registered commands reports
unregisteredfor everything — forgetting toregister cannot silently turn the lookup into a pass-through.
int— the CLI's own frozen exit-codecontract (
internal/cli/exitcodes.go), which is already a reviewed,documented classification of every way the CLI can fail. The classifier is
never handed an error message, so there is nothing for a path or a cell value
to travel down. An unmapped code is
unclassified, not the numberstringified into a value that appears on its own.
service.instance.idis a fresh random value per process, deliberately not thehostname (field hostnames are overwhelmingly
<firstname>-macbook, which §7.3forbids outright) and deliberately not persisted (a durable identifier we would
then have to answer erasure requests about).
The guard is derived, and mutation-proven
TestEveryEmittedStringComesFromAClosedSetdoes not hold a list of forbiddenkeys — a list like that agrees with itself and says nothing about the twentieth
attribute somebody adds. It walks what the code actually emits over a
cross-product of adversarial command paths and exit codes, and requires every
value to be an int or a member of a set assembled from the producer's own
declarations. A free-text channel of any kind fails it whether or not anyone
thought to forbid the thing travelling down it. It carries an attribute-count
anchor, because an inert loop over an empty payload reads exactly like a clean
sweep in a log.
TestNoFlagValueOrArgumentCanReachTheRecorddrives the real tree: everycommand, every flag set to a canary, canary args, canary environment, then the
whole payload searched.
13 mutations run, each reddened, each anchor asserted to have applied:
commandValuestops looking the path up (pass-through)ClassifyExitinvents a value outside the vocabularyRecordsmuggles the raw command into a second attributefaileddurationMSstops clampingtelemetryEnabledignores the opt-outtelemetryEnabledalways says no (feature dead)commandPathsstops recursing into subcommandstelemetryEnvrepairs an unknown environmentos.typedrops out ofresourceScopeNewstops stampinghost.archOne of these is worth calling out: "smuggle the raw command into a second
attribute" was caught by the telemetry-side test and NOT by the cli-side one,
which was asserting on a single key while the record carried the path verbatim.
That test now sweeps the whole payload. Without asserting the anchor applied,
that gap would have looked exactly like coverage.
What remains to be connected
The transport, and only the transport. The ticket says "rides the gateway
and token" — that wording is stale. The 17 Aug decision replaced the Collector
gateway with an ingest endpoint on the backend (RFC amendment
rfcs#28), which is
tracebloc/backend#1905, in flight and not yet available. So:
internal/cli.pendingSink()returnsnil— every event is validated anddropped (
SetSink's documented contract).wherever the binary was built.
pendingSink()returns the client that posts to it andnothing else in this PR changes.
The ticket body has been corrected so it no longer says "blocked by the gateway".
Would the three named field failures now be visible?
~/.local/bin, PATH advice only printedclientPR (tracebloc.install.cli_on_path). Nothing the CLI can emit sees it: a binary the shell never finds never runs.cluster inforeading the wrong kubeconfig contexttracebloc.cli.command=cluster infowitherror.typeno_secure_environment(exit 4) orlocal_environment(exit 3) becomes a countable rate instead of a support thread. Honest limit: a run that succeeds against a wrong-but-valid context still looks like a success, and no attribute in this closed set can tell them apart without emitting the context name, which is a path.apt-gethung on the dpkg lockThe CLI half of the DoD is the second row; the other two are the
clientPR.Also here
internal/telemetryjoinsscripts/coverage-floor.shat 95% (measures100.0%). It stopped being an unwired helper and became the thing that decides
what leaves a customer's machine, so a rotting test there is the guard going
quiet, not an ordinary coverage regression.
docs/troubleshooting.mdgains a "Usage reporting" section naming every fieldand the opt-out. That document's claim about which variables work is itself a
test — a user who exports a stale name believes they have opted out, and
nothing else would ever tell them.
Test plan
make checkgreen;make deadcodeclean (the whole package entered the buildgraph with this PR, so every function in it is now reachability-checked);
./scripts/coverage-floor.shgreen including the new entry; 13 mutations above.🤖 Generated with Claude Code
Note
Medium Risk
Touches default-on telemetry and the privacy boundary for what leaves customer machines; transport is still nil, but mis-wiring or enumeration drift could misreport commands or leak data if guards regress.
Overview
Adds one terminal telemetry event per CLI invocation from
cmd/tracebloc/main.goafterExecuteContextCreturns, recording command path (from the live cobra tree), exit code, duration, version, OS/arch, environment, and a per-process random instance id—without affecting exit behavior.Privacy and classification are structural:
internal/telemetry/outcome.gomaps commands via an enumerated path set (unregisteredotherwise) anderror.typevia frozen exit codes only; Ctrl-C (130) emitscancelledwith noerror.type.internal/cli/telemetry.gohandles opt-out (TRACEBLOC_NO_TELEMETRY,DO_NOT_TRACK), environment resolution, andpendingSink()returning nil so events are validated and dropped until backend#1905.Emitter changes stamp
os.type/host.archon the resource layer. Docs add a usage-reporting section indocs/troubleshooting.md. CI adds a 95% coverage floor forinternal/telemetry. Broad tests exercise the live command tree, canary args/flags, and doc/code opt-out parity.Reviewed by Cursor Bugbot for commit efff633. Bugbot is set up for automated code reviews on this repo. Configure here.