Conversation
Every $ld:ai:* event that executeAndTrack sends carries the LaunchDarkly environment id, because AI Config Monitoring needs it to match a trace to the config that produced it. The two payloads built outside that path did not: makeNodeTrackData, used by every native graph adapter, and the graph-level graphTrackData in graph.ts. Read the id through the same tryGetEnvironmentId helper, which is now exported inside the client package so graph.ts can call it too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A native graph adapter has no TrackData of its own: makeNodeTrackData describes one node, and the run as a whole is the graph flag. Add makeGraphTrackData, which names the graph flag as both the config key and the graph key, the same choice the SDK's own graph runner makes. Adapters pass it to setLdSpanAttributes in the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
toOpenAIAgents, toClaudeAgents, toLangGraph and toVercelAgents each open a launchdarkly.graph span that carried only the graph key and token counts. The AI Config Monitoring tab finds a trace by a feature_flag span event and the config key, so none of these runs appeared against their AI Config. Tag the span through setLdSpanAttributes, the same helper every handler already uses, so the graph span now carries launchdarkly.config.key, launchdarkly.variation.key, launchdarkly.run.id, the context keys, and the feature_flag event with feature_flag.set.id. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
toLangGraph opened its launchdarkly.graph span and never ended it, on success or failure. An unended span is never handed to the exporter, so LangGraph users got no graph span in their traces at all. The body now runs in try/finally, matching toVercelAgents. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
bugbot run |
The launchdarkly.graph span that graph() opens carried only launchdarkly.graph.key. It now goes through setLdSpanAttributes with the same track data its graph events use, so it has the config key, variation, run id, context keys and the feature_flag event, like the native adapters' graph span. invoke() runs the same walk as stream(), so one call site covers both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The $ld:ai:graph:invocation_success, invocation_failure, duration:total and total_tokens events (and path, in vercel-agents) from the native adapters carried the root node's config key, while their graph span and graph() use the graph key. They now use makeGraphTrackData, so a graph run reports the same config key whichever runner produced it. toOpenAIAgents and toClaudeAgents ended the graph span only on the run's own success and failure paths. An error while building the agents or sub-agent tools left it open, so it was never exported. Both bodies now end the span in a finally, as toLangGraph and toVercelAgents do. The adapters no longer set launchdarkly.graph.key by hand, since setLdSpanAttributes already does. makeGraphTrackData is marked @internal: only the adapters use it, and its signature changes if GraphDefinition gains the graph's variation metadata. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
bugbot run |
…-identity # Conflicts: # packages/client/src/index.ts
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 1cb2231. Configure here.
jeffdupont
left a comment
jeffdupont
left a comment
There was a problem hiding this comment.
Parity review alongside python #121 (launchdarkly/python-ai-sdk#121). This PR already handles most of what I raised there. Graph-level events in all four adapters now use the graph key through makeGraphTrackData. The graph() span gets setLdSpanAttributes (graph.ts:724). The hand-set launchdarkly.graph.key is gone, and every adapter ends its span in a finally. Tested at 1cb2231: yarn build, yarn test (1,174 tests, exit 0), yarn typecheck and biome check . all clean.
Nothing here blocks the merge. Two things I'd like settled before GA. Python has both of them too:
- Write the graph identity into
TESTING.md. The monorepo spec (main at c2cca4f) doesn't say that thelaunchdarkly.graphspan carrieslaunchdarkly.config.key, the run id and thefeature_flagevent. It also doesn't say whichconfigKeygraph-level events use. Both PRs now agree on the graph key, so now is a good time to write it down, before the next change in either SDK drifts. I didn't find a monorepo PR for it. - Graph-level events report a different variation depending on the runner.
graph()sends the graph flag's realvariationKeyandversion(graph.ts:133-142). The native adapters now sendvariationKey: ''andversion: 1(tracking.ts:133), where before this PR they sent the root node's. So one graph flag's$ld:ai:graph:invocation_successcarries a different variation depending on which runner sent it, and per-variation views of native runs come up empty. Before 1.0, either expose the graph flag's meta onGraphDefinitionor write this down in the spec as a known gap.
Smaller notes:
makeGraphTrackDatais still exported from the root (index.ts:44). The@internaltag is only documentation: the repo has nostripInternal, and the name is in the builtdist/index.d.ts. Python #121 has since dropped it from__all__(d693d68). JS can't do that yet, because the clientpackage.jsononly exports., so I'm fine with it merging as is. I'll add it to the 1.0 trim list so it moves to the internal entry point withmakeNodeTrackData.- Every
graph()run with the same context object gets the samelaunchdarkly.run.id. I reproduced it: twoinvoke()calls on onegraph()with the same context gave one run id on both spans and bothinvocation_successevents. The cause is thatrunIdis created inbuildGraph(graph.ts:134) and the build is cached per context (graph.ts:636-643). That predates this PR for events, but now it's on the span too. Python caches the same way (graph.py:898-909); I haven't run it there. The run id is what ties a trace to its events, so I'd fix this in both SDKs before GA. - Setup errors are handled differently in the two SDKs. Here a setup error (for example "Root agent ... was not built") ends the span with status UNSET, no recorded exception and no
invocation_failure. Python #121 opens the span only after setup, so it exports no span at all. Either approach is fine, but they should match.
| export { compose, globalRegistry, Registry } from './registry.js'; | ||
| export { registerAiSdkPackage } from './sdk-info.js'; | ||
| export { makeNodeTrackData, makeRunTrackData } from './tracking.js'; | ||
| export { makeGraphTrackData, makeNodeTrackData, makeRunTrackData } from './tracking.js'; |
There was a problem hiding this comment.
@internal doesn't keep this out of the public surface. There's no stripInternal in the tsconfigs, and makeGraphTrackData is in the built dist/index.d.ts. Python #121 dropped make_graph_track_data from __all__ in d693d68. JS can't do that until the client has an internal entry point (the exports map only has .), so this is fine for now. I'll add it to the 1.0 trim with makeNodeTrackData.
| export const makeGraphTrackData = (graphKey: string, runId: string): TrackData => ({ | ||
| runId, | ||
| configKey: graphKey, | ||
| variationKey: '', |
There was a problem hiding this comment.
Graph-level events from the native adapters now carry variationKey: '' and version: 1. Before this PR they carried the root node's variation, and graph() sends the graph flag's real ones (graph.ts:133-142). So the same event from the same graph flag reports a different variation depending on the runner. The doc comment explains why (GraphDefinition has no _ldMeta). Can we either add the meta before 1.0 or write the gap into TESTING.md? Python's make_graph_track_data has the same issue.
|
|
||
| const span = trace.getTracer('@launchdarkly/ai-server').startSpan('launchdarkly.graph', undefined, callerContext); | ||
| span.setAttribute('launchdarkly.graph.key', key); | ||
| setLdSpanAttributes(span, { __ld: graphTrackData, ldContext: context }); |
There was a problem hiding this comment.
graphTrackData.runId is created once in buildGraph (line 134), and the build is cached per context reference (lines 636-643). That means every run with the same context object reuses the same launchdarkly.run.id on this span. I checked with a scratch test: two invoke() calls with one context gave identical run ids on both launchdarkly.graph spans and both invocation_success events. The events behaved this way before this PR. Could the run id be made per run, the way the native adapters do it? Python looks the same (graph.py:898-909), but I haven't run it.
| try { | ||
| const startTime = Date.now(); | ||
| const runId = crypto.randomUUID(); | ||
| setLdSpanAttributes(span, { __ld: makeGraphTrackData(def.key, runId), ldContext }); |
There was a problem hiding this comment.
Since the setup code is inside the try, a setup error (for example "Root agent ... was not built") now ends and exports the span, which is good. But the span has status UNSET, no recorded exception, and no $ld:ai:graph:invocation_failure. Python #121 does it differently: it opens the span only after setup (d693d68), so a setup error produces no span at all. Either is fine, but the SDKs should match. The same applies to claude-agents and langchain-agents.
2 participants
Summary
Spans from the native graph adapters (
toOpenAIAgents,toClaudeAgents,toLangGraph,toVercelAgents) had nothing tying them to the graph's AI Config, so Monitoring couldn't find them by config key.launchdarkly.graphspan now getssetLdSpanAttributes, like every other handler: config key, run id, context keys and thefeature_flagevent. New helper:makeGraphTrackData.$ld:ai:graph:invocation_successand friends) now use the graph key asconfigKey, matching the span andgraph(). They used the root node's key.graph()runner's ownlaunchdarkly.graphspan gets the same identity.finally.toLangGraphnever ended it, andtoOpenAIAgents/toClaudeAgentsleft it open on setup errors.makeGraphTrackDatais marked@internal.Python twin: launchdarkly/python-ai-sdk#121.
Testing
🤖 Generated with Claude Code
Note
Overview
Native graph runners and the built-in
graph()client now taglaunchdarkly.graphspans with the same LaunchDarkly identity as other handlers viasetLdSpanAttributesand a newmakeGraphTrackDatahelper, so AI Config Monitoring can correlate traces to the graph flag (config key, run id, context keys,feature_flagevent with environment id).Graph-level
$ld:ai:graph:*events (success, failure, duration, tokens) now use the graph key asconfigKeyinstead of the root node.environmentIdis attached to graph- and node-level track payloads throughtryGetEnvironmentId.All native adapters (
toClaudeAgents,toLangGraph,toOpenAIAgents,toVercelAgents) end the graph span in afinallyblock so spans export on setup errors and LangGraph no longer leaks open spans.Reviewed by Cursor Bugbot for commit 1cb2231. Bugbot is set up for automated code reviews on this repo. Configure here.