feat: enforce otel-only tracing in service and lambda scaffolding [DEV-590] - #205
Merged
Conversation
Removes the X-Ray/OTel branch entirely from lambda templates (handler and worker layers) so every generated lambda, including ones added to legacy X-Ray services, uses OTel — forcing eventual migration instead of letting new code perpetuate X-Ray. Also fixes serverless.yml layer ARN references to point at the shared config/sls/layers.yml SSoT instead of hardcoded, stale ARNs, and runs `mage otel:fix` after scaffolding a new service so its otel-layer collector config and environment.yml otel section stay in sync with the monorepo's source of truth. Also bumps go.mod/go.sum dependencies (aws-sdk-go-v2, sentry-go, testify, x/crypto, huh, and related transitive deps) to go 1.25, and updates the pre-commit hook's pinned go.mod version to match.
negarciacamilo
marked this pull request as ready for review
July 23, 2026 15:08
…template OPENTELEMETRY_COLLECTOR_CONFIG_URI is one of the vars mage otel:fix injects into the otel:begin/otel:end section from the monorepo's SSoT (config/sls/otel-environment.yml). Hardcoding it in the template outside that section was redundant and would just get relocated on the first mage otel:fix run.
…ventions - Remove spans from ProvideResources and idempotency helpers (GenerateIdempotencyKey, SetIdempotency, DelIdempotency, deleteIdempotency) across all lambda types — per agents/observability.md §4/§4.5/§8, DI providers and private helpers must never be traced (orphaned root traces, cached singletons, no observability value), and >1 StartSpan per file is a red flag. - http worker.go.tmpl: propagate the span-carrying context back onto the echo request (ec.SetRequest) so any downstream call the developer adds inherits the handler span instead of getting a parentless one. - sqs worker.go.tmpl: keep the span-derived context available (was discarded) for consistency with the other lambda types. - Replace logger.Get() with log.Get().*Context() in idempotency error/panic paths so logs correlate with the active trace (dd.trace_id/dd.span_id), per §6. Also fixes the log package import path (pkg/log doesn't exist; the real package is projects/framev2/pkg/log) and a stray unresolved newCtx reference. - Remove a pre-existing unused log import in sqs worker.go.tmpl.
- domains providers/service.go.tmpl and providers/postgres/repository.go.tmpl: replace legacy logger.Get() with log.Get() (per §6, no *Context variant available here since these Provide functions don't receive a ctx). - service/postgres/search_one.go.tmpl: fix span operation name from snake_case "search_one" to kebab-case "search-one" per §4.3. Verified against real domains (searchsynonyms, betsettlements): service layer tracing (ServiceSpanName + LayerService) and repository layer being untraced (driver auto-instruments) already matched the established pattern — no further changes needed there. Dynamo domain scaffolding intentionally stays a bare skeleton (no generated CRUD methods), consistent with existing docs.
providers/generator.go.tmpl, providers/service.go.tmpl, and providers/postgres/repository.go.tmpl were loaded and unit-tested but never actually written to disk by newdomain/exec.go — the real DI files (service/provide.go, repository/provide.go) already follow the standardized sync.Once + providers.MustAs pattern used across real domains (verified against domains/betsettlements). Only providers/generators/nanoid/tableid, which is genuinely written to pkg/providers/generators/nanoid/tableid/, is kept.
…ml with real services - sqs/snssqs idempotency.go.tmpl: GenerateIdempotencyKey referenced base.CtxKeyRootID without importing pkg/consumer/base — generated code didn't compile. assertGoSyntax only parses syntax, so this slipped through. - Remove serverless-plugin-warmup end to end (dto field, form prompt, serverless.yml.tmpl, package.json.tmpl, tests) — 0 of 41 real services use it, it was dead scaffolding. - serverless.yml.tmpl: add the custom.group ([service, otel]) block, present in every real OTel service and absent from every X-Ray one; add serverless-plugin-lambda-deployments + custom.deploymentSettings, used by 38/41 services regardless of tracing mode. - package.json.tmpl: drop serverless-plugin-datadog — per observability.md rule 3, OTel services use the Datadog Extension layer directly and this plugin is legacy-X-Ray-only; its presence in many real services' package.json is migration debt, not the target state for a newly scaffolded service.
danteay
approved these changes
Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Removes the X-Ray tracing fallback from lambda and service scaffolding so every generated lambda and service is wired for OpenTelemetry only, including new lambdas added to existing legacy X-Ray services — this forces migration instead of letting new code perpetuate X-Ray. Also fixes stale/hardcoded otel-layer ARNs and a missing
serverless.ymlblock, aligns the generated OTel spans and logging with the monorepo'sobservability.mdconventions, removes a chunk of dead domain-scaffolding code found along the way, and drops a broken/unused plugin while adding one that was missing.Task Context
What is the current behavior?
new:lambdadetected whether the target service was already on OTel or X-Ray and generated the matching tracing code, meaning a legacy X-Ray service kept getting new X-Ray-instrumented lambdas indefinitely.workerlayer (resources.go,worker.go,idempotency.go) always used the legacy X-Ray tracer regardless of the service's tracing mode — this branch was never actually wired to OTel.serverless.yml.tmplhardcoded the OTel collector and Datadog extension layer ARNs directly instead of referencing the sharedconfig/sls/layers.yml, and those hardcoded ARNs were already stale. It also lacked thecustom.groupmetadata block andserverless-plugin-lambda-deployments, both present in nearly every real service.collector.yamland duplicated OTel-managed env vars inenvironment.yml, both of which can drift from the monorepo's SSoT.Providefunctions and private idempotency helpers, and worker spans in HTTP lambdas discarded their context — all against the monorepo'sobservability.mdconventions. The SQS/SNS+SQS idempotency template also referenced an undeclaredbasepackage, so generated code didn't compile (only caught by manual review — the test suite only checks template syntax, not full compilation).logger.Get()instead oflog.Get(), breaking log↔trace correlation; the same legacy logger was used in domain provider files.new:domaincomputed and unit-tested a wholeproviders/package (generator, service, postgres/repository) that was never actually written to disk — pure dead code.new:serviceoffered a "warmup" plugin option that 0 of 41 real services use, andpackage.json.tmplalways installedserverless-plugin-datadog, whichobservability.mdsays is legacy-X-Ray-only and should not be present for OTel services.What is the new behavior?
UseOteldetection/flag have been removed entirely.serverless.yml.tmplnow references the sharedconfig/sls/layers.ymlfor ARNs, adds thecustom.group/deploymentSettingsblocks andserverless-plugin-lambda-deployments, matching real services;new:servicerunsmage otel:fixpost-create so the collector config and environment section stay synced with the monorepo's source of truth.Providefunctions and idempotency helpers (handler/worker spans remain, since those are real entry points); HTTP worker spans now propagate their context onto the echo request so downstream calls nest correctly; fixed the missingbasepackage import so SQS/SNS+SQS lambdas actually compile.logger.Get()withlog.Get()(trace-correlated*Contextvariants) everywhere the legacy logger was used, in both lambda and domain templates; fixed the log package import path and a stray broken variable reference along the way; fixed a snake_case span operation name to kebab-case.providers/package templates and their loaders/tests — verified against a real domain (betsettlements) that the actually-generatedservice/provide.go+repository/provide.goalready match the standardized DI pattern used across the monorepo.serverless-plugin-datadogfrompackage.json.tmpl.Additional Context
Two other legacy X-Ray branches exist (
feat/otel-support,feat/otel-scaffolding-support) that appear superseded by work already merged into this branch.