feat(byoo-perf): add OTLP sink and telemetrygen load generation - #624
Open
shobham-nv wants to merge 2 commits into
Open
feat(byoo-perf): add OTLP sink and telemetrygen load generation#624shobham-nv wants to merge 2 commits into
shobham-nv wants to merge 2 commits into
Conversation
Add the in-cluster OTLP sink (pkg/sink) and telemetrygen load generator (pkg/loadgen), and wire `perf run` end-to-end: deploy the sink, deploy the authentic collector with its export redirected at the sink (backed by an export-credentials Secret over the collector secrets volume), drive load at the profile's rates, then clean up. - pkg/sink: stock collector-contrib sink accepting OTLP over gRPC/HTTP, discarding via the debug exporter, exposing receiver counters on a Prometheus endpoint, and gating readiness with health_check. - pkg/loadgen: single-shot telemetrygen Jobs (backoffLimit 0) per enabled signal targeting the collector receiver. - pkg/labels: shared labels so cleanup is scoped to suite-created objects. - pkg/deploy: DeploySink, RunLoad, export-credentials Secret + secrets-volume override, and Cleanup extended to jobs/configmaps/secrets. - Unit tests for all new packages and the run flag surface; make perf-test passes. Measurement and reporting land in the next milestone; the sink already exposes its counters for it to read. Signed-off-by: shobham <shobham@nvidia.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
--mode k3d (the default) now provisions a dedicated local k3d cluster, runs the suite against it, and deletes it afterwards (unless --retain, which keeps the resources and the cluster). --mode remote keeps the previous behavior of using the ambient kubeconfig/context. - pkg/k3d: thin, unit-testable wrapper over the k3d CLI (create/delete/list/ image-import) with an injectable command runner. Reuses an existing cluster of the same name so reruns are cheap. - run: provisions/tears down the cluster around the shape loop and points the deploy client at the k3d-<name> context. New flags --k3d-cluster and --import-images (load collector/sink/loadgen images from local Docker for non-pullable/local builds). Signed-off-by: shobham <shobham@nvidia.com>
4 tasks
sbaum1994
approved these changes
Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
Adds load generation and an in-cluster OTLP sink to the BYOO collector performance suite, and wires
perf runend-to-end.runnow deploys a sink, deploys the authentic collector with its export redirected at that sink, and drivestelemetrygenload at the selected profile's rates before cleaning up. Stacked on #623.Additional Details
The suite must drive real telemetry through the collector and let it drain, otherwise the collector backs up against the unreachable placeholder export endpoints and the numbers are meaningless. This PR adds the two building blocks and the wiring:
pkg/sink— a stockcollector-contribsink that accepts OTLP over gRPC (4317) and HTTP (4318), discards payloads via thedebugexporter, exposes itsotelcol_receiver_accepted_*counters on a Prometheus telemetry endpoint (8888), and gates readiness withhealth_check(13133). This is the destination the collector under test exports to.pkg/loadgen— single-shottelemetrygenJobs (one per enabled signal,backoffLimit: 0so a retry never replays load) that send OTLP into the collector receiver at the profile's rates forwarmup + window.pkg/labels— shared labels sodeploy,sink, andloadgentag objects identically and cleanup stays scoped.pkg/deploy—DeploySink,RunLoad(create + wait for completion), an export-credentialsSecretthat backs the collector's/etc/byoo-otel-collector/secretsvolume (so the generated${file:...}exporter config resolves and the collector can actually start), andCleanupextended to also remove jobs, config maps, and secrets.runredirects export viaProvider=OTEL_COLLECTOR+ endpoints pointed at the sink; new flags--sink-image,--loadgen-image,--skip-load.Limitations / to validate on a cluster
For the Reviewer
pkg/deploy/deploy.go(credentials Secret +mountSecretOverPath,RunLoad, broadenedCleanup) andcmd/perf/main.gorunShape.For QA
make perf-test(build + vet + unit tests, cluster-free) passes.perf run --shape container) should be exercised against a target cluster.Issues
NO-REF
Checklist