Skip to content

feat(byoo-perf): add OTLP sink and telemetrygen load generation - #624

Open
shobham-nv wants to merge 2 commits into
shobham/418-byoo-perf-deployfrom
shobham/419-byoo-perf-loadgen-sink
Open

feat(byoo-perf): add OTLP sink and telemetrygen load generation#624
shobham-nv wants to merge 2 commits into
shobham/418-byoo-perf-deployfrom
shobham/419-byoo-perf-loadgen-sink

Conversation

@shobham-nv

Copy link
Copy Markdown
Contributor

TL;DR

Adds load generation and an in-cluster OTLP sink to the BYOO collector performance suite, and wires perf run end-to-end. run now deploys a sink, deploys the authentic collector with its export redirected at that sink, and drives telemetrygen load at the selected profile's rates before cleaning up. Stacked on #623.

Additional Details

The suite must drive real telemetry through the collector and let it drain, otherwise the collector backs up against the unreachable placeholder export endpoints and the numbers are meaningless. This PR adds the two building blocks and the wiring:

  • pkg/sink — a stock collector-contrib sink that accepts OTLP over gRPC (4317) and HTTP (4318), discards payloads via the debug exporter, exposes its otelcol_receiver_accepted_* counters on a Prometheus telemetry endpoint (8888), and gates readiness with health_check (13133). This is the destination the collector under test exports to.
  • pkg/loadgen — single-shot telemetrygen Jobs (one per enabled signal, backoffLimit: 0 so a retry never replays load) that send OTLP into the collector receiver at the profile's rates for warmup + window.
  • pkg/labels — shared labels so deploy, sink, and loadgen tag objects identically and cleanup stays scoped.
  • pkg/deployDeploySink, RunLoad (create + wait for completion), an export-credentials Secret that backs the collector's /etc/byoo-otel-collector/secrets volume (so the generated ${file:...} exporter config resolves and the collector can actually start), and Cleanup extended to also remove jobs, config maps, and secrets.
  • run redirects export via Provider=OTEL_COLLECTOR + endpoints pointed at the sink; new flags --sink-image, --loadgen-image, --skip-load.

Limitations / to validate on a cluster

  • End-to-end startup was not run against a live cluster from here. The exporter-credentials + secrets-volume override and the sink/telemetrygen image tags are the first things to confirm on a real cluster.
  • Measurement and reporting are the next milestone; the sink already exposes its receiver counters for it to read.

For the Reviewer

  • Closest look: pkg/deploy/deploy.go (credentials Secret + mountSecretOverPath, RunLoad, broadened Cleanup) and cmd/perf/main.go runShape.
  • Sink/loadgen image defaults are stock upstream images and overridable via flags.

For QA

  • make perf-test (build + vet + unit tests, cluster-free) passes.
  • Manual end-to-end (perf run --shape container) should be exercised against a target cluster.

Issues

NO-REF

Checklist

  • I am familiar with the Contributing Guidelines.
  • I have signed off my commits for Developer Certificate of Origin (DCO) compliance.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

Add the in-cluster OTLP sink (pkg/sink) and telemetrygen load generator
(pkg/loadgen), and wire `perf run` end-to-end: deploy the sink, deploy the
authentic collector with its export redirected at the sink (backed by an
export-credentials Secret over the collector secrets volume), drive load at
the profile's rates, then clean up.

- pkg/sink: stock collector-contrib sink accepting OTLP over gRPC/HTTP,
  discarding via the debug exporter, exposing receiver counters on a
  Prometheus endpoint, and gating readiness with health_check.
- pkg/loadgen: single-shot telemetrygen Jobs (backoffLimit 0) per enabled
  signal targeting the collector receiver.
- pkg/labels: shared labels so cleanup is scoped to suite-created objects.
- pkg/deploy: DeploySink, RunLoad, export-credentials Secret + secrets-volume
  override, and Cleanup extended to jobs/configmaps/secrets.
- Unit tests for all new packages and the run flag surface; make perf-test
  passes.

Measurement and reporting land in the next milestone; the sink already
exposes its counters for it to read.

Signed-off-by: shobham <shobham@nvidia.com>
@shobham-nv
shobham-nv requested a review from a team as a code owner August 3, 2026 08:46
@shobham-nv
shobham-nv requested a review from apartha-nv August 3, 2026 08:46
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)
  • main

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8127bee9-b879-46a0-a4c2-97169c632e38

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

--mode k3d (the default) now provisions a dedicated local k3d cluster, runs
the suite against it, and deletes it afterwards (unless --retain, which keeps
the resources and the cluster). --mode remote keeps the previous behavior of
using the ambient kubeconfig/context.

- pkg/k3d: thin, unit-testable wrapper over the k3d CLI (create/delete/list/
  image-import) with an injectable command runner. Reuses an existing cluster
  of the same name so reruns are cheap.
- run: provisions/tears down the cluster around the shape loop and points the
  deploy client at the k3d-<name> context. New flags --k3d-cluster and
  --import-images (load collector/sink/loadgen images from local Docker for
  non-pullable/local builds).

Signed-off-by: shobham <shobham@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants