feat(stats)!: add telemetry for per-field cardinality limits by Eldolfin · Pull Request #2227 · DataDog/libdatadog

Eldolfin · 2026-07-10T15:15:12Z

What does this PR do?

Send telemetry about per-field cardinality collapses. But unlike #2212, now using a single count metric with all the offending fields in the bucket instead of separate count metrics.

Motivation

What inspired you to submit this pull request?

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Describe here in detail how the change can be validated.

datadog-datadog-prod-us1-2 · 2026-07-10T15:17:04Z

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 97.48%
• Overall Coverage: 74.73% (+0.02%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 69abc54 | Docs | Datadog PR Page | Give us feedback!}

github-actions · 2026-07-10T15:19:01Z

📚 Documentation Check Results

⚠️ 1857 documentation warning(s) found

📦 `datadog-ipc` - 981 warning(s)

📦 `libdd-trace-stats` - 876 warning(s)

Updated: 2026-07-23 09:53:15 UTC | Commit: fb758f0 | missing-docs job results

github-actions · 2026-07-10T15:19:23Z

🔒 Cargo Deny Results

⚠️ 5 issue(s) found, showing only errors (advisories, bans, sources)

📦 `datadog-ipc` - 3 error(s)

Show output

error[unmaintained]: Bincode is unmaintained
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:25:1
   │
25 │ bincode 1.3.3 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unmaintained advisory detected
   │
   ├ ID: RUSTSEC-2025-0141
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2025-0141
   ├ Due to a doxxing and harassment incident, the bincode team has taken the decision to cease development permanently.
     
     The team considers version 1.3.3 a complete version of bincode that is not in need of any updates.
     
     ## Alternatives to consider
     
     * [wincode](https://crates.io/crates/wincode)
     * [postcard](https://crates.io/crates/postcard)
     * [bitcode](https://crates.io/crates/bitcode)
     * [rkyv](https://crates.io/crates/rkyv)
   ├ Announcement: https://git.sr.ht/~stygianentity/bincode/tree/v3.0/item/README.md
   ├ Solution: No safe upgrade is available!
   ├ bincode v1.3.3
     └── datadog-ipc v0.1.0

error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:66:1
   │
66 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── (dev) datadog-ipc v0.1.0
                     ├── libdd-common v5.1.0
                     │   ├── datadog-ipc v0.1.0 (*)
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │       └── datadog-ipc v0.1.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── datadog-ipc v0.1.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:205:1
    │
205 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── (dev) libdd-common v5.1.0
      │   ├── datadog-ipc v0.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │       └── datadog-ipc v0.1.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── datadog-ipc v0.1.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── datadog-ipc v0.1.0 (*)
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

advisories FAILED, bans ok, sources ok

📦 `libdd-trace-stats` - 2 error(s)

Show output

error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:66:1
   │
66 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.1.0
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   ├── libdd-telemetry v6.0.0
                     │   │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-dogstatsd-client v4.0.0
                     │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:198:1
    │
198 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   ├── libdd-telemetry v6.0.0
      │   │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-dogstatsd-client v4.0.0
      │   │   └── libdd-trace-stats v6.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── libdd-telemetry v6.0.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

advisories FAILED, bans ok, sources ok

Updated: 2026-07-23 09:54:52 UTC | Commit: fb758f0 | dependency-check job results

pr-commenter · 2026-07-21T12:15:22Z

Benchmarks

Comparison

Benchmark execution time: 2026-07-23 10:02:51

Comparing candidate commit 69abc54 in PR branch oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator with baseline commit 85c73e9 in branch oscarld/stats-per-key-cardinality-limit.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Benchmark execution time: 2026-07-23 10:00:15

Comparing candidate commit 69abc54 in PR branch oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator with baseline commit 85c73e9 in branch oscarld/stats-per-key-cardinality-limit.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Candidate

Candidate benchmark details

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`69abc54`	1784800260	oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
trace_buffer/1_senders/10us_delay	execution_time	56.290ms	56.532ms ± 0.120ms	56.518ms ± 0.062ms	56.587ms	56.738ms	56.991ms	57.084ms	1.00%	1.618	4.445	0.21%	0.008ms	1	200
trace_buffer/1_senders/10us_delay	throughput	15766.152op/s	15920.224op/s ± 33.595op/s	15924.042op/s ± 17.504op/s	15939.256op/s	15960.922op/s	15972.237op/s	15988.762op/s	0.41%	-1.594	4.338	0.21%	2.376op/s	1	200
trace_buffer/1_senders/1us_delay	execution_time	50.029ms	50.091ms ± 0.035ms	50.083ms ± 0.013ms	50.108ms	50.151ms	50.194ms	50.201ms	0.24%	0.787	0.553	0.07%	0.002ms	1	200
trace_buffer/1_senders/1us_delay	throughput	17927.757op/s	17967.131op/s ± 12.473op/s	17970.009op/s ± 4.778op/s	17974.186op/s	17985.581op/s	17989.583op/s	17989.719op/s	0.11%	-0.783	0.546	0.07%	0.882op/s	1	200
trace_buffer/1_senders/no_delay	execution_time	334.638µs	339.794µs ± 4.270µs	339.238µs ± 1.527µs	340.885µs	344.510µs	351.373µs	387.186µs	14.13%	7.107	74.446	1.25%	0.302µs	1	200
trace_buffer/1_senders/no_delay	throughput	2324466.058op/s	2649045.545op/s ± 30734.359op/s	2653004.979op/s ± 11994.690op/s	2664163.163op/s	2678398.244op/s	2685863.365op/s	2689472.783op/s	1.37%	-6.194	60.893	1.16%	2173.247op/s	1	200
trace_buffer/2_senders/10us_delay	execution_time	56.562ms	56.960ms ± 0.219ms	56.907ms ± 0.143ms	57.100ms	57.382ms	57.594ms	57.632ms	1.27%	0.854	0.477	0.38%	0.016ms	1	200
trace_buffer/2_senders/10us_delay	throughput	31232.783op/s	31601.851op/s ± 121.257op/s	31630.610op/s ± 79.707op/s	31691.798op/s	31754.621op/s	31785.839op/s	31823.588op/s	0.61%	-0.834	0.430	0.38%	8.574op/s	1	200
trace_buffer/2_senders/1us_delay	execution_time	50.056ms	50.117ms ± 0.037ms	50.111ms ± 0.020ms	50.132ms	50.186ms	50.234ms	50.262ms	0.30%	1.255	2.006	0.07%	0.003ms	1	200
trace_buffer/2_senders/1us_delay	throughput	35812.563op/s	35915.976op/s ± 26.401op/s	35920.332op/s ± 14.441op/s	35934.252op/s	35947.456op/s	35958.300op/s	35959.588op/s	0.11%	-1.250	1.988	0.07%	1.867op/s	1	200
trace_buffer/2_senders/no_delay	execution_time	845.553µs	890.853µs ± 17.795µs	889.375µs ± 13.312µs	903.244µs	918.726µs	930.294µs	945.807µs	6.35%	0.073	-0.223	1.99%	1.258µs	1	200
trace_buffer/2_senders/no_delay	throughput	1903136.686op/s	2021338.311op/s ± 40359.729op/s	2023894.170op/s ± 30012.872op/s	2049199.106op/s	2089788.909op/s	2112973.948op/s	2128784.623op/s	5.18%	0.033	-0.238	1.99%	2853.864op/s	1	200
trace_buffer/4_senders/10us_delay	execution_time	57.094ms	57.425ms ± 0.155ms	57.419ms ± 0.080ms	57.502ms	57.668ms	57.771ms	58.326ms	1.58%	1.385	6.039	0.27%	0.011ms	1	200
trace_buffer/4_senders/10us_delay	throughput	61722.266op/s	62690.404op/s ± 168.596op/s	62697.522op/s ± 87.263op/s	62781.797op/s	62920.323op/s	62987.571op/s	63053.610op/s	0.57%	-1.336	5.749	0.27%	11.922op/s	1	200
trace_buffer/4_senders/1us_delay	execution_time	50.115ms	50.168ms ± 0.036ms	50.161ms ± 0.018ms	50.180ms	50.250ms	50.285ms	50.324ms	0.33%	1.484	2.507	0.07%	0.003ms	1	200
trace_buffer/4_senders/1us_delay	throughput	71537.003op/s	71758.283op/s ± 51.379op/s	71769.582op/s ± 25.300op/s	71791.061op/s	71821.658op/s	71829.208op/s	71834.968op/s	0.09%	-1.479	2.487	0.07%	3.633op/s	1	200
trace_buffer/4_senders/no_delay	execution_time	2.186ms	2.396ms ± 0.077ms	2.397ms ± 0.061ms	2.455ms	2.518ms	2.546ms	2.576ms	7.48%	-0.024	-0.695	3.22%	0.005ms	1	200
trace_buffer/4_senders/no_delay	throughput	1397245.341op/s	1504167.379op/s ± 48704.318op/s	1501772.061op/s ± 39163.123op/s	1541552.433op/s	1576621.021op/s	1603653.867op/s	1647066.951op/s	9.67%	0.152	-0.620	3.23%	3443.915op/s	1	200
trace_buffer/8_senders/10us_delay	execution_time	57.531ms	57.824ms ± 0.181ms	57.767ms ± 0.089ms	57.927ms	58.198ms	58.297ms	58.384ms	1.07%	1.027	0.335	0.31%	0.013ms	1	200
trace_buffer/8_senders/10us_delay	throughput	123320.634op/s	124517.303op/s ± 388.096op/s	124638.502op/s ± 191.497op/s	124799.013op/s	124983.403op/s	125034.411op/s	125149.945op/s	0.41%	-1.015	0.308	0.31%	27.442op/s	1	200
trace_buffer/8_senders/1us_delay	execution_time	50.203ms	50.293ms ± 0.054ms	50.278ms ± 0.030ms	50.319ms	50.398ms	50.490ms	50.559ms	0.56%	1.558	3.724	0.11%	0.004ms	1	200
trace_buffer/8_senders/1us_delay	throughput	142407.246op/s	143161.122op/s ± 153.569op/s	143204.431op/s ± 84.329op/s	143273.328op/s	143324.361op/s	143365.258op/s	143418.564op/s	0.15%	-1.548	3.669	0.11%	10.859op/s	1	200
trace_buffer/8_senders/no_delay	execution_time	6.067ms	6.487ms ± 0.119ms	6.514ms ± 0.074ms	6.572ms	6.634ms	6.682ms	6.725ms	3.23%	-1.033	1.465	1.83%	0.008ms	1	200
trace_buffer/8_senders/no_delay	throughput	1070663.918op/s	1110367.443op/s ± 20841.269op/s	1105239.046op/s ± 12550.943op/s	1121408.012op/s	1146291.046op/s	1181096.224op/s	1186679.262op/s	7.37%	1.170	1.897	1.87%	1473.700op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
trace_buffer/1_senders/10us_delay	execution_time	[56.516ms; 56.549ms] or [-0.029%; +0.029%]	None	None	None
trace_buffer/1_senders/10us_delay	throughput	[15915.568op/s; 15924.880op/s] or [-0.029%; +0.029%]	None	None	None
trace_buffer/1_senders/1us_delay	execution_time	[50.087ms; 50.096ms] or [-0.010%; +0.010%]	None	None	None
trace_buffer/1_senders/1us_delay	throughput	[17965.402op/s; 17968.859op/s] or [-0.010%; +0.010%]	None	None	None
trace_buffer/1_senders/no_delay	execution_time	[339.202µs; 340.386µs] or [-0.174%; +0.174%]	None	None	None
trace_buffer/1_senders/no_delay	throughput	[2644786.059op/s; 2653305.032op/s] or [-0.161%; +0.161%]	None	None	None
trace_buffer/2_senders/10us_delay	execution_time	[56.929ms; 56.990ms] or [-0.053%; +0.053%]	None	None	None
trace_buffer/2_senders/10us_delay	throughput	[31585.046op/s; 31618.657op/s] or [-0.053%; +0.053%]	None	None	None
trace_buffer/2_senders/1us_delay	execution_time	[50.112ms; 50.122ms] or [-0.010%; +0.010%]	None	None	None
trace_buffer/2_senders/1us_delay	throughput	[35912.317op/s; 35919.635op/s] or [-0.010%; +0.010%]	None	None	None
trace_buffer/2_senders/no_delay	execution_time	[888.386µs; 893.319µs] or [-0.277%; +0.277%]	None	None	None
trace_buffer/2_senders/no_delay	throughput	[2015744.841op/s; 2026931.782op/s] or [-0.277%; +0.277%]	None	None	None
trace_buffer/4_senders/10us_delay	execution_time	[57.404ms; 57.447ms] or [-0.037%; +0.037%]	None	None	None
trace_buffer/4_senders/10us_delay	throughput	[62667.038op/s; 62713.770op/s] or [-0.037%; +0.037%]	None	None	None
trace_buffer/4_senders/1us_delay	execution_time	[50.163ms; 50.173ms] or [-0.010%; +0.010%]	None	None	None
trace_buffer/4_senders/1us_delay	throughput	[71751.162op/s; 71765.403op/s] or [-0.010%; +0.010%]	None	None	None
trace_buffer/4_senders/no_delay	execution_time	[2.385ms; 2.407ms] or [-0.447%; +0.447%]	None	None	None
trace_buffer/4_senders/no_delay	throughput	[1497417.429op/s; 1510917.329op/s] or [-0.449%; +0.449%]	None	None	None
trace_buffer/8_senders/10us_delay	execution_time	[57.799ms; 57.849ms] or [-0.043%; +0.043%]	None	None	None
trace_buffer/8_senders/10us_delay	throughput	[124463.517op/s; 124571.089op/s] or [-0.043%; +0.043%]	None	None	None
trace_buffer/8_senders/1us_delay	execution_time	[50.286ms; 50.301ms] or [-0.015%; +0.015%]	None	None	None
trace_buffer/8_senders/1us_delay	throughput	[143139.839op/s; 143182.406op/s] or [-0.015%; +0.015%]	None	None	None
trace_buffer/8_senders/no_delay	execution_time	[6.470ms; 6.503ms] or [-0.255%; +0.255%]	None	None	None
trace_buffer/8_senders/no_delay	throughput	[1107479.044op/s; 1113255.842op/s] or [-0.260%; +0.260%]	None	None	None

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`69abc54`	1784800260	oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
concentrator/add_spans_to_concentrator	execution_time	9.602ms	9.629ms ± 0.017ms	9.625ms ± 0.010ms	9.640ms	9.662ms	9.674ms	9.693ms	0.71%	0.957	0.715	0.17%	0.001ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
concentrator/add_spans_to_concentrator	execution_time	[9.627ms; 9.631ms] or [-0.024%; +0.024%]	None	None	None

Group 2

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`69abc54`	1784800260	oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
write only interface	execution_time	1.061µs	1.079µs ± 0.007µs	1.079µs ± 0.005µs	1.083µs	1.089µs	1.091µs	1.093µs	1.21%	-0.323	-0.330	0.60%	0.000µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
write only interface	execution_time	[1.078µs; 1.080µs] or [-0.084%; +0.084%]	None	None	None

Group 3

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`69abc54`	1784800260	oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
two way interface	execution_time	15.238µs	15.605µs ± 0.209µs	15.576µs ± 0.098µs	15.674µs	15.921µs	16.388µs	16.951µs	8.83%	2.493	11.306	1.34%	0.015µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
two way interface	execution_time	[15.576µs; 15.634µs] or [-0.186%; +0.186%]	None	None	None

Baseline

Baseline benchmark details

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`85c73e9`	1784718237	oscarld/stats-per-key-cardinality-limit

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
trace_buffer/1_senders/10us_delay	execution_time	56.336ms	56.541ms ± 0.116ms	56.527ms ± 0.066ms	56.598ms	56.771ms	56.922ms	57.068ms	0.96%	1.266	2.707	0.21%	0.008ms	1	200
trace_buffer/1_senders/10us_delay	throughput	15770.582op/s	15917.647op/s ± 32.644op/s	15921.607op/s ± 18.609op/s	15939.505op/s	15956.769op/s	15973.483op/s	15975.575op/s	0.34%	-1.247	2.631	0.20%	2.308op/s	1	200
trace_buffer/1_senders/1us_delay	execution_time	50.017ms	50.079ms ± 0.019ms	50.080ms ± 0.007ms	50.086ms	50.112ms	50.136ms	50.150ms	0.14%	0.602	2.876	0.04%	0.001ms	1	200
trace_buffer/1_senders/1us_delay	throughput	17946.105op/s	17971.450op/s ± 6.773op/s	17971.308op/s ± 2.361op/s	17975.110op/s	17981.791op/s	17988.538op/s	17994.020op/s	0.13%	-0.597	2.869	0.04%	0.479op/s	1	200
trace_buffer/1_senders/no_delay	execution_time	332.610µs	340.504µs ± 5.693µs	339.889µs ± 1.933µs	341.860µs	345.000µs	351.566µs	408.496µs	20.19%	8.692	100.210	1.67%	0.403µs	1	200
trace_buffer/1_senders/no_delay	throughput	2203202.150op/s	2643785.205op/s ± 38970.392op/s	2647920.805op/s ± 15107.907op/s	2662415.678op/s	2676512.048op/s	2681791.658op/s	2705875.705op/s	2.19%	-7.426	80.168	1.47%	2755.623op/s	1	200
trace_buffer/2_senders/10us_delay	execution_time	56.562ms	56.981ms ± 0.219ms	56.948ms ± 0.117ms	57.079ms	57.471ms	57.663ms	57.675ms	1.28%	1.043	1.057	0.38%	0.016ms	1	200
trace_buffer/2_senders/10us_delay	throughput	31209.363op/s	31589.943op/s ± 121.200op/s	31607.864op/s ± 65.050op/s	31672.339op/s	31751.353op/s	31785.976op/s	31823.701op/s	0.68%	-1.020	1.005	0.38%	8.570op/s	1	200
trace_buffer/2_senders/1us_delay	execution_time	50.052ms	50.110ms ± 0.030ms	50.107ms ± 0.016ms	50.121ms	50.168ms	50.216ms	50.230ms	0.25%	1.400	2.692	0.06%	0.002ms	1	200
trace_buffer/2_senders/1us_delay	throughput	35835.162op/s	35921.119op/s ± 21.444op/s	35922.980op/s ± 11.265op/s	35935.377op/s	35946.940op/s	35952.486op/s	35962.747op/s	0.11%	-1.395	2.674	0.06%	1.516op/s	1	200
trace_buffer/2_senders/no_delay	execution_time	851.366µs	911.170µs ± 18.846µs	911.884µs ± 9.919µs	921.626µs	936.931µs	945.832µs	995.748µs	9.20%	0.152	2.707	2.06%	1.333µs	1	200
trace_buffer/2_senders/no_delay	throughput	1807686.383op/s	1976323.328op/s ± 40865.112op/s	1973935.595op/s ± 21393.184op/s	1996844.434op/s	2045953.471op/s	2088509.454op/s	2114248.508op/s	7.11%	0.122	2.224	2.06%	2889.600op/s	1	200
trace_buffer/4_senders/10us_delay	execution_time	57.087ms	57.453ms ± 0.156ms	57.423ms ± 0.085ms	57.535ms	57.777ms	57.879ms	58.024ms	1.05%	0.815	0.797	0.27%	0.011ms	1	200
trace_buffer/4_senders/10us_delay	throughput	62043.277op/s	62660.636op/s ± 169.559op/s	62692.586op/s ± 92.928op/s	62773.754op/s	62903.235op/s	62953.779op/s	63061.349op/s	0.59%	-0.798	0.763	0.27%	11.990op/s	1	200
trace_buffer/4_senders/1us_delay	execution_time	50.081ms	50.174ms ± 0.042ms	50.162ms ± 0.019ms	50.193ms	50.256ms	50.297ms	50.321ms	0.32%	1.094	1.224	0.08%	0.003ms	1	200
trace_buffer/4_senders/1us_delay	throughput	71541.272op/s	71749.797op/s ± 59.411op/s	71767.158op/s ± 26.631op/s	71786.465op/s	71818.794op/s	71842.105op/s	71883.974op/s	0.16%	-1.089	1.212	0.08%	4.201op/s	1	200
trace_buffer/4_senders/no_delay	execution_time	2.101ms	2.342ms ± 0.080ms	2.340ms ± 0.056ms	2.396ms	2.470ms	2.511ms	2.597ms	10.97%	0.021	0.172	3.40%	0.006ms	1	200
trace_buffer/4_senders/no_delay	throughput	1386074.761op/s	1538812.855op/s ± 52665.755op/s	1538163.043op/s ± 35796.758op/s	1574277.579op/s	1616915.918op/s	1674743.046op/s	1713853.986op/s	11.42%	0.203	0.265	3.41%	3724.031op/s	1	200
trace_buffer/8_senders/10us_delay	execution_time	57.592ms	57.873ms ± 0.209ms	57.806ms ± 0.135ms	58.038ms	58.265ms	58.390ms	58.419ms	1.06%	0.668	-0.649	0.36%	0.015ms	1	200
trace_buffer/8_senders/10us_delay	throughput	123247.920op/s	124412.906op/s ± 449.034op/s	124554.055op/s ± 291.945op/s	124805.874op/s	124926.075op/s	125010.185op/s	125017.908op/s	0.37%	-0.658	-0.667	0.36%	31.751op/s	1	200
trace_buffer/8_senders/1us_delay	execution_time	50.199ms	50.291ms ± 0.059ms	50.278ms ± 0.031ms	50.318ms	50.386ms	50.489ms	50.544ms	0.53%	1.431	2.678	0.12%	0.004ms	1	200
trace_buffer/8_senders/1us_delay	throughput	142450.796op/s	143165.634op/s ± 166.271op/s	143204.542op/s ± 88.203op/s	143276.384op/s	143354.913op/s	143386.920op/s	143429.846op/s	0.16%	-1.421	2.638	0.12%	11.757op/s	1	200
trace_buffer/8_senders/no_delay	execution_time	6.404ms	6.659ms ± 0.083ms	6.668ms ± 0.055ms	6.712ms	6.779ms	6.820ms	6.851ms	2.75%	-0.478	0.054	1.24%	0.006ms	1	200
trace_buffer/8_senders/no_delay	throughput	1050895.502op/s	1081483.571op/s ± 13564.688op/s	1079834.189op/s ± 8891.770op/s	1090356.685op/s	1105486.724op/s	1120796.098op/s	1124271.624op/s	4.12%	0.547	0.162	1.25%	959.168op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
trace_buffer/1_senders/10us_delay	execution_time	[56.525ms; 56.557ms] or [-0.028%; +0.028%]	None	None	None
trace_buffer/1_senders/10us_delay	throughput	[15913.123op/s; 15922.171op/s] or [-0.028%; +0.028%]	None	None	None
trace_buffer/1_senders/1us_delay	execution_time	[50.077ms; 50.082ms] or [-0.005%; +0.005%]	None	None	None
trace_buffer/1_senders/1us_delay	throughput	[17970.512op/s; 17972.389op/s] or [-0.005%; +0.005%]	None	None	None
trace_buffer/1_senders/no_delay	execution_time	[339.715µs; 341.293µs] or [-0.232%; +0.232%]	None	None	None
trace_buffer/1_senders/no_delay	throughput	[2638384.284op/s; 2649186.127op/s] or [-0.204%; +0.204%]	None	None	None
trace_buffer/2_senders/10us_delay	execution_time	[56.951ms; 57.011ms] or [-0.053%; +0.053%]	None	None	None
trace_buffer/2_senders/10us_delay	throughput	[31573.146op/s; 31606.740op/s] or [-0.053%; +0.053%]	None	None	None
trace_buffer/2_senders/1us_delay	execution_time	[50.106ms; 50.114ms] or [-0.008%; +0.008%]	None	None	None
trace_buffer/2_senders/1us_delay	throughput	[35918.147op/s; 35924.091op/s] or [-0.008%; +0.008%]	None	None	None
trace_buffer/2_senders/no_delay	execution_time	[908.558µs; 913.781µs] or [-0.287%; +0.287%]	None	None	None
trace_buffer/2_senders/no_delay	throughput	[1970659.816op/s; 1981986.839op/s] or [-0.287%; +0.287%]	None	None	None
trace_buffer/4_senders/10us_delay	execution_time	[57.431ms; 57.474ms] or [-0.038%; +0.038%]	None	None	None
trace_buffer/4_senders/10us_delay	throughput	[62637.137op/s; 62684.136op/s] or [-0.038%; +0.038%]	None	None	None
trace_buffer/4_senders/1us_delay	execution_time	[50.169ms; 50.180ms] or [-0.011%; +0.011%]	None	None	None
trace_buffer/4_senders/1us_delay	throughput	[71741.563op/s; 71758.031op/s] or [-0.011%; +0.011%]	None	None	None
trace_buffer/4_senders/no_delay	execution_time	[2.331ms; 2.353ms] or [-0.473%; +0.473%]	None	None	None
trace_buffer/4_senders/no_delay	throughput	[1531513.888op/s; 1546111.822op/s] or [-0.474%; +0.474%]	None	None	None
trace_buffer/8_senders/10us_delay	execution_time	[57.844ms; 57.902ms] or [-0.050%; +0.050%]	None	None	None
trace_buffer/8_senders/10us_delay	throughput	[124350.674op/s; 124475.138op/s] or [-0.050%; +0.050%]	None	None	None
trace_buffer/8_senders/1us_delay	execution_time	[50.283ms; 50.300ms] or [-0.016%; +0.016%]	None	None	None
trace_buffer/8_senders/1us_delay	throughput	[143142.590op/s; 143188.677op/s] or [-0.016%; +0.016%]	None	None	None
trace_buffer/8_senders/no_delay	execution_time	[6.647ms; 6.670ms] or [-0.173%; +0.173%]	None	None	None
trace_buffer/8_senders/no_delay	throughput	[1079603.636op/s; 1083363.506op/s] or [-0.174%; +0.174%]	None	None	None

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`85c73e9`	1784718237	oscarld/stats-per-key-cardinality-limit

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
concentrator/add_spans_to_concentrator	execution_time	9.547ms	9.582ms ± 0.016ms	9.580ms ± 0.011ms	9.593ms	9.611ms	9.627ms	9.636ms	0.59%	0.557	-0.005	0.17%	0.001ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
concentrator/add_spans_to_concentrator	execution_time	[9.580ms; 9.584ms] or [-0.024%; +0.024%]	None	None	None

Group 2

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`85c73e9`	1784718237	oscarld/stats-per-key-cardinality-limit

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
write only interface	execution_time	1.061µs	1.077µs ± 0.007µs	1.077µs ± 0.006µs	1.083µs	1.086µs	1.092µs	1.094µs	1.57%	-0.134	-0.881	0.65%	0.000µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
write only interface	execution_time	[1.076µs; 1.078µs] or [-0.091%; +0.091%]	None	None	None

Group 3

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	`85c73e9`	1784718237	oscarld/stats-per-key-cardinality-limit

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
two way interface	execution_time	14.987µs	15.687µs ± 0.209µs	15.657µs ± 0.086µs	15.751µs	15.995µs	16.220µs	17.583µs	12.30%	3.933	32.981	1.33%	0.015µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
two way interface	execution_time	[15.658µs; 15.716µs] or [-0.185%; +0.185%]	None	None	None

Eldolfin · 2026-07-21T12:24:22Z

@codex review

dd-octo-sts · 2026-07-21T12:37:21Z

Artifact Size Benchmark Report

aarch64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so	8.13 MB	8.13 MB	+0% (+48 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a	88.18 MB	88.30 MB	+.14% (+132.13 KB) 🔍

aarch64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so	10.93 MB	10.93 MB	+.03% (+4.25 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a	99.46 MB	99.59 MB	+.13% (+135.77 KB) 🔍

libdatadog-x64-windows

Artifact	Baseline	Commit	Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll	26.59 MB	26.62 MB	+.11% (+30.00 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib	89.18 KB	89.60 KB	+.47% (+430 B) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb	191.30 MB	191.44 MB	+.07% (+152.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib	1.04 GB	1.04 GB	+0% (+78.31 KB) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll	8.66 MB	8.67 MB	+.13% (+12.00 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib	89.18 KB	89.60 KB	+.47% (+430 B) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb	25.36 MB	25.39 MB	+.12% (+32.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib	50.55 MB	50.62 MB	+.13% (+69.18 KB) 🔍

libdatadog-x86-windows

Artifact	Baseline	Commit	Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll	23.16 MB	23.19 MB	+.12% (+30.50 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib	90.58 KB	91.00 KB	+.46% (+434 B) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb	195.94 MB	196.17 MB	+.11% (+232.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib	1.03 GB	1.03 GB	+.19% (+2.07 MB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll	6.68 MB	6.69 MB	+.18% (+13.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib	90.58 KB	91.00 KB	+.46% (+434 B) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb	27.26 MB	27.30 MB	+.14% (+40.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib	48.13 MB	48.19 MB	+.13% (+67.76 KB) 🔍

x86_64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a	78.64 MB	78.76 MB	+.14% (+118.42 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so	9.05 MB	9.07 MB	+.21% (+20.06 KB) 🔍

x86_64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a	94.14 MB	94.26 MB	+.12% (+124.96 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so	11.02 MB	11.03 MB	+.16% (+19.14 KB) 🔍

chatgpt-codex-connector · 2026-07-21T12:37:36Z

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: e1430135ee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

datadog-datadog-prod-us1-2 · 2026-07-21T14:24:23Z

Bits has a CI fix ready

🟢 Investigated · 🟢 Fix prepared · ⚪ Validation skipped · 🟠 Ready

test_collapsed_spans_telemetry dropped its HTTP mock before StatsExporter::send completed. The test now retains the mock for its full lifetime in libdd-trace-stats/src/stats_exporter.rs.

Commit fix to this PR

View in Datadog | Reviewed commit e47093e · Any feedback? Reach out in #deveng-pr-agent

feat: doc new module

Eldolfin · 2026-07-23T08:33:02Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 59adf71d1f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

yannham · 2026-07-24T08:55:50Z

+
+#[derive(Debug, Clone, Default, Copy)]
+// Note: slot 0 is a counter for non_collapsed spans. It's not used for emitting telemetry
+pub struct CollapsedFieldsMetrics(pub(super) [usize; COLLAPSED_FIELD_METRIC_SIZE]);


So if I understand correctly, you don't count each collapsed field individually, but you count sets of collapsed fields instead. I'm curious about the reason behind it. I would naively expect that counting each field individually would be enough.

yannham · 2026-07-24T09:00:24Z

                slot.insert();
            }
        }
+        self.collapsed_fields_metrics.0[collapsed_fields] += 1;


I think the API of collapsed fields is leaking. The consuming code here shouldn't have to dabble with C-like fieldbits and accessing the internal array of collapsed_fields_metrics, IMHO.

Instead, you could define something like e.g.

struct CollapsedFieldSet(usize); impl CollapsedFieldSet { const RESOURCE_NAME: usize = 1 << 1; // etc. fn add(&mut self, field: usize) { self.0 |= field; } }

Then you could use field_set.add(CollapsedFieldSet::RESOURCE_NAME), which doesn't depend on the implementation being a bit field. Similarly an increment(&mut self, field_set: CollapsedFieldSet) on CollapsedFieldMetrics could make the API nicer (whether you need to keep the internal array visible for sending the metrics is orthogonal, and is ok)

yannham · 2026-07-24T09:01:40Z

+//! Each collapsed field is assigned a bit; a span's mask is the OR of the bits for the
+//! fields collapsed on it, so masks double as an index into
+//! [`CollapsedFieldsMetrics`]'s counter array (one counter per distinct field combination).


This comment is hard to digest.

yannham · 2026-07-24T09:04:32Z

+                collapsed_field::ADDITIONAL_TAGS => {
+                    libdd_common::tag!("collapsed_spans", "additional_metric_tags")
+                }
+                // Unreachable: asserted just above that field is one of the 4 possible values


Nit: now the assert is debug only

Suggested change

// Unreachable: asserted just above that field is one of the 4 possible values

// Should be unreachable, but don't fail in prod if provided with an invalid field set

yannham · 2026-07-24T09:07:01Z

        force: bool,
        encode: impl Fn(StatsBucket, u64) -> T,
-    ) -> (Vec<(bool, T)>, u64) {
+    ) -> (Vec<(bool, T)>, u64, CollapsedFieldsMetrics) {


Nit: maybe this should be a proper struct with explicit field names? A 3 element tuple starts to be hard to read/guess.

github-actions Bot added sidecar data-pipeline labels Jul 10, 2026

Eldolfin changed the base branch from main to oscarld/stats-per-key-cardinality-limit July 10, 2026 15:15

github-actions Bot removed sidecar data-pipeline labels Jul 10, 2026

Eldolfin mentioned this pull request Jul 21, 2026

feat(stats)!: add telemetry for per-field cardinality limits #2212

Closed

Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator branch from 456c9bf to 295db59 Compare July 21, 2026 12:01

Eldolfin changed the title ~~feat(stats)!: add telemetry for per-field cardinality limits V2~~ feat(stats)!: add telemetry for per-field cardinality limits Jul 21, 2026

Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator branch from e47093e to cd350fa Compare July 22, 2026 11:00

Eldolfin added 9 commits July 22, 2026 13:05

feat: fields combination metrics, dogstatsd only

1d6f370

feat: cherry pick simple 1 collapsed field tests from old pr

d9740e2

fix: bitshift operations are hard...

171683e

cleanup

6ff1899

wip telemetry

0571862

add tests for multiple per-key collapse

4205724

refacto: extract telemetry stuff into its own module

7820438

feat: doc new module

fix: assert -> debug_assert, avoid useless unreachable!

76d3c4b

feat: telemetry for additional metric tags collapse

e648bf3

Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator branch from bf05f1a to e648bf3 Compare July 22, 2026 11:05

remove old fixme

59adf71

Eldolfin marked this pull request as ready for review July 22, 2026 11:17

Eldolfin requested a review from a team as a code owner July 22, 2026 11:17

chatgpt-codex-connector Bot reviewed Jul 23, 2026

View reviewed changes

Comment thread libdd-trace-stats/src/span_concentrator/cardinality_limit_telemetry.rs Outdated

fix: another assert -> debug_assert

69abc54

yannham reviewed Jul 24, 2026

View reviewed changes

	// Unreachable: asserted just above that field is one of the 4 possible values
	// Should be unreachable, but don't fail in prod if provided with an invalid field set

Uh oh!

Conversation

Eldolfin commented Jul 10, 2026

What does this PR do?

Motivation

Additional Notes

How to test the change?

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Jul 10, 2026 • edited by datadog-datadog-prod-us1 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📚 Documentation Check Results

📦 datadog-ipc - 981 warning(s)

📦 libdd-trace-stats - 876 warning(s)

Uh oh!

github-actions Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔒 Cargo Deny Results

📦 datadog-ipc - 3 error(s)

📦 libdd-trace-stats - 2 error(s)

Uh oh!

pr-commenter Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Comparison

Explanation

More details about the CI and significant changes

Explanation

More details about the CI and significant changes

Candidate

Group 1

Group 1

Group 2

Group 3

Baseline

Group 1

Group 1

Group 2

Group 3

Uh oh!

Eldolfin commented Jul 21, 2026

Uh oh!

dd-octo-sts Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Artifact Size Benchmark Report

Uh oh!

chatgpt-codex-connector Bot commented Jul 21, 2026

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Jul 21, 2026 • edited by datadog-datadog-prod-us1 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bits has a CI fix ready

Uh oh!

Eldolfin commented Jul 23, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

yannham Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

yannham Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

yannham Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

yannham Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

yannham Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

datadog-datadog-prod-us1-2 Bot commented Jul 10, 2026 •

edited by datadog-datadog-prod-us1 Bot

Loading

github-actions Bot commented Jul 10, 2026 •

edited

Loading

📦 `datadog-ipc` - 981 warning(s)

📦 `libdd-trace-stats` - 876 warning(s)

github-actions Bot commented Jul 10, 2026 •

edited

Loading

📦 `datadog-ipc` - 3 error(s)

📦 `libdd-trace-stats` - 2 error(s)

pr-commenter Bot commented Jul 21, 2026 •

edited

Loading

dd-octo-sts Bot commented Jul 21, 2026 •

edited

Loading

datadog-datadog-prod-us1-2 Bot commented Jul 21, 2026 •

edited by datadog-datadog-prod-us1 Bot

Loading