Add cache-envelope serialization benchmarks + manual workflow - #47
Merged
Merged
Conversation
Add a BenchmarkDotNet project (util/Benchmarks) measuring serialize/ deserialize time, allocations, and stored size for the NatsCache value envelope, establishing the JSON baseline ahead of the #37 binary-envelope change. In-memory only (no NATS/Docker), multi-targets net8.0;net10.0. - util/Benchmarks: Json_Serialize/Json_Deserialize across 128/1024/8192-byte payloads; `--sizes` report prints stored byte counts. JSON baseline is a local type so it stays fixed as the library evolves. - util/PerfTest: make ValueSizeBytes overridable via PERF_VALUE_SIZE_BYTES. - .github/workflows/benchmark.yml: workflow_dispatch job that runs `--sizes` and the full suite on both TFMs; kept manual since BDN is too slow/noisy to gate every PR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Matthew DeVenny <matt@codecargo.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a dedicated BenchmarkDotNet utility project to measure the current JSON-based cache-entry envelope’s serialized size, serialize/deserialize throughput, and allocations, plus a manual GitHub Actions workflow to run these benchmarks on demand. This establishes a stable performance baseline ahead of the planned binary-envelope change in #37.
Changes:
- Add
util/Benchmarks(multi-targetnet8.0;net10.0) with JSON serialize/deserialize benchmarks and a fast--sizesreport. - Add a manual
benchmarkworkflow to run the size report and benchmarks for both TFMs. - Allow
util/PerfTestpayload size to be overridden viaPERF_VALUE_SIZE_BYTESfor larger-value experiments.
Reviewed changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| util/PerfTest/PerfTest.cs | Adds env-var override for perf-test payload size and prints it in results. |
| util/Benchmarks/SizeReport.cs | Implements --sizes console report for JSON envelope byte size by payload size. |
| util/Benchmarks/README.md | Documents how to run the size report and BenchmarkDotNet suite. |
| util/Benchmarks/Program.cs | Adds CLI switch for --sizes, otherwise delegates to BenchmarkDotNet. |
| util/Benchmarks/packages.win-x64.lock.json | Adds deterministic NuGet lockfile for Windows x64 restores. |
| util/Benchmarks/packages.osx-arm64.lock.json | Adds deterministic NuGet lockfile for macOS arm64 restores. |
| util/Benchmarks/packages.linux-x64.lock.json | Adds deterministic NuGet lockfile for Linux x64 restores (used by CI cache). |
| util/Benchmarks/packages.linux-arm64.lock.json | Adds deterministic NuGet lockfile for Linux arm64 restores. |
| util/Benchmarks/JsonCacheEntry.cs | Introduces a frozen “legacy JSON envelope” model + source-gen JSON context. |
| util/Benchmarks/CacheEntrySerializationBenchmarks.cs | Adds JSON serialize/deserialize benchmarks with MemoryDiagnoser. |
| util/Benchmarks/Benchmarks.csproj | New BenchmarkDotNet executable project referencing the main library. |
| util/Benchmarks/BenchmarkData.cs | Provides deterministic payload/expiration test data for benchmarks and reports. |
| NatsDistributedCache.slnx | Adds the new Benchmarks project to the solution. |
| .gitignore | Ignores BenchmarkDotNet.Artifacts/ output directories. |
| .github/workflows/benchmark.yml | Adds manual workflow_dispatch job to run the size report + benchmarks on both TFMs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Bind the workflow_dispatch filter input to an env var and reference it as a
quoted shell variable ("${FILTER:-*}") instead of interpolating
${{ inputs.filter }} into the run script, so a value containing a quote
cannot break out and inject shell tokens. The :-* fallback also covers an
empty input.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Matthew DeVenny <matt@codecargo.com>
- Extend the payload params (and SizeReport) to 65536 and 262144 bytes so the large-value behavior is characterized (where the base64 size ratio asymptotes to 3/4 and JSON GC pressure grows); capped at 256 KiB since NATS' default max_payload is 1 MB. - Fix the README `--sizes` example to pass `-f net10.0`; without a framework, `dotnet run` fails on this multi-targeted project. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Matthew DeVenny <matt@codecargo.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a BenchmarkDotNet project (
util/Benchmarks) that measures theNatsCachevalue envelope — serialize/deserialize time, allocations, and stored size — establishing the JSON baseline ahead of the #37 binary-envelope change (the implementation is the stacked follow-up PR).In-memory only (no NATS server / Docker), multi-targets
net8.0;net10.0.What's here
util/Benchmarks—Json_Serialize/Json_Deserializeacross[Params(128, 1024, 8192)]payloads,MemoryDiagnoserfor allocations, and a--sizesreport for stored byte counts. The JSON baseline is a localJsonCacheEntryso it stays fixed as the library evolves.util/PerfTest—ValueSizeBytesis now overridable viaPERF_VALUE_SIZE_BYTESto exercise larger payloads..github/workflows/benchmark.yml— manualworkflow_dispatchjob that runs--sizes+ the full suite on both TFMs. Kept manual because BenchmarkDotNet is too slow / CPU-noisy to gate every PR.Run
dotnet run -c Release --project util/Benchmarks -- --sizes dotnet run -c Release --project util/Benchmarks -f net10.0 -- --filter '*'JSON baseline (this PR)
The follow-up #37 PR adds the
Binary_*variants and the head-to-head comparison.🤖 Generated with Claude Code