Skip to content

#38 add bucket auto creation option - #53

Merged
matthewdevenny merged 5 commits into
mainfrom
matt/38-bucket-autocreation
Jul 6, 2026
Merged

matthewdevenny merged 5 commits into
mainfrom
matt/38-bucket-autocreation

Conversation

@matthewdevenny

@matthewdevenny matthewdevenny commented Jul 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds opt-in automatic creation of the backing NATS KV bucket, so consumers no longer have to pre-create it before using the cache. Closes #38.

Previously NatsCache only called GetStoreAsync(bucketName), which fails if the bucket doesn't exist — every consumer had to run a manual CreateOrUpdateStoreAsync(...) step (the most prominent step in the README). This makes that step optional.

What's new

  • NatsCacheOptions.CreateBucketIfNotExists (default false) — when enabled, the bucket is created on first cache use if it is missing, with the settings per-key TTL requires: History = 1 and a non-zero LimitMarkerTTL.
  • NatsCacheOptions.ConfigureBucket (Func<NatsKVConfig, NatsKVConfig>) — optional hook to customize storage / replication / limits. NatsKVConfig is an immutable record, so it's used with a with expression (the same idiom already used for NatsOpts elsewhere in this repo):
    options.ConfigureBucket = cfg => cfg with { Storage = NatsKVStorageType.File, NumberOfReplicas = 3 };

Behavior / design notes

  • Create-only, never update. Only a missing bucket is created: GetStoreAsync is attempted first, and only if it reports the bucket is missing (JetStream "stream not found") does CreateStoreAsync create it. An existing operator-managed bucket is used as-is and never modified, so the option name matches the behavior. Probing with GetStoreAsync (rather than listing every bucket) keeps this O(1) and needs no stream-list permission; any other error propagates unchanged.
  • Lazy, on first use. Creation happens on the first cache operation, reusing the existing lazy store resolution and reset-on-failure retry. No IHostedService / startup coupling, so a brief NATS outage at boot doesn't fail host startup.
  • Defaults + override order. The library seeds History = 1 and a non-zero LimitMarkerTTL, then applies ConfigureBucket; the Bucket name is always re-asserted afterward. A ConfigureBucket that returns null throws a clear InvalidOperationException.
  • Disabled by default — existing behavior is unchanged when the flag is off.

Docs

README requirements and both examples now show the opt-in flag, plus a new "Automatic bucket creation" section documenting ConfigureBucket and its caveats. The util/ReadmeExample/* samples were updated to match (manual pre-create step removed).

Testing

  • Unit (net8.0 + net10.0): BuildBucketConfig defaults / hook override / null guard, and options plumbing + defaults-to-disabled contract.
  • Integration (real NATS via Aspire): auto-creates a missing bucket with History = 1 + LimitMarkerTTL; and verifies an existing operator-managed bucket is left unmodified.
  • Ran the ReadmeExample app end-to-end against a fresh NATS container — both the DistributedCache and HybridCache examples work with no manual bucket step.

🤖 Generated with Claude Code

Signed-off-by: Matthew DeVenny <matt@codecargo.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in capability for NatsCache to automatically create (and currently also update) the backing NATS KV bucket on first cache use, with cache-required defaults (LimitMarkerTTL, History = 1) and an optional configuration hook. This addresses issue #38 by making the README’s manual bucket pre-creation step optional.

Changes:

  • Introduces CreateBucketIfNotExists and ConfigureBucket on NatsCacheOptions, plus NatsCache logic to create/update the KV bucket on first use.
  • Updates README and util/ReadmeExample/* samples to demonstrate the new opt-in behavior (and remove mandatory pre-create snippets).
  • Adds unit + integration tests covering bucket config defaults, override hook behavior, and end-to-end auto-creation.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
util/ReadmeExample/HybridCache.Example.cs Updates sample to enable opt-in auto bucket creation; removes manual KV creation code.
util/ReadmeExample/HybridCache.cs Updates Aspire-based sample to enable opt-in auto bucket creation; removes manual KV creation.
util/ReadmeExample/DistributedCache.Example.cs Updates sample to enable opt-in auto bucket creation; removes manual KV creation code.
util/ReadmeExample/DistributedCache.cs Updates Aspire-based sample to enable opt-in auto bucket creation; removes manual KV creation.
test/UnitTests/Extensions/NatsDistributedCacheExtensionsTests.cs Adds tests ensuring new options are wired and default disabled contract holds.
test/UnitTests/Cache/BucketConfigUnitTests.cs Adds focused unit tests for BuildBucketConfig defaults and hook behavior.
test/IntegrationTests/Cache/BucketAutoCreationTests.cs Adds integration coverage validating bucket auto-creation on first cache operation.
src/NatsDistributedCache/NatsCacheOptions.cs Adds public options (CreateBucketIfNotExists, ConfigureBucket) with API docs.
src/NatsDistributedCache/NatsCache.cs Implements lazy create/update bucket behavior and BuildBucketConfig defaults/hook.
README.md Documents optional automatic bucket creation and adds configuration examples.
Comments suppressed due to low confidence (1)

README.md:18

  • The manual bucket-creation snippet only sets LimitMarkerTTL, but the new auto-creation path and docs state per-key TTL requires History = 1 as well. To avoid inconsistent guidance and make the requirements explicit, update the snippet to set History = 1 too.
    ```csharp
    // assuming an INatsConnection natsConnection
    var kvContext = natsConnection.CreateKeyValueStoreContext();
    await kvContext.CreateOrUpdateStoreAsync(new NatsKVConfig("cache") { LimitMarkerTTL = TimeSpan.FromSeconds(1) });

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/NatsDistributedCache/NatsCacheOptions.cs
Comment thread src/NatsDistributedCache/NatsCache.cs Outdated
- Only create a missing bucket; never update an existing (operator-managed)
  one, so the behavior matches the CreateBucketIfNotExists name. Check
  GetBucketNamesAsync and use CreateStoreAsync instead of
  CreateOrUpdateStoreAsync.
- Guard against ConfigureBucket returning null with a clear InvalidOperationException.
- Docs: reflect create-only semantics; show History = 1 in the manual snippet.
- Tests: add null-guard unit test and an integration test asserting an
  existing bucket is left unmodified.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Matthew DeVenny <matt@codecargo.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Comment thread src/NatsDistributedCache/NatsCache.cs Outdated
Comment thread README.md
- GetOrCreateStoreAsync now tries GetStoreAsync first and creates only on the
  JetStream "stream not found" error (ErrCode 10059). Avoids the O(n)
  GetBucketNamesAsync list call and its stream-list permission requirement;
  other errors (connectivity, auth) propagate unchanged.
- README: make the manual pre-create snippet self-contained with the required
  NATS.Client.KeyValueStore / NATS.Net usings.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Matthew DeVenny <matt@codecargo.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Comment thread src/NatsDistributedCache/NatsCache.cs Outdated
Comment thread src/NatsDistributedCache/NatsCache.cs
Use `ex.Error is { ErrCode: StreamNotFoundErrCode }` so a null Error makes the
catch filter evaluate false instead of throwing (which would silently skip
auto-creation of a missing bucket).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Matthew DeVenny <matt@codecargo.com>
@matthewdevenny
matthewdevenny marked this pull request as ready for review July 6, 2026 18:34
@matthewdevenny
matthewdevenny requested a review from mtmk July 6, 2026 18:34

@mtmk mtmk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with optional comment

Comment thread src/NatsDistributedCache/NatsCacheOptions.cs Outdated
Per review feedback, the hook only applies when a missing bucket is first
created (it's a no-op for an existing bucket). The clearer name reflects that.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Matthew DeVenny <matt@codecargo.com>
@matthewdevenny
matthewdevenny merged commit 7f2ce4c into main Jul 6, 2026
2 checks passed
@matthewdevenny
matthewdevenny deleted the matt/38-bucket-autocreation branch July 6, 2026 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: opt-in KV bucket auto-creation + bucket configuration

3 participants