perf(cache)!: read the local tier by span and key it by the name string

[cosmin-staicu]

Stacked on #216; its base is refactor/abstractions-implementations. Review and merge that one first.

Makes a read that hits the local tier allocate nothing, and gives callers a way to reach it without building a CacheKey.

The local tier is keyed by the name string

IMemoryCache's members take object, and CacheKey is a struct, so every lookup, set and remove boxed the key (32 B on x64), and the box stayed on as the entry's key. MemoryCacheSetter, MultilayerCache and MultilayerHashCache now pass CacheKey.Name: no box per lookup, one object less per resident entry, and MemoryCache's string-keyed dictionary instead of its object-keyed one, whose probes go through virtual Equals(object).

CacheKey equality is ordinal on Name, and the name is never null or empty by the time it reaches the tier: the caller's key is validated as before, and both entry builders now refuse a key strategy that composes an empty key with InvalidOperationException before any tier is touched (the local tier used to file every such key under one nameless entry). So the partition of keys is unchanged. Observable only through a custom IMemoryCacheFactory that shares one IMemoryCache with other string-keyed entries, which can now collide on equal text; a CHANGELOG Changed entry says so.

Reads by Span<char>

ICache.GetAsync<T>, ICache<T>.GetAsync, and GetItemAsync and GetAsync on IHashCache and IHashCache<T> gain a Span<char> overload, each with a default body that builds the key, so outside implementations keep compiling. NullCache and NullHashCache answer without building one.

• MultilayerCache and MultilayerHashCache, on .NET 9 and later: normalize the text on the stack, let the key strategy compose it through TryGetCacheKey<T>, normalize the result as WithName would, look the local tier up through MemoryCache.TryGetValue(ReadOnlySpan<char>), and answer from the entry. Anything else falls back to the string path: a strategy that declines, another IMemoryCache, an empty key (rejected as the string path rejects it) or one over 256 characters, a disconnected inner tier that is not served locally, Trace logging, or .NET 8. A hit and a fallback answer the same.

• Cache<T> and HashCache<T> forward the span, composing their own strategy's key on the stack the same way, and pass it through under the default one.

• GetItemAsync reads the field straight from the stored dictionary, only when it is the ordinal ImmutableDictionary the tier itself builds, so the answer is the one Filter would give.

• The parameter is Span<char>, not ReadOnlySpan<char>. The repo compiles with LangVersion 12, and a ReadOnlySpan<char> overload makes every existing GetAsync("literal") and GetAsync(stringVariable) call ambiguous against the CacheKey overload (CS0121, both are user-defined conversions). Span<char> has no conversion from string, and stackalloc plus TryWrite yields one anyway. A caller holding a ReadOnlySpan<char> writes new CacheKey(span).

• An already-cancelled token is observed where the key path observes it: after the empty-key check and before the strategy on MultilayerCache, first of all on MultilayerHashCache. The two overloads therefore throw the same exception for the same input, a throwing strategy included, and a cancelled read never reaches the strategy or the local tier.

• Not included: GetOrAddAsync, whose generator delegate allocates per call regardless.

Key strategies compose by span

ICacheKeyStrategy gains bool TryGetCacheKey<T>(ReadOnlySpan<char> key, Span<char> destination, out int written), the span form of GetCacheKey. The text arrives normalized as CacheKey normalizes it, the strategy writes the characters GetCacheKey would build, and the library normalizes the result as WithName would, so casing is not the strategy's concern; a test drives an upper-case-suffix strategy through both paths. DefaultCacheKeyStrategy copies, PrefixCacheKeyStrategy writes its prefix. The member has no default body: a strategy that cannot compose by span (a hash, a lookup) returns false in one line and its reads stay on the string path, rather than silently keeping every span read off the fast path.

Breaking for ICacheKeyStrategy implementations outside the library, which must add the member.

CacheKey from a span

CacheKey(ReadOnlySpan<char>) and CacheKey(ReadOnlySpan<char>, CacheKeyCasing) normalize while copying, so a key formatted on the stack costs one string rather than two. They share TryNormalize with the span reads, so a key written through one path and read through the other cannot differ; a test checks the span and string constructors agree, including trimming, non-ASCII lowercasing and names longer than the stack buffer.

Local writes

MemoryCacheSetter.Set fills the entry CreateEntry returns (expiration, change token, eviction callback, size, value) instead of copying a MemoryCacheEntryOptions and its two lists into it. PrefixCacheKeyStrategy concatenates its prefix + separator, computed once, instead of interpolating per key, and gains the internal span writer the reads use.

Benchmark

LocalHitBenchmark (new, MemoryDiagnoser, --job short --inProcess, net10.0): one entry held by the in-memory tier, read through ICache<string>.

| Local hit through ICache<string> | Before (#216) | After |

|---|---|---|

| GetAsync(CacheKey) | 174 ns, 32 B | 143 ns, 0 B |

| GetAsync(string) | 172 ns, 32 B | 147 ns, 0 B |

| GetAsync($"user:{id}") | 226 ns, 72 B | 179 ns, 40 B |

| GetAsync(Span<char>), TryWrite on the stack | n/a | 109 ns, 0 B |

| GetAsync(CacheKey) under PrefixCacheKeyStrategy | 229 ns, 80 B | 173 ns, 48 B |

| GetAsync(Span<char>) under PrefixCacheKeyStrategy | n/a | 170 ns, 0 B |

| SetAsync(CacheKey, value) | 1,199 ns, 1,152 B | 1,050 ns, 832 B |

The allocation column is exact; the means come from --job short and carry wide error bars. The 40 B and 48 B left are the formatted and the composed key string, which the span reads avoid. The prefixed span read pays the strategy call and two normalizations, so it matches the key read in time and wins on allocation.

Verification

• dotnet build -c Release -warnaserror: 0 warnings.

• Red checks: dropping the connection-state guard fails the disconnected theory; skipping normalization on the fast path fails the zero-allocation check on User:42; skipping normalization of a strategy's output fails the custom-strategy zero-allocation check; removing the fast path fails every zero-allocation test.

• dotnet test: 2067 passed on net8.0 and 2092 on net10.0, 0 failed, with the Redis integration tests on.

[github-actions]

🔎 Maintainer heads-up: automated triage flagged this PR as potentially material, so it may need a signed CLA in addition to the DCO sign-off.

Strong signals

• adds public API surface (PublicAPI.Unshipped.txt in src/UiPath.Caching.Abstractions, src/UiPath.Caching)

Other signals

• large production change (+331 lines under src/)

This is advisory only — the bot does not decide. Please judge against the CLA criteria (material, product-critical, patent-sensitive, corporate contributor, broad commercial use). Note that thresholds can be gamed by splitting PRs, so use your judgement.

• If a CLA is needed → add the cla-required label (a contributor comment with signing steps is posted automatically).
• If it is not needed → replace needs-cla-review with cla-not-required so later pushes don't re-flag it.

[Copilot]

Copilot review overview

🟡 Changes recommended

Span fast paths bypass cancellation checks in both multilayer cache implementations.

Review effort: Lite
Findings: 1 · 1

Open (2)

• Honor cancellation before span local-hit fast path · New
• Check cancellation before local cache fast path · New

What changed in this PR

Adds allocation-free span-based cache reads, string-keyed local entries, optimized writes, tests, documentation, and benchmarks.

Changes:

• Adds span overloads and span-based CacheKey normalization.
• Optimizes local cache lookup, prefix composition, and entry writes.
• Adds coverage, API documentation, changelog updates, and benchmarks.

File	Summary
tests/​UiPath.Caching.Tests/​SpanKeyReadTests.cs	Tests span reads and allocations.
tests/​UiPath.Caching.Tests/​PrefixCacheKeyStrategyTests.cs	Tests span key composition.
tests/​UiPath.Caching.Tests/​MultilayerHashCacheTests.cs	Tests hash cache behavior and string keys.
tests/​UiPath.Caching.Tests/​MultilayerCacheTryAddTests.cs	Updates local-key expectations.
tests/​UiPath.Caching.Tests/​MultilayerCacheTests.cs	Tests span reads and local keys.
tests/​UiPath.Caching.Tests/​MemoryCacheSetterTests.cs	Tests string-keyed entries.
tests/​UiPath.Caching.Tests/​LocalCacheSetterTests.cs	Tests local cache storage.
tests/​UiPath.Caching.Tests/​Fakes/​SpanReads.cs	Provides span-read test helpers.
tests/​UiPath.Caching.Tests/​Fakes/​InMemoryMultilayer.cs	Provides in-memory fixtures.
tests/​UiPath.Caching.Tests/​CacheKeyTests.cs	Tests span key normalization.
src/​UiPath.Caching/​SpanKey.cs	Composes normalized span keys.
src/​UiPath.Caching/​PublicAPI.Unshipped.txt	Records new APIs.
src/​UiPath.Caching/​PrefixCacheKeyStrategy.cs	Adds optimized span composition.
src/​UiPath.Caching/​MultilayerHashCache.cs	Adds hash span fast paths and string keys.
src/​UiPath.Caching/​MultilayerCache.cs	Adds span fast paths and string keys.
src/​UiPath.Caching/​MemoryCacheSetter.cs	Writes entries in place.
src/​UiPath.Caching/​HashCacheOfT.cs	Forwards typed hash span reads.
src/​UiPath.Caching/​HashCacheEntryBuilder.cs	Exposes the key strategy.
src/​UiPath.Caching/​CacheOfT.cs	Forwards typed span reads.
src/​UiPath.Caching/​CacheEntryBuilder.cs	Exposes the key strategy.
src/​UiPath.Caching.Abstractions/​PublicAPI.Unshipped.txt	Records new public APIs.
src/​UiPath.Caching.Abstractions/​NullHashCache.cs	Implements null hash span reads.
src/​UiPath.Caching.Abstractions/​NullCache.cs	Implements null span reads.
src/​UiPath.Caching.Abstractions/​IHashCacheOfT.cs	Adds typed hash span APIs.
src/​UiPath.Caching.Abstractions/​IHashCache.cs	Adds hash span APIs.
src/​UiPath.Caching.Abstractions/​ICacheOfT.cs	Adds typed span APIs.
src/​UiPath.Caching.Abstractions/​ICache.cs	Adds span read APIs.
src/​UiPath.Caching.Abstractions/​CacheKey.cs	Adds span normalization.
docs/​reference/​interfaces.md	Documents new APIs.
CHANGELOG.md	Documents behavior and performance changes.
benchmarks/​UiPath.Caching.Benchmarks/​README.md	Documents benchmark execution.
benchmarks/​UiPath.Caching.Benchmarks/​LocalHitBenchmark.cs	Adds local-hit benchmarks.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🟢 Approval recommended

No unresolved blocking issues were identified, and the changes are fully reviewed.

Review effort: Lite
Findings: None

Resolved since last review (2)

• Honor cancellation before span local-hit fast path
• Check cancellation before local cache fast path

[Copilot]

Copilot review overview

🟡 Changes recommended

Empty cancelled span reads currently differ from the established CacheKey overload behavior.

Review effort: Balanced
Findings: 1

Open (1)

• Validate normalized cache key before cancellation · New

[Copilot]

Copilot review overview

🔵 Needs a closer look

It changes public APIs and core cache-key storage semantics across several performance-sensitive paths.

Review effort: Balanced
Findings: None

Resolved since last review (1)

• Validate normalized cache key before cancellation

[Copilot]

Copilot review overview

🟡 Changes recommended

Span fast paths invoke custom key strategies before observing cancellation, producing behavior inconsistent with existing key-based reads.

Review effort: Balanced
Findings: 2

Open (2)

• Check cancellation before composing non-empty span cache keys · New
• Check cancellation before local lookup and span strategy invocation · New

Resolved since last review (1)

• Validate custom cache key strategy results before cache access

[Copilot]

Copilot review overview

🔵 Needs a closer look

It combines breaking public APIs with performance-sensitive cache-key and local-tier behavior on top of another unmerged pull request.

Review effort: Balanced
Findings: None

Resolved since last review (2)

• Check cancellation before local lookup and span strategy invocation
• Check cancellation before composing non-empty span cache keys