Skip to content

perf(cache): remove per-operation allocations on the Redis and memory hot paths - #214

Merged
cosmin-staicu merged 1 commit into
mainfrom
perf/hot-path-allocations
Sep 28, 2026
Merged

cosmin-staicu merged 1 commit into
mainfrom
perf/hot-path-allocations

Conversation

@cosmin-staicu

@cosmin-staicu cosmin-staicu commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Stacked on #213. Review and merge that one first; this PR's base is feat/resolve-key-prefix.

Removes per-operation allocations on the Redis command path and the local memory path, in six changes:

  1. Telemetry off still allocated. The caches call StartOperation(string, Type, string). NullTelemetryProvider never declared that overload, so the interface default built a TelemetryOperation, a Stopwatch and a scope string on every call. It now returns NullTelemetryOperation.Instance. TelemetryOperation also caches its metric names and times with Stopwatch timestamps.
  2. Polly wrapper. Pipelines were keyed by (Type, object?), which boxed the default value on every command, and the GetOrAdd factory captured a closure. Now there is one typed map per result type, and the read path passes the callback to Polly as state.
  3. Async lambdas. 43 single-command callbacks awaited the client task only to return it. They now wrap the task in a ValueTask.
  4. Key composition. string.Join(char, params string[]) is replaced by interpolation in the prefix key strategies.
  5. Comparers. TopicKey equality is now ordinal. Its name is already lowercased and it already hashes ordinally. The event-type sets and the hash field filter use OrdinalIgnoreCase, and the filter builds its result in a loop instead of LINQ with ToImmutableDictionary.
  6. Small cleanups. Array.ConvertAll replaces Select().ToArray() over arrays, and the internal builders and setters are sealed.

Benchmark

A single-operation BenchmarkDotNet run with MemoryDiagnoser against a local Redis on net10.0. The baseline is #213's head. Time on the Redis rows is dominated by the round trip, so the allocation column is the signal.

Operation Allocated before Allocated after Mean before Mean after
Redis GetAsync 3724 B 3123 B 760 µs 882 µs
Redis SetAsync 3020 B 2376 B 864 µs 742 µs
Multilayer GetAsync, memory hit 32 B 32 B 288 ns 201 ns
Hash GetAsync(fields), memory hit 896 B 576 B 1827 ns 801 ns
Multilayer SetAsync 14523 B 14448 B 1.98 ms 1.71 ms

The multilayer SetAsync row is noise-bound. It also counts allocations from the background stream reader, and repeated runs of either build ranged from 13.2 KB to 16.0 KB.

Behavior to review

  • Change 5. The known event types and hash field names now match with OrdinalIgnoreCase instead of InvariantCultureIgnoreCase. The event types are ASCII constants. Field names that differ only under culture rules no longer match.
  • Change 3. A cancellation checked before the command now throws synchronously instead of returning a faulted task. Every call site awaits inside a try, and Polly produces the same outcome either way.

One shape for the single-command members

SonarQube flagged 10.8% duplicated lines on this PR's new code: the thirteen single-command members of RedisCache, RedisHashCache and RedisSetCache (ContainsAsync, RemoveAsync, TimeToLiveAsync, ExpireTimeAsync, the set's CountAsync, ContainsItemAsync, RemoveItemAsync, RemoveItemsAsync) shared one 20-line skeleton around the pipeline call, and this PR touched two lines inside each. They now forward to RedisCacheBase.RunAsync, a protected helper that runs the command under the pipeline, times it, tracks the hit and logs a failure. The TTL and expiration reads, with the pre-Redis-7 fallback from TTL to expiration, live in the base as KeyTimeToLiveAsync and KeyExpireTimeAsync over one SupportsExpireTime, and RedisHashCache reads a hash's fields in one ReadFields method instead of two copies of the loop. Behaviour is unchanged, including a cancelled token being logged and answered with the fallback, and the TTL and expiration reads still composing the key inside the command so a null key is logged and answered the same way (two tests per cache pin both). The one difference: on a server without EXPIRETIME the fallback runs under the ExpireTimeAsync operation alone rather than also under a TimeToLiveAsync one.

Verification

  • dotnet build -c Release -warnaserror
  • dotnet test: 4001 passed on net8.0 and net10.0, with the Redis integration tests on (RUN_REDIS_INTEGRATION_TESTS=1).

@github-actions github-actions Bot added the needs-cla-review A maintainer should assess whether a signed CLA is required (see CONTRIBUTING.md) label Sep 25, 2026
@github-actions

Copy link
Copy Markdown

🔎 Maintainer heads-up: automated triage flagged this PR as potentially material, so it may need a signed CLA in addition to the DCO sign-off.

Strong signals

  • adds public API surface (PublicAPI.Unshipped.txt in src/UiPath.Caching.Abstractions)

This is advisory only — the bot does not decide. Please judge against the CLA criteria (material, product-critical, patent-sensitive, corporate contributor, broad commercial use). Note that thresholds can be gamed by splitting PRs, so use your judgement.

  • If a CLA is needed → add the cla-required label (a contributor comment with signing steps is posted automatically).
  • If it is not needed → replace needs-cla-review with cla-not-required so later pushes don't re-flag it.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

String L1 keys can corrupt shared application cache entries, and telemetry caching can retain collectible consumer types indefinitely.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity · 1 Medium severity

Open (2)
What changed in this PR

Reduces allocations across Redis command execution, telemetry, key composition, and multilayer memory-cache hot paths.

Changes:

  • Replaces async state machines and LINQ allocations with ValueTask wrappers and array conversions.
  • Optimizes telemetry, resilience-pipeline caching, key composition, and comparisons.
  • Uses string keys for local memory caching and updates corresponding tests.
File Description
tests/​UiPath.Caching.Tests/​MultilayerHashCacheTests.cs Updates memory-key expectations.
tests/​UiPath.Caching.Tests/​MultilayerCacheTryAddTests.cs Updates TryAdd memory-key assertions.
tests/​UiPath.Caching.Tests/​MultilayerCacheTests.cs Updates multilayer memory-key assertions.
tests/​UiPath.Caching.Tests/​MemoryCacheSetterTests.cs Verifies string-key eviction behavior.
tests/​UiPath.Caching.Tests/​LocalCacheSetterTests.cs Verifies string-key local caching.
src/​UiPath.Caching/​TaskObservation.cs Adds task-to-ValueTask conversion.
src/​UiPath.Caching/​Redis/​ShardPrefixRedisKeyStrategy.cs Avoids string.Join allocation.
src/​UiPath.Caching/​Redis/​RedisHashCache.cs Removes redundant async state machines.
src/​UiPath.Caching/​Redis/​RedisCache.cs Optimizes callbacks and array conversions.
src/​UiPath.Caching/​Redis/​PrefixRedisKeyStrategy.cs Uses interpolation for key composition.
src/​UiPath.Caching/​PrefixCacheKeyStrategy.cs Uses interpolation for prefixed keys.
src/​UiPath.Caching/​MultilayerHashCache.cs Uses string L1 keys and ordinal filtering.
src/​UiPath.Caching/​MultilayerCache.cs Optimizes arrays and string L1 keys.
src/​UiPath.Caching/​MemoryCacheSetter.cs Stores entries under string keys.
src/​UiPath.Caching/​LocalMemorySetter.cs Seals the local setter.
src/​UiPath.Caching/​HashLocalMemorySetter.cs Seals the hash setter.
src/​UiPath.Caching/​HashCacheEntryBuilder.cs Seals the hash builder.
src/​UiPath.Caching/​CacheEntryBuilder.cs Seals the cache builder.
src/​UiPath.Caching/​Broadcast/​Redis/​RedisStreamsTopic.cs Optimizes Redis stream callbacks.
src/​UiPath.Caching/​Broadcast/​Redis/​RedisPubSubTopic.cs Optimizes Pub/Sub callbacks.
src/​UiPath.Caching/​Broadcast/​ChangeTokenFactory.cs Uses ordinal event comparison.
src/​UiPath.Caching/​Broadcast/​CacheEventFactory.cs Uses ordinal event comparison.
src/​UiPath.Caching.Queue/​TaskObservation.cs Adds queue task conversion helper.
src/​UiPath.Caching.Queue/​RedisSetCache.cs Optimizes Redis set callbacks.
src/​UiPath.Caching.Polly/​ResiliencePipelineWrapper.cs Adds typed pipeline caches and state callbacks.
src/​UiPath.Caching.CloudEvents/​CloudCacheEventFactory.cs Uses ordinal event comparison.
src/​UiPath.Caching.Abstractions/​Telemetry/​TelemetryOperation.cs Caches metric names and timestamp state.
src/​UiPath.Caching.Abstractions/​Telemetry/​NullTelemetryProvider.cs Adds allocation-free overload.
src/​UiPath.Caching.Abstractions/​PublicAPI.Unshipped.txt Records the new public overload.
src/​UiPath.Caching.Abstractions/​CacheOfT.cs Replaces LINQ array conversions.
src/​UiPath.Caching.Abstractions/​Broadcast/​TopicKey.cs Makes equality ordinal.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/UiPath.Caching/MemoryCacheSetter.cs Outdated
Comment thread src/UiPath.Caching.Abstractions/Telemetry/TelemetryOperation.cs Outdated
@cosmin-staicu
cosmin-staicu force-pushed the perf/hot-path-allocations branch 2 times, most recently from 844c05a to c8e5f8c Compare September 25, 2026 21:28
@cosmin-staicu
cosmin-staicu requested a balanced review from Copilot September 25, 2026 21:29

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The static telemetry metric-name cache can grow indefinitely from public high-cardinality inputs.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (2)

Comment thread src/UiPath.Caching.Abstractions/Telemetry/TelemetryOperation.cs

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The allocation optimizations preserve established behavior and are supported by comprehensive build and test verification.

Review effort: Balanced
Findings: None

Resolved since last review (1)

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The allocation optimizations are internally consistent, preserve the documented semantics, and introduce no unresolved correctness issues.

Review effort: Balanced
Findings: None

@cosmin-staicu
cosmin-staicu force-pushed the perf/hot-path-allocations branch from 17f2e66 to 215df18 Compare September 26, 2026 12:35
@cosmin-staicu
cosmin-staicu force-pushed the feat/resolve-key-prefix branch from 79adbbb to f1900c3 Compare September 26, 2026 12:35
@cosmin-staicu
cosmin-staicu force-pushed the feat/resolve-key-prefix branch from f1900c3 to be904d1 Compare September 28, 2026 14:02
@cosmin-staicu
cosmin-staicu force-pushed the perf/hot-path-allocations branch from 215df18 to 84731d0 Compare September 28, 2026 14:19
@alinahornet
alinahornet self-requested a review September 28, 2026 14:21
alinahornet
alinahornet previously approved these changes Sep 28, 2026
Base automatically changed from feat/resolve-key-prefix to main September 28, 2026 14:25
@cosmin-staicu
cosmin-staicu dismissed alinahornet’s stale review September 28, 2026 14:25

The base branch was changed.

@cosmin-staicu
cosmin-staicu force-pushed the perf/hot-path-allocations branch 4 times, most recently from 73115bd to c292ef2 Compare September 28, 2026 15:19
… hot paths

- NullTelemetryProvider declares StartOperation(string, Type, string), so the
  interface default no longer builds a TelemetryOperation per call with
  telemetry off. TelemetryOperation caches its metric names and times with
  Stopwatch timestamps.
- The Polly wrapper keeps one typed pipeline map per result type, so a lookup
  neither boxes the default nor allocates a factory closure, and the read
  path passes its callback as Polly state.
- Single-command pipeline callbacks return the client task as a ValueTask
  instead of awaiting it in an async lambda.
- Prefixed keys are composed without a params array.
- TopicKey compares ordinally, consistent with its hash. The known event
  types and the hash field filter use OrdinalIgnoreCase, and the filter fills
  an immutable builder instead of running LINQ with two delegates.
- Array projections use Array.ConvertAll, and internal builders are sealed.

The single-command members of the three Redis caches, thirteen of them,
shared one 20-line skeleton around the pipeline call; they now forward to
RedisCacheBase.RunAsync. The TTL and expiration reads, with the
pre-Redis-7 fallback from TTL to expiration, live in the base as
KeyTimeToLiveAsync and KeyExpireTimeAsync over one SupportsExpireTime,
and RedisHashCache reads a hash's fields in one method instead of two
copies of the loop. Same behaviour, including a cancelled token being
logged and answered with the fallback, and the TTL and expiration reads
still composing the key inside the command so a null key is logged and
answered the same way. The one difference: on a server without
EXPIRETIME the fallback runs under the ExpireTimeAsync operation alone
rather than also under a TimeToLiveAsync one.

Signed-off-by: Cosmin Staicu <cosmin.staicu@uipath.com>
@cosmin-staicu
cosmin-staicu force-pushed the perf/hot-path-allocations branch from c292ef2 to b8965d3 Compare September 28, 2026 15:41
@sonarqubecloud

Copy link
Copy Markdown

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Base-class version detection now makes resolving RedisSetCache synchronously initiate and wait for a Redis connection.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)

Comment thread src/UiPath.Caching/Redis/RedisCacheBase.cs
@cosmin-staicu
cosmin-staicu merged commit aa52421 into main Sep 28, 2026
11 checks passed
@cosmin-staicu
cosmin-staicu deleted the perf/hot-path-allocations branch September 28, 2026 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-cla-review A maintainer should assess whether a signed CLA is required (see CONTRIBUTING.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants