Tag conditional responses with SHA-1 and hold the body in a pooled MemoryStream

[ipjohnson]

[ConditionalGet] now holds an untagged body in a pooled stream and tags it with SHA-1. Closes #433.

RequestBench measured the cost on container-h1 (run 36187074173, 0.40.0-rc1000). At 1,000 rps, etag.large minus json.large was +278 µs for Hardened, against −25 to +155 µs for the five ASP.NET Core frameworks. At 5,000 rps Hardened's etag.large p90 was 2.5 ms, against 1.2 to 1.6 ms.

Change

• ConditionalResponseStream holds a body that has no tag at its first write in a MemoryStream reserved from IMemoryStreamPool. It used a new MemoryStream() with no capacity. A 130 KiB body regrew that by doubling to 256 KiB on every request, which allocated about 0.5 MB and copied about 240 KB, with the two largest buffers on the large object heap. A returned stream keeps its capacity, so a steady body size stops regrowing it.
• ConditionalGetFilter is one instance per handler, so it resolves the pool from context.RequestServices on each request. It uses GetService. A container composed by hand with no pool gets a plain MemoryStream, as HardenedHtmlTemplate does.
• EntityTagHeader.ForContent hashes with SHA-1, still base64 in quotes. The response cache tags stored responses with the same method, so the two tags keep matching. ByPayload's cache key and the static content mount's tags stay SHA-256.
• The public surface does not change. ConditionalResponseStream is internal.

How the pooled stream always comes back

• One owner. The filter calls CompleteAsync in its finally, and CompleteAsync returns the stream in a finally of its own. The stream goes back after the body is copied to the transport, after a 304, when the chain threw, and when the transport write fails.
• Short holds. The stream is reserved at the write that holds the body back, not when the filter starts. A handler that writes its own tag reserves nothing, and neither does a response that writes nothing.
• Once only. The reference is cleared before the reservation is disposed. ItemPool does not guard against a second return.
• Nothing after the return. Every write after it throws ObjectDisposedException, so a caller that kept the body cannot write into a stream the pool has lent to another request.
• Never disposed. The pool resets Position on return, which throws on a closed stream, as the remarks on JsonSerializerImpl.DeserializeAsync describe. Nothing here disposes the pooled stream.

Measured

BenchmarkDotNet, in process on net10.0, on an M3 Pro. One GET goes through the chain. The handler writes 130 KiB in 16 KB pieces and sets no tag. The transport is Stream.Null, so the numbers are the filter's.

	Time	Allocated per request	Gen 2 GCs per 1,000 requests
main (1389e790)	89.3 µs	509,806 B	125
This change	47.4 µs	1,256 B	0
Without [ConditionalGet]	0.1 µs	936 B	0

With a 1,000-byte body the request took 651 ns on main and 631 ns with this change, and allocated 2,392 B and 1,256 B.

On the M3 Pro the gain is all from the buffer. SHA-1 and SHA-256 both hash at 3.1 GB/s there, and the hash is almost all of the 47 µs left. The RequestBench runner is an AMD EPYC 9V74, a Zen 4 part that also has SHA instructions, so the hash change should gain little there. It was not measured there. SHA-1 costs less per byte on a CPU without SHA instructions, and its tag is 28 characters rather than 44.

Upgrade

Every computed tag changes. A client holding a SHA-256 tag gets a 200 with the new tag on its first conditional request after the upgrade, and 304s after that. The shipped response cache store is in memory, so a restart empties it. A store an application wrote that outlives the deploy replays each entry's old tag until the entry expires, and that tag still matches. This belongs in the release notes.

Tests

ConditionalGetFilterTests gained seven tests. Six run against a counting pool that fills a returned stream with a marker and throws on a closed one:

• the body is reserved from the pool and returned once it is sent
• the stream is returned after a 304
• the stream is returned when the chain throws, after what was held is written
• the stream is returned when the transport write fails
• nothing is reserved for a handler's own tag or for an empty response
• a write after the return throws

The seventh uses the application's MemoryStreamPool. The next request gets the same stream, with its capacity and without the last body's bytes.

Six of the seven fail against main's filter. The one that checks nothing is reserved too early passes on main too. EntityTagHeaderTests pins the SHA-1 of the empty input.

dotnet build Hardened.slnx -c Release -p:ContinuousIntegrationBuild=true is clean and dotnet csharpier check . passes. The whole solution's tests pass locally with coverage, 9,895 of them with none skipped, and the coverage gate passes. ConditionalResponseStream has 24 of 24 branches covered, and Hardened.Web.Runtime stays at 100% of branches.

Docs

docs/guide/conditional-requests.md, docs/guide/response-caching.md and docs/design/response-caching.md say SHA-1. Each example tag is recomputed over the exact body shown beside it. The design page's example tag was 24 characters, which is neither hash, and is now a SHA-1. Its cost paragraph says the buffer is pooled. ByPayload's remark said the rest of the framework hashes with SHA-256, which is no longer true.

🤖 Generated with Claude Code