feat(sdk): add executor.cache, a host-pluggable key-value cache primitive#1248
Open
RhysSullivan wants to merge 1 commit into
Open
feat(sdk): add executor.cache, a host-pluggable key-value cache primitive#1248RhysSullivan wants to merge 1 commit into
RhysSullivan wants to merge 1 commit into
Conversation
ExecutorConfig.cache takes an Effect KeyValueStore; absent one the executor gets a bounded in-memory store with TTL expiry and LRU eviction (Clock-based, so tests pin TTL with the virtual TestClock). makeScopedExecutor reads an ambient KeyValueStore service optionally and threads it into createExecutor, so a host boot layer that provides one makes it durable. Ships the Cloudflare KV adapter (@executor-js/cloudflare/key-value-store) and wires the optional CACHE binding through the cloud seams layer; the wrangler binding stays commented out until the namespace is created, and everything falls back cleanly without it.
RhysSullivan
force-pushed
the
executor-cache-primitive
branch
from
4049627 to
651f44a
Compare
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
executor-marketing | 651f44a | Commit Preview URL Branch Preview URL |
Jul 02 2026, 01:41 AM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
executor-cloud | 651f44a | Jul 02 2026, 01:42 AM |
Contributor
Cloudflare preview
Sign-in is Cloudflare Access (one-time PIN to an allowed email). The preview has its own database and encryption key; it is destroyed when this PR closes. |
Greptile SummaryThis PR adds
Confidence Score: 4/5Safe to merge. The cache seam is additive — existing hosts without a KV namespace are completely unaffected, and the new path degrades gracefully to an in-memory store. The implementation is well-structured and the fallback/optional-service pattern is consistent with how RequestWebOrigin and RequestOrgSlug are handled elsewhere in makeScopedExecutor. The one minor rough edge is the void signal in clear, which means the paginated KV delete cannot be cooperatively cancelled on fiber interruption — not a correctness problem given clear is documented as off-limits for large namespaces in production, but tidier to drop the parameter. packages/hosts/cloudflare/src/key-value-store.ts — specifically the clear implementation's signal handling is worth a second look before clear ever gets a production caller. Important Files Changed
Sequence Diagram%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant Host as Cloud Host Boot
participant CSL as CloudExecutionSeamsLayer
participant SE as makeScopedExecutor
participant CE as createExecutor
participant Cache as executor.cache
Host->>CSL: env.CACHE present?
alt KV namespace provisioned
CSL->>CSL: layerCloudflareKeyValueStore(env.CACHE)
note over CSL: Layer<KeyValueStore> typed as Layer<never>
else not provisioned
CSL->>CSL: Layer.empty
end
CSL->>SE: merge into CloudExecutionSeamsLayer
SE->>SE: Effect.serviceOption(KeyValueStore)
alt service present
SE->>CE: "createExecutor({ cache: kvStore })"
CE->>Cache: use Cloudflare KV adapter
else service absent
SE->>CE: "createExecutor({})"
CE->>Cache: makeMemoryCacheStore() — bounded Map, 2048 cap, 10min TTL, LRU
end
CE-->>Host: executor.cache exposed on Executor surface
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant Host as Cloud Host Boot
participant CSL as CloudExecutionSeamsLayer
participant SE as makeScopedExecutor
participant CE as createExecutor
participant Cache as executor.cache
Host->>CSL: env.CACHE present?
alt KV namespace provisioned
CSL->>CSL: layerCloudflareKeyValueStore(env.CACHE)
note over CSL: Layer<KeyValueStore> typed as Layer<never>
else not provisioned
CSL->>CSL: Layer.empty
end
CSL->>SE: merge into CloudExecutionSeamsLayer
SE->>SE: Effect.serviceOption(KeyValueStore)
alt service present
SE->>CE: "createExecutor({ cache: kvStore })"
CE->>Cache: use Cloudflare KV adapter
else service absent
SE->>CE: "createExecutor({})"
CE->>Cache: makeMemoryCacheStore() — bounded Map, 2048 cap, 10min TTL, LRU
end
CE-->>Host: executor.cache exposed on Executor surface
Reviews (1): Last reviewed commit: "feat(sdk): add executor.cache, a host-pl..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+64
to
+70
| clear: Effect.tryPromise({ | ||
| try: async (signal) => { | ||
| void signal; | ||
| await deleteAllKeys(kv, await listAllKeys(kv)); | ||
| }, | ||
| catch: storeError("clear", undefined), | ||
| }), |
There was a problem hiding this comment.
signal parameter from Effect.tryPromise is acknowledged via void signal but never propagated to listAllKeys or deleteAllKeys. If the calling fiber is interrupted, the KV pagination loop and all batch deletes will still run to completion, holding open I/O in an Edge worker with strict CPU budgets. Since clear is an async function, it can simply drop the parameter — or better, pass the signal into the KV calls if Cloudflare's types permit it — to make the intent clearer and avoid the lint-suppress pattern.
Suggested change
| clear: Effect.tryPromise({ | |
| try: async (signal) => { | |
| void signal; | |
| await deleteAllKeys(kv, await listAllKeys(kv)); | |
| }, | |
| catch: storeError("clear", undefined), | |
| }), | |
| clear: Effect.tryPromise({ | |
| try: async () => { | |
| await deleteAllKeys(kv, await listAllKeys(kv)); | |
| }, | |
| catch: storeError("clear", undefined), | |
| }), |
@executor-js/cli
@executor-js/config
@executor-js/execution
@executor-js/sdk
@executor-js/plugin-file-secrets
@executor-js/plugin-graphql
@executor-js/plugin-keychain
@executor-js/plugin-mcp
@executor-js/plugin-onepassword
@executor-js/plugin-openapi
@executor-js/codemode-core
@executor-js/runtime-quickjs
executor
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the SDK cache seam future derived-data caching builds on. Supersedes #1118 (open, conflicting with main), rebuilt on current main with the same shape plus the pieces it deferred.
ExecutorConfig.cache?: KeyValueStore+executor.cacheon the executor surface, with a bounded in-memory TTL/LRU fallback. The fallback is Clock-based, so tests pin TTL with the virtual TestClock instead of patchingDate.now.@executor-js/cloudflare/key-value-store: KV-backed adapter + boot layer, mirroring the R2 blob-store binding.makeScopedExecutorreads the store optionally (Effect.serviceOption) and threads it intocreateExecutor— hosts without a KV tier are unaffected.CACHEKV binding (guarded likeBLOBS); the wrangler binding ships commented out until the namespace is created, and everything falls back cleanly without it.No consumers in this PR — the first one (OpenAPI spec-fetch caching) is stacked on top.
Tests
executor-cache.test.ts: fallback round-trip, host-store precedence, TestClock TTL expiry, LRU eviction.key-value-store.test.ts: KV adapter round-trip, paginated size, bounded-parallel clear.Stack