fix: re-land CIP-002 — stranded by the #20 squash-merge - #24
Merged
Conversation
Implements CIP-002: turns the c0mpute-store engine into a usable service and
puts it behind a CLI. Single node — every shard lands on the local disk and
host_hint stays None; cross-node placement is CIP-003.
c0mpute-store
- Tier (hot / standard / critical) carrying (k, parity) and CIP-001 pricing.
hot is RS k=1, which makes parity shards byte-identical copies: real 3-copy
replication with 1x repair amplification.
- Block layer: objects split into blocks, each independently RS-encoded, so
memory is bounded by block size and a range read touches only the blocks it
needs. CIP-007's random-access files depend on this.
- Manifest v2 (version, block_size, tier, blocks[]) with a shim that still
parses v1.
- block_size_for() scales 4 MiB -> 256 MiB with object size, keeping a 1 TiB
object's manifest under a megabyte instead of ~262k blocks.
- put_stream / read_stream / get_range. put is now a wrapper over put_stream
so there is one write path.
- Storage::list() for enumerating what a node holds.
c0mpute-gateway
- /storage/v1/{objects,shards,manifests,status} with Range support, tier
selection, idempotent PUT, single-flight per hash, and a disk budget.
- Commit-then-verify on every write: bytes must hash to the hash the caller
committed to, or 422.
- ed25519 signed-request auth (DIP-0007 shape) bound to method + path + body
hash, with a 5-minute skew window. Writes require it; reads do not, since
the hash is the capability.
c0mpute-cli
- `c0mpute storage put|get|ls|info|verify|rm|status|tiers|serve`, per
plugins/storage/module.toml. Commands that need later CIPs (volume, mount,
provide) are absent rather than stubbed, and --help says which CIP brings
each one.
Two bugs found by running it, not by the tests:
1. Data loss on rollback. A write that fails its hash commitment used to delete
every shard hash it touched. Shards are content-addressed and shared, so
re-uploading an existing object's bytes under a wrong hash produces the same
shard hashes — the rollback deleted the intact object's shards. One
malformed request destroyed real data. ChunkStore::put_new now reports
whether it created a chunk, and rollback only removes what it created.
Regression tests at both the store and HTTP level.
2. Tracing wrote to stdout, so `HASH=$(c0mpute storage put f)` captured log
lines and every scripted use broke. Diagnostics now go to stderr; daemon
mode is unaffected because it points both at the same log file.
Also: Config's api/storage/gateway sections get serde defaults, so a partial
config.toml that sets only [storage] root loads instead of erroring.
65 new tests (31 store, 12 auth, 22 HTTP integration); workspace is green and
clippy-clean on the touched crates. Verified end to end against a running
server: 12 MiB round-trips byte-identical, range reads match dd, and an object
still reconstructs after deleting 4 of 14 shards.
Claude-Session: https://claude.ai/code/session_01LsQAuvXkmyHTgnvquLHrRx
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
ThreatCrush Security Scan8 finding(s) HIGH/CRITICAL: 5 | MEDIUM: 3
Snippets are redacted; ThreatCrush never prints matched credential material. |
This was referenced Aug 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-lands the CIP-002 implementation, which never reached
masterdespite #21 showing MERGED.What happened
#20 (
feat/storage-cips→master) was squash-merged first, capturing the branch as it stood at that moment — the docs commit only. #21 then merged intofeat/storage-cipsafterwards. GitHub marks #21 merged because it reached its base branch; that base never reachedmasteragain.Result:
master's tip is the #20 squash (ed85c93), which containsdocs/prds/but none of the code. Two commits were stranded onfeat/storage-cips:Confirmed by file:
node/crates/c0mpute-store/src/tier.rsandnode/crates/c0mpute-gateway/src/storage_api.rsdo not exist onmaster.The fix
This branch is
origin/masterplus a cherry-pick of917d001— nothing else. It re-lands exactly the change you already reviewed and approved in #21, with no content changes.Only the CIP-002 commit is replanted.
a949ae2is deliberately left behind: its content is already onmasterinside the #20 squash, so replaying it would apply the docs on top of themselves.Do not merge
feat/storage-cipsinto master instead — that would do exactly that.Verified
cargo test --workspacegreen on this branch: 25 suites, including the 34 storage/auth/HTTP tests from CIP-002: storage HTTP API +c0mpute storageCLI #21.cargo build --workspaceclean.Follow-up
PR #23 (CIP-003) was stacked on
feat/cip-002-storage-api, which is also off-master. It is superseded by a branch rebuilt on top of this one; #23 is closed with a pointer rather than force-pushed.Once this merges,
feat/storage-cips,feat/cip-002-storage-apiandfeat/cip-003-placementare all safe to delete.🤖 Generated with Claude Code
https://claude.ai/code/session_01LsQAuvXkmyHTgnvquLHrRx