Skip to content

slave (M4): add datadir inspection and lifecycle checks - #40

Open
syntrust wants to merge 13 commits into
slave-m3from
slave-m4
Open

slave (M4): add datadir inspection and lifecycle checks#40
syntrust wants to merge 13 commits into
slave-m3from
slave-m4

Conversation

@syntrust

@syntrust syntrust commented Jul 15, 2026

Copy link
Copy Markdown

M4 of #17, stacked on #38 (base: slave-m3). Merge-forwarded onto the rewritten slave-m3 (post-#44/#26).

What

Adds slave inspect --datadir <d>: a read-only, config-free look at what a slave's datadir actually holds.

  • Reports every shard chaindb's stored minor genesis block — hash, height, state root, coinbase and its amount map, both gas limits, root-block linkage, cross-shard cursor — and the chain head.
  • Reports the EVM rule set stored alongside it.
  • Reports a shard only once its block is shown to hold together.
  • Tells an interrupted bootstrap apart from a datadir that is damaged or was never this shard's.
  • Reports a shard it cannot open, read, or validate without hiding the healthy ones, and exits non-zero.
  • Report on stdout, logs on stderr. A running slave holds its chaindb locks, so this is for a stopped node.

Test

  • Boots a real network's shards into a temp datadir and inspects it; the healthy report is checked against pyquarkchain's own create_minor_block() golden values, not against anything re-derived here.
  • Drives the failure paths by damaging an initialized datadir — tampered block, misplaced chaindb, a state this lifecycle cannot produce — and by an unwritable report, asserting each is surfaced and fails the command instead of passing as healthy.
  • Separately drops a stored rule set, the one anomaly the boot path treats as recoverable, and asserts it is reported without failing the command.
  • Covers shutdown for goroutine leaks, and the config-mismatch refusal end to end against the built binary.
  • Full suite green under -race, gofmt and vet clean.

syntrust and others added 5 commits July 15, 2026 17:30
DBDirName and ParseDBDirName make the shard-0x{full_shard_id} naming
convention a single point of truth: boot builds paths through it and the
inspect subcommand parses directory listings back through it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
slave inspect --datadir <d> is read-only and config-free: it scans for
shard-0x{full_shard_id}/ chaindb directories, opens each in pebble
read-only mode, and prints the stored genesis metadata record and chain
head. A shard that cannot be opened or read is reported inline without
aborting the remaining shards, with a non-zero exit if any failed. An
absent metadata record is reported as an interrupted bootstrap rather
than an error, since the next boot re-runs the fresh path.

The README gains the inspect section and a fixtures section covering
provenance and the pyquarkchain cross-validation command.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
goleak now wraps both boot/shutdown smoke-test packages. The ignore list
is empty on purpose: with metrics disabled neither geth nor pebble leaves
a background goroutine behind after Stop(), and keeping it empty means
the real chain's background work is heard here the day it lands.

A new subprocess test initializes a datadir from the mainnet config and
reruns the slave against it with the devnet config, asserting the run
exits 1 and names the stored genesis, the config-derived genesis, the db
path, and 'cluster config changed since initialization'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…mmand

Running the pyquarkchain cross-validation snippet with a bare system
python fails at import time (no aiohttp); it needs a virtualenv with
pyquarkchain's requirements installed. Both READMEs carrying the
command now say so. Verified against both networks: the venv run
reproduces the pinned mainnet 4036783e... and devnet 5ad443ef... hashes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@syntrust syntrust changed the title cmd/slave, qkc/shard, qkc/slave: add datadir inspect and fault coverage (M4) slave (M4): add datadir inspection and lifecycle checks Jul 15, 2026
@syntrust
syntrust marked this pull request as ready for review July 15, 2026 11:41
syntrust and others added 5 commits July 16, 2026 16:20
slave-m3 now carries the final fixture helpers and goleak TestMain in
run_test.go, so this branch's edits to slave_test.go and run_test.go dissolve
into the merge; TestRunGenesisMismatchExitsLoudly relocates into
inspect_test.go next to the initDataDir helper it depends on.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The README sections this branch reshuffled now originate upstream; its diff
reduces to the 'Inspecting a datadir' and follow-up checklist sections.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The inspect labels move with it (meta version -> record version,
genesis metadata -> genesis record), keeping the value column aligned.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
syntrust and others added 3 commits August 13, 2026 16:10
slave-m3 was merge-forwarded onto the base that #44 and #26 rewrote, so this
merge takes its tree wholesale and re-applies only what M4 owns, ported to it.
The GenesisRecord system this milestone was written against no longer exists.

- cmd/slave/inspectcmd.go is rewritten around shard.ReadGenesisBlock: it prints
  the real block (hash, height, state root, coinbase and its amount map, both
  gas limits, hash_prev_root_block, x-shard cursor) instead of a record's
  fields. The misplaced-chaindb check now comes from the block itself, comparing
  its branch's full shard id against the directory name.
- The inspect assertions are pinned against qkc/testdata/minor_genesis_golden.json,
  so what the report prints is pyquarkchain's own create_minor_block() output
  rather than a value derived a second way, and a new test covers the
  branch-versus-directory check.
- The README's inspect docs move under Subcommands next to config and genesis,
  with the current output, and the follow-up checklist is rewritten: no record or
  fingerprint to retire, the scaffolding is the single genesis block key.

Five old-world files this branch still carried are dropped rather than merged:
qkc/genesis/, qkc/shard/genesis.go and their tests, and qkc/types/tokenbalances.go.
The four commits superseded upstream were already dropped on slave-m3.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
inspect deserialized the stored genesis block and printed its fields, checking
only that the block's branch matched the directory. A minor block's hash is its
header's hash alone, so that left the fields hanging off the header unchecked:
a database whose meta had been replaced still reported the original,
authentic-looking block hash next to the substituted state root, and the command
exited 0 with "0 failed". The boot path catches this by comparing the stored
encoding against the config-derived block (shard.ReconcileGenesisBlock), but
inspect is config-free and has no such comparison to fall back on.

Rehash the meta and check it against the hash_meta the header commits to, which
is the check that closes the gap, and require block 0 to be block 0 with an
empty body and both sub-structs present. None of these needs the cluster config.

Move every check, including the misplaced-chaindb one, ahead of all printing, so
a shard that fails is described by its error alone. Printing first and reporting
after put a state root nobody should trust under a hash that looked genuine, and
contradicted what the README already claimed.

Stop reading "no genesis block" as an interrupted bootstrap unconditionally. The
block is written last, once the chain stands, so its absence means an interrupted
bootstrap only on a database holding nothing else; a head pointer with no genesis
under it is unreachable for this lifecycle, and may not be a shard chaindb at
all. Report it instead of claiming the next boot re-initializes it safely.

Read the head hash through Has/Get rather than rawdb.ReadHeadBlockHash, which
discards its Get error and answers a failed read with the zero hash — printed as
"none recorded", a claim about a database that could not be read.

Latch report write errors in a sticky writer and fail on them, so a report that
never reached the reader is not summarized as a success.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An initialized shard chaindb holds three things: the genesis block, the state it
commits to, and the EVM rule set. inspect reported the first two and said nothing
about the third, although it is the other half of what a reopen is checked
against — a shard can stand on exactly the right genesis and still refuse to boot
on an incompatible fork schedule, and nothing showed what schedule was stored.

Print the chain id and the fork schedule alongside the block. The schedule is
rendered from the stored encoding rather than a fixed list of fork fields, so a
rule set carrying a fork this build does not know about is shown rather than
silently dropped, and it is ordered by activation, which is how a schedule reads.

A datadir initialized before its rule set was written reports that instead of
looking complete. It is not a failure: ReconcileChainConfig treats a missing rule
set as recoverable and answers it by warning and writing one, and inspect follows
that judgement rather than inventing a stricter one.

Read the key directly instead of through rawdb.ReadChainConfig, which answers a
failed read and a malformed encoding alike with a nil config — indistinguishable
here from a rule set that was never stored, the same way ReadHeadBlockHash cannot
tell an unreadable head from an absent one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant