Repository navigation
bug: the file change counter (header offset 24) is never incremented — a long-lived sqlite3 reader keeps serving a stale cache #710
Description
Description
The file change counter at header offset 24 is never incremented on a write.
It is preserved rather than zeroed, so a file we write keeps whatever value
it had, and a file we create from scratch keeps its initial one.
It stays consistent with the version-valid-for number at offset 92, which is
why nothing has caught it: PRAGMA integrity_check does not look at either,
and our own reads have no cache to invalidate.
What the counter is for
It is the cache-invalidation signal for another connection. Stock SQLite
reads the change counter when it takes a SHARED lock; if it differs from the
value it saw last time, it discards its page cache and re-reads. Offset 92
holds the change-counter value for which the file-size field at offset 28 is
valid, which is why the two must move together.
So the failure mode is specific: a stock sqlite3 connection that has a
database open, caches pages, then sees our write, and keeps serving its
stale cache — because as far as it can tell nothing changed. No error, no
corruption on disk, just one reader answering from a version of the file that
no longer exists.
Why it matters
This is an interop defect, and interop is the entire premise of the project.
It cannot be seen by any test that only round-trips through our own code, and
it cannot be seen by integrity_check — it needs a long-lived oracle
connection spanning our write, which no test does today: every corpus test
invokes sqlite3 fresh per assertion, so it re-reads the header every time
and never consults its cache.
Found while auditing the embedding API, where it matters more than it used to:
a consumer keeping a stock sqlite3 reader alongside sqlite-rs writers is a
realistic migration path.
Scope
- Increment offset 24 on the first write within a transaction (SQLite does it
once per transaction, not once per page), and set offset 92 to the same
value when the file-size field is valid. src/header.rsowns both fields; the pager's commit path is where the bump
belongs.- Wrapping: it is a
u32and SQLite lets it wrap. Do not treat wrap as an
error.
Non-goals
- The schema cookie (offset 40), which is separate and already maintained by
feat: schema cookie + sqlite_master write maintenance #193's write path. - WAL-mode cache invalidation, which goes through the WAL index rather than
this counter.
Acceptance Criteria
- A test with a long-lived
sqlite3process — one that has read the
database, then observes our write, then reads again — sees the new rows.
This is the test the bug is invisible without; a freshsqlite3per
assertion cannot detect it - Offset 24 increments exactly once per committed transaction, not once
per modified page - Offset 92 equals offset 24 after a commit, so the offset-28 file size
stays valid - A rolled-back transaction leaves both unchanged
- A file we create from scratch has the same initial values stock sqlite3
gives a fresh database (byte-compare against an oracle-created empty
file) - Wrapping past
u32::MAXis not an error
Complexity
Estimate: small
Reasoning: Two header fields, one increment, in a commit path that already
writes page 1. The estimate is small for the fix and almost entirely about the
test: proving it requires a persistent oracle process rather than the
fire-and-forget sqlite3 invocation the corpus harness uses, so the harness
needs a way to hold a session open across our write.
Activity