Skip to content

ntx-builder: move to sqlite framework#2353

Open
SantiagoPittella wants to merge 5 commits into
nextfrom
migrate-ntx-builder-off-diesel
Open

ntx-builder: move to sqlite framework#2353
SantiagoPittella wants to merge 5 commits into
nextfrom
migrate-ntx-builder-off-diesel

Conversation

@SantiagoPittella

Copy link
Copy Markdown
Collaborator

Summary

Migrates the ntx-builder database layer off Diesel and onto the shared miden-node-db SQLite framework.

Why:

  • Removes the last Diesel dependency and its schema/codegen tooling from the ntx-builder.
  • Unifies persistence on one connection-pool + transaction model across the node.

How:

  • Replaces the Diesel query modules with plain query functions over ReadTx/WriteTx and hand-written SQL in db/sql/.
  • Drops the pinned "loop connection"; writes now serialize through the framework's single writer connection, reads use the shared reader pool.
  • On-disk layout is unchanged (same to_bytes()/read_from_bytes() codec, same 001_initial.sql schema), so existing databases stay compatible.

Changelog

changelog = "none"
reason    = "Internal change only."

@Mirko-von-Leipzig Mirko-von-Leipzig changed the title chore: mvoe ntx-builder to sqlite framework ntx-builder: move to sqlite framework Jul 17, 2026
Comment thread bin/ntx-builder/src/db/queries/note_scripts.rs Outdated
Comment thread bin/ntx-builder/src/db/queries/mod.rs Outdated

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some thoughts, leaving inline so we can discuss more easily.

First some organizational ones:

  • I think we should wrap the db crate's read and write tx again here, and then expose the queries / writes as methods instead of free functions.
  • I would prefer separate folder per sqlite query i.e. not bundling multiple per file, and no sql folder.

More important though:

  • Everywhere still has access to the Database struct.
    • This means everywhere can still write if they so choose.
  • I think the SQLite framework should not give back the Database struct on construction.
    • Instead return: (WriteTx, Pool<ReadTx>).
    • That way we cannot mix and match.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I implemented the wrapper struct (NtxDb) and replace the read/writes with methods, and did that folder organization that you mention. It looks cleaner now.

Regarding the other comments:

Everywhere still has access to the Database struct.

With the wrapper this was fixed, since all read/writes happens through the methods.

I think the SQLite framework should not give back the Database struct on construction.

I don't think I fully understand, so I want to clarify before changing things. Essentially in the ntx-builder we have the block subscription, the server, and the actor context. And only the block stream one (main loop) needs write access, so what I should do is to have 2 wrapper structs, one that only implements reading methods, and another with write capacity?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants