feat(import): carry supabase password hashes from a csv export - #45
Merged
Merged
Conversation
FerrisKey gained POST /realms/{realm}/users/{user_id}/credentials/import,
which stores a password hash produced by another identity provider and
re-encodes it as argon2id on the user's first successful login.
The request type carries salt and temporary even though the realm-import
caller sends neither: bcrypt and argon2 embed their salt in secret_data, and
an imported password must stay usable rather than force a reset. This crate
is published as the FerrisKey SDK, so it mirrors the endpoint rather than
only the one shape the CLI happens to need today.
`realm import --from supabase --source-passwords <file.csv>` migrates passwords, instead of leaving every imported account needing a reset. The export is unavoidable. Supabase's Auth Admin API never serves encrypted_password, so the hash can only come out of the auth.users table. A direct Postgres connection is the planned second path; the CSV is the first because it needs no new credential, no dependency beyond a CSV parser, and works with whatever export route the operator already has. Rows join onto users by auth.users.id, never by email: email is nullable in Supabase, so it is not a key. A hash the server would refuse never enters a blueprint. The prefix window ($2a$, $2b$, $2y$), the 4..=14 cost range and the 53-character body are the server's own rules, restated here so --dry-run tells the truth and a bad row is skipped with a note naming the account instead of aborting a migration halfway through. hash_iterations is read out of the hash itself, because the server rejects any value that differs from the encoded cost. An empty encrypted_password is counted, not warned about: those accounts sign in through a federated provider and have no password to carry. Warning per row would bury the real problems under the normal case. A failed import is a warning rather than a hard stop, unlike every other entity in apply_blueprint. A password is secondary to the user that now exists and is recoverable with `user set-password`, so a single malformed hash must not throw away the whole report after three thousand users. --dry-run redacts every hash in its json and yaml output. Printing the directory's credentials into a terminal that gets scrolled, logged and pasted into tickets is not an acceptable default for a preview. The csv crate earns its place: a `select *` export carries raw_user_meta_data as JSON, full of commas and quotes that a hand-rolled split would tear apart. Header lookup strips a UTF-8 BOM. Supabase's SQL editor emits one on its CSV download, and without this the friendliest export route would fail with "has no 'id' column" on a perfectly good file. Argon2 and Firebase-scrypt hashes, which a project that itself imported users into Supabase may hold, are not carried over yet even though FerrisKey accepts argon2. They are rejected visibly rather than silently mishandled.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The flag told operators a CSV of auth.users was needed but not how to get one, which is the part nobody can guess: the query has to run against a schema the dashboard does not browse, and the obvious `COPY ... TO` fails on a managed instance because it writes server-side. `\copy` is the psql command that writes locally, and the backslash is the whole difference. The connection string advice is there for the same reason. The direct connection Supabase shows first is IPv6-only without the paid IPv4 add-on, so an operator on an IPv4 network hits a timeout with no hint that the session pooler is the answer. The flag help carries it as well as the README because `--help` is where someone mid-migration actually looks. verbatim_doc_comment keeps clap from re-wrapping the command into something that no longer runs when pasted. Both also now say the file holds the whole directory's hashes and should be deleted after the import. Nothing in the tool can enforce that, so the only place it can live is the documentation.
…r user
A real migration against a FerrisKey binary predating the credentials/import
route produced forty-four identical "405 Method Not Allowed" warnings and still
announced the import as done, with zero passwords carried over and an exit code
of 0. Forty-four warnings read as forty-four problems; the actual problem was
one stale server, and the summary line that would have said so did not exist.
The 405 is not arbitrary. Without the import route, the POST falls through to
`credentials/{credential_id}` with credential_id="import", which is registered
for DELETE only, so axum answers 405 with `allow: DELETE`. A server built before
the route existed at all answers 404. Both mean the same thing, so both now end
the attempt: the verdict cannot change for the users that follow, and re-posting
the same doomed request once per account only slows the run down.
passwords_failed is new and always printed. A count of successes alone cannot
distinguish "this directory had no passwords to carry" from "every single one
was refused", and those deserve very different reactions from whoever reads the
report.
Per-user warnings are kept for per-user causes — a hash the server refuses names
the account, because there the account is the thing to go and look at.
Verified against a live server: three bcrypt hashes at costs 8, 10 and 12, in
all three prefix spellings, imported and counted. The 404/405 collapse itself is
covered by unit tests only; the server under test was rebuilt with the endpoint
before that path could be exercised end to end again.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #44.
realm import --from supabase --source-passwords <file.csv>migrates passwords, so imported accounts keep working instead of all needing a reset.Why a CSV, and not the API
Supabase's Auth Admin API never serves
encrypted_password— the hash exists only in theauth.userstable. A direct Postgres connection is the planned second path, agreed to come after this one. The CSV goes first because it needs no new credential, no database reachability, no dependency beyond a CSV parser, and it works with whatever export route the operator already has (SQL editor → Download CSV,psql, or an export Supabase support produced for them).What the server gives us
POST /realms/{realm}/users/{user_id}/credentials/import(added in the FerrisKey commit0c4d4616) takes{algorithm, secret_data, hash_iterations, salt?, temporary?}, stores the hash verbatim, and re-encodes it as argon2id on the user's first successful login. Supabase stores bcrypt, so no rehashing happens on our side and the migration is invisible to the end user.saltis deliberately never sent: bcrypt and argon2 embed their salt insidesecret_data, and the server turns an empty salt into SQLNULL.temporaryis alwaysfalse— an imported password must stay usable, not force the reset this change exists to avoid.Decisions a reviewer cannot read off the diff
Join on
auth.users.id, never on email. Email is nullable in Supabase (phone-only and anonymous accounts exist), so it is not a key. There is a test asserting an email-keyed export produces no credential rather than a wrong match.The server's validation rules are restated client-side. The
$2a$/$2b$/$2y$prefix window, the4..=14cost range and the 53-character body all come fromcore/src/infrastructure/repositories/password_hasher.rs. Duplicating them buys two things:--dry-runreports a truthful count, and a bad row is skipped with a note naming the account instead of surfacing as a 422 three thousand users into a migration.hash_iterationsis parsed out of the hash rather than taken on trust, because the server rejects any value that differs from the encoded cost.An empty
encrypted_passwordis counted, not warned about. Those accounts sign in through a federated provider and have no password to carry. Warning per row would bury the real problems under the normal case, so they get one summary line.A failed password import warns instead of aborting — deliberately unlike every other entity in
apply_blueprint. A password is secondary to the user that now exists, and it is recoverable withuser set-password; losing the whole report to one malformed hash is the worse outcome. A user who already has a password keeps it, reported underalready present.A server without the endpoint is reported once, not once per user. This was found the hard way: a real migration against a FerrisKey binary predating the
credentials/importroute produced 44 identical405 Method Not Allowedwarnings and still announced the import as done, with zero passwords carried and exit code 0. The 405 is not arbitrary — without the route, the POST falls through tocredentials/{credential_id}withcredential_id="import", which is registered for DELETE only, so axum answers 405 withallow: DELETE. A server built before the route existed answers 404. Both now end the attempt, since the verdict cannot change for the users that follow.passwords_failedis reported alongsidepasswords_importedand always printed: a count of successes alone cannot tell "this directory had no passwords to carry" apart from "every single one was refused".--dry-runredacts every hash in its-o json/-o yamloutput. Printing a directory's credentials into a terminal that gets scrolled, logged and pasted into tickets is not an acceptable default for a preview. Verified across all three output formats.The
csvcrate is the one new dependency. Aselect *export carriesraw_user_meta_dataas JSON, full of commas and quotes that a hand-rolled split would tear apart.tokioandrustlswere already in the tree, so the marginal cost is small.Header lookup strips a UTF-8 BOM. Found while reviewing the diff rather than from a failure: Supabase's SQL editor emits a BOM on its CSV download, and without this the friendliest export route would fail with
has no 'id' columnon a perfectly good file.Getting the export
The flag named a CSV of
auth.usersas an input without saying how to produce one, which is the part an operator cannot guess. Both--helpand the README now carry it:Two traps documented alongside it, because each fails in a way that gives no hint of the fix.
COPY … TOis server-side and a managed Supabase instance refuses it —\copyis the psql command that writes locally, and the backslash is the entire difference. And the direct connection string the dashboard offers first is IPv6-only without the paid IPv4 add-on, so an operator on an IPv4 network gets a timeout rather than an error pointing at the session pooler.The flag help uses
verbatim_doc_commentso clap does not re-wrap the command into something that no longer runs when pasted.-hstill shows a single line; the detail lands in--help.Both places also state that the export holds every hash in the directory and should be deleted once the import has run. Nothing in the tool can enforce that, so documentation is the only place it can live.
Not in this change
--source-db-url) — the agreed second pass.--source-passwordsis rejected outright on--from config|keycloak|zitadelrather than silently ignored. TheUserBlueprint.credentialfield it fills is source-agnostic and does work through theconfigsource, but that path is untested against a live server and undocumented on purpose.Verification
cargo test --workspace— 154 tests, green (23 new).cargo clippy --workspace --all-targets --all-features -- -D warnings— clean.--help; a missing file, a wrong-column file and a wrong--fromeach produce their own error message; a dry run reportspasswords: 1of 2 users and leaks the hash in none of the three output formats.$2a$,$2b$,$2y$), imported and counted.cargo fmt --checkstill reports diffs on this repo, including files this branch never touched — the tree was not formatted with the current rustfmt and CI does not gate on it. Every line this branch adds is fmt-clean; pre-existing lines were left alone so the diff stays reviewable.