Skip to content
This repository was archived by the owner on Aug 18, 2026. It is now read-only.

fix(postgresql): loosen requirements for getPrimaryKeyWhereClause - #103

Closed
vinhloc30796 wants to merge 2 commits into
streamingfast:developfrom
vinhloc30796:loc/fix/allow-text-to-array-conversion-in-pk-comparison
Closed

vinhloc30796 wants to merge 2 commits into
streamingfast:developfrom
vinhloc30796:loc/fix/allow-text-to-array-conversion-in-pk-comparison

Conversation

@vinhloc30796

@vinhloc30796 vinhloc30796 commented Aug 25, 2025 •

Copy link
Copy Markdown

substreams_history will look like so:

substreams=> select * from substreams_history limit 1;
-[ RECORD 1 ]-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
id         | 75756
op         | I
table_name | "public"."subtransaction"
pk         | {"block_number":"34830163","chain_id":"8453","trace_index":"ARRAY[]::bigint[]","transaction_hash":"0xd24deec488102fe7356efb5c034545f8d9c945b8881764c8b7f70a62a3c9d2a4"}
prev_value | 
block_num  | 34830163

Comment on lines +358 to +361
whereClause, err := d.getPrimaryKeyWhereClauseTyped(table, primaryKey, "")
if err != nil {
whereClause = getPrimaryKeyWhereClause(primaryKey, "")
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this a good way of doing it, can you add a comment explaining the reasoning here, we should also log the error minimally in DEBUG mode if it's expected to log a lot of time.

Comment on lines +327 to +329
if err != nil {
onClause = getPrimaryKeyWhereClause(primaryKey, table.nameEscaped)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, add comment as the why it's correct and log error.

Comment on lines +426 to +429
primaryKeySelector, err := d.getPrimaryKeyWhereClauseTyped(o.table, o.primaryKey, "")
if err != nil {
return "", err
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this backward compatible? The question is essentially, if there is already substreams_history entries`, are they going to be read correctly?

@GabrielCartier

Copy link
Copy Markdown
Contributor

Hi @vinhloc30796 — thanks for this, and sorry for the long silence.

This repository is being deprecated (see #133). The SQL sink now lives in the main substreams CLI as substreams sink postgres / substreams sink clickhouse, shipped in substreams v1.20.2. All future work happens there, so this PR can't land here.

The code moved as-is, so the port is mostly a path rewrite: db_changes/db/dialect_postgres.go is now sink/sql/db_changes/db/dialect_postgres.go in streamingfast/substreams.

The typed primary-key / array-literal handling isn't in the CLI yet, and the underlying problem is still real — if you're up for reopening it against develop there (with the review feedback from this thread addressed), we'll pick it up. The branch conflicts here anyway, so rebasing onto the new tree is roughly the same effort. If you'd rather not, tell us and we'll port it ourselves with credit to you.

Closing this one out.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants