fix(chainlink): incremental WHERE must be OR, not AND, across a FULL … - #10048
Open
burugupallyprem-coder wants to merge 2 commits into
Open
burugupallyprem-coder wants to merge 2 commits into
burugupallyprem-coder wants to merge 2 commits into
Conversation
…OUTER JOIN A node-day with no reverted transaction has reverted.block_time = NULL, so NULL >= cutoff is NULL and the row is dropped. Every incremental run degrades the FULL OUTER JOIN to an INNER JOIN. With incremental_strategy='merge' those node-days are never written. The other 13 models of the same family -- ocr_request_daily and automation_request_daily on every chain -- already use OR. This makes the 14 match the 13. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Bugbot needs on-demand usage enabledBugbot uses usage-based billing for this team and requires on-demand usage to be enabled. A team admin can enable on-demand usage in the Cursor dashboard. |
|
CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅ |
Author
|
I have read the CLA Document and I hereby sign the CLA |
burugupallyprem-coder
marked this pull request as ready for review
September 28, 2026 04:44
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
14 of the
chainlink_<chain>_*_request_dailymodels filter both sides of a FULL OUTER JOIN in the incrementalWHERE, joined withAND:incremental_predicate(col)expands tocol >= <cutoff>. A node-day with no reverted transaction hasreverted.block_time = NULL,NULL >= cutoffisNULL, and the row is dropped. So on every incremental run the FULL OUTER JOIN degrades to an INNER JOIN, and the only rows that survive are node/timestamp pairs that have both a fulfilled and a reverted transaction at the identicalblock_time.That coincidence is rare. With
incremental_strategy='merge'on(date_start, node_address), nothing is deleted — the table simply stops receiving new rows and continues to look healthy.The models' own output columns say unmatched rows were expected:
The fix
AND→OR, in 14 files. WithOR, a fulfilled-only row passes on the first disjunct (TRUE OR NULL=TRUE) and a stale pair still fails (FALSE OR FALSE=FALSE).This is not a new idea — 13 models in the same family already do it this way. Every
ocr_request_dailyandautomation_request_daily, on every chain, usesOR. Normalised for the product name,chainlink_arbitrum_fm_request_daily.sqlandchainlink_arbitrum_ocr_request_daily.sqlare identical files apart from the alias, apost_hook, and this one word. This PR makes the 14 match the 13.Measured on your production tables
Queried on Dune, 28 September 2026:
Scope and what this PR does not fix
14 files, one word each:
fm_request_dailyandvrf_request_dailyon arbitrum, avalanche_c, bnb, ethereum, fantom, gnosis, optimism and polygon.One thing deliberately left alone: the join is on
(block_time, node_address), which is not unique in either input — both upstream models declareunique_key = (tx_hash, tx_index, node_address), one row per transaction. A node with F fulfilled and R reverted transactions in the same block produces F×R rows and both counts report F×R. That is present on full refresh too and is a separate change; I have not measured how often it fires and did not want to mix it into a one-word fix. Happy to open it as its own issue.How I found it
I maintain a static analyser for grain and join defects in analytical SQL, and spellbook is one of the repositories I run it against. It flagged this shape, and I then checked it by hand against your own
ORmodels and by running the compiled CTE. Everything above is verifiable from the repo and from public Dune tables.