Fix wrong rows from rel property IN filters over rel table groups#699
Merged
Conversation
…DB#692) updateSelectedPos skipped writing the selection buffer whenever the current selection size was 1, assuming the state is flat. An unflat chunk with a single row also took that shortcut. The caller then activated the selection buffer through setToFiltered, exposing positions left over from a previous batch. A rel property IN filter over a rel table group could then duplicate a row from one endpoint table and drop the row from another. Take the shortcut only when the result state is really flat, so unflat single row batches write the buffer like any other batch. Adds a COPY based regression test that reproduces the corruption deterministically. The new case fails without the fix and passes with it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #692
Summary
A filter like
WHERE r.type IN ['CALLS']can return wrong rows on a rel table group. A row from one endpoint table came back twice and the row from another endpoint table was dropped. Equality filters on the same data were always correct.Cause
IN becomes list_contains, which has no native select implementation. The filter falls back to computing a boolean vector and then converting it into selected positions in updateSelectedPos. That conversion skips writing the selection buffer whenever the current selection has size 1, assuming the state is flat. But an unflat scan batch with a single row also takes that shortcut. The caller then calls setToFiltered, which switches the selection vector to the buffer that was never written. The buffer still holds positions from a previous batch, so the filter picks the wrong row. I confirmed this by stepping through the failing query in gdb and watching the selection vector across filter batches.
Fix
Take the shortcut only when the result state is really flat. Unflat single row batches now write the selection buffer like any other batch.