Skip to content

fix(filter)!: keep a comma inside a quoted list item - #142

Open
pdevito3 wants to merge 1 commit into
mainfrom
fm/qk-breaking-quoted-list-comma
Open

pdevito3 wants to merge 1 commit into
mainfrom
fm/qk-breaking-quoted-list-comma

Conversation

@pdevito3

@pdevito3 pdevito3 commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

For later consideration in a major version. Do not merge now. #134 restored the v1.14.2 list split to keep v1.x compatible (restore commit 01e8b50). This PR re-applies c90861c. The captain decides on this PR separately.

Summary

  • The list parser escapes , and \ in each quoted item before it joins the items.
  • SplitListItems splits the list only on a comma that is not escaped, then trims each item. The in and not-in paths use it.

v1.14.2 behavior (and main)

The list text is split on every comma with Split(','). A comma inside a quoted item also splits the item.

New behavior

A quoted item is one item, also when it holds a comma. An unquoted list ([20, 30, 40]) does not change.

Example

FilterParser.ParseFilter<TestingPerson>("""Title ^^ ["Warm, with syrup", "a\b"]""");
  • v1.14.2 and main: the list has three items, Warm, with syrup, and a\b. A person with the title Warm, with syrup does not match.
  • This PR: the list has two items, Warm, with syrup and a\b. The person with the title Warm, with syrup matches.

Justification

Quotes mark where a value starts and stops. A comma inside the quotes is part of the value. The old split made it impossible to filter by a value that holds a comma.

Migration

A caller that put many values in one quoted item must write each value as its own quoted item: ["Warm", "with syrup"].

README

The in operator example now tells that a quoted list item can hold a comma.

Tests

These tests come back from main before #134:

  • Unit FilterParsingRegressionTests.list_value_with_comma_is_split_into_items -> list_value_with_comma_is_one_item (theory, 5 cases: 4 cases expect one item again, the trim case is the same)
  • Integration FilterParsingRegressionTests.list_value_with_comma_is_split_into_items -> list_value_with_comma_is_one_item (Postgres, the in and not-in rows swap back)

dotnet test: 470 unit tests and 297 Postgres integration tests (Testcontainers) pass, 0 failures.

Rebase on main

This branch is rebased on current main. On main, the list parser also returns a flag for a dot-only number in the list. The rebase keeps this flag and escapes each item value with EscapeListItem.

A list value split on every comma, also on a comma inside a quoted item. Title ^^ ["Warm, with syrup"] matched Warm and with syrup, not the one title. Escape the commas in each quoted item and split the list only on the commas between items.

BREAKING CHANGE: a quoted list item with a comma is now one item. In and not-in filters with such an item return different rows.
@pdevito3
pdevito3 force-pushed the fm/qk-breaking-quoted-list-comma branch from 470a70f to f6037a3 Compare October 1, 2026 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant