Skip to content

fix(filter): count the nesting depth in the grammar - #157

Merged
pdevito3 merged 1 commit into
mainfrom
fm/qk-h1-nesting
Oct 1, 2026
Merged

pdevito3 merged 1 commit into
mainfrom
fm/qk-h1-nesting

Conversation

@pdevito3

@pdevito3 pdevito3 commented Oct 1, 2026

Copy link
Copy Markdown
Owner

Summary

MaxNestingDepth counted every ( and ) in the raw filter text, also inside quoted values. A quoted ) lowered the count. A filter such as Title == ")))…" || (((…Title == "Pancakes"…))) went past the limit and overflowed the call stack. .NET cannot catch a stack overflow, so the process stopped, even with both limits on (H1 in the review of main against v1.14.2).

The grammar now does the count. Each parenthesized group parser increments a per-thread counter when it enters the group and decrements it when it leaves.

 ParseFilter(input, config)
-  EnsureWithinParseLimits        # length check + raw '(' / ')' count
+  EnsureWithinInputLength        # length check only
+  set _maxNestingDepth for this parse
   ExprParser.Parse(input)
     AtomicExprParser
-      ExprParser.Contained('(', ')')
+      Grouped(ExprParser)        # logical group
     ArithmeticTermParser
-      ArithmeticExpressionParser.Contained('(', ')')
+      Grouped(ArithmeticExpressionParser)   # arithmetic group
     PropertyListParser
+      Grouped(properties)        # (A, B) > 3

+Grouped(inner) = '(' counted(inner) ')'
+  counted: depth++; if depth > max throw QueryKitNestingDepthExceededException; finally depth--
  • A ( or ) inside a quoted value is part of the value, so it does not change the count. A fix that only clamps the old count at zero does not stop ((((Title == "))))" && ((((…. The grammar count stops it.
  • Each group adds one level of recursion, so the limit also limits the depth of the call stack.
  • The defaults stay off (int.MaxValue). With the limits off, every input gives the same result as before.
  • With the limits on, a quoted ( no longer counts. For example, Title == "(((((((((" with MaxNestingDepth = 3 parsed before this change, but it threw QueryKitNestingDepthExceededException. Now it parses. Real groups count the same as before.
  • The README text, the IQueryKitParseLimits comment, and the wrong FilterParser comment ("Counting everything can only reject too much") are corrected. The README now also says that MaxNestingDepth does not limit a long flat && or || chain (L2). The grammar for that case is not changed.
  • The verify-querykit harness has a new depth-10 preset (MaxNestingDepth = 10, no length limit) to drive the overflow.

Evidence

Harness (qk run, public ApplyQueryKitFilter, memory and Postgres targets), preset depth-10:

Filter Before After
Title == "<20 x )>" || <20 x (>Title == "Pancakes"<20 x )> parses, [Pancakes], WHERE r."Title" IN ('))))…', 'Pancakes') QueryKitNestingDepthExceededException: … depth of 11 … maximum allowed depth of 10. (both targets)
((((((((Title == "))))))))" && ((((((((Title == "Pancakes")))))))))))))))) (clamp-proof shape) parses QueryKitNestingDepthExceededException (both targets)
same first shape with 5,000 levels (15,034 characters) Stack overflow., exit 134 QueryKitNestingDepthExceededException, exit 2
same first shape with 20,000 levels (60,034 characters) Stack overflow., exit 134 QueryKitNestingDepthExceededException, exit 2
((Title == ")))" || Title == "Pancakes" || Title == """(((""")) parses, [Pancakes]
limits off, first shape with 20 levels [Pancakes] [Pancakes]

New tests in ParseLimitsTests (run against the old FilterParser.cs, then the new one):

quoted_close_parentheses_before_a_group_do_not_lower_the_nesting_depth   fail -> pass
quoted_close_parentheses_inside_a_group_do_not_lower_the_nesting_depth   fail -> pass
repeated_quoted_close_parentheses_do_not_lower_the_nesting_depth         fail -> pass
quoted_parentheses_within_the_nesting_depth_keep_their_value             fail -> pass
quoted_open_parentheses_do_not_count_toward_the_nesting_depth            fail -> pass
arithmetic_groups_count_toward_the_nesting_depth                         pass -> pass
property_list_groups_count_toward_the_nesting_depth                      pass -> pass
deep_filter_with_quoted_close_parentheses_throws_instead_of_overflowing_the_stack
  (20,000 levels on a thread with a 1 MB stack)   "Test host process crashed : Stack overflow." -> pass

Full suites on net10.0: unit 410 passed (402 before, plus 8 new), integration (Postgres via Testcontainers) 283 passed. dotnet build QueryKit -c Release builds net6.0 to net10.0 with no warnings.

Merge Danger

Door: two-way

The change has no public API change and no data change. A revert brings back the old pre-scan.

Blast Radius: small

Only apps that set MaxNestingDepth see a difference. Two kinds of input now parse that threw before: a quoted ( that pushed the old count over the limit, and an input whose quoted ) hid real depth. The second kind now throws. That is the fix. The QueryKitNestingDepthExceededException now throws during parsing, not before it. The message format is the same. The out-of-scope #135 turns the limits on by default and will rebase on this change.

MaxNestingDepth counted every '(' and ')' in the raw filter text, also inside quoted values. A quoted ')' lowered the count, so a filter such as Title == ")))..." || (((...))) went past the limit and overflowed the call stack. The process stopped, even with both parse limits on.

The grammar now counts each parenthesized group (logical group, arithmetic group, and property list) when the parser enters it. Past the limit, the parser throws QueryKitNestingDepthExceededException. A '(' or ')' inside a quoted value does not change the count. The defaults stay off, so a filter without limits gives the same result as before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant