Skip to content

perf(filter)!: send filter values as parameters by default - #125

Open
pdevito3 wants to merge 1 commit into
mainfrom
fm/qk-breaking-parameterize
Open

pdevito3 wants to merge 1 commit into
mainfrom
fm/qk-breaking-parameterize

Conversation

@pdevito3

Copy link
Copy Markdown
Owner

For later consideration in a major version. Do not merge now. #122 made parameters opt-in to keep v1.x non-breaking. This PR makes them the default again.

The ParameterizeFilterValues setting stays. A consumer can set it to false to keep the v1.14.2 literals.

Summary

 QueryKitSettings
-  bool ParameterizeFilterValues = false
+  bool ParameterizeFilterValues = true

 FilterParser.ParseFilter
-  FilterValue.Parameterize = config is QueryKitConfiguration { ParameterizeFilterValues: true }
+  FilterValue.Parameterize = (config as QueryKitConfiguration)?.ParameterizeFilterValues ?? true
  • v1.14.2: Each filter value is an Expression.Constant. EF Core writes it into the SQL as a literal.
  • New (default): Each value that is not null is a field read on an internal holder. EF Core sends it as a parameter. null stays a constant, so the SQL keeps IS NULL.
  • Opt out: new QueryKitConfiguration(s => s.ParameterizeFilterValues = false) gives the v1.14.2 literals.
  • No config, or a custom IQueryKitConfiguration: gets parameters. These configurations cannot opt out, because the setting is only on QueryKitConfiguration.
  • Why: With literals, each new value makes new SQL text. EF Core then compiles a new query, and the database makes a new plan. With parameters, one filter shape uses one cached plan.

Evidence

Expression text for Title == "a" and Title == "b":

v1.14.2 and main:  x => (x.Title == "a")
                   x => (x.Title == "b")
this PR:           x => (x.Title == value(QueryKit.FilterValue`1[System.String]).Value)   (same text for both)

Generated SQL:

Npgsql, EF Core 10     v1.14.2 and main                 this PR
Title == "abc"         WHERE p."Title" = 'abc'          WHERE p."Title" = @Value
Active == true         WHERE p."Active"                 WHERE p."Active" = @Value
(Age + 5) > 10         WHERE p."Age" + 5 > 10           WHERE p."Age" + @Value > @Value0
Age ^^ [1, 2, 3]       WHERE p."Age" IN (1, 2, 3)       WHERE p."Age" = ANY (@Value)

SQL Server, EF Core 8  v1.14.2 and main                 this PR
Age ^^ [1, 2, 3]       WHERE [p].[Age] IN (1, 2, 3)     ... IN (SELECT [v].[value] FROM OPENJSON(@__Value_0) WITH ([value] int '$') AS [v])

Tests, changed from the restore tests:

unit/integration parameter tests      default config        -> values are parameters
filter_values_are_constants_when_parameters_are_off           LiteralConfig -> constants
filter_values_are_sql_literals_when_parameters_are_off        LiteralConfig -> SQL literals
in_list_is_a_literal_list_when_parameters_are_off_and_still_filters
complex_with_lots_of_types            expects (x.BirthMonth == January), the text on main before the restore

dotnet test on this branch: all unit and integration tests pass, 0 failures (Npgsql through Testcontainers).

Merge Danger

Door: two-way

A later release can change the default back. Consumers can opt out now with one setting.

Blast Radius: consumers

  • Apps that cache compiled filters or results with expression.ToString() as the key. Two different filters get the same key, so the app can return rows for the wrong filter. This fails silently.
  • Apps and tests that compare expression text or SQL text. Nullable enum values print January in place of new Nullable1(January)`.
  • Custom ExpressionVisitor code that reads ConstantExpression values now gets a MemberExpression on an internal type.
  • SQL Server at compatibility level 120 or less with in-list filters. EF Core 8 fails on every in-list, and EF Core 10 fails on very large in-lists (more than about 2100 values). The error is Microsoft.Data.SqlClient.SqlException: Incorrect syntax near '$'. Workarounds: UseSqlServer(conn, o => o.UseCompatibilityLevel(120)), or ParameterizeFilterValues = false.
  • Rows do not change. The v1.14.2 integration tests pass with parameters.

QueryKit wrote each filter value into the SQL as a literal. Each new value made new SQL text, so EF Core compiled a new query and the database made a new plan. ParameterizeFilterValues is now true by default, so one filter shape uses one cached plan. Set it to false to get the v1.14.2 literals.

BREAKING CHANGE: Filter values are now field reads that EF Core sends as SQL parameters. The expression text and the SQL text change, and two filters that differ only in values print the same expression text. In-lists become parameters, which use OPENJSON on SQL Server and need compatibility level 130 or more. Set ParameterizeFilterValues to false to keep the v1.14.2 behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant