Skip to content

fix(parser)!: read date values without an offset as utc - #116

Open
pdevito3 wants to merge 1 commit into
mainfrom
fm/qk-breaking-prs-114
Open

pdevito3 wants to merge 1 commit into
mainfrom
fm/qk-breaking-prs-114

Conversation

@pdevito3

Copy link
Copy Markdown
Owner

Summary

This PR reads a DateTime or DateTimeOffset filter value without an offset as UTC. This is a breaking change. It is one of the six breaking changes that were removed from #114 so that main stays compatible with v1.14.2.

Status: for later consideration. Do not merge this PR now. The captain decides on each breaking change separately. If this change is accepted, it needs a major version, or the opt-in setting below.

Old behavior (v1.14.2 and main)

  • The parser reads a value without an offset in the time zone of the server (DateTimeStyles.AssumeLocal).
  • A scalar DateTime value gets DateTimeKind.Local.
  • A list value and a custom operation value use AdjustToUniversal without AssumeUniversal. Thus they use a different rule from the scalar value.
  • On main, a scalar DateTime value is a query parameter (since perf: parameterize filter values and cache parsers and regexes #110). Npgsql rejects a Local DateTime for a timestamp with time zone column. As a result, the filter throws on Postgres in every server time zone.

New behavior

  • The parser reads a value without an offset as UTC (AssumeUniversal | AdjustToUniversal).
  • The parser converts a DateTime value with an offset to UTC.
  • The scalar path, the list path, and the custom operation path use the same two helpers, ParseDateTime and ParseDateTimeOffset.

Example

SpecificDateTime == 2022-07-01T00:00:03
  • Old: x => (x.SpecificDateTime == new DateTime(637922304030000000, Local)). On a server in UTC+3, this value is 2022-06-30T21:00:03Z.
  • New: x => (x.SpecificDateTime == new DateTime(637922304030000000, Utc)). The instant is 2022-07-01T00:00:03Z on every server.

Proof from the verify-querykit harness, on a machine in UTC+3. Pancakes has CreatedAt 2024-01-15 08:00 UTC.

Filter Target main This PR
CreatedAt == 2024-01-15T08:00:00 memory Pancakes (in memory, DateTime equality ignores the Kind) Pancakes
CreatedAt == 2024-01-15T08:00:00 Postgres ArgumentException: "Cannot write DateTime with Kind=Local to PostgreSQL type 'timestamp with time zone'" Pancakes, @Value='2024-01-15T08:00:00.0000000Z'
CreatedAt == 2024-01-15T10:00:00+02:00 memory and Postgres not run Pancakes, @Value='2024-01-15T08:00:00.0000000Z'

Justification

A filter string comes from a client, and the client does not know the time zone of the server. In v1.14.2, the same filter matches different rows on servers in different time zones. The results can also change after a deploy to a new region, or after a change to the container time zone.

Most APIs store and compare instants in UTC, and Npgsql requires UTC for timestamp with time zone. When the parser reads a value without an offset as UTC, the result depends only on the filter string. The change also removes the difference between scalar values and list values, which is a fault on its own.

Migration and opt-in alternative

  • A consumer that needs local time can send the offset in the value, for example 2022-07-01T00:00:03+03:00.
  • Alternative: add an opt-in setting, for example DateTimeKindForValuesWithoutOffset. The setting can keep the v1.14.2 default until the next major version. This PR does not add the setting.

Tests

  • FilterParserTests now expects Utc in 3 places, not Local.
  • New unit tests: date_time_without_offset_is_utc and date_time_list_value_matches_scalar_value.
  • New integration test: date_time_without_offset_is_utc.
  • dotnet test: 356 unit tests and 276 integration tests pass.

A DateTime or DateTimeOffset value without an offset was read in the time zone of the server. The same filter matched different rows on servers in different zones. List values and custom operation values also used a different rule from scalar values.

A value without an offset is now read as UTC. A DateTime value with an offset is converted to UTC. The scalar, list, and custom operation paths use the same two helpers.

BREAKING CHANGE: a DateTime or DateTimeOffset filter value without an offset is read as UTC, not in the time zone of the server. A scalar DateTime value now has DateTimeKind.Utc, not DateTimeKind.Local. To keep a local time, send the offset in the value, for example 2022-07-01T00:00:03+03:00.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant