Repository navigation
perf: turn pool_pre_ping off by default and rely on postgres_retry - #54
Merged
Merged
Conversation
Requires db-retry 0.8.0, whose retry costs about 0.14 us per call, and adds a test that a connection terminated while idle in the pool is recovered by the retry.
…ide the Transaction A lost connection on a read before the block left the session in an invalid transaction that nothing rolled back, so the retry failed with PendingRollbackError. With pool_pre_ping off, an idle connection that died in the pool hits this on the first statement.
This was referenced Oct 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
pool_pre_pingis now off by default, andpostgres_retryhandles a pooled connection that died while idle. This removes one round trip on every connection checkout. The setting is stillDB_POOL_PRE_PING, so a deployment can turn pre-ping back on.Why this is safe now
DBAPIErrorthat SQLAlchemy marksconnection_invalidated. That covers asyncpg'sInterfaceError: connection is closed, which a dead idle connection raises on its first use.pyproject.tomlnow requiresdb-retry>=0.8.0, and the lock drops tenacity.Fix: three use cases queried outside the Transaction
A retry only recovers if the retried function owns its whole transaction.
create_chat,create_messageandmark_readran reads beforeasync with self.transaction:create_chat:fetch_direct_by_keyfor direct chats.create_message:is_memberandfetch_by_idempotency_key.mark_read:fetch_memberandget_one_or_none.When one of those reads hit a lost connection, the session was left in an invalid transaction that nothing rolled back, so the retry failed with
PendingRollbackError. This was already true with pre-ping on, for a connection lost mid-request. With pre-ping off it also happens on the first statement after an idle connection dies.Those reads now run inside the block. An early return commits first, so
__aexit__only closes the session and the returned row stays loaded, as in the read use cases since #51. The direct-key validation increate_chatmoved to_direct_keyto keep__call__under ruff's complexity limit.Tests
tests/use_cases/test_connection_loss.py:lost_pooled_connectionfixture: it returns a connection to the pool, then terminates its backend. This is the case pre-ping used to cover. The existinglost_connectionfixture kills a connection the session already holds, which pre-ping never sees.create_messageandmark_read, run against both fixtures. A new test covers thecreate_chatdirect-key lookup.DB_RETRY_RETRIES_NUMBER=1, the pooled read tests fail withInterfaceError: connection is closed, so the default retry is what makes them pass.create_chat,create_messageandmark_readcases failed withPendingRollbackError._watch_backendand_terminate_backend.tests/test_settings.pychecks the new default.Lint and ty pass, and all 141 tests pass with 100% coverage.
Not done
I didn't rerun the benchmark against chat-app's endpoints. The gain comes from the db-retry measurements: a one-row
session.gettook 183 to 193 µs per request with pre-ping on, and 173 to 181 µs with pre-ping off pluspostgres_retryandTransaction. The gap grows with network latency between the app and Postgres.