test(fdv2): stop the recovery test spinning into an OutOfMemoryError - #404
Merged
Merged
Conversation
… tests fallbackAndRecoveryTasksWellBehaved handed the same MockQueuedSynchronizer back from each factory call. Recovery closes the active synchronizer and builds the primary again; the closed mock answers every next() with SHUTDOWN immediately, which advances to the other closed mock, and the orchestrator spun between the two until the test stopped it. The third changeset the test waits for could never arrive, so it spun for the whole ten-second await: locally about 630,000 factory calls and 3.8 million captured log lines, and on CI an OutOfMemoryError. Each factory call now builds a new synchronizer, as a real factory does, and the test asserts the third changeset arrives rather than letting the await time out silently. It finishes in about three seconds. orchestrationLogging_recovery_logsInfo had the same shape and gets the same change. Co-authored-by: Cursor <cursoragent@cursor.com>
…agTest Co-authored-by: Cursor <cursoragent@cursor.com>
abelonogov-ld
enabled auto-merge
September 23, 2026 22:54
tanderson-ld
approved these changes
Sep 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What failed
FDv2DataSourceTest.fallbackAndRecoveryTasksWellBehavedfailed on CI withjava.lang.OutOfMemoryError(seen on the tier-1 event-durability branch, but the test and the cause are onmain):Why
The test's synchronizer factories returned the same
MockQueuedSynchronizerinstance on every call. After fallback, the recovery timer closes the active synchronizer and builds the primary again. That returns the already-closed first mock, and a closed mock answers everynext()withSHUTDOWNimmediately. On AndroidSHUTDOWNadvances to the next synchronizer, which is the other closed mock, andSourceManagerwraps the index. So the orchestrator spun between two closed mocks with nothing blocking it. Each pass logs "Synchronizer '…' is starting." intoLogCaptureRuleand creates and cancels the condition timers.The test then waits up to 10 s for a third changeset. No closed mock can deliver it, and
awaitApplyCountreturns silently on timeout, so the spin ran for the whole wait. With a temporary probe I measured about 630,000 factory calls and 3.8 million captured log lines in that window, with only 2 changesets applied. It passes on a developer machine with enough heap and fails on CI.This is a test bug, not an SDK bug. A real factory builds a new synchronizer each time.
Fix
MockQueuedSynchronizer, as a real factory does. Recovery now gets a working primary that delivers the third changeset.getApplyCount() >= 3after the await. If the shared-instance pattern comes back, the test now fails with anAssertionError(checked by reverting the fix) instead of passing while it spins.orchestrationLogging_recovery_logsInfohad the same shape and gets the same change. It stopped as soon as the log line appeared, so it spun only briefly.FlagTestusesLong.valueOf/Integer.valueOfinstead of the deprecated boxing constructors, which removes the three test-compile deprecation warnings from the same CI log.Verification
fallbackAndRecoveryTasksWellBehavednow takes about 3.0 s, down from the full 10 s wait.FDv2DataSourceTestpassed 5/5 runs; the full unit suite passes (730 tests).Note
Overview
Fixes CI
OutOfMemoryErrorinFDv2DataSourceTest.fallbackAndRecoveryTasksWellBehavedby correcting how mock synchronizer factories behave.The fallback/recovery tests used factories that returned the same
MockQueuedSynchronizerinstances. After recovery closes a synchronizer, reusing it makes everynext()returnSHUTDOWNimmediately, so the orchestrator can spin between two closed mocks (logging and timer churn) while the test waits for a third changeset that never arrives.Changes: factory lambdas now construct a fresh
MockQueuedSynchronizeron each call (matching real factories), in bothfallbackAndRecoveryTasksWellBehavedandorchestrationLogging_recovery_logsInfo. The recovery test also assertsgetApplyCount() >= 3after the wait so a regression fails fast instead of timing out while spinning.Separate cleanup:
FlagTestreplaces deprecatednew Integer/new LongwithInteger.valueOf/Long.valueOfin assertions (compile warnings only).Reviewed by Cursor Bugbot for commit 486b53a. Bugbot is set up for automated code reviews on this repo. Configure here.