Repository navigation
Backport #7755 and #7861 to release-1.22 - #7874
Merged
CharlieTLe merged 2 commits intoOct 6, 2026
Merged
CharlieTLe merged 2 commits into
CharlieTLe merged 2 commits into
Conversation
… parquet converter (cortexproject#7755) Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com> Co-authored-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com> (cherry picked from commit 1f6ce6b) Signed-off-by: Charlie Le <charlie_le@apple.com>
… flake (cortexproject#7861) TestBlocksCleaner still flakes after cortexproject#7386. That change made the per-tenant cleanup wait for the visit marker HeartBeat goroutine to exit, closing the race between the heartbeat's final writes and the test's t.Cleanup. The heartbeat's first write can still overlap the cleaner's own deletions under the same tenant: the filesystem bucket's Delete removes each emptied parent directory with os.RemoveAll, and a concurrent Upload of <tenant>/markers/cleaner-visit-marker.json makes it fail with "unlinkat <bucket>/user-4/markers: directory not empty". deleteUserMarkedForDeletion then returns before deleting the tenant deletion mark, which is the failure seen on CI (user-4's mark still exists at blocks_cleaner_test.go:275). Object stores have no directories, so the race only affects the filesystem bucket, where a failed run is retried on the next cleanup cycle. Switch TestBlocksCleaner to the in-memory bucket, like cortexproject#7486 did for TestBlocksCleaner_ShouldRemoveBlocksOutsideRetentionPeriod, and assert that no cleaner visit marker is left once the initial cleanup has completed, so the test keeps guarding the cortexproject#7386 fix. Doing so exposed a production bug: DeleteTenantDeletionMark deletes the mark from the global and then the legacy per-tenant location and returns any error. The per-tenant mark is never there at that point (since cortexproject#5676 marks are only written to the global location, and the tenant's markers/ prefix was deleted just before), so on object stores whose Delete fails for a missing object (GCS, Azure, Swift, OCI) the final cleanup of every deleted tenant is reported as failed. Ignore not-found errors, like bucketindex.DeleteIndex and DeleteIndexSyncStatus already do, and assert in TestBlocksCleaner that the deleted-tenants run completes without failures. Fixes cortexproject#7564 Signed-off-by: Sandy Chen <Yuxuan.Chen@morganstanley.com> (cherry picked from commit cc57488) Signed-off-by: Charlie Le <charlie_le@apple.com>
CharlieTLe
added a commit
that referenced
this pull request
Oct 6, 2026
CharlieTLe
added a commit
that referenced
this pull request
Oct 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backports two bug fixes to
release-1.22. They would go out inv1.22.0-rc.3.#7861: Compactor tenant cleanup reported as failed
When the compactor does the final cleanup of a tenant marked for deletion, it reports the cleanup as failed on object stores that return an error when deleting a missing object (GCS, Azure, Swift, OCI). The fix is in
pkg/util/users/tenant_deletion_mark.go. The PR also fixes theTestBlocksCleanerflake. On master its CHANGELOG entry sits undermaster / unreleased. Here it is moved into the1.22.0 in progresssection.#7755: Parquet converter error messages
Error messages in the parquet converter referred to the compactor ring instead of the parquet converter ring. Only the message text changes. Master's CHANGELOG already lists this under
1.22.0 in progress, but the commit was never onrelease-1.22. Without this backport, 1.22.0 would ship without a fix its changelog claims.Both commits cherry-picked cleanly with
-x. The only manual edit is the CHANGELOG placement of #7861.Verification
go build -tags "netgo slicelabels" ./...: cleango test -tags "netgo slicelabels" ./pkg/parquetconverter/... ./pkg/util/users/... ./pkg/compactor/...: ok