Skip to content

CASSANDRA-19569 Advance the offline SSTable rewrite cache advice position - #5140

Open
cheeeee wants to merge 1 commit into
apache:trunkfrom
cheeeee:CASSANDRA-19569-trunk
Open

cheeeee wants to merge 1 commit into
apache:trunkfrom
cheeeee:CASSANDRA-19569-trunk

Conversation

@cheeeee

@cheeeee cheeeee commented Sep 10, 2026

Copy link
Copy Markdown

CASSANDRA-19569 reports sstableupgrade running far below disk throughput, with a profile dominated by
posix_fadvise.

SSTableRewriter.maybeReopenEarly is called on every append and is meant to skip the page cache of the
source readers once per preemptiveOpenInterval bytes of output. The offline branch never updated
currentlyOpenedEarlyAt, so once the output exceeded the interval the condition held for every appended
partition, and each append performed an index lookup plus a POSIX_FADV_DONTNEED over [0, current
position) on every source file. The online branch already records the position when it reopens early.

Change: record writer.getFilePointer() after the offline cache skip, so the advice is issued once per
interval. One line, plus a corrected field comment (the value is a byte position, not MiB).

Testing:

  • SSTableRewriterTest.testOfflineCacheSkippingAcrossWriterSwitches: Byteman on
    NativeLibrary.trySkipCache(String, long, long) records every advised range while an offline rewriter
    appends across three writers; asserts one advice per interval with the expected ranges. Without the fix
    it fails with an extra advice on the partition after the interval is first exceeded.
  • Existing SSTableRewriterTest suite (22 tests) passes.
  • Throughput on the same offline path (ext4, 50 MiB interval, JDK 11, median of three runs): 124 MB of
    ~1 KB partitions takes 6980 ms (17.0 MB/s) without the fix and 1894 ms (62.6 MB/s) with it; with
    ~84 B partitions, 5.8 MB/s against 18.0 MB/s, since the cost is per partition. bpftrace shows
    69,394 fadvise64 calls before against 3 after and 111,048 openat calls against 35,251, as
    trySkipCache(String, long, long) reopens the data file per call; only 253 ms of the difference is
    inside fadvise64. The ticket's ramdisk asymmetry did not reproduce: tmpfs is equally slow unpatched.

CASSANDRA-19569

Record the current output byte position after an offline cache-skip request
so later partitions wait for the next preemptive interval. Exercise request
ranges across populated and empty writer switches.

CASSANDRA-19569

Generated-by: Claude (Anthropic)
@cheeeee
cheeeee force-pushed the CASSANDRA-19569-trunk branch from db95802 to adc1524 Compare September 11, 2026 22:50
@cheeeee cheeeee changed the title CASSANDRA-19569: Fix currentlyOpenedEarlyAt not updating in offline SSTableRewriter CASSANDRA-19569 Advance the offline SSTable rewrite cache advice position Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant