Skip to content

CASSANDRA-19988 Honor background_read_disk_access_mode across SSTable maintenance and streaming - #5143

Open
cheeeee wants to merge 1 commit into
apache:trunkfrom
cheeeee:CASSANDRA-19988-trunk
Open

CASSANDRA-19988 Honor background_read_disk_access_mode across SSTable maintenance and streaming#5143
cheeeee wants to merge 1 commit into
apache:trunkfrom
cheeeee:CASSANDRA-19988-trunk

Conversation

@cheeeee

@cheeeee cheeeee commented Sep 10, 2026

Copy link
Copy Markdown

CASSANDRA-19988: extend direct-I/O reads from compaction to the other background operations that scan
whole SSTables, and rename the setting accordingly.

Changes:

  • compaction_read_disk_access_mode is renamed background_read_disk_access_mode (Config @Replaces; the
    old key is still accepted and reported as deprecated). Java accessors are renamed with it; the
    ticket's setting shipped in 6.0-alpha only.
  • Scrub, verify, SAI, SASI and Accord route index builds open their data readers with the configured
    background mode, as compaction already did.
  • Partial streaming (CassandraStreamWriter, CassandraCompressedStreamWriter) reads stored bytes through
    StreamingFileReader, which under direct mode opens the file with ExtendedOpenOption.DIRECT and stages
    reads in a 1 MiB buffer aligned to the filesystem block size (block size is looked up once per
    directory and cached), handling unaligned section starts and the unaligned tail at end of file.
    Checksums and compressed chunk boundaries are served from the staged bytes exactly as before.
  • Entire-SSTable streaming: sendfile from an O_DIRECT descriptor is either bounce-buffered or served
    from the page cache by the kernel, so under direct mode components are read through
    StreamingFileReader and written in 64 KiB batches; when the direct open did not actually take effect
    the reader is closed immediately and the zero-copy sendfile path is kept.
  • Fallback is decided from the actual open: if the JDK cannot enable direct I/O on the file (it sets
    O_DIRECT with fcntl after opening and throws IOException when the kernel rejects it) or the block
    size is unusable (0, not a power of two, or larger than the staging buffer), StreamingFileReader
    falls back to a buffered channel and logs a rate-limited WARN naming the file and reason. The
    streaming path inspects no errno values and probes no temporary file; scrub, verify and index builds
    go through the pre-existing FileHandle.supportsDirectIO() check that compaction already used.
  • cassandra.yaml/cassandra_latest.yaml document the new name, that on macOS DIRECT is only a cache
    retention hint (F_NOCACHE) and that tmpfs accepts it without effect; NEWS.txt gets a deprecation
    entry for the old key.

Testing:

  • BackgroundStreamingTest (4): the raw reader returns identical bytes in standard and direct mode
    across unaligned start/end boundaries of a file in the SSTable's data directory; compressed and
    uncompressed partial streams reproduce the stored chunks/checksum slices; an entire-SSTable direct
    stream reproduces every component of a compressed and an uncompressed SSTable.
  • CassandraEntireSSTableStreamWriterTest (2) and YamlConfigurationLoaderTest.backgroundReadModeAcceptsLegacyName.
  • Kernel-level check, Linux 6.19: on ext4 an O_DIRECT read of 8 MiB leaves 0 pages resident and an
    unaligned read is rejected with EINVAL; on tmpfs O_DIRECT is accepted but all pages stay resident
    (which is why the yaml calls it a no-op there); on a filesystem without O_DIRECT support the kernel
    rejects the flag with EINVAL, which the JDK surfaces as the IOException the fallback catches.
  • Not measured: streaming or scrub throughput under direct mode, and the page-cache effect on
    foreground reads. Incoming (receive-side) streaming is not covered by this change.

CASSANDRA-19988

Route maintenance readers through the background read setting and retain
the legacy YAML setting replacement. Use bounded aligned reads for direct
streaming of physical component bytes, including partial compressed chunks.
Keep buffered fallback and the existing zero-copy path for non-direct
entire-SSTable streaming.

CASSANDRA-19988

Generated-by: Claude (Anthropic)
@cheeeee
cheeeee force-pushed the CASSANDRA-19988-trunk branch from cd631b4 to e328bce Compare September 11, 2026 22:50
@cheeeee cheeeee changed the title CASSANDRA-19988: Direct IO support for background reads CASSANDRA-19988 Honor background_read_disk_access_mode across SSTable maintenance and streaming Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant