fix: bound in-flight memory at high thread counts (stacked on #723)

[dougnukem]

Stacked on OpenGene#723 (base branch is fix/backpressure-scales-with-threads-721).

Summary

packInMemLimit(threads) (added in OpenGene#723) can't safely drop below 2*threads
without reintroducing the OpenGene#721 deadlock, so it has no slack left to also cap
total buffered memory -- that limit already grows linearly with thread count
on purpose. This scales the pack size down instead once thread count
exceeds the point where packInMemLimit's own floor stops mattering, so
packInMemLimit(threads) * packSize(threads) (roughly the total buffered
read count) stays close to constant instead of growing linearly with
threads, with a floor (100 reads/pack) so packs don't shrink far enough for
per-pack overhead to dominate. See the comment on packSize() in
src/common.h for the exact reasoning.

This only changes behavior once thread count exceeds 16 (PACK_IN_MEM_LIMIT / 2) -- below that, packSize() returns the original constant 1000 and
nothing changes.

Validation

24-run sweep, fix/backpressure-scales-with-threads-721 (base) vs this
branch, using the corpus and scripts from OpenGene#725 (3 datasets x 4 thread counts
x 2 branches, on a 48-vCPU host):

• Correctness: zero output-digest mismatches across every run.
• Performance: timings match within noise at -w 1/16 (packSize is
  unchanged there); a small ~5-10% cost at -w 48 on some datasets, the
  expected tradeoff of processing more, smaller packs per unit of gzip
  output -- traded for materially bounded memory growth at high thread
  counts.

dataset	-w 1	-w 16	-w 32	-w 48
atac (base / this branch)	31.3s / 31.8s	7.9s / 7.6s	8.7s / 8.5s	9.5s / 10.5s
wgs (base / this branch)	38.4s / 39.0s	26.0s / 26.4s	26.5s / 26.8s	27.0s / 28.9s
synth (base / this branch)	9.1s / 9.0s	3.0s / 3.0s	3.0s / 3.1s	3.2s / 3.3s

Internal unit tests (fastp test) pass, and SE / interleaved-PE / non-interleaved-PE
modes were each spot-checked at 1/8/32/64 threads against the base branch
with matching output digests.

An idea I tried and reverted

I also attempted distributing packs to whichever worker queue currently
holds the fewest in-flight packs, instead of strict round-robin (the other
follow-up suggested alongside this one). I reverted it: the writer's output
ordering -- both the plain round-robin drain in WriterThread::output()
(mWorkingBufferList = (mWorkingBufferList+1) % threads) and the
parallel-pwrite offset-ring path's mNextSeq[t] sequencing -- both assume
pack round r always lands on worker r % threads. Distributing to the
least-full queue breaks that assumption and silently reorders output
records (caught via a digest diff between the two builds before this went
anywhere near a PR). Making that safe would mean reworking the writer to
track explicit per-chunk sequence numbers rather than inferring order from
worker index -- a materially larger and riskier change than scoped here, so
I'm not pursuing it further without more explicit interest in that scope.

🤖 Generated with Claude Code

[dougnukem]

Reopening against OpenGene/fastp:master directly so maintainers see it (GitHub can't target an unmerged branch in another fork's PR).