ci: thread-count correctness checks (would catch #721) and base-vs-head benchmarks

[dougnukem]

Stacked on #723 (the correctness check fails on master without it; see "Negative control" below).

Adds two checks to CI that would have caught #721, plus a benchmark job so performance changes come with numbers.

Correctness (ci.yml, both OSes)

• ./fastp test: the existing unit tests, which CI didn't run.
• .github/scripts/check_correctness.py: generates deterministic synthetic PE reads (TruSeq read-through, poly-G, Ns, degrading quality), then requires byte-identical output (decompressed) at -w 1,2,4,9,17,33,48 in 8 modes: PE gz, PE plain, SE, interleaved input, merge, failed/unpaired outputs, --stdout, and -s split (compared by sorted records). A run over 180s counts as a hang. It also checks that adapter auto-detection (SE, and PE with --detect_adapter_for_pe) finds the planted adapters.
• fastp caps -w to the core count, so on a 4-core runner -w 48 would silently run 4 workers. On Linux, fake_nprocs.c (LD_PRELOAD) reports 64 CPUs so high worker counts really run, and any run that still prints "Reduce worker threads" fails. The Linux build is fully static, so the check relinks the same objects against dynamic glibc for that step. macOS runs only the counts it can really run.

Runtime: 50s on ubuntu-latest, 18s on macOS.

Benchmark (bench.yml, ubuntu-latest)

• On PRs: builds base and head on the same runner and runs them interleaved (3 reps, alternating order) at -w 4. Each build runs on the first 0.3M and 1.2M pairs of the same input (--reads_to_process, no recompression), and a line through the two points gives a fixed cost (intercept) and a cost per pair (slope). Inputs are 1.2M synthetic pairs (used as both PE and SE) and the first 1.2M pairs of a public ATAC-seq run (SRR891268, from ENA).

• Projected to full size: subsets overstate changes to fixed-cost stages (adapter detection samples at most 256K reads), so the projection to a 50M-read run is fixed cost + cost per pair × 50M. On 3 complete public runs (4 and 16 cores) a line through two subset sizes predicted full-run CPU within 1–2% and wall within 2–4% (median error), against 3% and 11% for scaling a single subset. The fixed cost and the measured subset numbers are in collapsed sections.

• Regression gate: a cell regresses when its projected median is more than 10% worse than base and every head run is slower than every base run, so one noisy run can't trip it. Only projected wall, CPU per pair (µs) and peak RSS are gated. Any regression fails the regression gate step unless the PR has the perf-regression-ok label, which is the explicit sign-off. Adding or removing that label reruns the benchmark. The threshold can be set with the BENCH_THRESHOLD_PCT repo variable. Improvements over 5% (with no overlap) are marked 🟢 but never gated.

• PR comment: the report is posted as a single comment that's updated in place on each push. Fork PRs get a read-only token, so bench-comment.yml posts it from a workflow_run in the base repo. It never runs PR code, uses the artifact only as comment text, and finds the PR by head SHA.

• Structured output: bench-compare.json holds per-cell base, head, delta and verdict. The step outputs regressions and improvements and the job summary are also available.

• On pushes to master: charts head-only numbers over time with github-action-benchmark on gh-pages. It doesn't comment or fail on alerts; shared runners are too noisy to gate on.

• Noise: on the fork, upstream vs fix: scale reader backpressure with worker thread count (#721) #723 plus the adapter-detection index shows CPU per pair +4–5% (the backpressure fix costs 2–4% on 4-core runners) and nothing is flagged at the 10% threshold.

• Input robustness: ENA sometimes drops the connection mid-stream, so the download resumes with Range requests (and restarts from the top if the server answers 416). If an input still can't be fetched, that dataset is skipped with a warning, listed in the report, and the cache is saved only for a complete set.

Dependency builds move into a composite action (.github/actions/build-deps) shared by both workflows. The steps are unchanged.

Validation (on my fork)

• This branch: all green. Linux ran -w 1…48 with real workers, and every mode was identical across all thread counts. The benchmark job takes about 6 min with cached inputs, 9 min on a first run (it generates the synthetic data and downloads the public subset).
• Negative control: the same CI commits on unfixed master: Linux fails with HUNG at -w 33 and -w 48 in all 7 non-split modes (split doesn't hang) and passes everywhere else. The output digests at low -w match this branch's, so fix: scale reader backpressure with worker thread count (#721) #723 doesn't change output. That run takes about 45 min because every hang waits out the 180s timeout; with the fix in, the step takes 50s.
• Gate, on the earlier single-size version: a fork PR that adds a deliberate 2s startup sleep flagged 6 wall-time regressions (+42% to +131%) and left CPU unflagged, which is correct since sleep uses none. The gate failed; after the label was added it passed, and the same comment was updated to "accepted".
• Master tracking: with these workflows on the fork's master, the push run committed dev/bench/data.js to gh-pages and the chart renders.
• Two-size version, on fork PRs: the gap-search fix alone (Adapter trimming: one-gap search only compares the read start, so indel adapters are missed and reads that start like the adapter get trimmed #728) is flagged on all 3 datasets (CPU per pair +11% to +22%); the same fix with the early-exit search is within ±1% on the synthetic sets and −9% on ATAC; the adapter-detection index shows fixed cost −20% to −31% and nothing flagged.
• Not exercised yet: the fork-PR comment path (bench-comment.yml), which needs a PR from a different repository, and the download restart against the real server when several jobs fetch the same file at once (tested against a local server that drops connections and answers 416).

🤖 Generated with Claude Code