Skip to content

nf-core modules update - #448

Open
NicoDeVeaux wants to merge 11 commits into
nf-core:devfrom
NicoDeVeaux:nf-core-modules-update
Open

nf-core modules update#448
NicoDeVeaux wants to merge 11 commits into
nf-core:devfrom
NicoDeVeaux:nf-core-modules-update

Conversation

@NicoDeVeaux

@NicoDeVeaux NicoDeVeaux commented Jul 22, 2026 •
edited
Loading

Copy link
Copy Markdown

Description

Updates all nf-core modules and subworkflows to their latest upstream versions (except subread, which had to be pinned to version 2.0.1, see details below) and reconciles the resulting call-signature changes across the pipeline. No intended change in pipeline behavior. This implements Issue 447.

This was co-written with AI: Seqera Co-Scientist.

What changed and why

  • Updated all nf-core modules and subworkflows to their latest available revisions using:

    nf-core modules update --all --no-preview --update-deps
nf-core subworkflows update --all --no-preview --update-deps

    The resulting main.nf, meta.yml, and nf-test snapshot changes were retained, and the required ATAC-seq call sites were reconciled with the updated input and output signatures.

  • Reconciled call sites in workflows/atacseq.nf and the affected
    subworkflows/local/* and subworkflows/nf-core/* to the updated signatures.

  • Updated versions topic, but kept MultiQC version reporting as a named output rather than publishing it
    to the versions topic, avoiding a self-dependency because MultiQC consumes
    the collated version information as an input.

  • Add read groups to Chromap alignments, which Picard 3.4.0 MarkDuplicates now requires.

  • Pin subread to 2.0.1 so consensus peak counting keeps supporting mixed single-end/paired-end cohorts in a single featureCounts invocation.

  • Sort the consensus featureCounts BAM inputs so the count matrix column order is deterministic.

Verification:

Real-data A/B testing against dev was completed on AWS Batch using two
four-sample datasets.

  • Osmotic SRR (four-sample public subset): both runs produced the same
    72-process DAG and executed 185 tasks. MULTIQC completed as the terminal
    node in both runs, confirming resolution of the previous topic-channel
    deadlock on a second dataset. Scientific outputs are near-identical. The
    only differences are ±2 peaks in the two lowest-depth libraries, which
    have only single- or double-digit peak counts. Higher-signal samples are
    bit-identical.
  • HIV MDDC (GRCh38, four samples from PubMed ID 31968263): Peak calling is
    unaffected by the modules update — every per-library and per-replicate peak
    count and FRiP score matches dev to full precision. Per-library peak counts
    are 30,438 / 35,854 / 36,551 / 51,633, with FRiP scores of 0.216448 /
    0.230010 / 0.198316 / 0.250963. Consensus peak locations are effectively
    identical (dev↔branch Jaccard 0.995).
    ⚠️ Consensus quantification differed on the commit originally tested here
    (0116142, before the subread pin).     The featureCounts read counts over the
    consensus intervals were ~1.92× higher on that commit across all four samples
    From subread 2.0.2 onward, -p counts individual reads rather than fragments unless --countReadPairs is passed, so paired-end counts doubled.
    This is already addressed on the current head of this branch (5caa50bf),
    which pins subread back to 2.0.1     and restores the fragment-counting behaviour
    dev's snapshots encode. I re-ran the four HIV MDDC samples on that, and got 99.995% similarity to dev.

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/atacseq branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@NicoDeVeaux
NicoDeVeaux marked this pull request as draft July 22, 2026 02:07
NicoDeVeaux added a commit to NicoDeVeaux/atacseq-nfcore that referenced this pull request Jul 22, 2026
@NicoDeVeaux
ci: update setup-nextflow to v3.0.1
c9dd24f 
Use the release metadata fallback added in v3.0.1 so CI can install Nextflow while the nf-core metadata endpoint is empty. Update the changelog reference to PR nf-core#448 and remove the branch-local MultiQC deadlock entry.
@github-actions

github-actions Bot commented Jul 22, 2026 •
edited
Loading

Copy link
Copy Markdown

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 1513428

+| ✅ 274 tests passed       |+
#| ❔   4 tests were ignored |#
#| ❔   1 tests had warnings |#
!| ❗  37 tests had warnings |!
Details

❗ Test warnings:

  • readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
  • pipeline_todos - TODO string in nextflow.config: Optionally, you can add a pipeline-specific nf-core config at https://github.com/nf-core/configs
  • pipeline_todos - TODO string in nextflow.config: Update the field with the details of the contributors to your pipeline. New with Nextflow version 24.10.0
  • pipeline_todos - TODO string in nextflow.config: Specify any additional parameters here
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
  • pipeline_todos - TODO string in CONTRIBUTING.md: Add any pipeline specific contribution guidelines here, such as coding styles, procedures, checklists etc.
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • local_component_structure - tss_extract.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - genome_blacklist_regions.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - samplesheet_check.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - multiqc_custom_peaks.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - igv.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - plot_macs3_qc.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - star_genomegenerate.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - plot_homer_annotatepeaks.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - macs3_consensus.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - frip_score.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - deseq2_qc.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - bam_remove_orphans.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - bedtools_genomecov.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - bamtools_filter.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - get_autosomes.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - multiqc.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - gtf2bed.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - star_align.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - bam_bedgraph_bigwig_bedtools_ucsc.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - bed_consensus_quantify_qc_bedtools_featurecounts_deseq2.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - bam_shift_reads.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - bigwig_plot_deeptools.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - bam_peaks_call_qc_annotate_macs3_homer.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - align_star.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_genome.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - input_check.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - bam_filter_bamtools.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure

❔ Tests ignored:

  • nextflow_config - Config default ignored: params.bamtools_filter_pe_config
  • nextflow_config - Config default ignored: params.bamtools_filter_se_config
  • files_unchanged - File ignored due to lint config: .gitignore or .prettierignore
  • merge_markers - Ignoring file /home/runner/work/atacseq/atacseq/modules/nf-core/bowtie2/align/tests/main.nf.test.snap

❔ Tests fixed:

✅ Tests passed:

Run details

  • nf-core/tools version 4.0.3
  • Run at 2026-07-25 12:48:00

@NicoDeVeaux

NicoDeVeaux commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

I'm going to keep this PR in draft mode while I work on getting nf-test to pass

@NicoDeVeaux
Update nf-core modules and subworkflows to their latest versions
156defd 
Bump every nf-core module and subworkflow via `nf-core modules update --all`
/ `nf-core subworkflows update --all` and reconcile the resulting
call-signature changes in first-party code. No intended behaviour change.

- Reconcile call sites in workflows/atacseq.nf and the affected
  subworkflows/{local,nf-core}/*.
- Fix the ch_fasta_fai reference channel: add .first() so the genome
  reference broadcasts to every sample instead of being consumed once.
- Consolidate singleton reference-channel plumbing.
- Remove the orphaned custom/getchromsizes module, replaced by
  SAMTOOLS_FAIDX(..., get_sizes=true).
- Fix a terminal MultiQC hang: MultiQC must not publish to the `versions`
  topic because its own input depends on that topic being resolved, which
  creates a cycle that leaves the run hanging forever. Emit the version as
  a named output instead, matching the nf-core/multiqc convention.
- Replace fragile '2>&1 | tr | sed;sed' version extraction in local modules
  with concise single-command forms (still emitting topic: versions).
- Ignore a merge_markers false positive in the bowtie2 test snapshot: SAM
  base-quality strings contain runs of '<' that trip the check.
@NicoDeVeaux
fix: add read groups to chromap alignments
579e38f 
Chromap BAMs carried no @rg header, so Picard 3.4.0 MarkDuplicates aborted
with a NullPointerException (SAMRecord.getReadGroup() is null) once the
module update bumped Picard from 3.1.1.

Update the fastq_align_chromap subworkflow to d92e0f8, which adds
PICARD_ADDORREPLACEREADGROUPS and an update_readgroups input, install the
picard/addorreplacereadgroups module, enable read-group updating at the
call site in workflows/atacseq.nf, and configure the process in
conf/modules.config to mirror the existing BWA/Bowtie2 read-group mapping
(RGID=meta.id technical id, RGSM=biological sample, RGPL=ILLUMINA,
RGLB=meta.id, RGPU=1, RGCN=params.seq_center).

The linked bam_sort_stats_samtools dependency tree also advanced samtools
1.23.1 -> 1.24 as part of the subworkflow update.
@NicoDeVeaux
fix: pin subread to 2.0.1 for mixed single-end/paired-end peak counts
3812c92 
The module update bumped subread to 2.1.1, which strictly validates library
pairing per BAM: it rejects single-end reads when run in paired mode (-p) and
rejects paired-end reads when run in single-end mode. Consensus-peak
quantification passes a mixed single-end/paired-end cohort to one
SUBREAD_FEATURECOUNTS invocation, so 2.1.1 aborts with "No paired-end reads
were detected in paired-end read library".

Pin subread back to 2.0.1, which auto-detects pairing per BAM in a single
invocation and reproduces the read/fragment counts the existing snapshots
encode. This keeps the fix free of custom SE/PE-splitting and matrix-merging
code. Remove the pin once mixed-library counting is handled explicitly
(follow-up).

The pin makes the local module diverge from its registry git_sha, so it is
recorded as a tracked nf-core module patch (container + environment.yml) for
`nf-core modules lint` check_local_copy.
@NicoDeVeaux
fix: sort consensus featureCounts BAM inputs for deterministic column…
a7d2a31 
… order

The consensus BAM list is assembled from an unordered channel collect, so the
featureCounts invocation received its BAMs (and emitted its count columns) in a
run- and host-dependent order. The counts were identical, but the output file's
byte content - and therefore its snapshot md5 - varied between runs, making the
consensus featureCounts snapshot non-reproducible.

Sort the BAM list by filename before featureCounts so the column order (and the
output file) is deterministic and identical on any x86_64 host, matching CI.
@NicoDeVeaux
test: regenerate pipeline snapshots and ignore timestamped MarkDuplic…
dbfcabf 
…ates metrics

Regenerate all 7 pipeline nf-test snapshots against the updated modules
(subread 2.0.1 pin, samtools bumps, chromap read-group path, deterministic
consensus featureCounts column order). The previous snapshots were ~2 months
stale, from before the module update. Generated on GitHub-hosted ubuntu-latest
x86_64 runners via the update-snapshots workflow, so container digests and
architecture match CI.

Also extend .nftignore: Picard MarkDuplicates metrics headers carry a per-run
"Started on: <timestamp>" line, so their md5 is never reproducible. The
existing pattern only covered the default filename
(*.MarkDuplicates.metrics.txt), but this pipeline renames them via ext.prefix
to *.mkD.sorted.metrics.txt (library) and *.mRp.clN.sorted.metrics.txt
(replicate). These were the only non-deterministic files left once the
consensus featureCounts inputs were sorted.
@NicoDeVeaux
chore: apply the nf-core/tools 4.0.3 template update
e1dc032 
Merge the nf-core/tools 4.0.3 template output (the same content as nf-core#449) so
this branch lints clean against the current template rather than carrying
files_unchanged exemptions for the pre-existing drift in
.github/PULL_REQUEST_TEMPLATE.md and the two logo PNGs.

Keeps the branch-local .nf-core.yml entries (bowtie2 snapshot merge_markers
false positive, bamtools filter config defaults) and regenerates
ro-crate-metadata.json. Drop this commit if nf-core#449 lands on dev first.
@NicoDeVeaux
NicoDeVeaux force-pushed the nf-core-modules-update branch from 1513428 to e1dc032 Compare July 26, 2026 00:51
@github-actions

github-actions Bot commented Jul 26, 2026 •
edited
Loading

Copy link
Copy Markdown

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 5caa50b

+| ✅ 274 tests passed       |+
#| ❔   4 tests were ignored |#
!| ❗  37 tests had warnings |!
Details

❗ Test warnings:

  • readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
  • pipeline_todos - TODO string in nextflow.config: Optionally, you can add a pipeline-specific nf-core config at https://github.com/nf-core/configs
  • pipeline_todos - TODO string in nextflow.config: Update the field with the details of the contributors to your pipeline. New with Nextflow version 24.10.0
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
  • pipeline_todos - TODO string in nextflow.config: Specify any additional parameters here
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in CONTRIBUTING.md: Add any pipeline specific contribution guidelines here, such as coding styles, procedures, checklists etc.
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes
  • local_component_structure - samplesheet_check.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - frip_score.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - igv.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - tss_extract.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - bamtools_filter.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - get_autosomes.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - plot_macs3_qc.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - plot_homer_annotatepeaks.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - gtf2bed.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - deseq2_qc.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - genome_blacklist_regions.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - multiqc_custom_peaks.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - multiqc.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - bam_remove_orphans.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - star_genomegenerate.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - macs3_consensus.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - bedtools_genomecov.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - star_align.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - align_star.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - bed_consensus_quantify_qc_bedtools_featurecounts_deseq2.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - input_check.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - bam_bedgraph_bigwig_bedtools_ucsc.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - bam_shift_reads.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - bam_filter_bamtools.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - bigwig_plot_deeptools.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_genome.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - bam_peaks_call_qc_annotate_macs3_homer.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure

❔ Tests ignored:

  • nextflow_config - Config default ignored: params.bamtools_filter_pe_config
  • nextflow_config - Config default ignored: params.bamtools_filter_se_config
  • files_unchanged - File ignored due to lint config: .gitignore or .prettierignore
  • merge_markers - Ignoring file /home/runner/work/atacseq/atacseq/modules/nf-core/bowtie2/align/tests/main.nf.test.snap

✅ Tests passed:

Run details

  • nf-core/tools version 4.0.3
  • Run at 2026-08-21 19:37:17

@NicoDeVeaux
docs: record the module-update fixes in the changelog
b04867a 
Add entries for the Chromap read-group fix, the subread 2.0.1 pin and the
deterministic consensus featureCounts column order. The template 4.0.3 bump
is left to nf-core#449.
@NicoDeVeaux

NicoDeVeaux commented Jul 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

This PR depends on PR #449 merging first, so I'll keep it in a draft mode until then. Commit e1dc032 duplicates PR #449. It's what's keeping nf-core green right now, so I'll leave it until 449 actually merges, then rebase it out.

@NicoDeVeaux NicoDeVeaux mentioned this pull request Jul 26, 2026
Draft
11 tasks
NicoDeVeaux added a commit to NicoDeVeaux/atacseq-nfcore that referenced this pull request Jul 26, 2026
@NicoDeVeaux
Revert the subread 2.0.1 pin
b3c9ddb 
Restore subread to the registry version and drop the nf-core patch record.
The pin was the stopgap for mixed single-end/paired-end consensus counting;
the following commits handle mixed cohorts explicitly, so it is no longer
needed. Reverts the pin introduced in nf-core#448.
NicoDeVeaux added a commit to NicoDeVeaux/atacseq-nfcore that referenced this pull request Jul 26, 2026
@NicoDeVeaux
fix(consensus): keep deterministic ordering across the SE/PE split
47ad972 
The SE/PE split replaced the single sorted BAM list from nf-core#448 with two
per-endedness batches plus a merge step, each assembled from an unordered
channel collect. Sort all three by filename so the featureCounts column order,
the merged count matrix and its snapshot md5 stay reproducible across runs and
hosts, preserving the guarantee nf-core#448 established.
NicoDeVeaux added a commit to NicoDeVeaux/atacseq-nfcore that referenced this pull request Jul 28, 2026
@NicoDeVeaux
Revert the subread 2.0.1 pin
cdd71d3 
Restore subread to the registry version and drop the nf-core patch record. The
pin was the stopgap for mixed single-end/paired-end consensus counting; the
following commit handles mixed cohorts explicitly, so it is no longer needed.
Reverts the pin introduced in nf-core#448.
NicoDeVeaux added a commit to NicoDeVeaux/atacseq-nfcore that referenced this pull request Jul 28, 2026
@NicoDeVeaux
fix(consensus): count mixed SE/PE libraries per-batch instead of pinn…
56de4cb 
…ing subread

featureCounts (subread >= 2.1.0) applies paired-end mode (-p) per invocation and
aborts when one invocation mixes single-end and paired-end BAMs. The consensus
BAMs can span both library types, so split them by endedness, count each
homogeneous batch with the correct pairing flag, then merge the per-batch
matrices into a single consensus table for DESeq2 and MultiQC.

The joined tuple arity differs between the two instantiations of this
subworkflow: the merged-library caller carries a control-BAM column
([ meta, bam, control, peak ]) while the merged-replicate caller does not
([ meta, bams, peak ]). Index positionally to stay tolerant of both shapes, as
the pre-split implementation did.

The split replaced the single sorted BAM list from nf-core#448 with two per-endedness
batches plus a merge step, each assembled from an unordered channel collect, so
all three are sorted by filename. Without that the featureCounts column order,
the merged matrix and its snapshot md5 stop being reproducible across runs and
hosts — the guarantee nf-core#448 established.

Gating is unchanged: the join with ch_peaks keeps only samples that contributed
peaks, and combining with MACS3_CONSENSUS.out.saf still requires a consensus to
exist (>= 2 samples).
@NicoDeVeaux

NicoDeVeaux commented Jul 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

One thing I discovered while doing this:

  workflows/atacseq.nf:11   include { MULTIQC } from '../modules/local/multiqc'   ← what actually runs

  modules/local/multiqc.nf        multiqc 1.13   (dev: 1.13 — unchanged by 448)
  modules/nf-core/multiqc/        dev: 1.34  →  448: 1.35   ← installed, never included anywhere

So the pipeline runs a hand-rolled local MultiQC module pinned at 1.13, while the nf-core module sits in the tree unused. dev was already in exactly this state, and so I haven't changed anything, but I think there should be a follow-up issue to update multiqc.

@NicoDeVeaux
fix: quote seq_center when building read-group arguments
3025896 
seq_center is free text in the schema, so a value containing whitespace was
split by the shell. On the Chromap path this reached Picard
AddOrReplaceReadGroups as `--RGCN Broad Institute`, and Picard took `Broad` as
the value and rejected the remainder:

    Illegal argument value: Positional arguments were provided ',Institute}'
    but no positional argument is defined for this tool.

Reproduced against the picard 3.4.0 container this PR pins. Quoting the value
makes Picard receive it as a single argument.

Bowtie2 built `--rg CN:${params.seq_center}` the same way and had the same
defect; quoted too. BWA is unaffected because input_check.nf embeds the quotes
in meta.read_group at construction.

No test set seq_center, which is why the matrix stayed green over this. The
Chromap pipeline test now sets a value containing whitespace, so the path is
covered.
@NicoDeVeaux
docs: correct the consensus featureCounts output extension
b556d0d 
The updated subread module publishes *.featureCounts.tsv (and .tsv.summary);
the docs still described *.featureCounts.txt. Drop the matching .nftignore
patterns rather than renaming them: they are dead, and leaving the .tsv files
hashed by the pipeline snapshots is what gives the deterministic column order
useful coverage.
NicoDeVeaux added a commit to NicoDeVeaux/atacseq-nfcore that referenced this pull request Jul 29, 2026
@NicoDeVeaux
Revert the subread 2.0.1 pin
7391a78 
Restore subread to the registry version and drop the nf-core patch record. The
pin was the stopgap for mixed single-end/paired-end consensus counting; the
following commit handles mixed cohorts explicitly, so it is no longer needed.
Reverts the pin introduced in nf-core#448.
NicoDeVeaux added a commit to NicoDeVeaux/atacseq-nfcore that referenced this pull request Jul 29, 2026
@NicoDeVeaux
fix(consensus): count mixed SE/PE libraries per-batch instead of pinn…
bcb9877 
…ing subread

featureCounts (subread >= 2.1.0) applies paired-end mode (-p) per invocation and
aborts when one invocation mixes single-end and paired-end BAMs. The consensus
BAMs can span both library types, so split them by endedness, count each
homogeneous batch with the correct pairing flag, then merge the per-batch
matrices into a single consensus table for DESeq2 and MultiQC.

The joined tuple arity differs between the two instantiations of this
subworkflow: the merged-library caller carries a control-BAM column
([ meta, bam, control, peak ]) while the merged-replicate caller does not
([ meta, bams, peak ]). Index positionally to stay tolerant of both shapes, as
the pre-split implementation did.

The split replaced the single sorted BAM list from nf-core#448 with two per-endedness
batches plus a merge step, each assembled from an unordered channel collect, so
all three are sorted by filename. Without that the featureCounts column order,
the merged matrix and its snapshot md5 stop being reproducible across runs and
hosts — the guarantee nf-core#448 established.

Gating is unchanged: the join with ch_peaks keeps only samples that contributed
peaks, and combining with MACS3_CONSENSUS.out.saf still requires a consensus to
exist (>= 2 samples).
@NicoDeVeaux
fix: regenerate ro-crate-metadata.json against the real pipeline docs
250767f 
Applying the 4.0.3 template merge took nf-core-bot's RO-Crate wholesale, and the
bot generates its copy from the *template* README. That replaced the pipeline's
description with template boilerplate carrying "TODO nf-core" placeholders --
13,620 characters of real documentation down to 7,179.

Regenerate with `nf-core pipelines rocrate` so the file derives from this
repository's README instead. Verified: real description restored, 36 nodes, all
6 UUID @id references resolve, no dangling ids.

This makes the PR correct on its own rather than relying on a later template
merge to repair it.
NicoDeVeaux added a commit to NicoDeVeaux/atacseq-nfcore that referenced this pull request Aug 3, 2026
@NicoDeVeaux
Revert the subread 2.0.1 pin
ce08c91 
Restore subread to the registry version and drop the nf-core patch record. The
pin was the stopgap for mixed single-end/paired-end consensus counting; the
following commit handles mixed cohorts explicitly, so it is no longer needed.
Reverts the pin introduced in nf-core#448.
NicoDeVeaux added a commit to NicoDeVeaux/atacseq-nfcore that referenced this pull request Aug 3, 2026
@NicoDeVeaux
fix(consensus): count mixed SE/PE libraries per-batch instead of pinn…
8fc76c6 
…ing subread

featureCounts (subread >= 2.1.0) applies paired-end mode (-p) per invocation and
aborts when one invocation mixes single-end and paired-end BAMs. The consensus
BAMs can span both library types, so split them by endedness, count each
homogeneous batch with the correct pairing flag, then merge the per-batch
matrices into a single consensus table for DESeq2 and MultiQC.

The joined tuple arity differs between the two instantiations of this
subworkflow: the merged-library caller carries a control-BAM column
([ meta, bam, control, peak ]) while the merged-replicate caller does not
([ meta, bams, peak ]). Index positionally to stay tolerant of both shapes, as
the pre-split implementation did.

The split replaced the single sorted BAM list from nf-core#448 with two per-endedness
batches plus a merge step, each assembled from an unordered channel collect, so
all three are sorted by filename. Without that the featureCounts column order,
the merged matrix and its snapshot md5 stop being reproducible across runs and
hosts — the guarantee nf-core#448 established.

Gating is unchanged: the join with ch_peaks keeps only samples that contributed
peaks, and combining with MACS3_CONSENSUS.out.saf still requires a consensus to
exist (>= 2 samples).
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 4.0.3.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the Synchronisation documentation.

@NicoDeVeaux @claude
Merge branch 'dev' into nf-core-modules-update
5caa50b 
Resolves the ro-crate-metadata.json conflict by taking dev's copy: the two
sides are identical apart from generated UUIDs and timestamps.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@NicoDeVeaux
NicoDeVeaux marked this pull request as ready for review August 21, 2026 19:40
@NicoDeVeaux NicoDeVeaux mentioned this pull request Aug 21, 2026
Merged
4 tasks
@NicoDeVeaux
NicoDeVeaux marked this pull request as draft August 23, 2026 04:10
NicoDeVeaux added a commit to NicoDeVeaux/atacseq-nfcore that referenced this pull request Aug 23, 2026
@NicoDeVeaux
Count read pairs, not reads, in paired-end featureCounts
012517a 
Under subread 2.1.1 (which this PR restores by dropping the 2.0.1 pin from
nf-core#448), bare '-p' only declares paired-end input and counts individual reads;
counting fragments now requires '--countReadPairs'. Without it, consensus
peak counts for paired-end libraries roughly double relative to the 2.0.1
behaviour, inflating DESeq2 size factors and all downstream numbers.

Add '--countReadPairs' to the paired-end branch of the vendored
subread/featurecounts module via an nf-core modules patch.
NicoDeVeaux added a commit to NicoDeVeaux/atacseq-nfcore that referenced this pull request Aug 23, 2026
@NicoDeVeaux
Count read pairs, not reads, in paired-end featureCounts
4ddca8b 
Under subread 2.1.1 (which this PR restores by dropping the 2.0.1 pin from
nf-core#448), bare '-p' only declares paired-end input and counts individual reads;
counting fragments now requires '--countReadPairs'. Without it, consensus
peak counts for paired-end libraries roughly double relative to the 2.0.1
behaviour, inflating DESeq2 size factors and all downstream numbers.

Add '--countReadPairs' to the paired-end branch of the vendored
subread/featurecounts module via an nf-core modules patch.
@NicoDeVeaux
NicoDeVeaux marked this pull request as ready for review August 23, 2026 19:34
Kevin-Brockers
Kevin-Brockers requested changes Aug 25, 2026
View reviewed changes
Comment thread CHANGELOG.md

@Kevin-Brockers Kevin-Brockers Aug 25, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the updated software versions in the changelog as well

Kevin-Brockers
Kevin-Brockers reviewed Aug 25, 2026
View reviewed changes
Comment thread CHANGELOG.md
Note, since the pipeline is now using Nextflow DSL2, each process will be run with its own [Biocontainer](https://biocontainers.pro/#/registry). This means that on occasion it is entirely possible for the pipeline to be using different versions of the same tool. However, the overall software dependency changes compared to the last release have been listed below for reference.

| Dependency | Old version | New version |
| ---------- | ----------- | ----------- |

@Kevin-Brockers Kevin-Brockers Aug 25, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NicoDeVeaux see here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Kevin-Brockers Kevin-Brockers Kevin-Brockers requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@NicoDeVeaux @Kevin-Brockers