c19671dca9
* feat(subworkflows): Add align_bowtie2 subworkflow For testing CI setup * test(align_bowtie2): Add initial list of changes to test * test(align_bowtie2): Add initial test * refactor: Use tags to run subworkflows ci For every underlying module used by workflow and allow the modules pytest-modules definition be the source of truth. * refactor: Use individual directories for subworkflows * docs(align_bowtie2): Add initial meta.yml Copied most of it from the bowtie2/align module. * fix(align_bowtie2): Fix module include paths * test(bam_sort_samtools): Add initial test * ci(bam_sort_samtools): Add modules that trigger the tag * test(bam_stats_samtools): Add initial test * ci(bam_stats_samtools): Add keys to pick up changes * docs(bam_samtools): Add initial meta.yml * test(align_bowtie2): Fix path to subworkflow * test(align_bowtie2): Update entry point * fix(bam_sort_samtools): Update include paths * test(bam_sort_samtools): Fix path * style: Clean up addParams * test(samtools_sort): Add suffix for test * test(align_bowtie2): Add samtools_options for suffix * test(bam_stats_samtools): Update path * test(bam_stats_samtools): Use stats input Otherwise it's just an example of how it's used in the bam_sort_samtools subworkflow * ci(linting): Skip module linting of subworkflows * ci(linting): Clean up startsWith statement * test(bam_stats_samtools): Use single end test data for single end test * test(bam_stats_samtools): Add expected files * test(align_bowtie2): Add paired-end test * test(align_bowtie2): Sort order of output * test(align_bowtie2): Update hashes * docs(align_bowtie2): Fix typo * test(align_bowtie2): Update samtools output names * test(align_bowtie2): Remove md5sums for bam/bai * feat(subworkflows): Add nextflow.configs These can be used for default settings in the future. They can then be included in the conf/modules.config so that the params don't have to be duplicated in the root nextflow.config. * docs(subworkflows): Include modules instead of tools * fix: Update to versions * chore(align_bowtie2): Remove duplicate tag * style: Format yamls * test(subworkflows): Only check versions for modules * chore: Update subworkflows to match rnaseq dev * fix(subworkflows): Update paths * fix(bam_sort_samtools): Fix sort parameters for testing * Apply suggestions from code review Co-authored-by: Harshil Patel <drpatelh@users.noreply.github.com> * docs: Update TODOs with a message * ci: Try using a matrix for strategy * ci: Try passing an array * Revert "ci: Try passing an array" This reverts commit d3611fcd8332bbb9a8501e8dd299d0a623aaecaa. Co-authored-by: Harshil Patel <drpatelh@users.noreply.github.com> |
||
|---|---|---|
| .github | ||
| docs/images | ||
| modules | ||
| subworkflows/nf-core | ||
| tests | ||
| .editorconfig | ||
| .gitattributes | ||
| .gitignore | ||
| .markdownlint.yml | ||
| .nf-core.yml | ||
| LICENSE | ||
| README.md | ||
THIS REPOSITORY IS UNDER ACTIVE DEVELOPMENT. SYNTAX, ORGANISATION AND LAYOUT MAY CHANGE WITHOUT NOTICE!
A repository for hosting Nextflow DSL2 module files containing tool-specific process definitions and their associated documentation.
Table of contents
Using existing modules
The module files hosted in this repository define a set of processes for software tools such as fastqc, bwa, samtools etc. This allows you to share and add common functionality across multiple pipelines in a modular fashion.
We have written a helper command in the nf-core/tools package that uses the GitHub API to obtain the relevant information for the module files present in the modules/ directory of this repository. This includes using git commit hashes to track changes for reproducibility purposes, and to download and install all of the relevant module files.
-
Install the latest version of
nf-core/tools(>=2.0) -
List the available modules:
$ nf-core modules list remote ,--./,-. ___ __ __ __ ___ /,-._.--~\ |\ | |__ __ / ` / \ |__) |__ } { | \| | \__, \__/ | \ |___ \`-._,-`-, `._,._,' nf-core/tools version 2.0 INFO Modules available from nf-core/modules (master): pipeline_modules.py:164 ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Module Name ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ bandage/image │ │ bcftools/consensus │ │ bcftools/filter │ │ bcftools/isec │ ..truncated.. -
Install the module in your pipeline directory:
$ nf-core modules install fastqc ,--./,-. ___ __ __ __ ___ /,-._.--~\ |\ | |__ __ / ` / \ |__) |__ } { | \| | \__, \__/ | \ |___ \`-._,-`-, `._,._,' nf-core/tools version 2.0 INFO Installing fastqc pipeline_modules.py:213 INFO Downloaded 3 files to ./modules/nf-core/modules/fastqc pipeline_modules.py:236 -
Import the module in your Nextflow script:
#!/usr/bin/env nextflow nextflow.enable.dsl = 2 include { FASTQC } from './modules/nf-core/modules/fastqc/main' addParams( options: [:] ) -
Remove the module from the pipeline repository if required:
$ nf-core modules remove fastqc ,--./,-. ___ __ __ __ ___ /,-._.--~\ |\ | |__ __ / ` / \ |__) |__ } { | \| | \__, \__/ | \ |___ \`-._,-`-, `._,._,' nf-core/tools version 2.0 INFO Removing fastqc pipeline_modules.py:271 INFO Successfully removed fastqc pipeline_modules.py:285 -
Check that a locally installed nf-core module is up-to-date compared to the one hosted in this repo:
$ nf-core modules lint fastqc ,--./,-. ___ __ __ __ ___ /,-._.--~\ |\ | |__ __ / ` / \ |__) |__ } { | \| | \__, \__/ | \ |___ \`-._,-`-, `._,._,' nf-core/tools version 2.0 INFO Linting pipeline: . lint.py:104 INFO Linting module: fastqc lint.py:106 ╭─────────────────────────────────────────────────────────────────────────────────╮ │ [!] 1 Test Warning │ ╰─────────────────────────────────────────────────────────────────────────────────╯ ╭──────────────┬───────────────────────────────┬──────────────────────────────────╮ │ Module name │ Test message │ File path │ ├──────────────┼───────────────────────────────┼──────────────────────────────────┤ │ fastqc │ Local copy of module outdated │ modules/nf-core/modules/fastqc/ │ ╰──────────────┴────────────────────────────── ┴──────────────────────────────────╯ ╭──────────────────────╮ │ LINT RESULTS SUMMARY │ ├──────────────────────┤ │ [✔] 15 Tests Passed │ │ [!] 1 Test Warning │ │ [✗] 0 Test Failed │ ╰──────────────────────╯
Adding new modules
If you wish to contribute a new module, please see the documentation on the nf-core website.
Please be kind to our code reviewers and submit one pull request per module :)
Help
For further information or help, don't hesitate to get in touch on Slack #modules channel (you can join with this invite).
Citation
If you use the module files in this repository for your analysis please you can cite the nf-core publication as follows:
The nf-core framework for community-curated bioinformatics pipelines.
Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.
Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.