nf-core_modules/README.md

176 lines
9.1 KiB
Markdown
Raw Normal View History

2019-07-26 09:19:07 +00:00
# ![nf-core/modules](docs/images/nfcore-modules_logo.png)
Update all modules to new NF DSL2 syntax (#1099) * Add comment line for consistency * Remove all functions.nf * Remove include functions.nf and publishDir options * Replace options.args3 with task.ext.args3 - 3 modules * Replace options.args3 with task.ext.args3 - 17 modules * Replace {task.cpus} with task.cpus * Replace off on off off off off off off off on off on off on off off off on off off off on on off off off on on off off off off off off off on off off off off on off on on off off off on on on on off off off on off on on off on on off off on on on off on on off on off off off off on off off off on off off on off on off off off on on off on off on off off on off off off on off off off on off off off off on off off off on on on off on on off off on off on on on off on on off on on on off off off off off on on off off on off off off off off on off off on on off on on off on off off off on off off off off on on off on off off on off off on off on off off off off off off off off on on off on off off off.args with * Add def args = task.ext.args line to all modules in script section * Replace options.args with args and args_list * Initialise args2 and args3 properly * Replace container syntax * Revert container changes for cellranger/mkref * Replace getProcessName in all modules * Replace getSoftwareName in all modules * Unify modules using VERSION variable * Replae options.suffix with task.ext.suffix * Remove NF version restriction for CI * Bump NF version in README * Replace task.process.tokenize logic with task.process * Minor tweaks to unify syntax in tests main.nf * Add a separate nextflow.config for each module * Transfer remaining module options to nextflow.config * Remove addParams from tests main.nf * Remove TODO statements * Use -c to import module specific config * Bump NF version to 21.10.3 * Fix tests for artic/minion * Fix broken publishDir syntax * Standardise and fix obvious failing module tests * Remove kronatools to krona * Comment out tags in subworkflow test.yml * Fix failing module tests * Add consistent indentation to nextflow.config * Comment out subworklow definitions * Fix kallistobustools/ref * Fix rmarkdownnotebook * Fix jupyternotebook * Quote task.process * Add plink2/vcf to pytest_modules.yml * Remove NF_CORE_MODULES_TEST from pytest CI * Fix more tests * Move bacteroides_fragilis to prokaryotes folder * Fix cooler merge tests * Fix kallistobustools/count tests * Fix kallistobustools/ref tests * Update test_10x_1_fastq_gz file for kallistobustools/count tests * Fix bcftools/query tests * Fix delly/call tests * Fix cooler/zoomify tests * Fix csvtk/split tests * Fix gatk4/intervallisttools tests * Fix gatk4/variantfiltration * Fix pydamage/filter tests * Fix test data for unicycler * Fix gstama/collapse module * Fix leehom tests * Fix metaphlan3 tests * Fix pairtools/select tests * Update nextflow.config * Update nextflow.config * feat: update syntax * Fix arriba tests * Fix more failing tests * Update test syntax * Remove comments from tests nextflow.config * Apply suggestions from code review * Fix kallistobustools/count module * Update dumpsoftwareversions module * Update custom/dumpsoftwareversions * Add args2 to untar module * Update leftover modules * Remove last remaining addParams Co-authored-by: JoseEspinosa <kadomu@gmail.com> Co-authored-by: Gregor Sturm <mail@gregor-sturm.de> Co-authored-by: MaxUlysse <max.u.garcia@gmail.com>
2021-11-26 07:58:40 +00:00
[![Nextflow](https://img.shields.io/badge/nextflow%20DSL2-%E2%89%A521.10.3-23aa62.svg?labelColor=000000)](https://www.nextflow.io/)
2020-10-20 10:02:46 +00:00
[![run with conda](http://img.shields.io/badge/run%20with-conda-3EB049?labelColor=000000&logo=anaconda)](https://docs.conda.io/en/latest/)
[![run with docker](https://img.shields.io/badge/run%20with-docker-0db7ed?labelColor=000000&logo=docker)](https://www.docker.com/)
[![run with singularity](https://img.shields.io/badge/run%20with-singularity-1d355c.svg?labelColor=000000)](https://sylabs.io/docs/)
2020-10-20 10:04:12 +00:00
![GitHub Actions Coda Linting](https://github.com/nf-core/modules/workflows/Code%20Linting/badge.svg)
[![Get help on Slack](http://img.shields.io/badge/slack-nf--core%20%23modules-4A154B?labelColor=000000&logo=slack)](https://nfcore.slack.com/channels/modules)
2020-10-20 10:02:46 +00:00
[![Follow on Twitter](http://img.shields.io/badge/twitter-%40nf__core-1DA1F2?labelColor=000000&logo=twitter)](https://twitter.com/nf_core)
[![Watch on YouTube](http://img.shields.io/badge/youtube-nf--core-FF0000?labelColor=000000&logo=youtube)](https://www.youtube.com/c/nf-core)
> THIS REPOSITORY IS UNDER ACTIVE DEVELOPMENT. SYNTAX, ORGANISATION AND LAYOUT MAY CHANGE WITHOUT NOTICE!
2021-02-07 23:20:53 +00:00
2020-08-06 14:34:44 +00:00
A repository for hosting [Nextflow DSL2](https://www.nextflow.io/docs/latest/dsl2.html) module files containing tool-specific process definitions and their associated documentation.
2019-07-26 12:38:08 +00:00
2019-07-26 09:19:07 +00:00
## Table of contents
2020-07-11 11:48:11 +00:00
2020-07-14 08:51:19 +00:00
- [Using existing modules](#using-existing-modules)
- [Adding new modules](#adding-new-modules)
2020-07-14 08:51:19 +00:00
- [Help](#help)
2020-08-06 11:52:55 +00:00
- [Citation](#citation)
2019-07-26 09:19:07 +00:00
2019-07-26 12:38:08 +00:00
## Using existing modules
2019-07-26 09:19:07 +00:00
2020-08-06 12:33:16 +00:00
The module files hosted in this repository define a set of processes for software tools such as `fastqc`, `bwa`, `samtools` etc. This allows you to share and add common functionality across multiple pipelines in a modular fashion.
We have written a helper command in the `nf-core/tools` package that uses the GitHub API to obtain the relevant information for the module files present in the [`modules/`](modules/) directory of this repository. This includes using `git` commit hashes to track changes for reproducibility purposes, and to download and install all of the relevant module files.
2019-07-26 09:19:07 +00:00
1. Install the latest version of [`nf-core/tools`](https://github.com/nf-core/tools#installation) (`>=2.0`)
2020-08-06 11:27:01 +00:00
2. List the available modules:
2020-08-06 11:12:40 +00:00
```console
$ nf-core modules list remote
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/tools version 2.0
INFO Modules available from nf-core/modules (master): pipeline_modules.py:164
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Module Name ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ bandage/image │
│ bcftools/consensus │
│ bcftools/filter │
│ bcftools/isec │
..truncated..
```
2019-07-26 09:19:07 +00:00
2020-08-06 11:55:36 +00:00
3. Install the module in your pipeline directory:
2020-08-06 11:27:01 +00:00
```console
$ nf-core modules install fastqc
2020-08-06 11:27:01 +00:00
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
2020-08-06 11:27:01 +00:00
nf-core/tools version 2.0
2020-08-06 11:27:01 +00:00
INFO Installing fastqc pipeline_modules.py:213
INFO Downloaded 3 files to ./modules/nf-core/modules/fastqc pipeline_modules.py:236
```
2020-08-06 11:27:01 +00:00
2020-08-06 12:23:56 +00:00
4. Import the module in your Nextflow script:
```nextflow
#!/usr/bin/env nextflow
2020-08-06 12:23:56 +00:00
nextflow.enable.dsl = 2
2020-08-06 12:23:56 +00:00
include { FASTQC } from './modules/nf-core/modules/fastqc/main'
```
2020-08-06 12:23:56 +00:00
5. Remove the module from the pipeline repository if required:
2020-08-06 11:57:53 +00:00
```console
$ nf-core modules remove fastqc
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
2020-08-06 11:57:53 +00:00
nf-core/tools version 2.0
2020-08-06 11:57:53 +00:00
INFO Removing fastqc pipeline_modules.py:271
INFO Successfully removed fastqc pipeline_modules.py:285
```
2020-08-06 11:27:01 +00:00
6. Check that a locally installed nf-core module is up-to-date compared to the one hosted in this repo:
```console
$ nf-core modules lint fastqc
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/tools version 2.0
INFO Linting pipeline: . lint.py:104
INFO Linting module: fastqc lint.py:106
╭─────────────────────────────────────────────────────────────────────────────────╮
│ [!] 1 Test Warning │
╰─────────────────────────────────────────────────────────────────────────────────╯
╭──────────────┬───────────────────────────────┬──────────────────────────────────╮
│ Module name │ Test message │ File path │
├──────────────┼───────────────────────────────┼──────────────────────────────────┤
│ fastqc │ Local copy of module outdated │ modules/nf-core/modules/fastqc/ │
╰──────────────┴────────────────────────────── ┴──────────────────────────────────╯
╭──────────────────────╮
│ LINT RESULTS SUMMARY │
├──────────────────────┤
│ [✔] 15 Tests Passed │
│ [!] 1 Test Warning │
│ [✗] 0 Test Failed │
╰──────────────────────╯
```
## Adding new modules
2019-07-26 09:19:07 +00:00
If you wish to contribute a new module, please see the documentation on the [nf-core website](https://nf-co.re/developers/modules#writing-a-new-module-reference).
2020-08-06 12:33:16 +00:00
2021-09-20 11:00:11 +00:00
> Please be kind to our code reviewers and submit one pull request per module :)
2021-02-08 00:53:32 +00:00
2019-07-26 09:19:07 +00:00
## Help
2020-08-06 11:46:36 +00:00
For further information or help, don't hesitate to get in touch on [Slack `#modules` channel](https://nfcore.slack.com/channels/modules) (you can join with [this invite](https://nf-co.re/join/slack)).
2020-08-06 11:52:04 +00:00
## Citation
If you use the module files in this repository for your analysis please you can cite the `nf-core` publication as follows:
> **The nf-core framework for community-curated bioinformatics pipelines.**
>
> Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.
>
> _Nat Biotechnol._ 2020 Feb 13. doi: [10.1038/s41587-020-0439-x](https://dx.doi.org/10.1038/s41587-020-0439-x).
2020-08-06 12:33:16 +00:00
<!---
### Offline usage
If you want to use an existing module file available in `nf-core/modules`, and you're running on a system that has no internet connection, you'll need to download the repository (e.g. `git clone https://github.com/nf-core/modules.git`) and place it in a location that is visible to the file system on which you are running the pipeline. Then run the pipeline by creating a custom config file called e.g. `custom_module.conf` containing the following information:
```bash
include /path/to/downloaded/modules/directory/
```
Then you can run the pipeline by directly passing the additional config file with the `-c` parameter:
```bash
nextflow run /path/to/pipeline/ -c /path/to/custom_module.conf
```
> Note that the nf-core/tools helper package has a `download` command to download all required pipeline
> files + singularity containers + institutional configs + modules in one go for you, to make this process easier.
# New test data created for the module- sequenzautils/bam2seqz
The new test data is an output from another module- sequenzautils/bcwiggle- (which uses sarscov2 genome fasta file as an input).
2020-08-06 12:33:16 +00:00
-->