From eb9aeea7752c49081a838829446faefe55858164 Mon Sep 17 00:00:00 2001 From: drpatelh Date: Fri, 26 Jul 2019 13:30:26 +0100 Subject: [PATCH] Refine README --- README.md | 48 ++++++++++++++++-------------------------------- docs/fastqc.md | 2 ++ docs/template.md | 2 ++ nf/fastqc.nf | 2 ++ 4 files changed, 22 insertions(+), 32 deletions(-) create mode 100644 docs/template.md diff --git a/README.md b/README.md index 4b1fbc0c..5c13b7d5 100644 --- a/README.md +++ b/README.md @@ -16,68 +16,52 @@ A repository for hosting nextflow [`DSL2`](https://www.nextflow.io/docs/edge/dsl ## Using an existing module -The Nextflow [`-c`](https://www.nextflow.io/docs/latest/config.html) parameter can be used with nf-core pipelines in order to load custom config files that you have available locally. However, if you or other people within your organisation are likely to be running nf-core pipelines regularly it may be a good idea to use/create a custom config file that defines some generic settings unique to the computing environment within your organisation. +The Nextflow [`include`](https://www.nextflow.io/docs/edge/dsl2.html#modules-include) statement can be used within your pipelines in order to load module files that you have available locally. + +You should be able to get a good idea as to how other people are using module files by looking at pipelines available in nf-core e.g. [`nf-core/rnaseq`](https://github.com/nf-core/rnaseq/pull/162) ### Configuration and parameters -The config files hosted in this repository define a set of parameters which are specific to compute environments at different Institutions but generic enough to be used with all nf-core pipelines. - -All nf-core pipelines inherit the functionality provided by Nextflow, and as such custom config files can contain parameters/definitions that are available to both. For example, if you have the ability to use [Singularity](https://singularity.lbl.gov/) on your HPC you can add and customise the Nextflow [`singularity`](https://www.nextflow.io/docs/latest/config.html#scope-singularity) scope in your config file. Similarly, you can define a Nextflow [`executor`](https://www.nextflow.io/docs/latest/executor.html) depending on the job submission process available on your cluster. In contrast, the `params` section in your custom config file will typically define parameters that are specific to nf-core pipelines. - -You should be able to get a good idea as to how other people are customising the execution of their nf-core pipelines by looking at some of the config files in [`nf-core/modules`](https://github.com/nf-core/modules/tree/master/conf). +The module files hosted in this repository define a set of processes for software tools such as `fastc`, `trimgalore`, `bwa` etc. This allows you to share and add common functionality across multiple pipelines in a modular fashion. ### Offline usage -If you want to use an existing config available in `nf-core/modules`, and you're running on a system that has no internet connection, you'll need to download the config file and place it in a location that is visible to the file system on which you are running the pipeline. Then run the pipeline with `--custom_config_base` -or `params.custom_config_base` set to the location of the directory containing the repository files: +If you want to use an existing module file available in `nf-core/modules`, and you're running on a system that has no internet connection, you'll need to download the repository (e.g. `git clone https://github.com/nf-core/modules.git`) and place it in a location that is visible to the file system on which you are running the pipeline. Then run the pipeline by creating a custom config file called e.g. `custom_module.conf` containing the following information: ```bash -## Download and unzip the config files -cd /path/to/my/modules -wget https://github.com/nf-core/modules/archive/master.zip -unzip master.zip - -## Run the pipeline -cd /path/to/my/data -nextflow run /path/to/pipeline/ --custom_config_base /path/to/my/modules/modules-master/ +include /path/to/downloaded/modules/directory/ ``` -Alternatively, instead of using the configuration profiles from this repository, you can run your -pipeline directly calling the single institutional config file that you need with the `-c` parameter. +Then you can run the pipeline by directly passing the additional config file with the `-c` parameter: ```bash -nextflow run /path/to/pipeline/ -c /path/to/my/modules/modules-master/conf/my_config.config +nextflow run /path/to/pipeline/ -c /path/to/custom_module.conf ``` > Note that the nf-core/tools helper package has a `download` command to download all required pipeline -> files + singularity containers + institutional modules in one go for you, to make this process easier. +> files + singularity containers + institutional configs + modules in one go for you, to make this process easier. -## Adding a new config +## Adding a new module file -If you decide to upload your module file to `nf-core/module` then this will ensure that it will be automatically downloaded, and available at run-time to all nf-core pipelines, and to everyone within the community. You will simply have to specify `-profile ` in the command used to run the pipeline. See [`nf-core/modules`](https://github.com/nf-core/modules/tree/master/conf) for examples. +If you decide to upload your module file to `nf-core/modules` then this will ensure that it will be automatically downloaded, and available at run-time to all nf-core pipelines, and to everyone within the Nextflow community! See [`nf-core/modules`](https://github.com/nf-core/modules/tree/master/nf) for examples. -Please also make sure to add an extra `params` section with `params.config_profile_description`, `params.config_profile_contact` and `params.config_profile_url` set to reasonable values. Users will get information on who wrote the configuration profile then when executing a nf-core pipeline and can report back if there are things missing for example. +The definition and standards for module files is still under discussion amongst the community but hopefully, a description should be added here soon! ### Testing -If you want to add a new custom config file to `nf-core/modules` please test that your pipeline of choice runs as expected by using the [`-c`](https://www.nextflow.io/docs/latest/config.html) parameter. - -```bash -## Example command for nf-core/rnaseq -nextflow run nf-core/rnaseq --reads '*_R{1,2}.fastq.gz' --genome GRCh37 -c '[path to custom config]' -``` +If you want to add a new module config file to `nf-core/modules` please test that your pipeline of choice runs as expected by using the [`-include`](https://www.nextflow.io/docs/edge/dsl2.html#modules-include) statement with a local version of the module file. ### Documentation -You will have to create a [Markdown document](https://www.markdownguide.org/getting-started/) outlining the details required to use the custom config file within your organisation. You might orientate yourself using the [Template](docs/template.md) that we provide and filling out the information for your cluster there. +You will have to create a [Markdown document](https://www.markdownguide.org/getting-started/) outlining the details required to use the module file and extensive links to documentation for the tool(s) used in the module file. You can use the provided [`template.md`](docs/template.md) to guide you as to how to do this appropriately. See [`nf-core/modules/docs`](https://github.com/nf-core/modules/tree/master/docs) for examples. ### Uploading to `nf-core/modules` -[Fork](https://help.github.com/articles/fork-a-repo/) the `nf-core/modules` repository to your own GitHub account. Within the local clone of your fork add the custom config file to the [`modules/`](https://github.com/nf-core/modules/tree/master/modules) directory, and the documentation file to the [`docs/`](https://github.com/nf-core/modules/tree/master/docs) directory. Please keep the naming consistent between the module and documentation files e.g. `bwa.nf` and `bwa.md`, respectively. +[Fork](https://help.github.com/articles/fork-a-repo/) the `nf-core/modules` repository to your own GitHub account. Within the local clone of your fork add the module file to the [`nf/`](https://github.com/nf-core/modules/tree/master/nf) directory, and the documentation file to the [`docs/`](https://github.com/nf-core/modules/tree/master/docs) directory. Please keep the naming consistent between the module and documentation files e.g. `bwa.nf` and `bwa.md`, respectively. -Commit and push these changes to your local clone on GitHub, and then [create a pull request](https://help.github.com/articles/creating-a-pull-request-from-a-fork/) on the `nf-core/modules` GitHub repo with the appropriate information. +Commit and push these changes to your local clone on GitHub, and then [create a pull request](https://help.github.com/articles/creating-a-pull-request-from-a-fork/) on `nf-core/modules` GitHub repo with the appropriate information. We will be notified automatically when you have created your pull request, and providing that everything adheres to nf-core guidelines we will endeavour to approve your pull request as soon as possible. diff --git a/docs/fastqc.md b/docs/fastqc.md index e69de29b..d714627e 100644 --- a/docs/fastqc.md +++ b/docs/fastqc.md @@ -0,0 +1,2 @@ + +# COMING TO A NEXTFLOW PIPELINE NEAR YOU SOON! diff --git a/docs/template.md b/docs/template.md new file mode 100644 index 00000000..d714627e --- /dev/null +++ b/docs/template.md @@ -0,0 +1,2 @@ + +# COMING TO A NEXTFLOW PIPELINE NEAR YOU SOON! diff --git a/nf/fastqc.nf b/nf/fastqc.nf index e69de29b..d714627e 100644 --- a/nf/fastqc.nf +++ b/nf/fastqc.nf @@ -0,0 +1,2 @@ + +# COMING TO A NEXTFLOW PIPELINE NEAR YOU SOON!