diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 43c15cf..5fb2601 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -16,7 +16,7 @@ jobs: needs: test_all_profiles strategy: matrix: - profile: ['awsbatch', 'bigpurple', 'binac', 'cbe', 'ccga_dx', 'ccga', 'cfc', 'crick', 'denbi_qbic', 'genotoul', 'genouest', 'gis', 'hebbe', 'kraken', 'munin', 'pasteur', 'phoenix', 'prince', 'shh', 'uct_hex', 'uppmax', 'uzh'] + profile: ['awsbatch', 'bigpurple', 'binac', 'cbe', 'ccga_dx', 'ccga_med', 'ccga', 'cfc', 'crick', 'denbi_qbic', 'genotoul', 'genouest', 'gis', 'hebbe', 'kraken', 'munin', 'pasteur', 'phoenix', 'prince', 'shh', 'uct_hex', 'uppmax', 'uzh'] steps: - uses: actions/checkout@v1 - name: Install Nextflow diff --git a/README.md b/README.md index 8e1bbfd..0e58b09 100644 --- a/README.md +++ b/README.md @@ -6,19 +6,19 @@ A repository for hosting Nextflow configuration files containing custom paramete ## Table of contents -- [Using an existing config](#using-an-existing-config) - - [Configuration and parameters](#configuration-and-parameters) - - [Offline usage](#offline-usage) -- [Adding a new config](#adding-a-new-config) - - [Checking user hostnames](#checking-user-hostnames) - - [Testing](#testing) - - [Documentation](#documentation) - - [Uploading to `nf-core/configs`](#uploading-to-nf-coreconfigs) -- [Adding a new pipeline-specific config](#adding-a-new-pipeline-specific-config) - - [Pipeline-specific documentation](#pipeline-specific-documentation) - - [Enabling pipeline-specific configs within a pipeline](#enabling-pipeline-specific-configs-within-a-pipeline) - - [Create the pipeline-specific `nf-core/configs` files](#create-the-pipeline-specific-nf-coreconfigs-files) -- [Help](#help) +* [Using an existing config](#using-an-existing-config) + * [Configuration and parameters](#configuration-and-parameters) + * [Offline usage](#offline-usage) +* [Adding a new config](#adding-a-new-config) + * [Checking user hostnames](#checking-user-hostnames) + * [Testing](#testing) + * [Documentation](#documentation) + * [Uploading to `nf-core/configs`](#uploading-to-nf-coreconfigs) +* [Adding a new pipeline-specific config](#adding-a-new-pipeline-specific-config) + * [Pipeline-specific documentation](#pipeline-specific-documentation) + * [Enabling pipeline-specific configs within a pipeline](#enabling-pipeline-specific-configs-within-a-pipeline) + * [Create the pipeline-specific `nf-core/configs` files](#create-the-pipeline-specific-nf-coreconfigs-files) +* [Help](#help) ## Using an existing config @@ -93,30 +93,31 @@ See [`nf-core/configs/docs`](https://github.com/nf-core/configs/tree/master/docs Currently documentation is available for the following systems: -- [AWSBATCH](docs/awsbatch.md) -- [BIGPURPLE](docs/bigpurple.md) -- [BINAC](docs/binac.md) -- [CBE](docs/cbe.md) -- [CCGA](docs/ccga.md) -- [CCGA_DX](/docs/ccga_dx.md) -- [CFC](docs/binac.md) -- [CRICK](docs/crick.md) -- [CZBIOHUB_AWS](docs/czbiohub.md) -- [CZBIOHUB_AWS_HIGHPRIORITY](docs/czbiohub.md) -- [DENBI_QBIC](docs/denbi_qbic.md) -- [GENOTOUL](docs/genotoul.md) -- [GENOUEST](docs/genouest.md) -- [GIS](docs/gis.md) -- [HEBBE](docs/hebbe.md) -- [KRAKEN](docs/kraken.md) -- [MUNIN](docs/munin.md) -- [PASTEUR](docs/pasteur.md) -- [PHOENIX](docs/phoenix.md) -- [PRINCE](docs/prince.md) -- [SHH](docs/shh.md) -- [UCT_HEX](docs/uct_hex.md) -- [UPPMAX](docs/uppmax.md) -- [UZH](docs/uzh.md) +* [AWSBATCH](docs/awsbatch.md) +* [BIGPURPLE](docs/bigpurple.md) +* [BINAC](docs/binac.md) +* [CBE](docs/cbe.md) +* [CCGA](docs/ccga.md) +* [CCGA_DX](docs/ccga_dx.md) +* [CCGA_MED](docs/ccga_med.md) +* [CFC](docs/binac.md) +* [CRICK](docs/crick.md) +* [CZBIOHUB_AWS](docs/czbiohub.md) +* [CZBIOHUB_AWS_HIGHPRIORITY](docs/czbiohub.md) +* [DENBI_QBIC](docs/denbi_qbic.md) +* [GENOTOUL](docs/genotoul.md) +* [GENOUEST](docs/genouest.md) +* [GIS](docs/gis.md) +* [HEBBE](docs/hebbe.md) +* [KRAKEN](docs/kraken.md) +* [MUNIN](docs/munin.md) +* [PASTEUR](docs/pasteur.md) +* [PHOENIX](docs/phoenix.md) +* [PRINCE](docs/prince.md) +* [SHH](docs/shh.md) +* [UCT_HEX](docs/uct_hex.md) +* [UPPMAX](docs/uppmax.md) +* [UZH](docs/uzh.md) ### Uploading to `nf-core/configs` @@ -125,7 +126,14 @@ Within the local clone of your fork add the custom config file to the [`conf/`]( You will also need to edit and add your custom profile to the [`nfcore_custom.config`](https://github.com/nf-core/configs/blob/master/nfcore_custom.config) file in the top-level directory of the clone. You will also need to edit and add your custom profile to the [`README.md`](https://github.com/nf-core/configs/blob/master/README.md) file in the top-level directory of the clone. -Afterwards, make sure to edit the `.github/main.yml` file and add your profile name to the alphabetically sorted `profile:` scope. This way, it will be tested automatically using GitHub Actions. If you forget to do this, tests will fail and complain about that. +In order to ensure that the config file is tested automatically with GitHub Actions please add your profile name to the `profile:` scope in [`.github/workflows/main.yml`](.github/workflows/main.yml). If you forget to do this the tests will fail with the error: + +```bash +Run python ${GITHUB_WORKSPACE}/bin/cchecker.py ${GITHUB_WORKSPACE}/nfcore_custom.config ${GITHUB_WORKSPACE}/.github/workflows/main.yml +Tests don't seem to test these profiles properly. Please check whether you added the profile to the Github Actions testing YAML. +set(['']) +##[error]Process completed with exit code 1. +``` Commit and push these changes to your local clone on GitHub, and then [create a pull request](https://help.github.com/articles/creating-a-pull-request-from-a-fork/) on the `nf-core/configs` GitHub repo with the appropriate information. @@ -153,13 +161,13 @@ Note that pipeline-specific configs are not required and should only be added if Currently documentation is available for the following pipeline within the specific profile: -- ampliseq - - [BINAC](docs/pipeline/ampliseq/binac.md) -- sarek - - [MUNIN](docs/pipeline/sarek/munin.md) - - [UPPMAX](docs/pipeline/sarek/uppmax.md) -- eager - - [SHH](docs/pipeline/eager/shh.md) +* ampliseq + * [BINAC](docs/pipeline/ampliseq/binac.md) +* eager + * [SHH](docs/pipeline/eager/shh.md) +* sarek + * [MUNIN](docs/pipeline/sarek/munin.md) + * [UPPMAX](docs/pipeline/sarek/uppmax.md) ### Enabling pipeline-specific configs within a pipeline @@ -188,7 +196,7 @@ We will be notified automatically when you have created your pull request, and p [Fork](https://help.github.com/articles/fork-a-repo/) the [`nf-core/configs`](https://github.com/nf-core/configs/) repository to your own GitHub account. And add or edit the following files in the local clone of your fork. -- `pipeline/.config` +* `pipeline/.config` If not already created, create the `pipeline/.config` file, and add your custom profile to the profile scope @@ -198,18 +206,18 @@ profiles { } ``` -- `conf/pipeline//.config` +* `conf/pipeline//.config` Add the custom configuration file to the `conf/pipeline//` directory. Make sure to add an extra `params` section with `params.config_profile_description`, `params.config_profile_contact` to the top of `pipeline/.config` and set to reasonable values. Users will get information on who wrote the pipeline-specific configuration profile then when executing the nf-core pipeline and can report back if there are things missing for example. -- `docs/pipeline//.md` +* `docs/pipeline//.md` Add the documentation file to the `docs/pipeline//` directory. You will also need to edit and add your custom profile to the [`README.md`](https://github.com/nf-core/configs/blob/master/README.md) file in the top-level directory of the clone. -- `README.md` +* `README.md` Edit this file, and add the new pipeline-specific institutional profile to the list in the section Pipeline specific documentation diff --git a/assets/nf-core-configs_social_preview.png b/assets/nf-core-configs_social_preview.png new file mode 100644 index 0000000..20b5bc4 Binary files /dev/null and b/assets/nf-core-configs_social_preview.png differ diff --git a/assets/nf-core-configs_social_preview.svg b/assets/nf-core-configs_social_preview.svg new file mode 100644 index 0000000..235bc55 --- /dev/null +++ b/assets/nf-core-configs_social_preview.svg @@ -0,0 +1,450 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + Config files used to define parameters specific tocompute environments at different Institutions + configs + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/conf/binac.config b/conf/binac.config index 21f2459..d3624df 100644 --- a/conf/binac.config +++ b/conf/binac.config @@ -13,14 +13,14 @@ singularity { process { beforeScript = 'module load devel/singularity/3.4.2' executor = 'pbs' - queue = { task.memory >= 128.GB ? 'smp': task.time <= 20.m ? 'tiny' : task.time <= 48.h ? 'short' : task.time <= 168.h ? 'short' : 'long'} + queue = { task.memory >= 128.GB ? 'smp': task.time <= 20.m ? 'tiny' : task.time > 48.h ? 'long' : 'short'} } params { igenomes_base = '/nfsmounts/igenomes' max_memory = 1000.GB max_cpus = 28 - max_time = 48.h + max_time = 168.h } weblog{ diff --git a/conf/ccga_med.config b/conf/ccga_med.config new file mode 100644 index 0000000..d55fde2 --- /dev/null +++ b/conf/ccga_med.config @@ -0,0 +1,39 @@ +//Profile config names for nf-core/configs +params { + config_profile_description = 'CCGA Med cluster profile provided by nf-core/configs.' + config_profile_contact = 'Marc Hoeppner (@marchoeppner)' + config_profile_url = 'https://www.ccga.uni-kiel.de/' +} + +/* + * ------------------------------------------------- + * Nextflow config file for CCGA cluster in Kiel + * ------------------------------------------------- + */ + +singularity { + enabled = true + runOptions = "-B /work_ifs -B /scratch" + cacheDir = "/work_ifs/ikmb_repository/singularity_cache/" +} + +executor { + queueSize=100 +} + +process { + + // Global process config + executor = 'slurm' + queue = 'all' + +} + +params { + // illumina iGenomes reference file paths on RZCluster + igenomes_base = '/work_ifs/ikmb_repository/references/iGenomes/references/' + saveReference = true + max_memory = 250.GB + max_cpus = 24 + max_time = 120.h +} diff --git a/docs/ccga_med.md b/docs/ccga_med.md new file mode 100644 index 0000000..9aa4236 --- /dev/null +++ b/docs/ccga_med.md @@ -0,0 +1,16 @@ +# nf-core/configs: CCGA Med Cluster Configuration + +Deployment and testing of nf-core pipelines at the CCGA Med cluster is on-going. + +To use, run the pipeline with `-profile ccga_med`. This will download and launch the [`ccga_med.config`](../conf/ccga_med.config) which has been pre-configured with a setup suitable for the CCGA cluster. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Singularity image before execution of the pipeline. + +Before running the pipeline you will need to load Nextflow and Singularity using the environment module system on the cluster. You can do this by issuing the commands below: + +```bash +## Load Nextflow and Singularity environment modules +module purge +module load nextflow +module load singularity +``` + +>NB: Access to the CCGA Med cluster is restricted to IKMB/CCGA employees. Please talk to Marc Hoeppner to get access (@marchoeppner). diff --git a/nfcore_custom.config b/nfcore_custom.config index 404d11e..8b4a3a9 100644 --- a/nfcore_custom.config +++ b/nfcore_custom.config @@ -19,6 +19,7 @@ profiles { cbe { includeConfig "${params.custom_config_base}/conf/cbe.config" } ccga { includeConfig "${params.custom_config_base}/conf/ccga.config" } ccga_dx { includeConfig "${params.custom_config_base}/conf/ccga_dx.config" } + ccga_med { includeConfig "${params.custom_config_base}/conf/ccga_med.config" } cfc { includeConfig "${params.custom_config_base}/conf/cfc.config" } crick { includeConfig "${params.custom_config_base}/conf/crick.config" } czbiohub_aws { includeConfig "${params.custom_config_base}/conf/czbiohub_aws.config" }