millironx/nf-configs
1
0
Fork 0
mirror of https://github.com/MillironX/nf-configs.git synced 2024-11-22 08:29:54 +00:00
Code Issues Releases Packages Activity

Merge pull request #232 from jfy133/update-mpcdf

Browse source 
Update mpcdf to use singularity
This commit is contained in:
Alexander Peltzer 2021-04-16 09:28:46 +02:00 committed by GitHub
commit a525be112f
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 24 additions and 19 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
conf/mpcdf.config
View file

@ -1,24 +1,28 @@
params { params {
config_profile_description = 'MPCDF HPC profiles (unoffically) provided by nf-core/configs.' config_profile_description = 'MPCDF HPC profiles (unoffically) provided by nf-core/configs.'
config_profile_contact = 'James Fellows Yates (@jfy133)' config_profile_contact = 'James Fellows Yates (@jfy133)'
config_profile_url = 'https://www.mpcdf.mpg.de/services/computing' config_profile_url = 'https://www.mpcdf.mpg.de/services/supercomputing'
} }
profiles { profiles {
cobra { cobra {
// Does not have singularity! Conda module must be used, but it is
// recommended to set NXF_CONDA_CACHEDIR var in ~/.bash{_profile,rc}
// To create common cache dir
process { process {
beforeScript = 'module load anaconda/3/2020.02' beforeScript = 'module load singularity'
executor = 'slurm' executor = 'slurm'
} }
executor { executor {
queueSize = 8 queueSize = 8
} }
// Set $NXF_SINGULARITY_CACHEDIR in your ~/.bash_profile
// to stop downloading the same image for every run
singularity {
enabled = true
autoMounts = true
}
params { params {
config_profile_description = 'MPCDF cobra profile (unofficially) provided by nf-core/configs.' config_profile_description = 'MPCDF cobra profile (unofficially) provided by nf-core/configs.'
@ -28,18 +32,22 @@ profiles {
} }
} }
raven { raven {
// Does not have singularity! Conda module must be used, but it is
// recommended to set NXF_CONDA_CACHEDIR var in ~/.bash{_profile,rc}
// to create common cache dir
process { process {
beforeScript = 'module load anaconda/3/2020.02' beforeScript = 'module load singularity'
executor = 'slurm' executor = 'slurm'
} }
executor { executor {
queueSize = 8 queueSize = 8
} }
// Set $NXF_SINGULARITY_CACHEDIR in your ~/.bash_profile
// to stop downloading the same image for every run
singularity {
enabled = true
autoMounts = true
}
params { params {
config_profile_description = 'MPCDF raven profile (unofficially) provided by nf-core/configs.' config_profile_description = 'MPCDF raven profile (unofficially) provided by nf-core/configs.'
@ -47,5 +55,8 @@ profiles {
max_cpus = 192 max_cpus = 192
max_time = 24.h max_time = 24.h
} }
}
debug {
cleanup = false
} }
} }

14
docs/mpcdf.md
View file

@ -4,18 +4,16 @@ All nf-core pipelines have been successfully configured for use on the HPCs at [
> :warning: these profiles are not officially supported by the MPCDF. > :warning: these profiles are not officially supported by the MPCDF.
To run Nextflow, the `jdk` module must be loaded. To use the nf-core profile(s), run the pipeline with `-profile <cluster>,mpcdf`. To run Nextflow, the `jdk` module must be loaded. To use the nf-core profile(s), run the pipeline with `-profile mpcdf,<cluster>`.
Currently the following clusters are supported: cobra, raven Currently profiles for the following clusters are supported: `cobra`, `raven`
All profiles use `singularity` as the corresponding containerEngine. To prevent repeatedly downloading the same singularity image for every pipeline run, for all profiles we recommend specifying a cache location in your `~/.bash_profile` with the `$NXF_SINGULARITY_CACHEDIR` bash variable.
>NB: Nextflow will need to submit the jobs via SLURM to the clusters and as such the commands above will have to be executed on one of the head nodes. Check the [MPCDF documentation](https://www.mpcdf.mpg.de/services/computing). >NB: Nextflow will need to submit the jobs via SLURM to the clusters and as such the commands above will have to be executed on one of the head nodes. Check the [MPCDF documentation](https://www.mpcdf.mpg.de/services/computing).
## cobra ## cobra
Cobra does not currently support singularity, therefore the anaconda/module is loaded for each process.
Due to this, we also recommend setting the `$NXF_CONDA_CACHEDIR` to a location of your choice to store all environments (so to prevent nextflow building the environment on every run).
To use: `-profile cobra,mpcdf` To use: `-profile cobra,mpcdf`
Sets the following parameters: Sets the following parameters:
@ -31,10 +29,6 @@ Sets the following parameters:
## raven ## raven
Raven does not currently support singularity, therefore `module load anaconda/3/2020.02` is loaded for each process.
Due to this, we also recommend setting the `$NXF_CONDA_CACHEDIR` to a location of your choice to store all environments (so to prevent nextflow building the environment on every run).
To use: `-profile raven,mpcdf` To use: `-profile raven,mpcdf`
Sets the following parameters: Sets the following parameters: