millironx/nf-configs
1
0
Fork 0
mirror of https://github.com/MillironX/nf-configs.git synced 2024-11-22 00:26:03 +00:00
Code Issues Releases Packages Activity

Merge pull request #35 from jfy133/master

Browse source 
Updates to SHH config
This commit is contained in:
Alexander Peltzer 2019-03-15 09:56:44 +01:00 committed by GitHub
commit d95b464e13
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 11 additions and 15 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
conf/shh.config
View file

@ -7,7 +7,7 @@ params {
singularity {
enabled = true
cacheDir = "/projects1/users/$USER/nextflow/nf_cache/singularity/"
cacheDir = "/projects1/singularity_scratch/cache/"
}
process {
@ -16,7 +16,7 @@ process {
}
params {
max_memory = 734.GB
max_cpus = 64
max_memory = 256.GB
max_cpus = 32
max_time = 2.h
}

20
docs/shh.md
View file

@ -1,21 +1,17 @@
# nf-core/configs: SHH Configuration
All nf-core pipelines have been successfully configured for use on the Department of Archaeogenetic's SDAG cluster at the [Max Planck Institute for the Science of Human History (MPI-SHH)](http://shh.mpg.de).
All nf-core pipelines have been successfully configured for use on the Department of Archaeogenetic's SDAG/CDAG clusters at the [Max Planck Institute for the Science of Human History (MPI-SHH)](http://shh.mpg.de).
To use, run the pipeline with `-profile shh`. This will download and launch the [`shh.config`](../conf/shh.config) which has been pre-configured with a setup suitable for the SDAG cluster. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Singularity image before execution of the pipeline.
Note that the configuration file is currently optimised for `nf-core/eager`. It
will submit to the medium queue but with a walltime of 48 hours.
## Preparation
Before running the pipeline you will need to create a the following folder in your `/projects1/users/` directory. This will be used to store the singularity software images, which will take up too much space for your home directory.
This should be named as follows, replacing `<your_user>` with your username:
To use, run the pipeline with `-profile shh`. This will download and launch the [`shh.config`](../conf/shh.config) which has been pre-configured with a setup suitable for the SDAG and CDAG clusters. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Singularity image before execution of the pipeline. The image will currently be centrally stored here:
```bash
"/projects1/users/<your_user>/nextflow/nf_cache/singularity/"
/projects1/singularity_scratch/cache/
```
however this will likely change to a read-only directory in the future that will be managed by IT.
Note that **the configuration file is currently optimised for `nf-core/eager`**. It will submit to the short queue but with a walltime of 2 hours.
>NB: You will need an account and VPN access to use the cluster at MPI-SHH in order to run the pipeline. If in doubt contact IT.
>NB: Nextflow will need to submit the jobs via the job scheduler to the HPC cluster and as such the commands above will have to be executed on one of the lhead nodes. If in doubt contact IT.
>NB: Nextflow will need to submit the jobs via SLURM to the clusters and as such the commands above will have to be executed on one of the head nodes. If in doubt contact IT.