mirror of
https://github.com/MillironX/nf-configs.git
synced 2024-11-21 16:16:04 +00:00
added nyu prince and bigpurple md files, added singularity cacheDir to prince and bigpurple configs
This commit is contained in:
parent
4a915e330a
commit
4ba831705d
4 changed files with 53 additions and 5 deletions
|
@ -1,17 +1,20 @@
|
||||||
|
singularityDir = "/gpfs/scratch/${USER}/singularity_images_nextflow"
|
||||||
|
|
||||||
params {
|
params {
|
||||||
config_profile_description = """
|
config_profile_description = """
|
||||||
NYU School of Medicine BigPurple cluster profile provided by nf-core/configs.
|
NYU School of Medicine BigPurple cluster profile provided by nf-core/configs.
|
||||||
module load both singularity/3.1 and squashfs-tools/4.3 before running the pipeline with this profile!!
|
module load both singularity/3.1 and squashfs-tools/4.3 before running the pipeline with this profile!!
|
||||||
Run from your scratch or lab directory - Nextflow makes a lot of files!!
|
Run from your scratch or lab directory - Nextflow makes a lot of files!!
|
||||||
Also consider running the pipeline on a compute node the first time, as it will be pulling the docker image, which will be converted into a singularity image, which is heavy on the login node. Subsequent runs can be done on the login node, as the docker image will only be pulled and converted once. By default the images will be stored in ./work/singularity
|
Also consider running the pipeline on a compute node (srun --pty /bin/bash -t=01:00:00) the first time, as it will be pulling the docker image, which will be converted into a singularity image, which is heavy on the login node and will take some time. Subsequent runs can be done on the login node, as the docker image will only be pulled and converted once. By default the images will be stored in $singularityDir
|
||||||
""".stripIndents()
|
""".stripIndent()
|
||||||
config_profile_contact = 'Tobias Schraink (@tobsecret)'
|
config_profile_contact = 'Tobias Schraink (@tobsecret)'
|
||||||
config_profile_url = 'https://genome.med.nyu.edu/hpcf/wiki/Manual:Cluster_User_Guide'
|
config_profile_url = 'https://github.com/nf-core/configs/blob/master/docs/bigpurple.md'
|
||||||
}
|
}
|
||||||
|
|
||||||
singularity {
|
singularity {
|
||||||
enabled = true
|
enabled = true
|
||||||
autoMounts = true
|
autoMounts = true
|
||||||
|
cacheDir = singularityDir
|
||||||
}
|
}
|
||||||
|
|
||||||
process {
|
process {
|
||||||
|
|
|
@ -1,15 +1,18 @@
|
||||||
|
singularityDir = "$SCRATCH/singularity_images_nextflow"
|
||||||
|
|
||||||
params {
|
params {
|
||||||
config_profile_description = """
|
config_profile_description = """
|
||||||
NYU prince cluster profile provided by nf-core/configs.
|
NYU prince cluster profile provided by nf-core/configs.
|
||||||
Run from your scratch directory, the output files may be large!
|
Run from your scratch directory, the output files may be large!
|
||||||
Please consider running the pipeline on a compute node the first time, as it will be pulling the docker image, which will be converted into a singularity image, which is heavy on the login node. Subsequent runs can be done on the login node, as the docker image will only be pulled and converted once. By default the images will be stored in ./work/singularity
|
Please consider running the pipeline on a compute node the first time, as it will be pulling the docker image, which will be converted into a singularity image, which is heavy on the login node. Subsequent runs can be done on the login node, as the docker image will only be pulled and converted once. By default the images will be stored in $singularityDir
|
||||||
""".stripIndent()
|
""".stripIndent()
|
||||||
config_profile_contact = 'Tobias Schraink (@tobsecret)'
|
config_profile_contact = 'Tobias Schraink (@tobsecret)'
|
||||||
config_profile_url = 'https://wikis.nyu.edu/display/NYUHPC/Clusters+-+Prince'
|
config_profile_url = 'https://github.com/nf-core/configs/blob/master/docs/prince.md'
|
||||||
}
|
}
|
||||||
|
|
||||||
singularity {
|
singularity {
|
||||||
enabled = true
|
enabled = true
|
||||||
|
cacheDir = singularityDir
|
||||||
}
|
}
|
||||||
|
|
||||||
process {
|
process {
|
||||||
|
|
23
docs/bigpurple.md
Normal file
23
docs/bigpurple.md
Normal file
|
@ -0,0 +1,23 @@
|
||||||
|
# nf-core/configs: BigPurple Configuration
|
||||||
|
## nf-core pipelines that use this repo
|
||||||
|
All nf-core pipelines that use this config repo (which is most), can be run on BigPurple. **Before** running a pipeline for the first time, go into an interactive slurm session on a compute node (`srun --pty --time=02:00:00 -c 2`), as the docker image for the pipeline will need to be pulled and converted. Once in the interactive session:
|
||||||
|
|
||||||
|
```
|
||||||
|
module load singularity/3.1
|
||||||
|
module load squashfs-tools/4.3
|
||||||
|
```
|
||||||
|
|
||||||
|
Now, run the pipeline of your choice with `-profile bigpurple`. This will download and launch the bigpurple.config which has been pre-configured with a setup suitable for the BigPurple cluster. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a singularity image before execution of the pipeline.
|
||||||
|
An example commandline:
|
||||||
|
|
||||||
|
`nextflow run nf-core/<pipeline name> -profile bigpurple <additional flags>`
|
||||||
|
|
||||||
|
## nf-core pipelines that do not use this repo
|
||||||
|
If the pipeline has not yet been configured to use this config, then you will have to do it manually.
|
||||||
|
git clone this repo, copy the `bigpurple.config` from the conf folder and then you can invoke the pipeline like this:
|
||||||
|
|
||||||
|
`nextflow run nf-core/<pipeline name> -c bigpurple.config <additional flags>`
|
||||||
|
|
||||||
|
|
||||||
|
>NB: You will need an account to use the HPC cluster BigPurple in order to run the pipeline. If in doubt contact MCIT.
|
||||||
|
>NB: You will need to install nextflow in your home directory - instructions are on nextflow.io (or ask the writer of this profile). The reason there is no module for nextflow on the cluster, is that the development cycle of nextflow is rapid and it's easy to update yourself: `nextflow self-update`
|
19
docs/prince.md
Normal file
19
docs/prince.md
Normal file
|
@ -0,0 +1,19 @@
|
||||||
|
# nf-core/configs: Prince Configuration
|
||||||
|
## nf-core pipelines that use this repo
|
||||||
|
All nf-core pipelines that use this config repo (which is most), can be run on prince. **Before** running a pipeline for the first time, go into an interactive slurm session on a compute node (`srun --pty --time=02:00:00 -c 2`), as the docker image for the pipeline will need to be pulled and converted.
|
||||||
|
|
||||||
|
Now, run the pipeline of your choice with `-profile prince`. This will download and launch the prince.config which has been pre-configured with a setup suitable for the prince cluster. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a singularity image before execution of the pipeline. This step **takes time**!!
|
||||||
|
An example commandline:
|
||||||
|
|
||||||
|
`nextflow run nf-core/<pipeline name> -profile prince <additional flags>`
|
||||||
|
|
||||||
|
## nf-core pipelines that do not use this repo
|
||||||
|
If the pipeline has not yet been configured to use this config, then you will have to do it manually.
|
||||||
|
git clone this repo, copy the `prince.config` from the conf folder and then you can invoke the pipeline like this:
|
||||||
|
|
||||||
|
`nextflow run nf-core/<pipeline name> -c prince.config <additional flags>`
|
||||||
|
|
||||||
|
|
||||||
|
>NB: You will need an account to use the HPC cluster Prince in order to run the pipeline. If in doubt contact the HPC admins.
|
||||||
|
>NB: Rather than using the nextflow module, I recommend you install nextflow in your home directory - instructions are on nextflow.io (or ask the writer of this profile). The reason this is better than using the module for nextflow on the cluster, is that the development cycle of nextflow is rapid and it's easy to update your installation yourself: `nextflow self-update`
|
||||||
|
|
Loading…
Reference in a new issue