millironx/nf-configs
1
0
Fork 0
mirror of https://github.com/MillironX/nf-configs.git synced 2024-11-22 00:26:03 +00:00
Code Issues Releases Packages Activity

Merge branch 'nf-core:master' into sbc_sharc

Browse source
This commit is contained in:
Lewis Quayle 2022-09-28 10:49:05 +01:00 committed by GitHub
commit 1eea9b464a
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 10 additions and 9 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
conf/biohpc_gen.config
View file

@ -11,9 +11,8 @@ env {
process { process {
executor = 'slurm' executor = 'slurm'
queue = { task.memory <= 1536.GB ? (task.time > 2.d || task.memory > 384.GB ? 'biohpc_gen_production' : 'biohpc_gen_normal') : 'biohpc_gen_highmem' } queue = { task.memory <= 1536.GB ? (task.time > 2.d || task.memory > 384.GB ? 'biohpc_gen_production' : 'biohpc_gen_normal') : 'biohpc_gen_highmem' }
beforeScript = 'module use /dss/dsslegfs02/pn73se/pn73se-dss-0000/spack/modules/x86_avx2/linux*' module = 'charliecloud/0.25'
module = 'charliecloud/0.22:miniconda3'
} }
charliecloud { charliecloud {
@ -21,7 +20,7 @@ charliecloud {
} }
params { params {
params.max_time = 14.d params.max_time = 14.d
params.max_cpus = 80 params.max_cpus = 80
params.max_memory = 3.TB params.max_memory = 3.TB
} }

3
conf/sage.config
View file

@ -29,9 +29,12 @@ aws {
region = "us-east-1" region = "us-east-1"
client { client {
uploadChunkSize = 209715200 uploadChunkSize = 209715200
uploadMaxThreads = 4
} }
batch { batch {
maxParallelTransfers = 1 maxParallelTransfers = 1
maxTransferAttempts = 5
delayBetweenAttempts = '120 sec'
} }
} }
executor { executor {

6
docs/biohpc_gen.md
View file

@ -4,14 +4,12 @@ All nf-core pipelines have been successfully configured for use on the BioHPC Ge
To use, run the pipeline with `-profile biohpc_gen`. This will download and launch the [`biohpc_gen.config`](../conf/biohpc_gen.config) which has been pre-configured with a setup suitable for the biohpc_gen cluster. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Charliecloud container before execution of the pipeline. To use, run the pipeline with `-profile biohpc_gen`. This will download and launch the [`biohpc_gen.config`](../conf/biohpc_gen.config) which has been pre-configured with a setup suitable for the biohpc_gen cluster. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Charliecloud container before execution of the pipeline.
Before running the pipeline you will need to load Nextflow and Charliecloud using the environment module system on biohpc_gen. You can do this by issuing the commands below: Before running the pipeline you will need to load Nextflow and Charliecloud using the environment module system on a login node. You can do this by issuing the commands below:
```bash ```bash
## Load Nextflow and Charliecloud environment modules ## Load Nextflow and Charliecloud environment modules
module purge module load nextflow/21.04.3 charliecloud/0.25
module load nextflow charliecloud/0.22
``` ```
> NB: Charliecloud support requires Nextflow version `21.03.0-edge` or later.
> NB: You will need an account to use the LRZ Linux cluster as well as group access to the biohpc_gen cluster in order to run nf-core pipelines. > NB: You will need an account to use the LRZ Linux cluster as well as group access to the biohpc_gen cluster in order to run nf-core pipelines.
> NB: Nextflow will need to submit the jobs via the job scheduler to the HPC cluster and as such the commands above will have to be executed on one of the login nodes. > NB: Nextflow will need to submit the jobs via the job scheduler to the HPC cluster and as such the commands above will have to be executed on one of the login nodes.

1
docs/sage.md
View file

@ -8,6 +8,7 @@ This global configuration includes the following tweaks:
- Enable retries by default when exit codes relate to insufficient memory - Enable retries by default when exit codes relate to insufficient memory
- Allow pending jobs to finish if the number of retries are exhausted - Allow pending jobs to finish if the number of retries are exhausted
- Increase the amount of time allowed for file transfers - Increase the amount of time allowed for file transfers
- Improve reliability of file transfers with retries and reduced concurrency
- Increase the default chunk size for multipart uploads to S3 - Increase the default chunk size for multipart uploads to S3
- Slow down job submission rate to avoid overwhelming any APIs - Slow down job submission rate to avoid overwhelming any APIs
- Define the `check_max()` function, which is missing in Sarek v2 - Define the `check_max()` function, which is missing in Sarek v2