1
0
Fork 0
mirror of https://github.com/MillironX/nf-configs.git synced 2024-11-21 16:16:04 +00:00

Merge branch 'master' into master

This commit is contained in:
Alexander Peltzer 2020-05-29 11:43:47 +02:00 committed by GitHub
commit f877644059
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
38 changed files with 518 additions and 101 deletions

View file

@ -16,7 +16,7 @@ jobs:
needs: test_all_profiles needs: test_all_profiles
strategy: strategy:
matrix: matrix:
profile: ['awsbatch', 'bigpurple', 'binac', 'cbe', 'ccga_dx', 'ccga_med', 'ccga', 'cfc', 'crick', 'denbi_qbic', 'genotoul', 'genouest', 'gis', 'hebbe', 'kraken', 'munin', 'pasteur', 'phoenix', 'prince', 'shh', 'uct_hex', 'uppmax', 'uzh'] profile: ['awsbatch', 'bi','bigpurple', 'binac', 'cbe', 'ccga_dx', 'ccga_med', 'cfc', 'cfc_dev', 'crick', 'denbi_qbic', 'ebc', 'genotoul', 'genouest', 'gis', 'google', 'hebbe', 'kraken', 'munin', 'pasteur', 'phoenix', 'prince', 'shh', 'uct_hex', 'uppmax', 'utd_ganymede', 'uzh']
steps: steps:
- uses: actions/checkout@v1 - uses: actions/checkout@v1
- name: Install Nextflow - name: Install Nextflow
@ -26,4 +26,5 @@ jobs:
- name: Check ${{ matrix.profile }} profile - name: Check ${{ matrix.profile }} profile
env: env:
SCRATCH: '~' SCRATCH: '~'
NXF_GLOBAL_CONFIG: awsbatch.config
run: nextflow run ${GITHUB_WORKSPACE}/configtest.nf --custom_config_base=${GITHUB_WORKSPACE} -profile ${{ matrix.profile }} run: nextflow run ${GITHUB_WORKSPACE}/configtest.nf --custom_config_base=${GITHUB_WORKSPACE} -profile ${{ matrix.profile }}

View file

@ -15,6 +15,7 @@ A repository for hosting Nextflow configuration files containing custom paramete
* [Documentation](#documentation) * [Documentation](#documentation)
* [Uploading to `nf-core/configs`](#uploading-to-nf-coreconfigs) * [Uploading to `nf-core/configs`](#uploading-to-nf-coreconfigs)
* [Adding a new pipeline-specific config](#adding-a-new-pipeline-specific-config) * [Adding a new pipeline-specific config](#adding-a-new-pipeline-specific-config)
* [Pipeline-specific institutional documentation](#pipeline-specific-institutional-documentation)
* [Pipeline-specific documentation](#pipeline-specific-documentation) * [Pipeline-specific documentation](#pipeline-specific-documentation)
* [Enabling pipeline-specific configs within a pipeline](#enabling-pipeline-specific-configs-within-a-pipeline) * [Enabling pipeline-specific configs within a pipeline](#enabling-pipeline-specific-configs-within-a-pipeline)
* [Create the pipeline-specific `nf-core/configs` files](#create-the-pipeline-specific-nf-coreconfigs-files) * [Create the pipeline-specific `nf-core/configs` files](#create-the-pipeline-specific-nf-coreconfigs-files)
@ -95,19 +96,21 @@ Currently documentation is available for the following systems:
* [AWSBATCH](docs/awsbatch.md) * [AWSBATCH](docs/awsbatch.md)
* [BIGPURPLE](docs/bigpurple.md) * [BIGPURPLE](docs/bigpurple.md)
* [BI](docs/bi.md)
* [BINAC](docs/binac.md) * [BINAC](docs/binac.md)
* [CBE](docs/cbe.md) * [CBE](docs/cbe.md)
* [CCGA](docs/ccga.md)
* [CCGA_DX](docs/ccga_dx.md) * [CCGA_DX](docs/ccga_dx.md)
* [CCGA_MED](docs/ccga_med.md) * [CCGA_MED](docs/ccga_med.md)
* [CFC](docs/binac.md) * [CFC](docs/cfc.md)
* [CRICK](docs/crick.md) * [CRICK](docs/crick.md)
* [CZBIOHUB_AWS](docs/czbiohub.md) * [CZBIOHUB_AWS](docs/czbiohub.md)
* [CZBIOHUB_AWS_HIGHPRIORITY](docs/czbiohub.md) * [CZBIOHUB_AWS_HIGHPRIORITY](docs/czbiohub.md)
* [DENBI_QBIC](docs/denbi_qbic.md) * [DENBI_QBIC](docs/denbi_qbic.md)
* [EBC](docs/ebc.md)
* [GENOTOUL](docs/genotoul.md) * [GENOTOUL](docs/genotoul.md)
* [GENOUEST](docs/genouest.md) * [GENOUEST](docs/genouest.md)
* [GIS](docs/gis.md) * [GIS](docs/gis.md)
* [GOOGLE](docs/google.md)
* [HEBBE](docs/hebbe.md) * [HEBBE](docs/hebbe.md)
* [KRAKEN](docs/kraken.md) * [KRAKEN](docs/kraken.md)
* [MUNIN](docs/munin.md) * [MUNIN](docs/munin.md)
@ -117,6 +120,7 @@ Currently documentation is available for the following systems:
* [SHH](docs/shh.md) * [SHH](docs/shh.md)
* [UCT_HEX](docs/uct_hex.md) * [UCT_HEX](docs/uct_hex.md)
* [UPPMAX](docs/uppmax.md) * [UPPMAX](docs/uppmax.md)
* [UTD_GANYMEDE](docs/utd_ganymede.md)
* [UZH](docs/uzh.md) * [UZH](docs/uzh.md)
### Uploading to `nf-core/configs` ### Uploading to `nf-core/configs`
@ -157,18 +161,28 @@ Each configuration file will add new params and overwrite the params already exi
Note that pipeline-specific configs are not required and should only be added if needed. Note that pipeline-specific configs are not required and should only be added if needed.
### Pipeline-specific documentation ### Pipeline-specific institutional documentation
Currently documentation is available for the following pipeline within the specific profile: Currently documentation is available for the following pipelines within specific profiles:
* ampliseq * ampliseq
* [BINAC](docs/pipeline/ampliseq/binac.md) * [BINAC](docs/pipeline/ampliseq/binac.md)
* [UPPMAX](docs/pipeline/ampliseq/uppmax.md)
* eager * eager
* [SHH](docs/pipeline/eager/shh.md) * [SHH](docs/pipeline/eager/shh.md)
* rnafusion
* [MUNIN](docs/pipeline/rnafusion/munin.md)
* sarek * sarek
* [MUNIN](docs/pipeline/sarek/munin.md) * [MUNIN](docs/pipeline/sarek/munin.md)
* [UPPMAX](docs/pipeline/sarek/uppmax.md) * [UPPMAX](docs/pipeline/sarek/uppmax.md)
### Pipeline-specific documentation
Currently documentation is available for the following pipeline:
* viralrecon
* [genomes](docs/pipeline/viralrecon/genomes.md)
### Enabling pipeline-specific configs within a pipeline ### Enabling pipeline-specific configs within a pipeline
:warning: **This has to be done on a fork of the `nf-core/<PIPELINE>` repository.** :warning: **This has to be done on a fork of the `nf-core/<PIPELINE>` repository.**

18
conf/bi.config Normal file
View file

@ -0,0 +1,18 @@
params{
config_profile_description = 'Boehringer Ingelheim internal profile provided by nf-core/configs.'
config_profile_contact = 'Alexander Peltzer (@apeltzer)'
config_profile_url = 'https://www.boehringer-ingelheim.com/'
}
params.globalConfig = determine_global_config()
includeConfig params.globalConfig
def determine_global_config() {
if( System.getenv('NXF_GLOBAL_CONFIG') == null)
{
def errorMessage = "ERROR: Environment variable NXF_GLOBAL_CONFIG is missing. Set it to point to global.config file."
System.err.println(errorMessage)
throw new Exception(errorMessage)
}
return System.getenv('NXF_GLOBAL_CONFIG')
}

View file

@ -14,7 +14,7 @@ process {
singularity { singularity {
enabled = true enabled = true
cacheDir = '/scratch-cbe/shared/containers' cacheDir = '/resources/containers'
} }
params { params {

View file

@ -1,41 +0,0 @@
//Profile config names for nf-core/configs
params {
config_profile_description = 'CCGA cluster profile provided by nf-core/configs.'
config_profile_contact = 'Marc Hoeppner (@marchoeppner)'
config_profile_url = 'https://www.ccga.uni-kiel.de/'
}
/*
* -------------------------------------------------
* Nextflow config file for CCGA cluster in Kiel
* -------------------------------------------------
*/
singularity {
enabled = true
runOptions = "-B /ifs -B /scratch -B /work_beegfs"
cacheDir = "/ifs/data/nfs_share/ikmb_repository/singularity_cache/"
}
executor {
queueSize=100
}
process {
// Global process config
executor = 'slurm'
queue = 'ikmb_a'
clusterOptions = { "--qos=ikmb_a" }
}
params {
// illumina iGenomes reference file paths on RZCluster
igenomes_base = '/ifs/data/nfs_share/ikmb_repository/references/iGenomes/references/'
saveReference = true
max_memory = 128.GB
max_cpus = 16
max_time = 120.h
}

View file

@ -1,7 +1,7 @@
//Profile config names for nf-core/configs //Profile config names for nf-core/configs
params { params {
config_profile_description = 'QBiC Core Facility cluster profile provided by nf-core/configs.' config_profile_description = 'QBiC Core Facility cluster profile provided by nf-core/configs.'
config_profile_contact = 'Alexander Peltzer (@apeltzer)' config_profile_contact = 'Gisela Gabernet (@ggabernet)'
config_profile_url = 'http://qbic.uni-tuebingen.de/' config_profile_url = 'http://qbic.uni-tuebingen.de/'
} }
@ -13,6 +13,8 @@ singularity {
process { process {
beforeScript = 'module load devel/singularity/3.4.2' beforeScript = 'module load devel/singularity/3.4.2'
executor = 'slurm' executor = 'slurm'
queue = { task.memory > 60.GB || task.cpus > 20 ? 'qbic' : 'compute' }
scratch = 'true'
} }
weblog{ weblog{
@ -22,7 +24,7 @@ weblog{
params { params {
igenomes_base = '/nfsmounts/igenomes' igenomes_base = '/nfsmounts/igenomes'
max_memory = 498.GB max_memory = 1999.GB
max_cpus = 20 max_cpus = 128
max_time = 140.h max_time = 140.h
} }

29
conf/cfc_dev.config Normal file
View file

@ -0,0 +1,29 @@
//Profile config names for nf-core/configs
params {
config_profile_description = 'QBiC Core Facility cluster dev profile without container cache provided by nf-core/configs.'
config_profile_contact = 'Gisela Gabernet (@ggabernet)'
config_profile_url = 'http://qbic.uni-tuebingen.de/'
}
singularity {
enabled = true
}
process {
beforeScript = 'module load devel/singularity/3.4.2'
executor = 'slurm'
queue = { task.memory > 60.GB || task.cpus > 20 ? 'qbic' : 'compute' }
scratch = 'true'
}
weblog{
enabled = true
url = 'https://services.qbic.uni-tuebingen.de/flowstore/workflows'
}
params {
igenomes_base = '/nfsmounts/igenomes'
max_memory = 1999.GB
max_cpus = 128
max_time = 140.h
}

View file

@ -50,6 +50,7 @@ params {
// No final slash because it's added later // No final slash because it's added later
gencode_base = "s3://czbiohub-reference/gencode" gencode_base = "s3://czbiohub-reference/gencode"
transgenes_base = "s3://czbiohub-reference/transgenes" transgenes_base = "s3://czbiohub-reference/transgenes"
refseq_base = "s3://czbiohub-reference/ncbi/genomes/refseq/"
// AWS configurations // AWS configurations
awsregion = "us-west-2" awsregion = "us-west-2"
@ -79,6 +80,12 @@ params {
transcript_fasta = "${params.gencode_base}/mouse/vM21/gencode.vM21.transcripts.ERCC92.fa" transcript_fasta = "${params.gencode_base}/mouse/vM21/gencode.vM21.transcripts.ERCC92.fa"
star = "${params.gencode_base}/mouse/vM21/STARIndex/" star = "${params.gencode_base}/mouse/vM21/STARIndex/"
} }
'AaegL5.0' {
fasta = "${params.refseq_base}/invertebrate/Aedes_aegypti/GCF_002204515.2_AaegL5.0/nf-core--rnaseq/reference_genome/GCF_002204515.2_AaegL5.0_genomic.fna"
gtf = "${params.refseq_base}/invertebrate/Aedes_aegypti/GCF_002204515.2_AaegL5.0/nf-core--rnaseq/reference_genome/GCF_002204515.2_AaegL5.0_genomic.gtf"
bed = "${params.refseq_base}/invertebrate/Aedes_aegypti/GCF_002204515.2_AaegL5.0/nf-core--rnaseq/reference_genome/GCF_002204515.2_AaegL5.0_genomic.bed"
star = "${params.refseq_base}/invertebrate/Aedes_aegypti/GCF_002204515.2_AaegL5.0/nf-core--rnaseq/reference_genome/star/"
}
} }
transgenes { transgenes {

25
conf/ebc.config Normal file
View file

@ -0,0 +1,25 @@
//Profile config names for nf-core/configs
params {
config_profile_description = 'Generic Estonian Biocentre profile provided by nf-core/configs.'
config_profile_contact = 'Marcel Keller (@marcel-keller)'
config_profile_url = 'https://genomics.ut.ee/en/about-us/estonian-biocentre'
}
cleanup = true
conda {
cacheDir = '/ebc_data/nf-core/conda'
}
process {
executor = 'slurm'
conda = "$baseDir/environment.yml"
beforeScript = 'module load nextflow'
}
executor {
queueSize = 16
}
params {
max_memory = 12.GB
max_cpus = 20
max_time = 120.h
}

21
conf/google.config Normal file
View file

@ -0,0 +1,21 @@
// Nextflow config file for running on Google Cloud Life Sciences
params {
config_profile_description = 'Google Cloud Life Sciences Profile'
config_profile_contact = 'Evan Floden, Seqera Labs (@evanfloden)'
config_profile_url = 'https://cloud.google.com/life-sciences'
google_zone = 'europe-west2-c'
google_bucket = false
google_debug = false
google_preemptible = true
}
process.executor = 'google-lifesciences'
google.zone = params.google_zone
google.lifeSciences.debug = params.google_debug
workDir = params.google_bucket
google.lifeSciences.preemptible = params.google_preemptible
if (google.lifeSciences.preemptible) {
process.errorStrategy = { task.exitStatus==14 ? 'retry' : 'terminate' }
process.maxRetries = 5
}

View file

@ -0,0 +1,15 @@
// Profile config names for nf-core/configs
params {
// Specific nf-core/configs params
config_profile_contact = 'Daniel Lundin (daniel.lundin@lnu.se)'
config_profile_description = 'nf-core/ampliseq UPPMAX profile provided by nf-core/configs'
}
withName: make_SILVA_132_16S_classifier {
clusterOptions = { "-A $params.project -C fat -p node -N 1 ${params.clusterOptions ?: ''}" }
}
withName: classifier {
clusterOptions = { "-A $params.project -C fat -p node -N 1 ${params.clusterOptions ?: ''}" }
}

View file

@ -5,6 +5,10 @@ params {
config_profile_contact = 'James Fellows Yates (@jfy133)' config_profile_contact = 'James Fellows Yates (@jfy133)'
config_profile_description = 'nf-core/eager SHH profile provided by nf-core/configs' config_profile_description = 'nf-core/eager SHH profile provided by nf-core/configs'
igenomes_base = "/projects1/public_data/igenomes/" igenomes_base = "/projects1/public_data/igenomes/"
// default BWA
bwaalnn = 0.04
bwaalnl = 32
} }
// Specific nf-core/eager process configuration // Specific nf-core/eager process configuration
@ -17,3 +21,27 @@ process {
queue = { task.memory > 756.GB ? 'supercruncher' : 'long' } queue = { task.memory > 756.GB ? 'supercruncher' : 'long' }
} }
} }
profiles {
pathogen_loose {
params {
config_profile_description = 'Pathogen (loose) MPI-SHH profile, provided by nf-core/configs.'
bwaalnn = 0.01
bwaalnl = 16
}
}
pathogen_strict {
params {
config_profile_description = 'Pathogen (strict) MPI-SHH SDAG profile, provided by nf-core/configs.'
bwaalnn = 0.1
bwaalnl = 32
}
}
human {
params {
config_profile_description = 'Human MPI-SHH SDAG profile, provided by nf-core/configs.'
bwaalnn = 0.01
bwaalnl = 16500
}
}
}

View file

@ -0,0 +1,10 @@
// rnafusion/munin specific profile config
params {
max_cpus = 24
max_memory = 256.GB
max_time = 72.h
// Paths
genomes_base = '/data1/references/rnafusion/dev/'
}

View file

@ -1,4 +1,4 @@
// Profile config names for nf-core/configs // sarek/munin specific profile config
params { params {
// Specific nf-core/configs params // Specific nf-core/configs params
@ -7,16 +7,22 @@ params {
// Specific nf-core/sarek params // Specific nf-core/sarek params
annotation_cache = true annotation_cache = true
cadd_cache = true
cadd_indels = '/data1/cache/CADD/v1.4/InDels.tsv.gz'
cadd_indels_tbi = '/data1/cache/CADD/v1.4/InDels.tsv.gz.tbi'
cadd_wg_snvs = '/data1/cache/CADD/v1.4/whole_genome_SNVs.tsv.gz'
cadd_wg_snvs_tbi = '/data1/cache/CADD/v1.4/whole_genome_SNVs.tsv.gz.tbi'
pon = '/data1/PON/vcfs/BTB.PON.vcf.gz' pon = '/data1/PON/vcfs/BTB.PON.vcf.gz'
pon_index = '/data1/PON/vcfs/BTB.PON.vcf.gz.tbi' pon_index = '/data1/PON/vcfs/BTB.PON.vcf.gz.tbi'
snpEff_cache = '/data1/cache/snpEff/' snpeff_cache = '/data1/cache/snpEff/'
vep_cache = '/data1/cache/VEP/' vep_cache = '/data1/cache/VEP/'
vep_cache_version = '95'
} }
// Specific nf-core/sarek process configuration // Specific nf-core/sarek process configuration
process { process {
withLabel:sentieon { withLabel:sentieon {
module = {params.sentieon ? 'sentieon/201808.05' : null} module = {params.sentieon ? 'sentieon/201911.00' : null}
container = {params.sentieon ? null : container} container = {params.sentieon ? null : container}
} }
} }

View file

@ -9,9 +9,13 @@ params {
igenomeIgnore = true igenomeIgnore = true
genomes_base = params.genome == 'GRCh37' ? '/sw/data/uppnex/ToolBox/ReferenceAssemblies/hg38make/bundle/2.8/b37' : '/sw/data/uppnex/ToolBox/hg38bundle' genomes_base = params.genome == 'GRCh37' ? '/sw/data/uppnex/ToolBox/ReferenceAssemblies/hg38make/bundle/2.8/b37' : '/sw/data/uppnex/ToolBox/hg38bundle'
} }
def hostname = "hostname".execute().text.trim()
if (hostname ==~ "r.*") { if (hostname ==~ "r.*") {
params.singleCPUmem = 6400.MB params.singleCPUmem = 6400.MB
} }
if (hostname ==~ "i.*") { if (hostname ==~ "i.*") {
params.singleCPUmem = 15.GB params.singleCPUmem = 15.GB
} }

View file

@ -0,0 +1,20 @@
/*
* -------------------------------------------------
* nfcore/viralrecon custom profile Nextflow config file
* -------------------------------------------------
* Defines viral reference genomes for all environments.
*/
params {
// Genome reference file paths
genomes {
'NC_045512.2' {
fasta = "https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/genome/NC_045512.2/GCF_009858895.2_ASM985889v3_genomic.200409.fna.gz"
gff = "https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/genome/NC_045512.2/GCF_009858895.2_ASM985889v3_genomic.200409.gff.gz"
}
'MN908947.3' {
fasta = "https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/genome/MN908947.3/GCA_009858895.3_ASM985889v3_genomic.200409.fna.gz"
gff = "https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/genome/MN908947.3/GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz"
}
}
}

View file

@ -3,6 +3,11 @@ params {
config_profile_description = 'Generic MPI-SHH cluster(s) profile provided by nf-core/configs.' config_profile_description = 'Generic MPI-SHH cluster(s) profile provided by nf-core/configs.'
config_profile_contact = 'James Fellows Yates (@jfy133), Maxime Borry (@Maxibor)' config_profile_contact = 'James Fellows Yates (@jfy133), Maxime Borry (@Maxibor)'
config_profile_url = 'https://shh.mpg.de' config_profile_url = 'https://shh.mpg.de'
max_memory = 256.GB
max_cpus = 32
max_time = 720.h
//Illumina iGenomes reference file path
igenomes_base = "/projects1/public_data/igenomes/"
} }
cleanup = true cleanup = true
@ -23,21 +28,17 @@ executor {
queueSize = 16 queueSize = 16
} }
params {
max_memory = 256.GB
max_cpus = 32
max_time = 720.h
//Illumina iGenomes reference file path
igenomes_base = "/projects1/public_data/igenomes/"
}
profiles { profiles {
cdag { cdag {
config_profile_description = 'MPI-SHH CDAG profile, provided by nf-core/configs.' params {
config_profile_description = 'CDAG MPI-SHH profile, provided by nf-core/configs.'
}
} }
sdag { sdag {
config_profile_description = 'MPI-SHH SDAG profile, provided by nf-core/configs.' params {
config_profile_description = 'SDAG MPI-SHH profile, provided by nf-core/configs.'
max_memory = 2.TB max_memory = 2.TB
max_cpus = 128 max_cpus = 128
} }
} }
}

View file

@ -26,7 +26,7 @@ params {
def hostname = "hostname".execute().text.trim() def hostname = "hostname".execute().text.trim()
if (hostname ==~ "b.*") { if (hostname ==~ "b.*" || hostname ==~ "s.*") {
params.max_memory = 109.GB params.max_memory = 109.GB
} }

24
conf/utd_ganymede.config Normal file
View file

@ -0,0 +1,24 @@
//Profile config names for nf-core/configs
params {
config_profile_description = 'University of Texas at Dallas HPC cluster profile provided by nf-core/configs'
config_profile_contact = 'Edmund Miller(@emiller88)'
config_profile_url = 'http://docs.oithpc.utdallas.edu/'
}
singularity {
enabled = true
envWhitelist='SINGULARITY_BINDPATH'
autoMounts = true
}
process {
beforeScript = 'module load singularity/2.4.5'
executor = 'slurm'
queue = 'genomics'
}
params {
max_memory = 32.GB
max_cpus = 16
max_time = 48.h
}

9
docs/bi.md Normal file
View file

@ -0,0 +1,9 @@
# nf-core/configs: BI Configuration
All nf-core pipelines have been successfully configured for use at Boehringer Ingelheim.
To use, run the pipeline with `-profile bi`. This will download and launch the [`bi.config`](../conf/bi.config) which has been pre-configured with a setup suitable for the BI systems. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Singularity image before execution of the pipeline.
Before running the pipeline you will need to follow the internal documentation to run Nextflow on our systems. Similar to that, you need to set an environment variable `NXF_GLOBAL_CONFIG` to the path of the internal global config which is not publicly available here.
>NB: Nextflow will need to submit the jobs via the job scheduler to the HPC cluster and as such the commands above will have to be executed on one of the login nodes. If in doubt contact IT.

View file

@ -1,18 +0,0 @@
# nf-core/configs: CCGA Configuration
Deployment and testing of nf-core pipelines at the CCGA cluster is on-going.
To use, run the pipeline with `-profile ccga`. This will download and launch the [`ccga.config`](../conf/ccga.config) which has been pre-configured with a setup suitable for the CCGA cluster. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Singularity image before execution of the pipeline.
Before running the pipeline you will need to load Nextflow and Singularity using the environment module system on the cluster. You can do this by issuing the commands below:
```bash
## Load Nextflow and Singularity environment modules
module purge
module load IKMB
module load Java/1.8.0
module load Nextflow
module load singularity3.1.0
```
>NB: Access to the CCGA cluster is restricted to IKMB/CCGA employes. Please talk to Marc Hoeppner to get access (@marchoeppner).

View file

@ -10,8 +10,10 @@ Before running the pipeline you will need to load Nextflow and Singularity using
## Load Nextflow and Singularity environment modules ## Load Nextflow and Singularity environment modules
module purge module purge
module load devel/java_jdk/1.8.0u121 module load devel/java_jdk/1.8.0u121
module load qbic/singularity_slurm/3.0.3 module load devel/singularity/3.4.2
``` ```
>NB: You will need an account to use the HPC cluster CFC in order to run the pipeline. If in doubt contact IT. >NB: You will need an account to use the HPC cluster CFC in order to run the pipeline. If in doubt contact IT.
>NB: Nextflow will need to submit the jobs via the job scheduler to the HPC cluster and as such the commands above will have to be executed on one of the login nodes. If in doubt contact IT. >NB: Nextflow will need to submit the jobs via the job scheduler to the HPC cluster and as such the commands above will have to be executed on one of the login nodes. If in doubt contact IT.
The queues are set to be `qbic` or `compute` and will be chosen automatically for you depending on your job submission.

11
docs/ebc.md Normal file
View file

@ -0,0 +1,11 @@
# nf-core/configs: EBC Configuration
All nf-core pipelines have been successfully configured for use on the [Estonian Biocentre (EBC)](https://genomics.ut.ee/en/about-us/estonian-biocentre) cluster at the [High Performance Computing Center](https://hpc.ut.ee/en) of the the University of Tartu.
To use, run the pipeline with `-profile ebc`. This will download and launch the [`ebc.config`](../conf/ebc.config) which has been pre-configured with a setup suitable for the EBC cluster. Using this profile, currently, a conda environment containing all of the required software will be downloaded and stored in a central location.
> :warning: You must install your own [conda binary](conda.io) to run nf-core pipelines in a conda environment. Running with singularity will be added soon.
The profile will put a maximum job limit of 12 GB, 20 CPUs and a maximum wall time of 120 hours.
NB: You will need an account to use the HPC cluster on EBC cluster in order to run the pipeline. If in doubt contact IT.
NB: Nextflow will need to submit the jobs via the SLURM scheduler to the HPC cluster and as such the commands above will have to be executed on one of the head nodes. If in doubt contact IT.

37
docs/google.md Normal file
View file

@ -0,0 +1,37 @@
# nf-core/configs: Google Cloud Life Sciences Configuration
To be used with the `google` profile by specifying the `-profile google` when running nf-core pipelines.
![Google Cloud](images/google-cloud-logo.svg "https://cloud.google.com/life-sciences/docs/tutorials/nextflow")
## Quick Start
```bash
GOOGLE_APPLICATION_CREDENTIALS=<your_key>.json
NXF_MODE=google
nextflow run nf-core/rnaseq -profile test,google --google_bucket <gs://your_bucket/work>
```
### Required Parameters
#### `--google-bucket`
The Google Cloud Storage bucket location to be used as a Nextflow work directory. Can also be specified with (`-w gs://your_bucket/work`).
### Optional Parameters
#### `--google-zone`
The Google zone where the computation is executed in Compute Engine VMs. Multiple zones can be provided separating them by a comma. Default (`europe-west2-c`).
#### `--google-preemptible`
Enables the usage of preemptible virtual machines with a retry error statergy for up to 5 retries. Default (`true`).
#### `--google-debug`
Copies the /google debug directory from the VM to the task bucket directory. Useful for debugging. Default (`false`).
## Cloud Life Sciences Setup
Please refer to the [Google Cloud](https://cloud.google.com/life-sciences/docs/tutorials/nextflow) and [Nextflow](https://www.nextflow.io/docs/latest/google.html#cloud-life-sciences) documentation which describe how to setup the Google Cloud environment.

View file

@ -0,0 +1,96 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Generator: Adobe Illustrator 23.0.1, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd" [
<!ENTITY ns_extend "http://ns.adobe.com/Extensibility/1.0/">
<!ENTITY ns_ai "http://ns.adobe.com/AdobeIllustrator/10.0/">
<!ENTITY ns_graphs "http://ns.adobe.com/Graphs/1.0/">
<!ENTITY ns_vars "http://ns.adobe.com/Variables/1.0/">
<!ENTITY ns_imrep "http://ns.adobe.com/ImageReplacement/1.0/">
<!ENTITY ns_sfw "http://ns.adobe.com/SaveForWeb/1.0/">
<!ENTITY ns_custom "http://ns.adobe.com/GenericCustomNamespace/1.0/">
<!ENTITY ns_adobe_xpath "http://ns.adobe.com/XPath/1.0/">
]>
<svg version="1.1" id="Art" xmlns:x="&ns_extend;" xmlns:i="&ns_ai;" xmlns:graph="&ns_graphs;"
xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 8986.8 1407.9"
style="enable-background:new 0 0 8986.8 1407.9;" xml:space="preserve">
<style type="text/css">
.st0{fill:#EA4335;}
.st1{fill:#4285F4;}
.st2{fill:#34A853;}
.st3{fill:#FBBC05;}
.st4{fill:#5F6368;}
</style>
<metadata>
<sfw xmlns="&ns_sfw;">
<slices></slices>
<sliceSourceBounds bottomLeftOrigin="true" height="1407.9" width="8986.8" x="-4403.9" y="-689.9"></sliceSourceBounds>
</sfw>
</metadata>
<title>Cloud_Logo_Nav</title>
<g>
<g>
<path class="st0" d="M1057.1,369.2h50.8l144.8-144.8l7.1-61.5C990.3-75,579-49.3,341.1,220.2C275,295,227.1,384.1,200.9,480.5
c16.1-6.6,34-7.7,50.8-3l289.6-47.8c0,0,14.7-24.4,22.4-22.9c128.8-141.5,345.6-158,494.4-37.6L1057.1,369.2z"/>
<path class="st1" d="M1459,480.5c-33.3-122.6-101.6-232.8-196.6-317l-203.2,203.2c85.8,70.1,134.7,175.8,132.6,286.6v36.1
c99.9,0,180.9,81,180.9,180.9s-81,180.9-180.9,180.9H830l-36.1,36.6v217l36.1,36.1h361.8c259.8,2,472.1-207,474.1-466.8
C1667.1,716.3,1589.4,568.7,1459,480.5z"/>
<path class="st2" d="M467.7,1338.6h361.8V1049H467.7c-25.8,0-51.2-5.6-74.7-16.3l-50.8,15.8l-145.8,144.8l-12.7,50.8
C265.4,1305.9,365.2,1339.1,467.7,1338.6z"/>
<path class="st3" d="M467.7,399.2C207.8,400.7-1.5,612.6,0,872.5c0.9,145.1,68.6,281.7,183.7,370.1l209.8-209.8
c-91-41.1-131.5-148.3-90.4-239.3s148.3-131.5,239.3-90.4c40.1,18.1,72.2,50.3,90.4,90.4l209.8-209.8
C753.4,466.9,614.6,398.6,467.7,399.2z"/>
</g>
<g>
<g>
<path class="st4" d="M2629.1,1136.4c-125.9,1.6-246.9-48.1-335.3-137.7c-91.3-85.8-142.3-206-140.7-331.3
c-1.5-125.3,49.5-245.4,140.7-331.3c88-90.5,209.1-141.2,335.3-140.2c120.2-1.6,236.1,45.1,321.6,129.6l-90.4,92
c-61.9-59.9-145.1-92.7-231.2-91.5c-90.3-1.1-177,35.8-238.8,101.6c-64.2,63.7-99.6,150.9-98.1,241.3
c-0.8,89.4,34.6,175.3,98.1,238.3c130.8,130.4,341.7,132.9,475.6,5.6c39.9-43.6,64.3-99.2,69.6-158h-304.9V624.8H3062
c4.2,26.2,5.9,52.7,5.1,79.3c5.4,110.9-34.8,219.2-111.3,299.8C2870.7,1092.4,2751.9,1140.6,2629.1,1136.4z M3632.1,1050
c-120.3,115.3-310.1,115.3-430.4,0c-58.3-56.5-90.1-134.8-87.9-215.9c-2.2-81.1,29.7-159.4,87.9-215.9c120.4-115,310-115,430.4,0
c58.2,56.5,90.1,134.8,87.9,215.9c2.1,81.2-30,159.5-88.4,215.9H3632.1z M3296.7,965.2c62.1,66.4,166.3,69.8,232.7,7.6
c2.6-2.5,5.2-5,7.6-7.6c33.9-35.1,52.2-82.3,50.8-131.1c1.5-48.9-16.8-96.4-50.8-131.6c-63.9-66.6-169.7-68.9-236.4-5
c-1.7,1.6-3.4,3.3-5,5c-34,35.2-52.3,82.7-50.8,131.6c-1.3,48.8,17.2,96.1,51.3,131.1H3296.7z M4293.6,1050
c-120.3,115.3-310.1,115.3-430.4,0c-58.3-56.5-90.1-134.8-87.9-215.9c-2.2-81.1,29.7-159.4,87.9-215.9
c120.3-115.3,310.1-115.3,430.4,0c58.2,56.5,90.1,134.8,87.9,215.9C4383.8,915.2,4351.9,993.5,4293.6,1050z M3958.3,965.2
c62.1,66.4,166.3,69.8,232.7,7.6c2.6-2.5,5.2-5,7.6-7.6c33.9-35.1,52.2-82.3,50.8-131.1c1.5-48.9-16.8-96.4-50.8-131.6
c-63.9-66.6-169.7-68.9-236.3-5c-1.7,1.6-3.4,3.3-5,5c-34,35.2-52.3,82.7-50.8,131.6c-1.3,48.8,17.2,96.1,51.3,131.1H3958.3z
M4728.5,1407.7c-60.9,1.9-120.6-17.1-169.2-53.9c-43.1-32.4-77.1-75.5-98.6-125l115.8-48.3c12.6,29.6,32.3,55.6,57.4,75.7
c26.8,21.6,60.5,33,95,32c44.2,2.7,87.4-14,118.4-45.7c30.6-36.9,45.7-84.3,42.2-132.1v-43.7h-4.6c-39.3,46.6-98.2,72-159,68.6
c-77.1,0.2-150.8-31.8-203.2-88.4c-56.8-56.1-88.2-133-86.9-212.9c-1.4-80.3,30-157.7,86.9-214.4c52.3-56.8,126-89.1,203.2-88.9
c32.3-0.1,64.3,7.1,93.5,20.8c25.3,11,47.8,27.5,66.1,48.3h4.6v-48.3h126v541.1c6.7,86.5-22.5,172-80.8,236.3
C4879.8,1382,4805.3,1410.4,4728.5,1407.7z M4737.7,1017.5c43.9,1,86-17.7,114.8-50.8c31.3-35.9,47.9-82.4,46.2-130.1
c1.9-48.5-14.6-95.8-46.2-132.6c-28.8-33.2-70.9-51.8-114.8-50.8c-45.1-0.9-88.4,17.6-118.9,50.8
c-33.6,35.5-51.9,82.8-50.8,131.6c-1,48.4,17.2,95.2,50.8,130.1C4649.1,999.2,4692.4,1018.1,4737.7,1017.5z M5240.2,230v888.1
h-132.6V230L5240.2,230z M5604,1136.4c-80.3,2.1-157.8-29.4-213.9-86.9c-56.9-57-88-134.9-85.9-215.4
c-3-80.7,26.9-159.2,82.8-217.5c52.8-55.6,126.6-86.4,203.2-84.9c34.9-0.4,69.4,6.4,101.6,19.8c29,11.7,55.4,29,77.7,50.8
c19.2,18.5,36.2,39.1,50.8,61.5c11.8,18.9,21.8,38.8,30,59.4l13.7,34.6L5457.6,925c25.2,57.8,83.3,94.2,146.3,91.5
c60.9,0.2,117.4-31.7,148.9-83.8l101.6,68.6c-25.9,37.1-59.1,68.6-97.6,92.5C5711.2,1122.6,5658,1137.4,5604,1136.4z
M5434.3,824.9l270.3-112.3c-8.3-19.8-23.2-36.1-42.2-46.2c-21.2-12-45.2-18.1-69.6-17.8c-41.3,1.1-80.6,18.4-109.2,48.3
C5448.9,730.1,5430.8,777,5434.3,824.9z"/>
</g>
<g>
<path class="st4" d="M6558.7,1136.4c-231.2,4-421.8-180.2-425.8-411.4c-0.1-5.6-0.1-11.3,0.1-16.9
c-5.4-231.2,177.7-422.9,408.8-428.3c5.6-0.1,11.3-0.1,16.9-0.1c115.6-3.6,226.6,45.8,301.3,134.1l-73.2,71.1
c-54.9-69.1-139.5-107.8-227.6-104.2c-84.5-1.9-166.2,30.8-226.1,90.4c-62.3,62.7-95.6,148.5-92,236.8
c-3.7,88.3,29.7,174.1,92,236.8c59.9,59.7,141.6,92.3,226.1,90.4c98.4,1.1,191.9-42.7,254-118.9l73.2,73.2
c-38.3,45.8-86.4,82.4-140.7,107.2C6687.1,1123.8,6623.2,1137.4,6558.7,1136.4z"/>
<path class="st4" d="M7083,1118.1h-105.2V297.5H7083V1118.1z"/>
<path class="st4" d="M7254.8,624.2c114.7-113.9,299.9-113.9,414.6,0c54.4,57.5,83.7,134.3,81.3,213.4
c2.4,79.1-26.9,155.9-81.3,213.4c-114.7,113.9-299.9,113.9-414.6,0c-54.4-57.5-83.7-134.3-81.3-213.4
C7171.1,758.5,7200.3,681.7,7254.8,624.2z M7333,985c67.3,71.1,179.5,74.2,250.7,6.9c2.4-2.2,4.7-4.6,6.9-6.9
c37.1-39.8,56.7-92.9,54.4-147.3c2.4-54.4-17.2-107.5-54.4-147.3c-67.3-71.1-179.5-74.2-250.7-6.9c-2.4,2.2-4.7,4.6-6.9,6.9
c-37.1,39.8-56.7,92.9-54.4,147.3c-2.1,54.3,17.7,107.2,54.9,146.8L7333,985z"/>
<path class="st4" d="M8333.4,1118.1h-101.6v-77.7h-3c-18.1,29.5-43.7,53.7-74.2,70.1c-31.6,18.3-67.6,28-104.2,27.9
c-60.2,4.1-118.9-19.5-159.5-64c-38.5-48.7-57.8-109.8-54.4-171.7V557.2h105.2v326.2c0,104.7,46.2,157,138.7,157
c41.7,1.3,81.4-17.6,106.7-50.8c27.3-34.5,41.8-77.4,41.2-121.4v-311h105.2V1118.1z"/>
<path class="st4" d="M8690.6,1136.4c-72.2,0.2-140.9-31.1-188-85.9c-52.7-58.2-80.9-134.4-78.8-212.9
c-2.1-78.5,26.1-154.7,78.8-212.9c47.1-54.7,115.8-86.1,188-85.9c40-0.6,79.5,8.8,114.8,27.4c31.1,15.6,57.4,39.3,76.2,68.6h4.6
l-4.6-77.7V297.5h105.2v820.6h-101.6v-77.7h-4.6c-18.8,29.3-45.1,53-76.2,68.6C8769.4,1127.4,8730.3,1136.9,8690.6,1136.4z
M8707.9,1040.4c48.1,0.7,94.2-19.3,126.5-54.9c35.7-40.7,54-93.8,50.8-147.9c3.2-54-15.1-107.2-50.8-147.9
c-63.8-70.2-172.4-75.3-242.6-11.4c-4,3.6-7.8,7.5-11.4,11.4c-35.8,40.4-54.1,93.4-50.8,147.3c-3.3,53.9,15,106.9,50.8,147.3
C8612.8,1020.7,8659.3,1041.1,8707.9,1040.4z"/>
</g>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 7.2 KiB

View file

@ -6,7 +6,7 @@ Extra specific configuration for the ampliseq pipeline.
To use, run the pipeline with `-profile binac`. To use, run the pipeline with `-profile binac`.
This will download and launch the ampliseq specific [`binac.config`](../conf/pipeline/ampliseq/binac.config) which has been pre-configured with a setup suitable for the BINAC cluster. This will download and launch the ampliseq specific [`binac.config`](../../../conf/pipeline/ampliseq/binac.config) which has been pre-configured with a setup suitable for the BINAC cluster.
Example: `nextflow run nf-core/ampliseq -profile binac` Example: `nextflow run nf-core/ampliseq -profile binac`

View file

@ -0,0 +1,17 @@
# nf-core/configs: uppmax ampliseq specific configuration
Extra specific configuration for the ampliseq pipeline.
## Usage
To use, run the pipeline with `-profile uppmax`.
This will download and launch the ampliseq specific [`uppmax.config`](../../../conf/pipeline/ampliseq/uppmax.config) which has been pre-configured with a setup suitable for the UPPMAX cluster.
Example: `nextflow run nf-core/ampliseq -profile uppmax`
## ampliseq specific configurations for uppmax
Specific configurations for UPPMAX has been made for ampliseq.
* Makes sure that a fat node is allocated for training and applying a Bayesian classifier.

View file

@ -6,7 +6,7 @@ Extra specific configuration for eager pipeline
To use, run the pipeline with `-profile shh`. To use, run the pipeline with `-profile shh`.
This will download and launch the eager specific [`shh.config`](../conf/pipeline/eager/shh.config) which has been pre-configured with a setup suitable for the shh cluster. This will download and launch the eager specific [`shh.config`](../../../conf/pipeline/eager/shh.config) which has been pre-configured with a setup suitable for the shh cluster.
Example: `nextflow run nf-core/eager -profile shh` Example: `nextflow run nf-core/eager -profile shh`
@ -14,4 +14,5 @@ Example: `nextflow run nf-core/eager -profile shh`
Specific configurations for shh has been made for eager. Specific configurations for shh has been made for eager.
* If running with the MALT module turned on, the MALT process by default will be sent to the long queue with a resource requirement minimum of 725GB and 64 cores. If this fails, the process will be tried once more only and sent to the supercruncher queue. The module will not retry after this, and pipeline will fail. * If running with the MALT module turned on, the MALT process by default will be sent to the long queue with a resource requirement minimum of 725GB and 64 cores. If this fails, the process will be tried once more only and sent to the supercruncher queue. The module will not retry after this, and pipeline will fail. Note, this will only work on SDAG.
* Provides additional group specific profiles, which adapt the `bwa aln` mapping parameters to each context: `pathogens_loose` (`-l 0.01 -n 16`), `pathogens_strict` (`-l 32, -n 0.1`) and `human` (`-l 16500, -n 0.01`).

View file

@ -0,0 +1,18 @@
# nf-core/configs: MUNIN rnafusion specific configuration
Extra specific configuration for rnafusion pipeline
## Usage
To use, run the pipeline with `-profile munin`.
This will download and launch the rnafusion specific [`munin.config`](../../../conf/pipeline/rnafusion/munin.config) which has been pre-configured with a setup suitable for the `MUNIN` cluster.
Example: `nextflow run nf-core/rnafusion -profile munin`
## rnafusion specific configurations for MUNIN
Specific configurations for `MUNIN` has been made for rnafusion.
* `cpus`, `memory` and `time` max requirements.
* Paths to specific references and indexes

View file

@ -6,17 +6,22 @@ Extra specific configuration for sarek pipeline
To use, run the pipeline with `-profile munin`. To use, run the pipeline with `-profile munin`.
This will download and launch the sarek specific [`munin.config`](../conf/pipeline/sarek/munin.config) which has been pre-configured with a setup suitable for the MUNIN cluster. This will download and launch the sarek specific [`munin.config`](../../../conf/pipeline/sarek/munin.config) which has been pre-configured with a setup suitable for the `MUNIN` cluster.
Example: `nextflow run nf-core/sarek -profile munin` Example: `nextflow run nf-core/sarek -profile munin`
## Sarek specific configurations for MUNIN ## Sarek specific configurations for MUNIN
Specific configurations for MUNIN has been made for sarek. Specific configurations for `MUNIN` has been made for sarek.
* Params `annotation_cache` set to `true` * Params `annotation_cache` and `cadd_cache` set to `true`
* Path to `snpEff_cache`: `/data1/cache/snpEff/` * Params `vep_cache_version` set to `95`
* Path to `snpeff_cache`: `/data1/cache/snpEff/`
* Path to `vep_cache`: `/data1/cache/VEP/` * Path to `vep_cache`: `/data1/cache/VEP/`
* Path to `pon`: `/data1/PON/vcfs/BTB.PON.vcf.gz` * Path to `pon`: `/data1/PON/vcfs/BTB.PON.vcf.gz`
* Path to `pon_index`: `/data1/PON/vcfs/BTB.PON.vcf.gz.tbi` * Path to `pon_index`: `/data1/PON/vcfs/BTB.PON.vcf.gz.tbi`
* Path to `cadd_indels`: `/data1/cache/CADD/v1.4/InDels.tsv.gz`
* Path to `cadd_indels_tbi`: `/data1/cache/CADD/v1.4/InDels.tsv.gz.tbi`
* Path to `cadd_wg_snvs`: `/data1/cache/CADD/v1.4/whole_genome_SNVs.tsv.gz`
* Path to `cadd_wg_snvs_tbi`: `/data1/cache/CADD/v1.4/whole_genome_SNVs.tsv.gz.tbi`
* Load module `Sentieon` for Processes with `sentieon` labels * Load module `Sentieon` for Processes with `sentieon` labels

View file

@ -6,7 +6,7 @@ Extra specific configuration for sarek pipeline
To use, run the pipeline with `-profile uppmax`. To use, run the pipeline with `-profile uppmax`.
This will download and launch the sarek specific [`uppmax.config`](../conf/pipeline/sarek/uppmax.config) which has been pre-configured with a setup suitable for uppmax clusters. This will download and launch the sarek specific [`uppmax.config`](../../../conf/pipeline/sarek/uppmax.config) which has been pre-configured with a setup suitable for uppmax clusters.
Example: `nextflow run nf-core/sarek -profile uppmax` Example: `nextflow run nf-core/sarek -profile uppmax`

View file

@ -0,0 +1,9 @@
# nf-core/configs: viralrecon specific configuration
Extra specific configuration for viralrecon pipeline
## Usage
Will be used automatically when running the pipeline with the shared configs in the nf-core/configs repository
This will download and launch the viralrecon specific [`viralrecon.config`](../../../conf/pipeline/viralrecon/genomes.config) which has been pre-configured with custom genomes.

View file

@ -2,7 +2,7 @@
All nf-core pipelines have been successfully configured for use on the Department of Archaeogenetic's SDAG/CDAG clusters at the [Max Planck Institute for the Science of Human History (MPI-SHH)](http://shh.mpg.de). All nf-core pipelines have been successfully configured for use on the Department of Archaeogenetic's SDAG/CDAG clusters at the [Max Planck Institute for the Science of Human History (MPI-SHH)](http://shh.mpg.de).
To use, run the pipeline with `-profile ssh`. You can further with optimise submissions by specifying which cluster you are using with `-profile shh,sdag` or `-profile ssh,cdag`. This will download and launch the [`shh.config`](../conf/shh.config) which has been pre-configured with a setup suitable for the SDAG and CDAG clusters respectively. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Singularity image before execution of the pipeline. The image will currently be centrally stored here: To use, run the pipeline with `-profile shh`. You can further with optimise submissions by specifying which cluster you are using with `-profile shh,sdag` or `-profile shh,cdag`. This will download and launch the [`shh.config`](../conf/shh.config) which has been pre-configured with a setup suitable for the SDAG and CDAG clusters respectively. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Singularity image before execution of the pipeline. The image will currently be centrally stored here:
```bash ```bash
/projects1/singularity_scratch/cache/ /projects1/singularity_scratch/cache/
@ -10,7 +10,7 @@ To use, run the pipeline with `-profile ssh`. You can further with optimise subm
however this will likely change to a read-only directory in the future that will be managed by the IT team. however this will likely change to a read-only directory in the future that will be managed by the IT team.
This configuration will automatically choose the correct SLURM queue (`short`,`medium`,`long`) depending on the time and memory required by each process. `-profile ssh,sdag` additionally allows for submission of jobs to the `supercruncher` queue when a job's requested memory exceeds 756GB. This configuration will automatically choose the correct SLURM queue (`short`,`medium`,`long`) depending on the time and memory required by each process. `-profile shh,sdag` additionally allows for submission of jobs to the `supercruncher` queue when a job's requested memory exceeds 756GB.
>NB: You will need an account and VPN access to use the cluster at MPI-SHH in order to run the pipeline. If in doubt contact the IT team. >NB: You will need an account and VPN access to use the cluster at MPI-SHH in order to run the pipeline. If in doubt contact the IT team.
>NB: Nextflow will need to submit the jobs via SLURM to the clusters and as such the commands above will have to be executed on one of the head nodes. If in doubt contact IT. >NB: Nextflow will need to submit the jobs via SLURM to the clusters and as such the commands above will have to be executed on one of the head nodes. If in doubt contact IT.

18
docs/utd_ganymede.md Normal file
View file

@ -0,0 +1,18 @@
# nf-core/configs: UTD Ganymede Configuration
All nf-core pipelines have been successfully configured for use on the Ganymede HPC cluster at the [The Univeristy of Texas at Dallas](https://www.utdallas.edu/).
To use, run the pipeline with `-profile utd_ganymede`. This will download and launch the [`utd_ganymede.config`](../conf/utd_ganymede.config) which has been pre-configured with a setup suitable for the Ganymede HPC cluster. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Singularity image before execution of the pipeline.
Before running the pipeline you will need to load Singularity using the environment module system on Ganymede. You can do this by issuing the commands below:
```bash
## Singularity environment modules
module purge
module load singularity
```
All of the intermediate files required to run the pipeline will be stored in the `work/` directory. It is recommended to delete this directory after the pipeline has finished successfully because it can get quite large, and all of the main output files will be saved in the `results/` directory anyway.
>NB: You will need an account to use the HPC cluster on Ganymede in order to run the pipeline. If in doubt contact Ganymedeadmins.
>NB: Nextflow will need to submit the jobs via SLURM to the HPC cluster and as such the commands above will have to be executed on one of the login nodes. If in doubt contact GanymedeAdmins.

View file

@ -11,20 +11,23 @@
//Please use a new line per include Config section to allow easier linting/parsing. Thank you. //Please use a new line per include Config section to allow easier linting/parsing. Thank you.
profiles { profiles {
awsbatch { includeConfig "${params.custom_config_base}/conf/awsbatch.config" } awsbatch { includeConfig "${params.custom_config_base}/conf/awsbatch.config" }
bi { includeConfig "${params.custom_config_base}/conf/bi.config" }
bigpurple { includeConfig "${params.custom_config_base}/conf/bigpurple.config" } bigpurple { includeConfig "${params.custom_config_base}/conf/bigpurple.config" }
binac { includeConfig "${params.custom_config_base}/conf/binac.config" } binac { includeConfig "${params.custom_config_base}/conf/binac.config" }
cbe { includeConfig "${params.custom_config_base}/conf/cbe.config" } cbe { includeConfig "${params.custom_config_base}/conf/cbe.config" }
ccga { includeConfig "${params.custom_config_base}/conf/ccga.config" }
ccga_dx { includeConfig "${params.custom_config_base}/conf/ccga_dx.config" } ccga_dx { includeConfig "${params.custom_config_base}/conf/ccga_dx.config" }
ccga_med { includeConfig "${params.custom_config_base}/conf/ccga_med.config" } ccga_med { includeConfig "${params.custom_config_base}/conf/ccga_med.config" }
cfc { includeConfig "${params.custom_config_base}/conf/cfc.config" } cfc { includeConfig "${params.custom_config_base}/conf/cfc.config" }
cfc_dev { includeConfig "${params.custom_config_base}/conf/cfc_dev.config" }
crick { includeConfig "${params.custom_config_base}/conf/crick.config" } crick { includeConfig "${params.custom_config_base}/conf/crick.config" }
czbiohub_aws { includeConfig "${params.custom_config_base}/conf/czbiohub_aws.config" } czbiohub_aws { includeConfig "${params.custom_config_base}/conf/czbiohub_aws.config" }
czbiohub_aws_highpriority { czbiohub_aws_highpriority {
includeConfig "${params.custom_config_base}/conf/czbiohub_aws.config"; includeConfig "${params.custom_config_base}/conf/czbiohub_aws.config";
includeConfig "${params.custom_config_base}/conf/czbiohub_aws_highpriority.config"} includeConfig "${params.custom_config_base}/conf/czbiohub_aws_highpriority.config"}
ebc { includeConfig "${params.custom_config_base}/conf/ebc.config" }
icr_davros { includeConfig "${params.custom_config_base}/conf/icr_davros.config" } icr_davros { includeConfig "${params.custom_config_base}/conf/icr_davros.config" }
genotoul { includeConfig "${params.custom_config_base}/conf/genotoul.config" } genotoul { includeConfig "${params.custom_config_base}/conf/genotoul.config" }
google { includeConfig "${params.custom_config_base}/conf/google.config" }
denbi_qbic { includeConfig "${params.custom_config_base}/conf/denbi_qbic.config" } denbi_qbic { includeConfig "${params.custom_config_base}/conf/denbi_qbic.config" }
genouest { includeConfig "${params.custom_config_base}/conf/genouest.config" } genouest { includeConfig "${params.custom_config_base}/conf/genouest.config" }
gis { includeConfig "${params.custom_config_base}/conf/gis.config" } gis { includeConfig "${params.custom_config_base}/conf/gis.config" }
@ -37,6 +40,7 @@ profiles {
shh { includeConfig "${params.custom_config_base}/conf/shh.config" } shh { includeConfig "${params.custom_config_base}/conf/shh.config" }
uct_hex { includeConfig "${params.custom_config_base}/conf/uct_hex.config" } uct_hex { includeConfig "${params.custom_config_base}/conf/uct_hex.config" }
uppmax { includeConfig "${params.custom_config_base}/conf/uppmax.config" } uppmax { includeConfig "${params.custom_config_base}/conf/uppmax.config" }
utd_ganymede { includeConfig "${params.custom_config_base}/conf/utd_ganymede.config" }
uzh { includeConfig "${params.custom_config_base}/conf/uzh.config" } uzh { includeConfig "${params.custom_config_base}/conf/uzh.config" }
} }
@ -46,11 +50,14 @@ profiles {
params { params {
// This is a groovy map, not a nextflow parameter set // This is a groovy map, not a nextflow parameter set
hostnames = [ hostnames = [
binac: ['.binac.uni-tuebingen.de'],
cbe: ['.cbe.vbc.ac.at'], cbe: ['.cbe.vbc.ac.at'],
cfc: ['.hpc.uni-tuebingen.de'],
crick: ['.thecrick.org'], crick: ['.thecrick.org'],
icr_davros: ['.davros.compute.estate'], icr_davros: ['.davros.compute.estate'],
genotoul: ['.genologin1.toulouse.inra.fr', '.genologin2.toulouse.inra.fr'], genotoul: ['.genologin1.toulouse.inra.fr', '.genologin2.toulouse.inra.fr'],
genouest: ['.genouest.org'], genouest: ['.genouest.org'],
uppmax: ['.uppmax.uu.se'] uppmax: ['.uppmax.uu.se'],
utd_ganymede: ['ganymede.utdallas.edu']
] ]
} }

View file

@ -10,4 +10,5 @@
profiles { profiles {
binac { includeConfig "${params.custom_config_base}/conf/pipeline/ampliseq/binac.config" } binac { includeConfig "${params.custom_config_base}/conf/pipeline/ampliseq/binac.config" }
uppmax { includeConfig "${params.custom_config_base}/conf/pipeline/ampliseq/uppmax.config" }
} }

13
pipeline/rnafusion.config Normal file
View file

@ -0,0 +1,13 @@
/*
* -------------------------------------------------
* nfcore/rnafusion custom profile Nextflow config file
* -------------------------------------------------
* Config options for custom environments.
* Cluster-specific config options should be saved
* in the conf/pipeline/rnafusion folder and imported
* under a profile name here.
*/
profiles {
munin { includeConfig "${params.custom_config_base}/conf/pipeline/rnafusion/munin.config" }
}

View file

@ -0,0 +1,7 @@
/*
* -------------------------------------------------
* nfcore/viralrecon custom profile Nextflow config file
* -------------------------------------------------
*/
includeConfig "${params.custom_config_base}/conf/pipeline/viralrecon/genomes.config"