mirror of
https://github.com/MillironX/nf-configs.git
synced 2024-11-10 20:13:09 +00:00
cbe: send SIGUSR2 upon job termination
Previously, if a process hit the walltime limit and received SIGKILL from the slurm scheduler, singularity did not properly propagate such (soft) kill signal. This prevented the exit code to be caught, e.g for resubmission purposes. This commit introduces a workaround using slurms --signal directive to send SIGUSR2 to the singularity process itself (instead of container child processes, which presumably was happening before). Effectively, once a job reaches walltime limit, this will result in exitcode 140 which is typically caught by the errorStrategy in nf-core pipelines See also: https://slurm.schedmd.com/sbatch.html#OPT_signal https://github.com/nextflow-io/nextflow/issues/2163 https://github.com/nextflow-io/nextflow/issues/1561
This commit is contained in:
parent
9cca0eaf85
commit
f5c36dde3e
1 changed files with 1 additions and 1 deletions
|
@ -8,7 +8,7 @@ params {
|
|||
process {
|
||||
executor = 'slurm'
|
||||
queue = { task.memory <= 170.GB ? 'c' : 'm' }
|
||||
clusterOptions = { task.time <= 1.h ? '--qos rapid' : task.time <= 8.h ? '--qos short': task.time <= 48.h ? '--qos medium' : '--qos long' }
|
||||
clusterOptions = { def qos = task.time <= 1.h ? '--qos rapid' : { task.time <= 8.h ? '--qos short': { task.time <= 48.h ? '--qos medium' : '--qos long' } }; qos << ' --signal B:USR2' }
|
||||
module = 'anaconda3/2019.10'
|
||||
}
|
||||
|
||||
|
|
Loading…
Reference in a new issue