millironx/nf-configs
1
0
Fork 0
mirror of https://github.com/MillironX/nf-configs.git synced 2024-11-21 16:16:04 +00:00
Code Issues Releases Packages Activity

Improve reliability of file transfers

Browse source
This commit is contained in:
Bruno Grande 2022-09-17 10:14:04 -07:00
parent c9c4d02f0d
commit 8bb9361359
2 changed files with 4 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
conf/sage.config
View file

@ -29,9 +29,12 @@ aws {
region = "us-east-1"
client {
uploadChunkSize = 209715200
uploadMaxThreads = 4
}
batch {
maxParallelTransfers = 1
maxTransferAttempts = 5
delayBetweenAttempts = '120 sec'
}
}
executor {

1
docs/sage.md
View file

@ -8,6 +8,7 @@ This global configuration includes the following tweaks:
- Enable retries by default when exit codes relate to insufficient memory
- Allow pending jobs to finish if the number of retries are exhausted
- Increase the amount of time allowed for file transfers
- Improve reliability of file transfers with retries and reduced concurrency
- Increase the default chunk size for multipart uploads to S3
- Slow down job submission rate to avoid overwhelming any APIs
- Define the `check_max()` function, which is missing in Sarek v2