* [POC] Get subworkflows working again - bam_stats_samtools
* Comment out aliased anchors
* Add explicit anchor to pytest_modules.yml
* test(subworkflows): Remove anchors
There are two options to get the tests to trigger:
1. Add the module anchors to the subworkflows (less things to update and maintain because we only have to update the module triggers)
2. Add the tags to the pytest workflow spec.
This is option 2.
Co-authored-by: Edmund Miller <edmund.a.miller@protonmail.com>
* added code for subworkflow fgbio call umi consensus
* ironing out a few typos etc
* fixing last things
* fixed md5sum - lets see if it changes
* removing file accidentally deleted
* tidy indents
* added bwamem2 alternative
* fixed entry for both tests
* changed name second test workflow entry
* fixed workflow entry names
* fixed md5sum for file generated with bwamem2
* added syntax new DSL2
* added new config location in test command line
* added new config location in test command line
* use of prefix instead of suffix because modules have been changed in this way
* explicit alias to bwa mem1 to avoid confusion
* removed param that should be an ext optional argument in fgbio groupreadsbyumi
* missing colon in config
* missing colon in module config too
* order list alphabetically
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
* remove params from body
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
* improving readability of input structure
Co-authored-by: Mahesh Binzer-Panchal <mahesh.binzer-panchal@nbis.se>
* reverting to mandatory input
* fixed tests and workflow take values
* remove param
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
* simplify tests params
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
* formatting inputs for readability
* factoring in changes to bwamem2_mem and bwa_mem sort/view inputs
* updating test md5sum for grouped file following code update in bwamem
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
Co-authored-by: Maxime U. Garcia <max.u.garcia@gmail.com>
Co-authored-by: Mahesh Binzer-Panchal <mahesh.binzer-panchal@nbis.se>
* greatly simplify syntax
* feat: add subworkflows to annotate (+ bgzip/tabix index) with ensemblvep and snpeff
* feat: get versions from all tools
* add commented infor for new annotation modules
* Add comment line for consistency
* Remove all functions.nf
* Remove include functions.nf and publishDir options
* Replace options.args3 with task.ext.args3 - 3 modules
* Replace options.args3 with task.ext.args3 - 17 modules
* Replace {task.cpus} with task.cpus
* Replace off on off off off off off off off on off on off on off off off on off off off on on off off off on on off off off off off off off on off off off off on off on on off off off on on on on off off off on off on on off on on off off on on on off on on off on off off off off on off off off on off off on off on off off off on on off on off on off off on off off off on off off off on off off off off on off off off on on on off on on off off on off on on on off on on off on on on off off off off off on on off off on off off off off off on off off on on off on on off on off off off on off off off off on on off on off off on off off on off on off off off off off off off off on on off on off off off.args with
* Add def args = task.ext.args line to all modules in script section
* Replace options.args with args and args_list
* Initialise args2 and args3 properly
* Replace container syntax
* Revert container changes for cellranger/mkref
* Replace getProcessName in all modules
* Replace getSoftwareName in all modules
* Unify modules using VERSION variable
* Replae options.suffix with task.ext.suffix
* Remove NF version restriction for CI
* Bump NF version in README
* Replace task.process.tokenize logic with task.process
* Minor tweaks to unify syntax in tests main.nf
* Add a separate nextflow.config for each module
* Transfer remaining module options to nextflow.config
* Remove addParams from tests main.nf
* Remove TODO statements
* Use -c to import module specific config
* Bump NF version to 21.10.3
* Fix tests for artic/minion
* Fix broken publishDir syntax
* Standardise and fix obvious failing module tests
* Remove kronatools to krona
* Comment out tags in subworkflow test.yml
* Fix failing module tests
* Add consistent indentation to nextflow.config
* Comment out subworklow definitions
* Fix kallistobustools/ref
* Fix rmarkdownnotebook
* Fix jupyternotebook
* Quote task.process
* Add plink2/vcf to pytest_modules.yml
* Remove NF_CORE_MODULES_TEST from pytest CI
* Fix more tests
* Move bacteroides_fragilis to prokaryotes folder
* Fix cooler merge tests
* Fix kallistobustools/count tests
* Fix kallistobustools/ref tests
* Update test_10x_1_fastq_gz file for kallistobustools/count tests
* Fix bcftools/query tests
* Fix delly/call tests
* Fix cooler/zoomify tests
* Fix csvtk/split tests
* Fix gatk4/intervallisttools tests
* Fix gatk4/variantfiltration
* Fix pydamage/filter tests
* Fix test data for unicycler
* Fix gstama/collapse module
* Fix leehom tests
* Fix metaphlan3 tests
* Fix pairtools/select tests
* Update nextflow.config
* Update nextflow.config
* feat: update syntax
* Fix arriba tests
* Fix more failing tests
* Update test syntax
* Remove comments from tests nextflow.config
* Apply suggestions from code review
* Fix kallistobustools/count module
* Update dumpsoftwareversions module
* Update custom/dumpsoftwareversions
* Add args2 to untar module
* Update leftover modules
* Remove last remaining addParams
Co-authored-by: JoseEspinosa <kadomu@gmail.com>
Co-authored-by: Gregor Sturm <mail@gregor-sturm.de>
Co-authored-by: MaxUlysse <max.u.garcia@gmail.com>
* initial commit to setup branch
* workflow finished
* Update nextflow.config
* tumour to tumor, getpileup passed as nomral and tumor
* paired_somatic renamed to tumor_normal_somatic
* Apply suggestions from code review
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
* Update subworkflows/nf-core/gatk_tumor_normal_somatic_variant_calling/main.nf
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
* updated index names in meta.yml
* changed index file names in main script and test
* Apply suggestions from code review
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
* Apply suggestions from code review
* fixed bug from changes
* Apply suggestions from code review
* tests should now work after the yml update
* Update pytest_modules.yml
Co-authored-by: GCJMackenzie <gavin.mackenzie@nibsc.org>
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
Co-authored-by: Maxime U. Garcia <max.u.garcia@gmail.com>
* initial commit to set up new branch
* save changes to checkout
* workflow working, still needs test.yml and meta.yml, also fix versions file
* subworkflow finished
* Update pytest_subworkflows.yml
* Update pytest_subworkflows.yml
* Update pytest_subworkflows.yml
* fix config subworkflow name
* Update main.nf
* Update pytest_subworkflows.yml
* fixed md5sum issue likely caused by gatk version update
* tumour changed to tumor
* old dir deleted
* Comments added to explain use of placeholders '[]'
* updated index names, input channel renamed to input
* Apply suggestions from code review
* updated to perform new subworkflow testing
Co-authored-by: GCJMackenzie <gavin.mackenzie@nibsc.org>
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
* ci: Remove pytest_subworkflows
* ci(bam_sort_samtools): Depend on paths-filter instead of pytest-workflow
Co-authored-by: Harshil Patel <drpatelhh@gmail.com>
Co-authored-by: Gregor Sturm <mail@gregor-sturm.de>
Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>
* ci: Revert back to one job branch
* ci(align_bowtie2): Run tests that depend on bam_sort_samtools
* ci: Fix anchor not being created yet
* ci: Update sra_fastq tags and pytest_modules
* fix(bam_sort_samtools): Update nextflow.config with params
* test(subworkflows): Update gatk_create_som_pon tags
* ci: Point to subworkflow_hacks branch of nf-core tools
Co-authored-by: Harshil Patel <drpatelhh@gmail.com>
Co-authored-by: Gregor Sturm <mail@gregor-sturm.de>
Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>
* feat: update gatk4 from 4.2.0.0 to 4.2.3.0
* update md5checksum
* commit all files
* actually checksum was good, but I suspect something fishy with the tests
* commiting changes to switch branch
* commit to setup remote branch
* first draft of the sompon workflow
* keep branch in line with gendb bugfixing
* Update test.yml
* tidy up main.nf
* fixed md5sum
Co-authored-by: GCJMackenzie <gavin.mackenzie@nibsc.org>
* feat(subworkflows): Add align_bowtie2 subworkflow
For testing CI setup
* test(align_bowtie2): Add initial list of changes to test
* test(align_bowtie2): Add initial test
* refactor: Use tags to run subworkflows ci
For every underlying module used by workflow and allow the modules
pytest-modules definition be the source of truth.
* refactor: Use individual directories for subworkflows
* docs(align_bowtie2): Add initial meta.yml
Copied most of it from the bowtie2/align module.
* fix(align_bowtie2): Fix module include paths
* test(bam_sort_samtools): Add initial test
* ci(bam_sort_samtools): Add modules that trigger the tag
* test(bam_stats_samtools): Add initial test
* ci(bam_stats_samtools): Add keys to pick up changes
* docs(bam_samtools): Add initial meta.yml
* test(align_bowtie2): Fix path to subworkflow
* test(align_bowtie2): Update entry point
* fix(bam_sort_samtools): Update include paths
* test(bam_sort_samtools): Fix path
* style: Clean up addParams
* test(samtools_sort): Add suffix for test
* test(align_bowtie2): Add samtools_options for suffix
* test(bam_stats_samtools): Update path
* test(bam_stats_samtools): Use stats input
Otherwise it's just an example of how it's used in the bam_sort_samtools subworkflow
* ci(linting): Skip module linting of subworkflows
* ci(linting): Clean up startsWith statement
* test(bam_stats_samtools): Use single end test data for single end test
* test(bam_stats_samtools): Add expected files
* test(align_bowtie2): Add paired-end test
* test(align_bowtie2): Sort order of output
* test(align_bowtie2): Update hashes
* docs(align_bowtie2): Fix typo
* test(align_bowtie2): Update samtools output names
* test(align_bowtie2): Remove md5sums for bam/bai
* feat(subworkflows): Add nextflow.configs
These can be used for default settings in the future. They can then be
included in the conf/modules.config so that the params don't have to be
duplicated in the root nextflow.config.
* docs(subworkflows): Include modules instead of tools
* fix: Update to versions
* chore(align_bowtie2): Remove duplicate tag
* style: Format yamls
* test(subworkflows): Only check versions for modules
* chore: Update subworkflows to match rnaseq dev
* fix(subworkflows): Update paths
* fix(bam_sort_samtools): Fix sort parameters for testing
* Apply suggestions from code review
Co-authored-by: Harshil Patel <drpatelh@users.noreply.github.com>
* docs: Update TODOs with a message
* ci: Try using a matrix for strategy
* ci: Try passing an array
* Revert "ci: Try passing an array"
This reverts commit d3611fcd8332bbb9a8501e8dd299d0a623aaecaa.
Co-authored-by: Harshil Patel <drpatelh@users.noreply.github.com>
