Commit graph

32 commits

Author SHA1 Message Date
Harshil Patel
b3e322064e
[POC] Get subworkflows working again - bam_stats_samtools (#2097)
* [POC] Get subworkflows working again - bam_stats_samtools

* Comment out aliased anchors

* Add explicit anchor to pytest_modules.yml

* test(subworkflows): Remove anchors

There are two options to get the tests to trigger:

1. Add the module anchors to the subworkflows (less things to update and maintain because we only have to update the module triggers)

2. Add the tags to the pytest workflow spec.

This is option 2.

Co-authored-by: Edmund Miller <edmund.a.miller@protonmail.com>
2022-09-28 19:20:20 +01:00
Moritz E. Beber
74b11ccc44 refactor: rename and update subworkflow 2022-05-11 19:24:16 +02:00
MaxUlysse
40dd662fd2 update ensemblvep and snpeff modules 2022-05-05 12:57:14 +02:00
Matthias De Smet
7e391c3a3b
Update subworkflows/nf-core/bam_qc_picard/main.nf
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
2022-04-29 14:26:06 +02:00
Matthias De Smet
343c0ebe20
Update subworkflows/nf-core/bam_qc_picard/main.nf
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
2022-04-29 14:26:00 +02:00
Matthias De Smet
20e3767657
Update subworkflows/nf-core/bam_qc_picard/main.nf
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
2022-04-29 14:24:25 +02:00
Matthias De Smet
71ccf0e207
Update subworkflows/nf-core/bam_qc_picard/main.nf
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
2022-04-29 14:24:19 +02:00
Matthias De Smet
73600b0339
Update subworkflows/nf-core/bam_qc_picard/main.nf
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
2022-04-29 14:20:15 +02:00
Matthias De Smet
4618c542e9 fix metrics output 2022-04-28 13:59:46 +02:00
Matthias De Smet
2fc857955f fix index omission 2022-04-28 13:53:48 +02:00
Matthias De Smet
8ad861a645 add fasta index 2022-04-28 13:52:03 +02:00
Matthias De Smet
55dfe1d8ab fix hsmetrics input 2022-04-28 13:43:59 +02:00
Matthias De Smet
e0a8af8693 fix outputs 2022-04-28 13:42:24 +02:00
Matthias De Smet
dee8ec9e91 fix typos 2022-04-28 12:53:17 +02:00
Matthias De Smet
881e9db4bf update tests 2022-04-28 10:19:30 +02:00
Matthias De Smet
3cbf5c63e5
Update subworkflows/nf-core/bam_qc_picard/main.nf
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
2022-04-28 10:09:25 +02:00
Matthias De Smet
8fb8199f31 fix copilot suggestion 2022-04-28 10:05:07 +02:00
Matthias De Smet
2b29ff5883 drop params.options 2022-04-28 10:03:02 +02:00
Matthias De Smet
d85f9c7011 add subworkflow bam_qc_picard 2022-04-21 13:16:23 +02:00
Edmund Miller
13cc32399c
feat(homer): Add groseq subworkflow (#1492)
* feat(homer): Add groseq subworkflow

* fix(homer): Update groseq paths

* test(homer): Update groseq bam md5sums

* test(homer): Update bed process args wildcard

* test(homer): Update groseq bed md5s

* style: Run prettier

* style(homer): Align comments

Co-authored-by: Friederike Hanssen <Friederike.hanssen@qbic.uni-tuebingen.de>

* docs(homer): Add groseq meta.yml

Co-authored-by: Friederike Hanssen <Friederike.hanssen@qbic.uni-tuebingen.de>
2022-04-04 18:21:37 +00:00
Matthias Hörtenhuber
e745e167c1
Fix formatting in yaml files, add yamllint config (#1279)
* fix yml formatting

* allow fastq.gz and fq.gz as file input, add meta.yml and test

* fix yaml files

* Revert "allow fastq.gz and fq.gz as file input, add meta.yml and test"

This reverts commit 34002d7a7a8c7f7bb4600c3377f35c87849f71a4.

* prettier magic!

* fix comments for yamllint

* remove node version number

* fix linting errors

Co-authored-by: Harshil Patel <drpatelh@users.noreply.github.com>
2022-02-15 11:15:27 +00:00
Francesco L
a68c563e54
Added UMI sub-workflow (#1098)
* added code for subworkflow fgbio call umi consensus

* ironing out a few typos etc

* fixing last things

* fixed md5sum - lets see if it changes

* removing file accidentally deleted

* tidy indents

* added bwamem2 alternative

* fixed entry for both tests

* changed name second test workflow entry

* fixed workflow entry names

* fixed md5sum for file generated with bwamem2

* added syntax new DSL2

* added new config location in test command line

* added new config location in test command line

* use of prefix instead of suffix because modules have been changed in this way

* explicit alias to bwa mem1 to avoid confusion

* removed param that should be an ext optional argument in fgbio groupreadsbyumi

* missing colon in config

* missing colon in module config too

* order list alphabetically

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

* remove params from body

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

* improving readability of input structure

Co-authored-by: Mahesh Binzer-Panchal <mahesh.binzer-panchal@nbis.se>

* reverting to mandatory input

* fixed tests and workflow take values

* remove param

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

* simplify tests params

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

* formatting inputs for readability

* factoring in changes to bwamem2_mem and bwa_mem sort/view inputs

* updating test md5sum for grouped file following code update in bwamem

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
Co-authored-by: Maxime U. Garcia <max.u.garcia@gmail.com>
Co-authored-by: Mahesh Binzer-Panchal <mahesh.binzer-panchal@nbis.se>
2021-12-09 11:16:40 +01:00
Maxime U. Garcia
31d4099f38
Add subworkflows for ensemblvep and snpeff (#1124)
* greatly simplify syntax

* feat: add subworkflows to annotate (+ bgzip/tabix index) with ensemblvep and snpeff

* feat: get versions from all tools

* add commented infor for new annotation modules
2021-12-07 17:12:35 +01:00
Robert A. Petit III
f052dc445c
use underscores in anchors and references (#1080)
* use underscores in anchors and references

* Dummy change to trigger CI

* use dev branch

* underscore anchor
2021-11-18 18:09:09 +00:00
GCJMackenzie
5b975cc20d
Add gatk somatic paired calling subworkflow (#1067)
* initial commit to setup branch

* workflow finished

* Update nextflow.config

* tumour to tumor, getpileup passed as nomral and tumor

* paired_somatic renamed to tumor_normal_somatic

* Apply suggestions from code review

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

* Update subworkflows/nf-core/gatk_tumor_normal_somatic_variant_calling/main.nf

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

* updated index names in meta.yml

* changed index file names in main script and test

* Apply suggestions from code review

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

* Apply suggestions from code review

* fixed bug from changes

* Apply suggestions from code review

* tests should now work after the yml update

* Update pytest_modules.yml

Co-authored-by: GCJMackenzie <gavin.mackenzie@nibsc.org>
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
Co-authored-by: Maxime U. Garcia <max.u.garcia@gmail.com>
2021-11-17 11:34:07 +01:00
GCJMackenzie
071b1d50a8
Add gatk somatic tumour calling subworkflow (#1064)
* initial commit to set up new branch

* save changes to checkout

* workflow working, still needs test.yml and meta.yml, also fix versions file

* subworkflow finished

* Update pytest_subworkflows.yml

* Update pytest_subworkflows.yml

* Update pytest_subworkflows.yml

* fix config subworkflow name

* Update main.nf

* Update pytest_subworkflows.yml

* fixed md5sum issue likely caused by gatk version update

* tumour changed to tumor

* old dir deleted

* Comments added to explain use of placeholders '[]'

* updated index names, input channel renamed to input

* Apply suggestions from code review

* updated to perform new subworkflow testing

Co-authored-by: GCJMackenzie <gavin.mackenzie@nibsc.org>
Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
2021-11-17 11:07:17 +01:00
Edmund Miller
2af071ed0d
Fix subworkflows seperate from modules (#933)
* ci: Remove pytest_subworkflows

* ci(bam_sort_samtools): Depend on paths-filter instead of pytest-workflow

Co-authored-by: Harshil Patel <drpatelhh@gmail.com>
Co-authored-by: Gregor Sturm <mail@gregor-sturm.de>
Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

* ci: Revert back to one job branch

* ci(align_bowtie2): Run tests that depend on bam_sort_samtools

* ci: Fix anchor not being created yet

* ci: Update sra_fastq tags and pytest_modules

* fix(bam_sort_samtools): Update nextflow.config with params

* test(subworkflows): Update gatk_create_som_pon tags

* ci: Point to subworkflow_hacks branch of nf-core tools

Co-authored-by: Harshil Patel <drpatelhh@gmail.com>
Co-authored-by: Gregor Sturm <mail@gregor-sturm.de>
Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>
2021-11-15 18:18:46 +00:00
Maxime U. Garcia
3426834744
feat: fastaidx -> fai (#1073) 2021-11-15 18:03:02 +01:00
GCJMackenzie
9573cb1bec
Add panel of normals subworkflow (#1044)
* commiting changes to switch branch

* commit to setup remote branch

* first draft of the sompon workflow

* keep branch in line with gendb bugfixing

* Update test.yml

* tidy up main.nf

* fixed md5sum

Co-authored-by: GCJMackenzie <gavin.mackenzie@nibsc.org>
2021-11-09 10:16:43 +00:00
FriederikeHanssen
ac1e6df076
Update to allow cram + update needed to use the gatk4 modules in sarek (#976)
* Make samtools/merge cram compliant

* samtools/stats cram compliance

* update yml file

* samtools/view to deal with crams

* Update tests to make sure cram works

* also fix tmp dir and min mem in one go

* basequalityrecal test for cram + min mem + tmpdir

* update haplotypecaller for sarek

* update haplotype yml

* update markdup to allow multiple bams, take out params to be passed with options.args

* remove TODO statement

* Remove variable md5sum

* add emtpy input to stats module in subworkflows

* subworkflows seem to work now on my side

* Apply code review

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

* replace bam with input to be more inclusive

* rename everywhere

* rename input

* remove variable checksum

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
2021-10-29 13:01:05 +02:00
Moritz E. Beber
c912117972
feat: add sub-workflow for SRA (#836)
* feat: add sub-workflow for SRA
* Combine prefetch and fasterq-dump into one sub-workflow
* tests: add sub-workflow to pytest config
2021-10-14 11:50:31 +02:00
Edmund Miller
c19671dca9
Subworkflow Infrastructure (#662)
* feat(subworkflows): Add align_bowtie2 subworkflow

For testing CI setup

* test(align_bowtie2): Add initial list of changes to test

* test(align_bowtie2): Add initial test

* refactor: Use tags to run subworkflows ci

For every underlying module used by workflow and allow the modules
pytest-modules definition be the source of truth.

* refactor: Use individual directories for subworkflows

* docs(align_bowtie2): Add initial meta.yml

Copied most of it from the bowtie2/align module.

* fix(align_bowtie2): Fix module include paths

* test(bam_sort_samtools): Add initial test

* ci(bam_sort_samtools): Add modules that trigger the tag

* test(bam_stats_samtools): Add initial test

* ci(bam_stats_samtools): Add keys to pick up changes

* docs(bam_samtools): Add initial meta.yml

* test(align_bowtie2): Fix path to subworkflow

* test(align_bowtie2): Update entry point

* fix(bam_sort_samtools): Update include paths

* test(bam_sort_samtools): Fix path

* style: Clean up addParams

* test(samtools_sort): Add suffix for test

* test(align_bowtie2): Add samtools_options for suffix

* test(bam_stats_samtools): Update path

* test(bam_stats_samtools): Use stats input

Otherwise it's just an example of how it's used in the bam_sort_samtools subworkflow

* ci(linting): Skip module linting of subworkflows

* ci(linting): Clean up startsWith statement

* test(bam_stats_samtools): Use single end test data for single end test

* test(bam_stats_samtools): Add expected files

* test(align_bowtie2): Add paired-end test

* test(align_bowtie2): Sort order of output

* test(align_bowtie2): Update hashes

* docs(align_bowtie2): Fix typo

* test(align_bowtie2): Update samtools output names

* test(align_bowtie2): Remove md5sums for bam/bai

* feat(subworkflows): Add nextflow.configs

These can be used for default settings in the future. They can then be
included in the conf/modules.config so that the params don't have to be
duplicated in the root nextflow.config.

* docs(subworkflows): Include modules instead of tools

* fix: Update to versions

* chore(align_bowtie2): Remove duplicate tag

* style: Format yamls

* test(subworkflows): Only check versions for modules

* chore: Update subworkflows to match rnaseq dev

* fix(subworkflows): Update paths

* fix(bam_sort_samtools): Fix sort parameters for testing

* Apply suggestions from code review

Co-authored-by: Harshil Patel <drpatelh@users.noreply.github.com>

* docs: Update TODOs with a message

* ci: Try using a matrix for strategy

* ci: Try passing an array

* Revert "ci: Try passing an array"

This reverts commit d3611fcd8332bbb9a8501e8dd299d0a623aaecaa.

Co-authored-by: Harshil Patel <drpatelh@users.noreply.github.com>
2021-10-08 16:02:42 +01:00