Commit graph

40 commits

Author SHA1 Message Date
nickhsmith
454e0acc09
Mpileup intervals (#1955)
* update tests

* update

* update

* make the manta inputs consistant for germline/somatic/tumoronly

* match chromosomes to cram file (chr21)

* undo genotypegvfs

* undo genotypegvfs

* include intervals

* update md5 with interval

Co-authored-by: Smith Nicholas <smith@in.tum.de>
2022-08-16 14:16:35 +02:00
Matthieu Muffato
31409f5e72
Added the samtools/dict module (#1922)
* Added the samtools/dict module

* samtools/dict is single-threaded
2022-07-26 22:13:19 +01:00
nvnieuwk
720027275c
small update to samtools/merge (#1896) 2022-07-18 11:37:45 +02:00
Matthias De Smet
ecece498f1
samtools/*stats*: use prefix for outputs (#1788)
* use prefix in flagstat

* use prefix in idxstats

* use prefix in stats

* update test.yml
2022-06-16 13:42:01 +02:00
WackerO
dc6c09e083 samtools mpileup now bgzips mpileup files 2022-06-11 13:32:53 +02:00
Rike
ef5f145342 Generalize Bam2Cram do either conversion 2022-05-09 15:22:54 +02:00
Lucpen
8e5fda3b10
Merge branch 'master' into correcting_stubs 2022-05-06 15:37:08 +02:00
Lucpen
971a17a3ab fix datatest for stubs 2022-05-06 15:34:18 +02:00
Lucpen
0fa36e6777 fix added correct entry workflow in test.yml 2022-05-06 15:26:34 +02:00
ljmesi
be8ce6de2a Remove variable md5sum from the tests 2022-05-05 08:32:45 +02:00
ljmesi
a39fa4e003 Fix checksum for one file 2022-05-03 15:47:49 +02:00
ljmesi
034352dbbf Merge branch 'add_compression_to_bam2fq' of https://github.com/Genomic-Medicine-Linkoping/modules into add_compression_to_bam2fq 2022-05-03 15:18:12 +02:00
ljmesi
5510ea39fe Test --no-name to solve inconsistent checksums 2022-05-03 15:18:09 +02:00
Lauri Mesilaakso
773e6f8e93
Merge branch 'master' into add_compression_to_bam2fq 2022-05-03 11:13:55 +03:00
ljmesi
59d6e6e1dd Add updated md5sums 2022-05-03 09:56:50 +02:00
Lucpen
3a4e415fe2 chores adding dummy files for stubs 2022-05-02 13:30:18 +02:00
Lucpen
08d5acbeb6 feat added tests for stubs 2022-05-02 11:59:08 +02:00
Maxime U. Garcia
569e07f0af
add samtools/bamtocram modules (#1561)
* add new samtools/bamtocram module

* fix md5sum

* remove md5sum

* Update modules/samtools/bamtocram/main.nf

Co-authored-by: James A. Fellows Yates <jfy133@gmail.com>

Co-authored-by: James A. Fellows Yates <jfy133@gmail.com>
2022-04-25 14:32:49 +02:00
Lucpen
12afb6b0fa
Update samtools view to add input path(index) (#1539)
* feat added index as input, to allow module to be used for subsampling

* fix test

* feat added index to meta.yml

* Update modules/samtools/view/meta.yml

feat corrected description of idea pattern file in meta.yml

Co-authored-by: James A. Fellows Yates <jfy133@gmail.com>

Co-authored-by: James A. Fellows Yates <jfy133@gmail.com>
2022-04-22 10:01:47 +02:00
Maxime U. Garcia
705f8c9ac4
Add samtools/collatefastq module (#1536)
* add samtools/collatefastq module

* update yml file

* improve output
2022-04-20 10:05:17 +02:00
FriederikeHanssen
897c33d5da
Samtools version update (#1507)
* Fix typo

* update version to 1.15.1

* Fix md5sums

* update mulled containers

* update md5sums

* update md5sums
2022-04-11 14:26:28 +02:00
FriederikeHanssen
b5825fe6b3
Mpileup also likes intervals (#1445)
* Mpileup also likes intervals

* Also update meta yml with inclusive input and intervals

Co-authored-by: Sateesh Peri <33637490+sateeshperi@users.noreply.github.com>
Co-authored-by: Maxime U. Garcia <max.u.garcia@gmail.com>
2022-03-25 07:46:51 +01:00
FriederikeHanssen
1ad73f1b2a
update samtools version to 1.15 (#1358)
* update samtools version to 1.15

* Update checksums
2022-02-28 12:42:29 +01:00
Harshil Patel
e751e5040a
Bump software versions for viralrecon modules (#1198)
* Bump software versions for viralrecon modules

* Remove custom params.save_unaligned from bowtie2_align

* Unify samtools modules and error if input and output names are the same

* Fix ALL the tests
2022-01-07 17:52:39 +00:00
Harshil Patel
0fafaeebf5
Revert PR #1147 (#1166)
* Revert PR #1147

* Fix md5sum for crai file
2021-12-13 16:15:20 +00:00
Maxime Borry
e0aa89141f
Add meta information to samtools/faidx (#1114)
* add meta to samtools/faidx
2021-12-06 11:37:04 +01:00
Harshil Patel
9d0cad583b
Change syntax from task.ext.suffix to tast.ext.prefix in all modules (#1110)
* Add comment line for consistency

* Remove all functions.nf

* Remove include functions.nf and publishDir options

* Replace options.args3 with task.ext.args3 - 3 modules

* Replace options.args3 with task.ext.args3 - 17 modules

* Replace {task.cpus} with task.cpus

* Replace off on off off off off off off off on off on off on off off off on off off off on on off off off on on off off off off off off off on off off off off on off on on off off off on on on on off off off on off on on off on on off off on on on off on on off on off off off off on off off off on off off on off on off off off on on off on off on off off on off off off on off off off on off off off off on off off off on on on off on on off off on off on on on off on on off on on on off off off off off on on off off on off off off off off on off off on on off on on off on off off off on off off off off on on off on off off on off off on off on off off off off off off off off on on off on off off off.args with

* Add def args = task.ext.args line to all modules in script section

* Replace options.args with args and args_list

* Initialise args2 and args3 properly

* Replace container syntax

* Revert container changes for cellranger/mkref

* Replace getProcessName in all modules

* Replace getSoftwareName in all modules

* Unify modules using VERSION variable

* Replae options.suffix with task.ext.suffix

* Remove NF version restriction for CI

* Bump NF version in README

* Replace task.process.tokenize logic with task.process

* Minor tweaks to unify syntax in tests main.nf

* Add a separate nextflow.config for each module

* Transfer remaining module options to nextflow.config

* Remove addParams from tests main.nf

* Remove TODO statements

* Use -c to import module specific config

* Bump NF version to 21.10.3

* Fix tests for artic/minion

* Fix broken publishDir syntax

* Standardise and fix obvious failing module tests

* Remove kronatools to krona

* Comment out tags in subworkflow test.yml

* Fix failing module tests

* Add consistent indentation to nextflow.config

* Comment out subworklow definitions

* Fix kallistobustools/ref

* Fix rmarkdownnotebook

* Fix jupyternotebook

* Quote task.process

* Add plink2/vcf to pytest_modules.yml

* Remove NF_CORE_MODULES_TEST from pytest CI

* Fix more tests

* Move bacteroides_fragilis to prokaryotes folder

* Fix cooler merge tests

* Fix kallistobustools/count tests

* Fix kallistobustools/ref tests

* Update test_10x_1_fastq_gz file for kallistobustools/count tests

* Fix bcftools/query tests

* Fix delly/call tests

* Fix cooler/zoomify tests

* Fix csvtk/split tests

* Fix gatk4/intervallisttools tests

* Fix gatk4/variantfiltration

* Fix pydamage/filter tests

* Fix test data for unicycler

* Fix gstama/collapse module

* Fix leehom tests

* Fix metaphlan3 tests

* Fix pairtools/select tests

* Update nextflow.config

* Update nextflow.config

* feat: update syntax

* Fix arriba tests

* Fix more failing tests

* Update test syntax

* Remove comments from tests nextflow.config

* Apply suggestions from code review

* Fix kallistobustools/count module

* Update dumpsoftwareversions module

* Update custom/dumpsoftwareversions

* Add args2 to untar module

* Update leftover modules

* Remove last remaining addParams

* Change syntax from task.ext.suffix to tast.ext.prefix

* Change nextflow.config in all tests to use ext.prefix instead of ext.suffix

Co-authored-by: JoseEspinosa <kadomu@gmail.com>
Co-authored-by: Gregor Sturm <mail@gregor-sturm.de>
Co-authored-by: MaxUlysse <max.u.garcia@gmail.com>
2021-12-02 12:39:55 +00:00
Harshil Patel
20d8250d9f
Update all modules to new NF DSL2 syntax (#1099)
* Add comment line for consistency

* Remove all functions.nf

* Remove include functions.nf and publishDir options

* Replace options.args3 with task.ext.args3 - 3 modules

* Replace options.args3 with task.ext.args3 - 17 modules

* Replace {task.cpus} with task.cpus

* Replace off on off off off off off off off on off on off on off off off on off off off on on off off off on on off off off off off off off on off off off off on off on on off off off on on on on off off off on off on on off on on off off on on on off on on off on off off off off on off off off on off off on off on off off off on on off on off on off off on off off off on off off off on off off off off on off off off on on on off on on off off on off on on on off on on off on on on off off off off off on on off off on off off off off off on off off on on off on on off on off off off on off off off off on on off on off off on off off on off on off off off off off off off off on on off on off off off.args with

* Add def args = task.ext.args line to all modules in script section

* Replace options.args with args and args_list

* Initialise args2 and args3 properly

* Replace container syntax

* Revert container changes for cellranger/mkref

* Replace getProcessName in all modules

* Replace getSoftwareName in all modules

* Unify modules using VERSION variable

* Replae options.suffix with task.ext.suffix

* Remove NF version restriction for CI

* Bump NF version in README

* Replace task.process.tokenize logic with task.process

* Minor tweaks to unify syntax in tests main.nf

* Add a separate nextflow.config for each module

* Transfer remaining module options to nextflow.config

* Remove addParams from tests main.nf

* Remove TODO statements

* Use -c to import module specific config

* Bump NF version to 21.10.3

* Fix tests for artic/minion

* Fix broken publishDir syntax

* Standardise and fix obvious failing module tests

* Remove kronatools to krona

* Comment out tags in subworkflow test.yml

* Fix failing module tests

* Add consistent indentation to nextflow.config

* Comment out subworklow definitions

* Fix kallistobustools/ref

* Fix rmarkdownnotebook

* Fix jupyternotebook

* Quote task.process

* Add plink2/vcf to pytest_modules.yml

* Remove NF_CORE_MODULES_TEST from pytest CI

* Fix more tests

* Move bacteroides_fragilis to prokaryotes folder

* Fix cooler merge tests

* Fix kallistobustools/count tests

* Fix kallistobustools/ref tests

* Update test_10x_1_fastq_gz file for kallistobustools/count tests

* Fix bcftools/query tests

* Fix delly/call tests

* Fix cooler/zoomify tests

* Fix csvtk/split tests

* Fix gatk4/intervallisttools tests

* Fix gatk4/variantfiltration

* Fix pydamage/filter tests

* Fix test data for unicycler

* Fix gstama/collapse module

* Fix leehom tests

* Fix metaphlan3 tests

* Fix pairtools/select tests

* Update nextflow.config

* Update nextflow.config

* feat: update syntax

* Fix arriba tests

* Fix more failing tests

* Update test syntax

* Remove comments from tests nextflow.config

* Apply suggestions from code review

* Fix kallistobustools/count module

* Update dumpsoftwareversions module

* Update custom/dumpsoftwareversions

* Add args2 to untar module

* Update leftover modules

* Remove last remaining addParams

Co-authored-by: JoseEspinosa <kadomu@gmail.com>
Co-authored-by: Gregor Sturm <mail@gregor-sturm.de>
Co-authored-by: MaxUlysse <max.u.garcia@gmail.com>
2021-11-26 07:58:40 +00:00
FriederikeHanssen
19035c99d1
Add thread option to samtools modules (#1069)
* add thread option to samtools modules

* fix tests

* fix tests

* fix tests

* Fix naming to fix tests

* cpus-1 to account for mainthread

* remove thread in ampliconclip, docu doesn't report this param

* add -1 to all other applicable samtools modules

* Update samtools version

* Update checksums

* retrigger GHA after update

* Update modules/samtools/fastq/main.nf

Co-authored-by: Patrick Hüther <patrick.huether@gmail.com>

* Update modules/samtools/fixmate/main.nf

Co-authored-by: Patrick Hüther <patrick.huether@gmail.com>

* Update modules/samtools/flagstat/main.nf

Co-authored-by: Patrick Hüther <patrick.huether@gmail.com>

* Update modules/samtools/index/main.nf

Co-authored-by: Patrick Hüther <patrick.huether@gmail.com>

* Update modules/samtools/merge/main.nf

Co-authored-by: Patrick Hüther <patrick.huether@gmail.com>

* Update modules/samtools/stats/main.nf

Co-authored-by: Patrick Hüther <patrick.huether@gmail.com>

* Update modules/samtools/view/main.nf

Co-authored-by: Patrick Hüther <patrick.huether@gmail.com>

* Fix md5sum fixmate

* Fix md5sums

* sth funny with the fixmate checksums

* more md5sums updates

Co-authored-by: Patrick Hüther <patrick.huether@gmail.com>
2021-11-18 22:24:47 +01:00
Maxime U. Garcia
1abe23e159
Add CRAM to samtools/index (#1049)
* feat: update samtools/index

* feat: add test

* fix: lint

* fix: output file + md5
2021-11-10 10:30:48 +01:00
Simon Pearce
81d65d4978
Samtools fixmate module (#991)
* Samtools fixmate module

* Update modules/samtools/fixmate/meta.yml

Co-authored-by: James A. Fellows Yates <jfy133@gmail.com>

* Update modules/samtools/fixmate/main.nf

Co-authored-by: James A. Fellows Yates <jfy133@gmail.com>

* Update main.nf

* Update modules/samtools/fixmate/meta.yml

Co-authored-by: James A. Fellows Yates <jfy133@gmail.com>

* Update meta.yml

Co-authored-by: Simon Pearce <simon.pearce@cruk.manchester.ac.uk>
Co-authored-by: James A. Fellows Yates <jfy133@gmail.com>
2021-11-01 12:12:14 +00:00
FriederikeHanssen
ac1e6df076
Update to allow cram + update needed to use the gatk4 modules in sarek (#976)
* Make samtools/merge cram compliant

* samtools/stats cram compliance

* update yml file

* samtools/view to deal with crams

* Update tests to make sure cram works

* also fix tmp dir and min mem in one go

* basequalityrecal test for cram + min mem + tmpdir

* update haplotypecaller for sarek

* update haplotype yml

* update markdup to allow multiple bams, take out params to be passed with options.args

* remove TODO statement

* Remove variable md5sum

* add emtpy input to stats module in subworkflows

* subworkflows seem to work now on my side

* Apply code review

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

* replace bam with input to be more inclusive

* rename everywhere

* rename input

* remove variable checksum

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>
2021-10-29 13:01:05 +02:00
Francesco L
4b8c7ac7bd
Bam2fq (#958)
* added template for module

* update main

* added specific code

* wrong variable name in else script

* added tests for both split and nosplit

* docker test successful - updating yaml

* adding echo to version print
2021-10-29 10:46:34 +02:00
louperelo
263bbe56d2
add new module samtools/depth (#950)
* add new module samtools_depth

* fixed main.nf for samtools/depth

* Apply suggestions from code review

Co-authored-by: James A. Fellows Yates <jfy133@gmail.com>

Co-authored-by: James A. Fellows Yates <jfy133@gmail.com>
2021-10-28 12:53:21 +02:00
Edmund Miller
c19671dca9
Subworkflow Infrastructure (#662)
* feat(subworkflows): Add align_bowtie2 subworkflow

For testing CI setup

* test(align_bowtie2): Add initial list of changes to test

* test(align_bowtie2): Add initial test

* refactor: Use tags to run subworkflows ci

For every underlying module used by workflow and allow the modules
pytest-modules definition be the source of truth.

* refactor: Use individual directories for subworkflows

* docs(align_bowtie2): Add initial meta.yml

Copied most of it from the bowtie2/align module.

* fix(align_bowtie2): Fix module include paths

* test(bam_sort_samtools): Add initial test

* ci(bam_sort_samtools): Add modules that trigger the tag

* test(bam_stats_samtools): Add initial test

* ci(bam_stats_samtools): Add keys to pick up changes

* docs(bam_samtools): Add initial meta.yml

* test(align_bowtie2): Fix path to subworkflow

* test(align_bowtie2): Update entry point

* fix(bam_sort_samtools): Update include paths

* test(bam_sort_samtools): Fix path

* style: Clean up addParams

* test(samtools_sort): Add suffix for test

* test(align_bowtie2): Add samtools_options for suffix

* test(bam_stats_samtools): Update path

* test(bam_stats_samtools): Use stats input

Otherwise it's just an example of how it's used in the bam_sort_samtools subworkflow

* ci(linting): Skip module linting of subworkflows

* ci(linting): Clean up startsWith statement

* test(bam_stats_samtools): Use single end test data for single end test

* test(bam_stats_samtools): Add expected files

* test(align_bowtie2): Add paired-end test

* test(align_bowtie2): Sort order of output

* test(align_bowtie2): Update hashes

* docs(align_bowtie2): Fix typo

* test(align_bowtie2): Update samtools output names

* test(align_bowtie2): Remove md5sums for bam/bai

* feat(subworkflows): Add nextflow.configs

These can be used for default settings in the future. They can then be
included in the conf/modules.config so that the params don't have to be
duplicated in the root nextflow.config.

* docs(subworkflows): Include modules instead of tools

* fix: Update to versions

* chore(align_bowtie2): Remove duplicate tag

* style: Format yamls

* test(subworkflows): Only check versions for modules

* chore: Update subworkflows to match rnaseq dev

* fix(subworkflows): Update paths

* fix(bam_sort_samtools): Fix sort parameters for testing

* Apply suggestions from code review

Co-authored-by: Harshil Patel <drpatelh@users.noreply.github.com>

* docs: Update TODOs with a message

* ci: Try using a matrix for strategy

* ci: Try passing an array

* Revert "ci: Try passing an array"

This reverts commit d3611fcd8332bbb9a8501e8dd299d0a623aaecaa.

Co-authored-by: Harshil Patel <drpatelh@users.noreply.github.com>
2021-10-08 16:02:42 +01:00
Harshil Patel
43c2779258
Fix version commands (#749)
* Fix version commands

* Fix version commands: again
2021-09-27 22:10:37 +01:00
Harshil Patel
c5235a983d
nf-core modules bump-versions for all modules (#630)
* Add blacklist of modules that shouldn't be updated to .nf-core.yml

* nf-core modules bump-versions for all modules

* Remove TODO statements identified by linting

* Fix md5sums for failing tests

* Fix more tests
2021-07-28 09:10:44 +01:00
Johnathan D
2e619add87
Add samtools ampliconclip (#590)
* created template for `samtools/ampliconclip` (#584)

* All tests passing (#584)

* Linting fixed (#584)

* Final linting fixed (#584)

* Optional output flags moved to `input` (#584)

* typo fix (#584)

* Apply suggestions from code review

Co-authored-by: Harshil Patel <drpatelh@users.noreply.github.com>
2021-07-20 21:31:31 +01:00
Maxime U. Garcia
adfb8a3d65
fix: update samtools_merge (#582)
* fix: update samtools_merge

* Update modules/samtools/merge/main.nf

Co-authored-by: Harshil Patel <drpatelh@users.noreply.github.com>

Co-authored-by: Harshil Patel <drpatelh@users.noreply.github.com>
2021-07-16 14:32:17 +02:00
Harshil Patel
e937c7950a
Rename software/ directory to modules/ to re-organise module structure (#567)
* Update README

* Rename pytest_software.yml to pytest_modules.yml

* Rename main software directory to modules

* Remove deprecated modules

* Rename tests software to modules

* Replace paths for tests in pytest_modules.yml

* Replace software with modules in Github Actions

* Replace software with modules in main.nf tests

* Rename software to modules in test.yml
2021-07-07 10:10:18 +01:00