nf-core_modules/.github/workflows/pytest-workflow.yml

108 lines
3.1 KiB
YAML
Raw Normal View History

2021-01-22 19:32:54 +00:00
name: Pytest-workflow
on:
push:
branches: [master]
pull_request:
branches: [master]
2021-01-22 19:32:54 +00:00
jobs:
changes:
name: Check for changes
runs-on: ubuntu-latest
outputs:
# Expose matched filters as job 'modules' output variable
modules: ${{ steps.filter.outputs.changes }}
steps:
- uses: actions/checkout@v2
- uses: dorny/paths-filter@v2
id: filter
with:
filters: "tests/config/pytest_modules.yml"
2021-01-24 05:39:32 +00:00
test:
2021-02-02 22:26:24 +00:00
runs-on: ubuntu-20.04
2021-02-02 22:46:56 +00:00
name: ${{ matrix.tags }} ${{ matrix.profile }} ${{ matrix.nxf_version }}
needs: changes
if: needs.changes.outputs.modules != '[]'
2021-01-22 19:32:54 +00:00
strategy:
2021-02-02 23:08:10 +00:00
fail-fast: false
2021-01-22 19:32:54 +00:00
matrix:
nxf_version: ["21.04.0"]
tags: ["${{ fromJson(needs.changes.outputs.modules) }}"]
profile: ["docker", "singularity", "conda"]
2021-01-22 19:32:54 +00:00
env:
NXF_ANSI_LOG: false
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: "3.x"
2021-01-24 17:25:13 +00:00
- uses: actions/cache@v2
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: "3.x"
- name: Install Python dependencies
run: python -m pip install --upgrade pip pytest-workflow
2021-01-24 17:35:24 +00:00
- uses: actions/cache@v2
with:
path: /usr/local/bin/nextflow
key: ${{ runner.os }}-nextflow-${{ matrix.nxf_version }}
restore-keys: |
${{ runner.os }}-nextflow-
- name: Install Nextflow
env:
NXF_VER: ${{ matrix.nxf_version }}
CAPSULE_LOG: none
run: |
wget -qO- get.nextflow.io | bash
sudo mv nextflow /usr/local/bin/
- name: Set up Singularity
if: matrix.profile == 'singularity'
uses: eWaterCycle/setup-singularity@v5
with:
singularity-version: 3.7.1
2021-02-02 23:13:35 +00:00
- name: Setup miniconda
if: matrix.profile == 'conda'
uses: conda-incubator/setup-miniconda@v2
with:
auto-update-conda: true
2021-02-03 12:07:39 +00:00
channels: conda-forge,bioconda,defaults
python-version: ${{ matrix.python-version }}
2021-02-03 12:57:55 +00:00
- name: Conda clean
if: matrix.profile == 'conda'
run: conda clean -a
# Test the module
- name: Run pytest-workflow
# only use one thread for pytest-workflow to avoid race condition on conda cache.
run: NF_CORE_MODULES_TEST=1 TMPDIR=~ PROFILE=${{ matrix.profile }} pytest --tag ${{ matrix.tags }} --symlink --kwdof
2021-01-24 05:39:51 +00:00
- name: Upload logs on failure
if: failure()
2021-01-24 05:39:51 +00:00
uses: actions/upload-artifact@v2
with:
Homer Modules (#75) * feat(homer): Add initial makeTagDirectory * feat(homer): Add initial findPeaks module * feat(homer): Update with new options See 1d30e2c21affedc742680e8e04d60c6481d9cd11 * fix(homer): Correct findpeaks process name * fix(homer): Takes a bam file instead of bed * feat(homer): Add initial makeTagDirectory test * fix(homer): Hardcode genome and configureHomer I'd like to modularize configureHomer, but I need to figure out how exactly the genomes work. * fix(homer): bam => bed Bam requires samtools to be present, which it's not in this docker image * feat(homer): Add initial configureHomer script * ci(homer): Add initial test * test(homer): Reproducible configuration workaround - I can't run both tests(one file and two files) at the same time because it breaks - I can't copy the genome stuff from the configurehomer module because it's read only - So I can't make the makeTagDirectory module depend on configureHomer * test(homer): Add placeholder annotatepeaks The required inputs are necessarily required for all workflows from what I've used, but I'll need to look at the actual docs * test(homer): Add missing B.bed * test(homer): Rename two => groseq Then all of the various workflows that homer provides can be e2e tested * feat(homer): Add initial makeUCSCfile module * test(homer): Add start to makeUCSCfile testing * chore(homer): Add various cleanups * test(homer): Rewrite annotatepeaks Not passing yet * test(homer): Rewrite configurehomer * test(homer): Rewrite findpeaks Still failing * test(homer): Rewrite makeucscfile Not passing yet * test(homer): Rewrite maketagdirectory All homer modules now follow the new structure. Time to make them pass. * test(homer): Fix typo for workflow name * fix(homer): Use correct container * fix(homer): Accept fasta in maketagdirectory Apparently all of the homer stuff can just take any old fasta and you don't need to configure the genome ahead of time with configureHomer * test(homer): makeTagDirectory passes now * fix(homer): Update containers in makeucscfile * test(homer): Rewrite makeucscfile Takes input from maketagdirectory which is how the module should be used * fix(homer): Update makeUCSCFile bedgraph path * test(homer): Update makeucscfile expected output * fix(homer): Update containers in findpeaks * fix(homer): Change findpeaks args The user is just going to have to know what they're doing for now * test(homer): findPeaks rewrite with tagDir input * test(homer): Update expected files for findPeaks And bump filters * style: Appease editorconfig * ci: Remove old workflow * tests(homer): Add md5sums * test(homer): Add meta test * style(homer): Capitalize HOMER * docs(homer): Add maketagdirectory meta.yml * docs(homer): Add makeucscfile meta.yml * docs(homer): Add findpeaks meta.yml * test(homer): Update to new test data standards * chore: Remove stuff that got revived in the rebase * chore: software => modules * test(homer): Update tags * test(homer): Update annotatepeaks * ci: Fix uploading of artifacts GitHub actions doesn't like the / in the tags * test(homer): Remove annotate md5sum This is failing and breaking new tests * test(homer): Use bams instead of beds * test(homer): Fix meta maketagdirectory * test(homer): Fix input in all tests * test(homer): Move back to bed files Forgot samtools isn't present * chore(homer): Add TODOs for tests * test(homer): Add bed format arg * test(homer): Update md5sums * test(homer): Fix tags tsvs * style(homer): Appease nf-core linting * docs(homer): Be in line with what is in the main.nf file Co-authored-by: Kevin Menden <kevin.menden@live.com> Co-authored-by: Kevin Menden <kevin.menden@live.com>
2021-09-08 15:40:34 +00:00
name: logs-${{ matrix.profile }}-${{ matrix.nxf_version }}
2021-01-24 05:39:51 +00:00
path: |
/home/runner/pytest_workflow_*/*/.nextflow.log
/home/runner/pytest_workflow_*/*/log.out
/home/runner/pytest_workflow_*/*/log.err
/home/runner/pytest_workflow_*/*/work