-
Notifications
You must be signed in to change notification settings - Fork 950
Add module t1k/run #9555
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Add module t1k/run #9555
Conversation
prototaxites
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good! Mostly a few thoughts about catching/documenting/testing the different input types.
| if (is_bam) { | ||
| // BAM input | ||
| input_args = "-b ${input}" | ||
| } | ||
| else if (meta.single_end) { | ||
| // Single-end FASTQ | ||
| input_args = "-u ${input}" | ||
| } | ||
| else if (!meta.single_end && input.size() == 2) { | ||
| // Paired-end FASTQ | ||
| input_args = "-1 ${input[0]} -2 ${input[1]}" | ||
| } | ||
| else if (!meta.single_end && input.size() == 1) { | ||
| // Interleaved FASTQ | ||
| input_args = "-i ${input}" | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add a final else {} that errors if none of these conditions are met?
| e.g. `[ id:'sample1', single_end:false ]` | ||
| - input: | ||
| type: file | ||
| description: BAM or FASTQ file(s). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe mention that it can be either a single bam, two paired-end fastq, an interleaved fastq or a single-end fastq, and anything else will error?
| assert process.success | ||
| assertAll( | ||
| { assert snapshot( | ||
| process.out.genotype_tsv.get(0).get(1), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there any scope to test anything about the other inputs - some contains statements, or just checking their existence?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because the tool targets only specific genes, the test dataset does not produce meaningful results. As a result, most output files are empty, except for genotype_tsv.
I will keep the current setup and just add assertions to validate only the presence and names of the remaining output files.
| tag "t1k" | ||
| tag "t1k/run" | ||
|
|
||
| test("homo_sapiens - paired_end - 10x ") { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any possiblilty with the available test data to write tests for the other input formats - bam, single-end, etc.?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @prototaxites! I can try to add tests for single-end and interleaved FASTQ inputs using the available test data.
For BAM input, this is not currently possible, as it requires a coordinate file created form t1k-build which again uses highly specific data that do not exist in the test-data repository.
* Add vcf_impute_minimac4 * Update linting * Update test * Fix linting * Update minimac4 sbwf * Remove tag * Remove tag * Fix linting * Add comment * Update snapshot * Fix nf-test
…subworkflow (nf-core#9559) Add BBSplit stats to MultiQC files in fastq_qc_trim_filter_setstrandedness Pass BBSplit stats output to MultiQC for visualization of read binning statistics. MultiQC 1.33+ includes support for parsing BBSplit stats.txt files and displaying per-sample read distribution across reference genomes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* proper stub for gz and stub test added * topic output syntax and tests update * meta yml updated with topics and ontologies * meta file curated * version bump to latest * update nf-tests properly * adding self to maintainers * removed Z flag as is deprecated after v4.10 * conda bug with different pre-built python version fixed
* update manta germline * topics convertinversion * topics convertinversion * topics manta/somatic * topics manta/tumoronly
syntax updates and topics of jasminesv
* update yaml * update main.nf * modified test runs * update bedmethyltobigwig tests * update main * update snapshot * fix linting * update snapshots * remove config * update module_args * [automated] Fix linting with Prettier * changed name * update main --------- Co-authored-by: ra25wog <jin.khoo@campus.lmu.de> Co-authored-by: nf-core-bot <core@nf-co.re>
…-core#9566) * Standarize and alignment * Fix glimpse2 sbwf test * Fix test * Add comment * Update snapshot --------- Co-authored-by: LouisLeNezet <louislenezet@gmaio.com>
* Remove .view() * Remove unecessary tags
* testing solo trim-galore container, without adding extra cutadapt and pigz * Syntax updates and topic version for manta modules (nf-core#9556) * update manta germline * topics convertinversion * topics convertinversion * topics manta/somatic * topics manta/tumoronly * Syntax updates and topics of jasminesv (nf-core#9554) syntax updates and topics of jasminesv * Update `Modkit pileup` (nf-core#9553) * update yaml * update main.nf * modified test runs * update bedmethyltobigwig tests * update main * update snapshot * fix linting * update snapshots * remove config * update module_args * [automated] Fix linting with Prettier * changed name * update main --------- Co-authored-by: ra25wog <jin.khoo@campus.lmu.de> Co-authored-by: nf-core-bot <core@nf-co.re> * Standarize and alignment for all imputation and alignment modules (nf-core#9566) * Standarize and alignment * Fix glimpse2 sbwf test * Fix test * Add comment * Update snapshot --------- Co-authored-by: LouisLeNezet <louislenezet@gmaio.com> * Update Infrastructural dependencies * Remove .view() (nf-core#9567) * Bump strdrop to 0.3.1 (nf-core#9565) * Remove unecessary tags (nf-core#9568) * Remove .view() * Remove unecessary tags * latest container, with cutadapt 5.2 * new output syntax, nf-tests updated, meta updated * meta yml lint fixed * trying to fix lint * lint fix with nf-core tools 3.6.0dev * removing TRIMGALORE versions output from the FASTQ_FASTQC_UMITOOLS_TRIMGALORE subworkflow --------- Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> Co-authored-by: Jinn <155078830+jkh00@users.noreply.github.com> Co-authored-by: ra25wog <jin.khoo@campus.lmu.de> Co-authored-by: nf-core-bot <core@nf-co.re> Co-authored-by: Louis Le Nézet <58640615+LouisLeNezet@users.noreply.github.com> Co-authored-by: LouisLeNezet <louislenezet@gmaio.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Felix Lenner <52530259+fellen31@users.noreply.github.com>
* add 3 required ribocode modules * GTFUpdate; meta in inputs; No Gzip; * return snaps * nf-test passing * Update RiboCode versions to use topics * Update RiboCode versions to use topics -New lines at EOF * Update RiboCode versions to use topics - Prettier * RiboCode update snapshots * Update RiboCode snapshots to container versions * Bump to latest ribocode build with pyfasta and ribocode fixes * Improve RiboCode module tests and fix review issues - Remove defaults channel from environment.yml files - Fix keywords to lowercase in meta.yml files - Add stub tests to all modules - Update version snapshots from 1.2.14 to 1.2.15 - Remove duplicate snapshot entries - Optimize test setup using pre-made test data: - metaplots/ribocode: use UNTAR for annotation instead of full setup chain - prepare: use pre-made updated GTF instead of running gtfupdate Temporarily points to test-datasets branch pending merge of: nf-core/test-datasets#1820 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Update RiboCode snapshot to GHA runner architecture Update md5 hashes to match linux/amd64 output from CI runners. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add schema reference to ribocode/ribocode meta.yml 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Revert nf-test.config to standard test-datasets paths test-datasets PR nf-core#1820 has been merged, reverting to nf-core/test-datasets. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Jonathan Manning <jonathan.manning@seqera.io> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Bump stranger * meta.yaml * linting not happy * linting is happy, but I'm not * pin same python version as used in container
* add containers for fastqc and multiqc * sort containers and add https for singularity * fix build_id naming * some more key names fixing * use underscores
) * Updated rseqc/bamstat: Added BAI support per nf-core#9460 & topics-based versions * Updated rseqc/inferexperiment: Added BAI support per nf-core#9460 & topics-based versions * Updated rseqc/innerdistance: Added BAI support per nf-core#9460 & topics-based versions * Updated rseqc/junctionannotation: Added BAI support per nf-core#9460 & topics-based versions * Updated rseqc/junctionsaturation: Added BAI support per nf-core#9460 & topics-based versions * Updated rseqc/readdistribution: Added BAI support per nf-core#9460 & topics-based versions * Updated rseqc/readduplication: Added BAI support per nf-core#9460 & topics-based versions * Updated rseqc/tin: Added topics-based versions * Updated bam_rseqc subworkflow - BAI support, channel syntax fix - topics-based versions * Failed snapshot update - linting error * Snapshot Update & Channel syntax fix * Channel syntax update * Test & Snapshot Updated - version assertion to circumvent nf-core tools potential bug * Fixed null entries in snapshot * Updated subworkflow assertions on optional files (& snapshot) - migration of versions channels to topics * Linting failures fixes - dedicated versions assertions
) - Add SEQKIT_STATS output to multiqc_files in fastq_remove_rrna (when using ribodetector for rRNA removal) - Add umi_log output to multiqc_files in fastq_qc_trim_filter_setstrandedness for both trimgalore and fastp paths 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* adding uncompression functionality * new versions output syntax * meta.yml updated
* Added fq2bam * test fix * typo * PE channel * tidying * gpu snap * test fix for snaps * gpu snapshot * PR feedback * tests * cpu snapshot * tests * last snapshot * fix channels * last snapshot * PR feedback * PR feedback * new cpu snapshot * changing test * changing test * new gpu snapshot * [automated] Update gpu snapshot * PR feedback * PR feedback * PR feedback * PR feedback * PR feedback * PR feedback * PR feedback * PR feedback * PR feedback * PR feedback * PR feedback * containerOptions back * genome test name * genome test name * wrong comment * fix tests * new snapshot * Add use_gpu channel * new snapshot * [automated] Fix linting with Prettier * fix lint * Remove containerOptions --------- Co-authored-by: root <root@ip-172-31-44-88.us-west-2.compute.internal> Co-authored-by: root <root@ip-172-31-20-12.us-west-2.compute.internal> Co-authored-by: nf-core-bot <core@nf-co.re> Co-authored-by: Joon Klaps <joon.klaps@kuleuven.be> Co-authored-by: Eduard Casas <eduard.casas@WMG-FN0Q7KY4X6.local>
….yml (nf-core#9588) Replace biocontainers with Seqera Wave containers built from the exact environment.yml specifications. This ensures conda and container environments match. Key changes: - Pin r-base=4.3 in all RSEQC environment.yml files (r-base=3.5 has no ARM build available) - Use Wave-generated containers built from environment.yml for both Docker and Singularity - Standardize all 9 RSEQC modules to use identical dependencies The previous biocontainers (rseqc:5.0.4--pyhdfd78af_1) were not built with the R pin specified in environment.yml, causing inconsistency between conda and container execution modes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Allow gzipped inputs to modkit/bedmethyltobigwig * Update modules/nf-core/modkit/bedmethyltobigwig/main.nf Co-authored-by: Felix Kummer <felix.kummer@hu-berlin.de> * harshil --------- Co-authored-by: Felix Kummer <felix.kummer@hu-berlin.de>
* feat(ribowaltz): Export underlying data from QC plots Add TSV export for data underlying P-site region and frame distribution plots. This makes the raw data available for downstream analysis and reporting tools like MultiQC. Changes: - Export psite_region.tsv from region_psite() in save_psite_region_plot - Export frames.tsv from frame_psite() in save_frame_plots - Export frames_stratified.tsv from frame_psite_length() in save_frame_plots - Add ribowaltz_qc_data output channel for TSV files - Update stub outputs to include new TSV files - Add test assertions for new output channel 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore(ribowaltz): Update meta.yml with ribowaltz_qc_data output 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat(ribowaltz): Export underlying data for all QC plots Add TSV data exports for all riboWaltz QC plots to enable downstream processing and MultiQC custom content integration: - length_bins_for_psite.tsv (P-site offset identification bins) - ends_heatmap.tsv (read extremities heatmap) - codon_usage.tsv (codon usage barplot) - length_distribution.tsv (read length distribution) - metaprofile_psite.tsv (P-site metaprofile around start/stop codons) These complement existing exports (psite_region, frames, frames_stratified). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(ribowaltz): Use correct key for ends_heatmap data export The rends_heat function returns data in 'count_dt' not 'dt'. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Swap bcftools/mpileup to topic * Fix linting * Update snapshots
* Swap mpileup to topics * Apply suggestions from code review * Add Harshil back as author * Update snapshots
* Fix string path * Update subworkflows/nf-core/vcf_impute_beagle5/main.nf Co-authored-by: Felix Lenner <52530259+fellen31@users.noreply.github.com> --------- Co-authored-by: Felix Lenner <52530259+fellen31@users.noreply.github.com>
* Bump genmod to 3.10.2 and switch to topics * Add md5sum for stable subcommands
nf-core#9606) * Modify output files in FASTP module to use 'R1' and 'R2' conventions rather than just '1' or '2' * Update testing snapshots --------- Co-authored-by: Tatiana Gelaf Romer <romert01@pfizer.com>
* Update MLST from 2.23 to 2.25 * test: update snapshots for mlst 2.25 --------- Co-authored-by: Subham Verma <subhamverma@S.local>
…9586) Update PR template check list item WRT version numbers Updating the PR template per the blog: ``` We are adopting topic channels to replace the versions.yml. For now both options are valid, but this will change. No more mixing of version channels. ``` Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com>
* Start of topics conversion * Swap ngscheckmate to topic * Remove trailing whitespace * Update snapshots * Update test * Update snapshots * Skip two submodules on conda * Address comments * Update snapshot * Update snapshot --------- Co-authored-by: Louis Le Nézet <58640615+LouisLeNezet@users.noreply.github.com>
* Update samtools dict/faidx/merge * Update subworkflows * Update meta * Update snapshots * Prettier * Add config back * Add config * Prettier
* Allow `dream` to process non-formula contrasts * Add test case without formula * Update `meta.yml` * Fix test with unstable file * Update subworkflow to allow non-formula contrasts with DREAM * Update modules/nf-core/variancepartition/dream/templates/dream.R Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update modules/nf-core/variancepartition/dream/templates/dream.R Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Correct format of function * Fix contrast building and code flow * Update tests * Modify logic for contrast_string building * Make contrast_string compulsory with formula * Update tests * Refactor contrast_string construction logic Simplified the construction of contrast_string by removing unnecessary checks for treatment levels in the design matrix. * Make formula mutually exclusive to contrast tuple * Modify tests with mutually exclusive varaibles * Revert changes in first test case * Update tests * Fix tests --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Add vcfpgloader/load module High-throughput VCF to PostgreSQL loader using asyncpg for bulk variant ingestion. * Remove throughput metric from meta.yml description * fix(vcfpgloader): fix CI failures for conda, singularity, and docker - Add pip as explicit conda dependency in environment.yml - Simplify container directive (remove oras:// for singularity compatibility) - Remove integration test requiring PostgreSQL database 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: re-trigger CI for bioconda 0.5.3 propagation * chore: reorder deps to bust conda cache for 0.5.3 * fix: specify exact bioconda build string to force 0.5.3 * chore: sort environment.yml dependencies (linter) * fix: remove version from snapshot (varies by profile) * fix(vcfpgloader): restore version checking in tests Add back versions to snapshot assertions and update snapshot file with expected version output format for lint compliance. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * fix(vcfpgloader): use correct findAll pattern for topic versions Use process.out.findAll { key, val -> key.startsWith("versions")} pattern for topic-based versioning, matching nf-core conventions. Update snapshot with versions_vcfpgloader object format. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * feat(vcfpgloader): bump to v0.5.4 - Update environment.yml to bioconda::vcf-pg-loader=0.5.4 - Update container to ghcr.io/zacharyr41/vcf-pg-loader:0.5.4 - Add sed to parse version number from --version output - Update snapshot for 0.5.4 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * ci: retrigger after bioconda 0.5.4 merge * fix(vcfpgloader): update meta.yml version format Add type: eval and description to version command entries in outputs and topics sections. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * style(vcfpgloader): apply nextflow lint formatting * fix(vcfpgloader): add export to env() output variables Add explicit `export` to ROWS_LOADED assignments as recommended for env() output qualifiers. Note: `nextflow lint` still reports false positive errors for env() outputs - this is a known limitation of static analysis on shell scripts. The linter cannot verify variable definitions inside heredoc script blocks. Other nf-core modules (e.g., genescopefk) have the same lint warning. The code functions correctly at runtime. * Update modules/nf-core/vcfpgloader/load/tests/nextflow.config Co-authored-by: Famke Bäuerle <45968370+famosab@users.noreply.github.com> * chore(vcfpgloader): remove unused tags.yml * Update modules/nf-core/vcfpgloader/load/tests/main.nf.test Co-authored-by: Famke Bäuerle <45968370+famosab@users.noreply.github.com> * Update modules/nf-core/vcfpgloader/load/tests/main.nf.test Co-authored-by: Famke Bäuerle <45968370+famosab@users.noreply.github.com> * refactor(vcfpgloader): consolidate inputs into single tuple * feat(vcfpgloader): switch to BioContainers - Use BioContainers URLs instead of personal ghcr.io - Remove jq dependency, use Python for JSON parsing - Fix sed quoting for nf-core lint compatibility 🤖 Generated with [Claude Code](https://claude.ai/code) * fix(vcfpgloader): update snapshots to include row_count output Add missing row_count and log outputs to test snapshots to match the process outputs being asserted in tests. --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Zachary Rothstein <zacharyr41@users.noreply.github.com> Co-authored-by: Famke Bäuerle <45968370+famosab@users.noreply.github.com>
Replace busybox with coreutils sort
|
Really sorry for the mess above :( I think I messed up during rebase. |
PR checklist
This is a pull request to include the submodule t1k/run. Here is the tool repository: https://github.com/mourisl/T1K
versions.ymlfile.labelnf-core modules test <MODULE> --profile dockernf-core modules test <MODULE> --profile singularitynf-core modules test <MODULE> --profile condanf-core subworkflows test <SUBWORKFLOW> --profile dockernf-core subworkflows test <SUBWORKFLOW> --profile singularitynf-core subworkflows test <SUBWORKFLOW> --profile conda