The genomeqc template follows an nf-core/genomeqc run from the assemblies as they are to the genes the genomes share:
- Overview: the run in four key figures, live findings and four figures, each linked to the tab that explains it
- Data & QC: the MultiQC report, how contiguous each assembly is, and whether it carries foreign or adaptor sequence
- Genomes: BUSCO completeness, a karyotype of conserved genes and telomeric repeats, repeat content, gene models, and the orthogroups the genomes share
The persistent Sample filters (the group and the genome) sit in the left panel
and narrow every tab through the genome hub, one row per assembly.
The design table is optional
The samplesheet is not published and has no design column, so the design is a
table you pass (METADATA_FILE: the assembly name in a sample column, every
other column a factor such as a family). Without it the group filters and the
group split fall away and the scatters draw in one colour.
Quick start¶
depictio ingest /path/to/genomeqc_results \
--template nf-core/genomeqc/latest \
--var METADATA_FILE=/path/to/genome_design.tsv
The results directory is the only required value.
| Variable | Default | Role |
|---|---|---|
METADATA_FILE |
unset | design table, one row per assembly; without it the group filter, colours and splits fall away |
GROUP_COL |
first factor column of the design table | column the dashboard groups and filters genomes by |
GROUP_COL_DISPLAY |
title-cased GROUP_COL |
reader-facing label of GROUP_COL in filters and cards |
depictio config nextflow --install # once per machine
nextflow run nf-core/genomeqc -r 1.0.1 -profile docker --outdir results
No depictio ingest, and no template named: the pipeline ingests its own
output directory when it finishes and resolves this template from its own
manifest. See Nextflow trigger. The
trigger passes no --var, so the run has no design table.
Reference¶
The template reads the summary tables of each step: QUAST, BUSCO (assembly and
proteins), tidk, AGAT, the pipeline's gene overlap counts, OrthoFinder's
statistics, FCS-GX, FCS-adaptor, Tiara, RepeatMasker, and the native MultiQC
report, which holds QUAST only. It never reads the masked FASTAs, the RepeatMasker
.out tables or the tens of thousands of per-orthogroup files OrthoFinder writes.
QUAST and MultiQC label an assembly after its FASTA, every other tool after the
assembly name, and the hub joins the two.
Every tool collection but QUAST is optional, and a tool that did not run drops what depends on it:
| Run | What goes |
|---|---|
| No annotation in the samplesheet | the Annotation and Orthology tabs, with the Annotate and Compare steps |
--skip_tidk |
the Karyotype tab (sequence lengths come from tidk) |
| No FCS-GX database | the Contamination tab and the Screen step |
No --te repeatmasker |
the Repeats tab and the Mask step |
--skip_busco |
the Completeness and Karyotype tabs |
Self-adapting layout
The dashboard adapts to whatever the run actually produced: components bound to pruned or unparsed data collections are hidden, tabs left with no real visualizations are dropped entirely, and the remaining components are re-packed so there are no empty rows.
Template variables¶
Variables you provide when running the template: DATA_ROOT is the results directory given to depictio ingest, the others are set with --var NAME=value:
| Variable | Required | Description |
|---|---|---|
DATA_ROOT |
✓ | Root directory of one genomeqc run (quast/, busco/, tidk/, agat/, orthofinder/, decontamination/, multiqc/, pipeline_info/). |
METADATA_FILE |
— | Optional design table (TSV): assembly name in a column named sample (or in the first column), one column per factor such as a family or a clade. Joined into the genome hub: enables the design filters, the group colours and the group splits. |
METADATA_ID_COL |
— | Design table assembly-name column. Auto-detected as the first column; defaults to 'sample'. |
GROUP_COL |
— | Design column the dashboards group and filter genomes by. Auto-detected as the first factor column of METADATA_FILE. |
Auto-detected (set from the run's metadata / params.json; the route flags drive Conditional routes below): GROUP_COL_DISPLAY, ANNOTATION_COLS
Data collections¶
25 data collections — 5 required 20 optional · 18 direct 7 derived.
Origin tells you whether a collection is real pipeline data or a reshape of it: direct = a pipeline output (scanned, or a recipe that reads raw files); derived = a recipe that reshapes one or more direct collections into the layout a visualization needs (no new measurement). Reads shows what produces it: a recipe .py transform, or a raw file scanned off disk. (A direct collection can still have a recipe — one that merely parses/cleans the raw file; derived means the recipe reshapes another collection.)
| Tag | Origin | Type | Reads | Status |
|---|---|---|---|---|
metadata |
direct | Table | file |
optional |
multiqc_data |
direct |  MultiQC |
file |
required |
quast_assembly_raw |
direct | Table | file |
required |
quast_assembly_report |
derived | Table | recipe |
required |
quast_length_ladder |
derived | Table | recipe |
required |
busco_batch_raw |
direct | Table | file |
optional |
busco_batch_summary |
derived | Table | recipe |
optional |
busco_composition |
derived | Table | recipe |
optional |
tidk_search_windows |
direct | Table | recipe |
optional |
busco_karyotype |
derived | Table | recipe |
optional |
agat_statistics |
direct | Table | recipe |
optional |
agat_gene_models |
direct | Table | recipe |
optional |
gene_overlaps |
direct | Table | recipe |
optional |
orthofinder_species_stats |
direct | Table | recipe |
optional |
orthofinder_orthogroup_matrix |
direct | Table | recipe |
optional |
orthofinder_orthologue_matrix |
direct | Table | recipe |
optional |
ortho_seq_count |
direct | Table | file |
optional |
fcsgx_contamination |
direct | Table | file |
optional |
fcsgx_actions |
direct | Table | recipe |
optional |
fcsadaptor_actions |
direct | Table | recipe |
optional |
tiara_classifications |
direct | Table | recipe |
optional |
tiara_class_counts |
derived | Table | recipe |
optional |
repeatmasker_repeat_classes |
direct | Table | recipe |
optional |
repeatmasker_masking_summary |
direct | Table | recipe |
optional |
genomes |
derived | Table | recipe |
required |
Conditional routes¶
Rows are data collections; columns are the variables you set or params.json flags auto-detected from the run. Each filled cell is the effect of setting that variable; an empty cell means that variable leaves the collection unchanged. (24 collections are unaffected by any variable — present on every run.)
+ included− removed⇄ repointed
| Data collection | METADATA_FILE |
|---|---|
metadata |
+ |
Cross-DC links¶
26 links — selecting a value in the source collection filters the target. The join column is shown after the source.
| Source · column | Target | Filters | |
|---|---|---|---|
genomes · assembly_label |
→ | multiqc_data |
Filter the MultiQC panels by the genomes picked in the left panel |
genomes · assembly_label |
→ | quast_assembly_report |
Filter the QUAST statistics by the genomes picked in the left panel |
genomes · assembly_label |
→ | quast_length_ladder |
Filter the length ladder by the genomes picked in the left panel |
genomes · genome |
→ | busco_batch_summary |
Filter the BUSCO scores by the genomes picked in the left panel |
genomes · genome |
→ | busco_composition |
Filter the BUSCO classes by the genomes picked in the left panel |
genomes · genome |
→ | busco_karyotype |
Filter the BUSCO karyotype by the genomes picked in the left panel |
genomes · genome |
→ | tidk_search_windows |
Filter the telomeric repeat windows by the genomes picked in the left panel |
genomes · genome |
→ | agat_statistics |
Filter the AGAT statistics by the genomes picked in the left panel |
genomes · genome |
→ | agat_gene_models |
Filter the gene models by the genomes picked in the left panel |
genomes · genome |
→ | gene_overlaps |
Filter the gene overlaps by the genomes picked in the left panel |
genomes · genome |
→ | orthofinder_species_stats |
Filter the orthogroup assignment by the genomes picked in the left panel |
genomes · genome |
→ | orthofinder_orthologue_matrix |
Keep the rows of the orthologue matrix of the genomes picked in the left panel |
genomes · genome |
→ | fcsgx_actions |
Filter the FCS-GX calls by the genomes picked in the left panel |
genomes · genome |
→ | fcsadaptor_actions |
Filter the FCS-adaptor calls by the genomes picked in the left panel |
genomes · genome |
→ | tiara_classifications |
Filter the Tiara classes by the genomes picked in the left panel |
genomes · genome |
→ | tiara_class_counts |
Filter the Tiara class counts by the genomes picked in the left panel |
genomes · genome |
→ | repeatmasker_repeat_classes |
Filter the repeat classes by the genomes picked in the left panel |
genomes · genome |
→ | repeatmasker_masking_summary |
Filter the masking summary by the genomes picked in the left panel |
genomes · genome |
→ | fcsgx_contamination |
Filter the foreign share by the genomes picked in the left panel |
genomes · genome |
→ | ortho_seq_count |
Filter the orthologue-carrying sequences by the genomes picked in the left panel |
quast_assembly_report · assembly_id |
→ | genomes |
Carry the assemblies kept by the Assembly tab's QUAST sliders to the hub and the length ladder |
fcsgx_actions · genome |
→ | genomes |
Carry the genomes holding the picked FCS-GX action to the hub, the foreign share and the adaptor calls |
tiara_classifications · genome |
→ | genomes |
Carry the genomes holding the picked Tiara class to the hub, the foreign share and the FCS calls |
tiara_classifications · tiara_class |
→ | tiara_class_counts |
Carry the Tiara class pick to the class counts |
busco_batch_summary · assembly_id |
→ | genomes |
Carry the assemblies kept by the Completeness tab's BUSCO slider to the hub and the BUSCO classes |
busco_karyotype · sequence |
→ | tidk_search_windows |
Carry the sequences kept by the Karyotype tab's length slider to the telomeric repeat windows |
Recipes¶
Each recipe reshapes raw pipeline output into a tidy table. The name links to its source; Output lists the validated OUTPUT_SCHEMA columns.
| Recipe | Transforms | Output |
|---|---|---|
agat/gene_models.py |
The gene models of each annotation, one row per assembly and feature type. | genome, feature_type, genes, transcripts, transcripts_per_gene, exons, exons_per_transcript, single_exon_genes, single_exon_gene_pct, overlapping_genes, mean_gene_length, median_gene_length, mean_transcript_length, mean_exon_length, mean_intron_length, mean_cds_length, total_gene_length |
agat/statistics.py |
Every number of an AGAT agat_sp_statistics.pl report, as a long table. |
genome, feature_type, isoforms, metric, value, metric_order |
busco/batch_summary.py |
One row per assessed assembly, from BUSCO's batch_summary.txt. |
assembly_id, input_file, lineage, busco_mode, complete_pct, single_pct, duplicated_pct, fragmented_pct, missing_pct, n_markers, stop_codon_pct, scaffold_n50, contig_n50, percent_gaps, n_scaffolds |
busco/composition.py |
BUSCO scores as four classes per assembly, for a composition bar. | assembly_id, lineage, busco_mode, run_id, busco_class, rank, class_order, percent |
fcsadaptor/actions.py |
Adaptor and vector calls of FCS-adaptor, from its *.fcs_adaptor_report.txt. |
genome, seq_id, seq_len, action, start, end, flagged_bp, adaptor, source_type |
fcsgx/actions.py |
Contamination calls of FCS-GX, from its *.fcs_gx_report.txt. |
genome, seq_id, start, end, seq_len, action, division, division_group, contaminant_cov, top_taxon, flagged_bp |
nf-core/genomeqc/busco_karyotype.py |
Complete BUSCOs per sequence, drawn to scale: the nf-core/genomeqc karyotype. | genome, busco_mode, sequence, start, end, sequence_length, complete_buscos, buscos_per_mb, busco_share_pct, sequence_rank |
nf-core/genomeqc/gene_overlaps.py |
Overlapping genes per annotation, from nf-core/genomeqc's GENEOVERLAPS step. | genome, total_genes, unresolved_strand_genes, contained_same_strand, contained_opposite_strand, overlapping_genes, overlapping_pct, contained_same_strand_pct, not_overlapping_genes, overlapping_not_nested |
nf-core/genomeqc/genomes.py |
The nf-core/genomeqc genome hub: one row per assembly, every per-genome number. | genome, assembly_label, total_length, n_contigs, largest_contig, n50, n90, l50, l90, gc_percent, ns_per_100_kbp, busco_lineage, busco_complete_pct, busco_single_pct, busco_duplicated_pct, busco_fragmented_pct, busco_missing_pct, protein_complete_pct, protein_duplicated_pct, protein_missing_pct, sequences_with_busco, repeat_unit, genes, transcripts, transcripts_per_gene, exons_per_transcript, single_exon_gene_pct, mean_gene_length, mean_cds_length, overlapping_pct, contained_same_strand, genes_in_orthogroups_pct, unassigned_genes, species_specific_orthogroups, syntenic_sequences, contaminant_pct, gx_flagged_ranges, gx_flagged_bp, adaptor_ranges, tiara_sequences, prokaryotic_sequences, unknown_sequences, masked_pct, interspersed_pct |
nf-core/genomeqc/tiara_class_counts.py |
Sequences per Tiara class and assembly, from the per-sequence Tiara table. | genome, tiara_class, sequences, share_pct |
orthofinder/orthogroup_matrix.py |
Which genomes each orthogroup reaches, from Orthogroups.GeneCount.tsv. |
orthogroup, n_genomes, total_genes, copy_class, sharing |
orthofinder/orthologue_matrix.py |
One-to-one orthologues between every pair of species, as a square matrix. | genome |
orthofinder/species_stats.py |
How much of each proteome OrthoFinder placed, from Statistics_PerSpecies.tsv. |
genome, genes, genes_in_orthogroups, unassigned_genes, genes_in_orthogroups_pct, unassigned_genes_pct, orthogroups, orthogroups_pct, species_specific_orthogroups, genes_in_species_specific, genes_in_species_specific_pct, genes_in_shared_orthogroups |
quast/assembly_report.py |
One row per assembly, from QUAST's transposed report. | assembly_id, assembler, sample, n_contigs, total_length, largest_contig, n50, n75, l50, l75, gc_percent, ns_per_100_kbp, predicted_rrna_genes, contigs_over_1kb, length_over_1kb |
quast/length_ladder.py |
The contig-length ladder of each assembly, one row per length threshold. | assembly_id, assembler, sample, min_contig_length, n_contigs, total_length, length_fraction |
repeatmasker/masking_summary.py |
How much of each assembly RepeatMasker masked, from the header of its .tbl. |
genome, sequences, total_length, length_excl_n, gc_pct, masked_bp, masked_pct, interspersed_pct, retroelements_pct, dna_transposons_pct, unclassified_pct, simple_repeats_pct |
repeatmasker/repeat_classes.py |
Repeat classes and families of each assembly, from RepeatMasker's .tbl. |
genome, repeat_class, parent_class, rank, class_order, elements, length_bp, genome_pct |
tiara/classifications.py |
Tiara's class for every sequence of an assembly. | genome, stage, sequence, class_fst_stage, class_snd_stage, tiara_class, is_eukaryotic |
tidk/search_windows.py |
Telomeric repeats per window along every sequence, from tidk search. |
genome, sequence, start, end, forward_repeats, reverse_repeats, repeats, repeat_unit, search, sequence_length |
Dashboard tabs¶
One dashboard: the Overview, then eight child tabs in two groups, read as a funnel from the assemblies to the genes the genomes share. Each tab below carries the same icon and colour the dashboard gives it, so the page and the app read alike.
| Group | Tabs |
|---|---|
| Data & QC | MultiQC, Assembly, Contamination |
| Genomes | Completeness, Karyotype, Repeats, Annotation, Orthology |
Each child tab opens with a short intro and a strip of four cards, then at most three open sections; tables follow, collapsed. The Sample sheet (the genome hub) is pinned, collapsed, to the bottom of every child tab.
Genome assembly and annotation quality, from contiguity to the orthogroups the genomes share.
The run lists the genome count, the BUSCO lineage, the annotation validator, the OrthoFinder version and the repeat annotation, and Pipeline walks six steps (measure, screen, mask, complete, annotate, compare), each linked to its parameters and its tab. Four key figures: the genomes, the N50, the complete BUSCOs and the N bases per 100 kb. The findings are live rows plus four figures: contiguity, the BUSCO classes, the gene models and each genome's genes by orthogroup.
Data & QC · What does the MultiQC report say about the assemblies?
MultiQC panels only: QUAST's assembly statistics, the sequences in QUAST's
length bins and the general statistics. A Report scope filter narrows the
panels to report rows, and the pinned genome filters reach them through the
FASTA label.
Data & QC · How contiguous is each assembly?
The N50, the L50, the assembly length and the GC content. Then the sequences against the N50 of every assembly on log axes, sized by length and coloured by group, with the record of the picked genome beside it, and the share of each assembly kept above QUAST's minimum lengths. Filters: N50 and sequence count.
Data & QC · Does any assembly carry foreign or adaptor sequence?
The highest foreign share FCS-GX reports, the bases it flags, the adaptor bases
and the sequences Tiara classified. Then the foreign share per genome beside
Tiara's classes per genome, both stacked bars whose axis starts near the
smallest bottom share (a // marks the cut; the Zoom switch shows the whole
bar). The FCS-GX, FCS-adaptor and Tiara tables are collapsed.
Genomes · How much of the conserved gene set does each genome hold?
The complete BUSCOs in the assembly (a threshold at 95%) and in the proteins, the duplicated ones and the four classes summed. Then one stacked bar per BUSCO run, each assembly beside its proteins, and the assembly score against the protein score per genome. Filter: complete BUSCOs.
Genomes · Where do the conserved genes and telomeric repeats sit?
The sequences holding a BUSCO, the BUSCOs placed, the BUSCOs per Mb and the 10 kb windows holding the tidk repeat. Then two ideograms, one karyotype per genome each, the 24 longest sequences drawn to their length: the first shaded by complete BUSCOs per Mb, the second by copies of the searched repeat per 10 kb window. A genome picker sits above the chart, and a click on a bar opens it zoomed. genomeqc publishes no gene positions, so a whole sequence is shaded, not single genes.
Genomes · How much of each genome is repeats, and of which kinds?
The masked share, the interspersed share, the simple repeats and the elements found. Then the share of each assembly that every repeat class takes; the rank switch opens the families. The masking summary and every class and family line are collapsed. Filter: repeat class.
Genomes · How many genes does each annotation hold, and how are they built?
The protein-coding genes, the exons per mRNA, the single-exon genes and the overlapping genes. Then every transcript type of every annotation by genes and exons per transcript, and the genes per type beside each annotation's genes split by overlap (none, another gene, nested on the opposite or the same strand). The gene models per type and every AGAT statistic are collapsed.
Genomes · Which orthogroups do the genomes share?
The orthogroups by copy class, the core every genome shares, the share of genes placed for the genome with the fewest, and the orthogroups of one genome. Then each genome's genes in shared, genome-specific or no orthogroups, an UpSet of the genomes each orthogroup reaches, and the one-to-one orthologues of every pair of genomes. Filter: copy class.
A pick on a genome reaches every per-genome collection through the project links.
Running the pipeline¶
Depictio reads the output of nf-core/genomeqc, it does not run the pipeline. Run the pipeline first:
nextflow run nf-core/genomeqc -r 1.0.1 \
--input samplesheet.csv \
--te repeatmasker \
--outdir results \
-profile docker
A tool that did not run leaves its tab out (see the table above): the
Contamination tab needs the FCS-GX database (--gxdb and --gxdb_manifest) and
the Repeats tab needs --te repeatmasker. Then point Depictio at the results:
depictio ingest results/ \
--template nf-core/genomeqc/latest \
--var METADATA_FILE=genome_design.tsv
See nf-co.re/genomeqc/usage for full pipeline documentation.
Required data structure¶
Point depictio ingest at the directory holding the pipeline output. Depictio
scans recursively and matches on file name.
<DATA_ROOT>/
├── input/genome_metadata.tsv # --var METADATA_FILE (optional design table)
├── pipeline_info/
├── multiqc/multiqc_data/multiqc.parquet # QUAST only
├── quast/<a>/<a>.quast/transposed_report.tsv # required
├── busco/{genome,protein}/<a>/stats/*-busco.batch_summary.txt
├── busco/<mode>/<a>/ideogram/*.csv
├── tidk/<a>/search/<kind>/<a>.tsv
├── agat/<a>/stats/<a>.stats.txt
├── gene_overlaps/<a>.counts.tsv
├── orthofinder/ # summary tables only
├── ortho_seq_count/species_ortho_seq_count.tsv
├── fcsgx/fcs_table.tsv
├── decontamination/{fcs-gx,fcs-adaptor,tiara}/
└── repeatmasker/<a>/<a>.tbl
Validation runs¶
The repository ships
download_test_data.sh,
which fetches the subset of nf-core's AWS megatest run that the template needs
(files over 50 MB are skipped) and copies the vendored design table into input/:
bash depictio/projects/nf-core/genomeqc/1.0.1/download_test_data.sh \
/tmp/genomeqc_test
depictio ingest /tmp/genomeqc_test \
--template nf-core/genomeqc/latest \
--var METADATA_FILE=/tmp/genomeqc_test/input/genome_metadata.tsv
The run is
s3://nf-core-awsmegatests/genomeqc/results-bce44e5b558e21b38d4dd2b6521302246b4dae6d/,
the 1.0.1 test_full run: three Hymenoptera RefSeq assemblies with their
annotations, the hymenoptera_odb10 BUSCO lineage, FCS-GX on its test-only
database and RepeatMasker with RepeatModeler. The FCS-GX database is a smoke
test, so its contamination shares are not a real screen.
Additional resources¶
- nf-co.re/genomeqc: official pipeline documentation
- nf-co.re/genomeqc/results: AWS test results
- Template System Reference: YAML format, variables, conditionals
- Recipes: how to read, test, and write recipes
Authorship¶
weber8thomas
Reviewing a template on your own data, or taking over a role here, is a contribution in itself: the contributing guide says what each one involves.