AIRR Repertoire Analysis

Template version latest

The airrflow template covers the reporting half of a standard nf-core/airrflow run, from the raw reads through to the clones:

Overview: the run in four key figures, live findings and four figures, each linked to the tab that explains it
Data & QC: fastp trimming and both FastQC runs from the pipeline's MultiQC report, and every read followed through pRESTO and Change-O
Repertoire: V gene usage at family and gene resolution, CDR3 lengths and V by J pairing
Clonality: Hill diversity profiles, clonal expansion and homeostasis, and the clones samples of one subject share

No external metadata file

Everything the template reads comes from the run itself: the validated samplesheet the pipeline writes to pipeline_info/samplesheet.valid.tsv is the hub data collection, so there is nothing to prepare. The one column to name is the condition, GROUP_COL, which defaults to treatment, the column airrflow's samplesheet schema documents for it; a sample sheet that names it differently passes --var GROUP_COL=<column>. 5.1.1 keeps the 5.1.0 output layout and has no AWS megatest run: the 5.1.1 template was validated on an EMBL HPC run of the release, and the 5.1.0 megatest still serves to try it.

Quick start¶

Point at a finished runFrom the pipeline itself (v1.10.0+)

depictio ingest /path/to/airrflow_results \
  --template nf-core/airrflow/latest

The results directory is the only thing you have to pass. The samplesheet is picked up from {DATA_ROOT}/pipeline_info/samplesheet.valid.tsv; pass --var SAMPLESHEET_FILE=... to point somewhere else.

The condition filter reads the samplesheet column named by GROUP_COL (default treatment, labelled Condition). A samplesheet that keeps the condition in another free column names it, and can relabel it:

depictio ingest /path/to/airrflow_results \
  --template nf-core/airrflow/latest \
  --var GROUP_COL=intervention \
  --var GROUP_COL_DISPLAY=Intervention

Variable	Default	Meaning
`DATA_ROOT`	required	Root of the airrflow output directory
`SAMPLESHEET_FILE`	`{DATA_ROOT}/pipeline_info/samplesheet.valid.tsv`	The validated samplesheet, the hub data collection
`GROUP_COL`	`treatment`	Samplesheet column holding the condition; must exist in the samplesheet
`GROUP_COL_DISPLAY`	`Condition`	Reader-facing label of that column in filter titles
`SKIP_CLONAL_ANALYSIS`, `SKIP_REPORT`, `SKIP_THRESHOLD_REPORT`, `SKIP_MULTIQC`, `ASSEMBLED_MODE`	unset	Mirror the run's route flags; see Conditional routes below

depictio config nextflow --install     # once per machine
nextflow run nf-core/airrflow -r 5.1.0 -profile docker --outdir results

No depictio ingest, and no template named: the pipeline ingests its own output directory when it finishes and resolves this template from its own manifest. See Nextflow trigger.

Reference¶

The template reads the MultiQC parquet, the two sequence-count logs, the V usage tables from the repertoire comparison report, and the enchantR clonal analysis tables (diversity, clone sizes, pairwise overlap, distance threshold). Most tiles are catalog renders bound to enchantr, airrflow's own R package rather than an nf-core module, so the tile chrome names where each panel comes from.

The route flags in the Conditional routes table are not read back from params.json yet, so a run started with --skip_clonal_analysis, --skip_report, --skip_report_threshold, --skip_multiqc or --mode assembled needs the matching --var SKIP_...=true (or --var ASSEMBLED_MODE=true). Omitting one still ingests: the affected collections are optional and simply come up empty.

Self-adapting layout

The dashboard adapts to whatever the run actually produced: components bound to pruned or unparsed data collections are hidden, tabs left with no real visualizations are dropped entirely, and the remaining components are re-packed so there are no empty rows.

Template variables¶

Variables you provide when running the template: DATA_ROOT is the results directory given to depictio ingest, the others are set with --var NAME=value:

Variable	Required	Description
`DATA_ROOT`	✓	Root directory containing airrflow pipeline output (multiqc/, clonal_analysis/, repertoire_comparison/, parsed_logs/)
`SAMPLESHEET_FILE`	—	Path to the validated samplesheet. Defaults to {DATA_ROOT}/pipeline_info/samplesheet.valid.tsv, which every airrflow run writes.
`SKIP_CLONAL_ANALYSIS`	—	Set when the run used --skip_clonal_analysis (params.json skip_clonal_analysis). Prunes every clonal_analysis/ derived DC.
`SKIP_REPORT`	—	Set when the run used --skip_report (params.json skip_report). Prunes the repertoire_comparison/ and parsed_logs/ derived DCs: the pRESTO log parser runs inside the report step.
`SKIP_THRESHOLD_REPORT`	—	Set when the run used --skip_report_threshold, or an explicit --clonal_threshold instead of 'auto'. Prunes the threshold summary DC.
`SKIP_MULTIQC`	—	Set when the run used --skip_multiqc (params.json skip_multiqc). Prunes the MultiQC DC, which drops the MultiQC tab; the other tabs are unaffected.
`ASSEMBLED_MODE`	—	Set when the run used --mode assembled (params.json mode). Prunes the pRESTO log DCs: assembled input skips read pre-processing entirely.
`GROUP_COL`	—	Samplesheet column holding the condition the dashboard filters and breaks down by (default: treatment). Must exist in pipeline_info/samplesheet.valid.tsv.
`GROUP_COL_DISPLAY`	—	Reader-facing label of GROUP_COL in filter titles and descriptions (default: Condition).

Data collections¶

20 data collections — 13 required 7 optional · 20 direct 0 derived.

Origin tells you whether a collection is real pipeline data or a reshape of it: direct = a pipeline output (scanned, or a recipe that reads raw files); derived = a recipe that reshapes one or more direct collections into the layout a visualization needs (no new measurement). Reads shows what produces it: a recipe .py transform, whose name links to its source on GitHub, or a raw file scanned off disk. (A direct collection can still have a recipe — one that merely parses/cleans the raw file; derived means the recipe reshapes another collection.)

Tag	Origin	Type	Reads	Status
`multiqc_data`	direct	MultiQC	file `(?:.*/)?multiqc(?:/[^/]+)?/multiqc_data/multiqc.parquet$`	required
`samplesheet`	direct	Table	file `{DATA_ROOT}/pipeline_info/samplesheet.valid.tsv`	required
`sequence_counts`	direct	Table	recipe `enchantr/sequence_counts.py`	required
`sequence_fates`	direct	Table	recipe `enchantr/sequence_fates.py`	required
`annotation_counts_assembled`	direct	Table	recipe `nf-core/airrflow/annotation_counts.py`	required
`annotation_counts_noclonal`	direct	Table	recipe `nf-core/airrflow/annotation_counts.py`	required
`repertoire_summary`	direct	Table	recipe `enchantr/repertoire_summary.py`	required
`repertoire_summary_noreport`	direct	Table	recipe `enchantr/repertoire_summary.py`	required
`clonal_diversity`	direct	Table	recipe `enchantr/clonal_diversity.py`	optional
`diversity_orders`	direct	Table	recipe `enchantr/diversity_orders.py`	optional
`clonal_abundance`	direct	Table	recipe `enchantr/clonal_abundance.py`	optional
`clone_sizes`	direct	Table	recipe `enchantr/clone_sizes.py`	required
`clone_sets`	direct	Table	recipe `enchantr/clone_sets.py`	required
`clonal_overlap`	direct	Table	recipe `enchantr/clonal_overlap.py`	optional
`v_gene_usage`	direct	Table	recipe `enchantr/v_gene_usage.py`	required
`v_gene_matrix`	direct	Table	recipe `enchantr/v_gene_matrix.py`	required
`v_gene_usage_noclonal`	direct	Table	recipe `enchantr/v_gene_usage.py`	required
`cdr3_spectratype`	direct	Table	recipe `nf-core/airrflow/cdr3_spectratype.py`	optional
`vj_usage_matrix`	direct	Table	recipe `nf-core/airrflow/vj_usage_matrix.py`	optional
`threshold_summary`	direct	Table	recipe `enchantr/threshold_summary.py`	optional

Conditional routes¶

Rows are data collections; columns are the variables you set or params.json flags auto-detected from the run. Each filled cell is the effect of setting that variable; an empty cell means that variable leaves the collection unchanged. (0 collections are unaffected by any variable — present on every run.)

+ included− removed⇄ repointed

Data collection	`SAMPLESHEET_FILE`	`SKIP_CLONAL_ANALYSIS`	`SKIP_REPORT`	`SKIP_THRESHOLD_REPORT`	`SKIP_MULTIQC`	`ASSEMBLED_MODE`
`multiqc_data`					−
`samplesheet`	⇄
`sequence_counts`			−			−
`sequence_fates`			−			−
`annotation_counts_assembled`			−			+
`annotation_counts_noclonal`		+	−
`repertoire_summary`		−
`repertoire_summary_noreport`		−	+
`clonal_diversity`		−
`diversity_orders`		−
`clonal_abundance`		−
`clone_sizes`		−
`clone_sets`		−
`clonal_overlap`		−
`v_gene_usage`			−
`v_gene_matrix`			−
`v_gene_usage_noclonal`		+	−
`cdr3_spectratype`		−
`vj_usage_matrix`		−
`threshold_summary`		−		−

Cross-DC links¶

24 links — selecting a value in the source collection filters the target. The join column is shown after the source.

Source · column		Target	Filters
`samplesheet` · sample_id	→	`multiqc_data`	Filter the MultiQC panels by samplesheet samples
`samplesheet` · sample_id	→	`sequence_counts`	Filter the sequence processing counts by samplesheet samples
`samplesheet` · sample_id	→	`sequence_fates`	Filter the attrition flow by samplesheet samples
`samplesheet` · sample_id	→	`annotation_counts_assembled`	Filter the assembled-route annotation counts by samplesheet samples
`samplesheet` · sample_id	→	`annotation_counts_noclonal`	Filter the no-clonal-route annotation counts by samplesheet samples
`samplesheet` · sample_id	→	`v_gene_usage_noclonal`	Filter the no-clonal-route V gene usage by samplesheet samples
`samplesheet` · sample_id	→	`repertoire_summary_noreport`	Filter the no-report-route repertoire summary by samplesheet samples
`samplesheet` · sample_id	→	`repertoire_summary`	Filter the repertoire summary by samplesheet samples
`samplesheet` · sample_id	→	`clonal_diversity`	Filter the diversity profiles by samplesheet samples
`samplesheet` · sample_id	→	`diversity_orders`	Filter the ranked diversity bars by samplesheet samples
`samplesheet` · sample_id	→	`clone_sizes`	Filter the clone size table by samplesheet samples
`samplesheet` · sample_id	→	`v_gene_usage`	Filter the V gene usage by samplesheet samples
`samplesheet` · sample_id	→	`cdr3_spectratype`	Filter the CDR3 spectratypes by samplesheet samples
`samplesheet` · subject_id	→	`vj_usage_matrix`	Filter the V-J pairing grid by samplesheet donors
`samplesheet` · sample_id	→	`clonal_overlap`	Filter the clonal overlap matrix by samplesheet samples
`samplesheet` · sample_id	→	`v_gene_matrix`	Filter the V gene matrix by samplesheet samples
`repertoire_summary` · sample_id	→	`clone_sizes`	Filter the clone size table by the repertoire summary selection
`repertoire_summary` · sample_id	→	`clonal_diversity`	Filter the diversity profiles by the repertoire summary selection
`repertoire_summary` · sample_id	→	`diversity_orders`	Filter the ranked diversity bars by the repertoire summary selection
`repertoire_summary` · sample_id	→	`clonal_overlap`	Filter the clonal overlap matrix by the repertoire summary selection
`repertoire_summary` · sample_id	→	`clonal_abundance`	Filter the rank abundance curves by the repertoire summary selection
`samplesheet` · sample_id	→	`clonal_abundance`	Filter the rank abundance curves by samplesheet samples
`samplesheet` · sample_id	→	`clone_sets`	Narrow the shared clone UpSet to the samplesheet samples
`samplesheet` · subject_id	→	`threshold_summary`	Filter the fitted clonal threshold by samplesheet subjects

Recipes¶

Each recipe reshapes raw pipeline output into a tidy table. The name links to its source; Output lists the validated OUTPUT_SCHEMA columns.

Recipe	Transforms	Output
`enchantr/clonal_abundance.py`	Rank-abundance curve per sample with its bootstrap confidence band.	`sample_id`, `subject_id`, `clone_id`, `rank`, `p`, `p_sd`, `lower`, `upper`
`enchantr/clonal_diversity.py`	Hill diversity profiles per sample, from the enchantR repertoire analysis report.	`sample_id`, `subject_id`, `q`, `d`, `d_sd`, `d_lower`, `d_upper`, `e`, `e_lower`, `e_upper`
`enchantr/clonal_overlap.py`	Sample by sample matrix of shared clones, for a clustered heatmap.	`sample_id`, `subject_id`
`enchantr/clone_sets.py`	Clone by sample presence matrix (0/1) for an UpSet plot of shared clones.	`clone`, `subject_id`, `n_samples`, `total_sequences`
`enchantr/clone_sizes.py`	Clone sizes with a within-sample rank and a clonal-homeostasis size class.	`sample_id`, `subject_id`, `clone_id`, `seq_count`, `seq_freq`, `rank`, `size_class`
`enchantr/diversity_orders.py`	Hill diversity at the three named orders, from the enchantR repertoire analysis report.	`sample_id`, `subject_id`, `order`, `q`, `d`, `d_lower`, `d_upper`
`enchantr/repertoire_summary.py`	One row per sample: clone counts, clone sizes and the three Hill numbers.	`sample_id`, `subject_id`, `sequences`, `number_of_clones`, `sequences_per_clone`, `clone_size_count_min`, `clone_size_count_median`, `clone_size_count_max`, `richness`, `richness_lower`, `richness_upper`, `shannon`, `shannon_lower`, `shannon_upper`, `simpson`, `simpson_lower`, `simpson_upper`, `evenness`, `evenness_lower`, `evenness_upper`
`enchantr/sequence_counts.py`	Sequences left after every pRESTO and Change-O step, one wide row per sample.	`sample_id`, `subject_id`, `sequences`, `retention`
`enchantr/sequence_fates.py`	Read fates through the airrflow pre-processing funnel, shaped for a Sankey.	`sample_id`, `subject_id`, `input`, `quality`, `paired`, `consensus`, `assembled`, `representative`, `reads`
`enchantr/threshold_summary.py`	Clonal distance threshold per subject, from enchantR's find-threshold report.	`subject_id`, `model`, `cutoff`, `loglk`, `threshold`, `sensitivity`, `specificity`, `pvalue`, `mean_threshold`
`enchantr/v_gene_matrix.py`	Sample by V gene usage matrix (sequence fractions) for a clustered heatmap.	`row_id`, `sample_id`, `subject_id`, `locus`
`enchantr/v_gene_usage.py`	V gene usage per sample, at family and gene resolution, in long format.	`sample_id`, `subject_id`, `locus`, `rank`, `taxon`, `v_family`, `seq_count`, `locus_count`, `seq_freq`
`nf-core/airrflow/annotation_counts.py`	Sequences left after each V(D)J annotation and QC step, one row per sample.	`sample_id`, `input`, `collapsed`
`nf-core/airrflow/cdr3_spectratype.py`	CDR3 length spectratype per sample, from the AIRR rearrangement table.	`sample_id`, `subject_id`, `locus`, `cdr3_aa_length`, `sequences`, `fraction`
`nf-core/airrflow/vj_usage_matrix.py`	V by J gene pairing matrix per subject, from the AIRR rearrangement table.	`row_id`, `subject_id`, `locus`, `v_gene`

Template variables¶

Variables you provide when running the template: DATA_ROOT is the results directory given to depictio ingest, the others are set with --var NAME=value:

Variable	Required	Description
`DATA_ROOT`	✓	Root directory containing airrflow pipeline output (multiqc/, clonal_analysis/, repertoire_comparison/, parsed_logs/)
`SAMPLESHEET_FILE`	—	Path to the validated samplesheet. Defaults to {DATA_ROOT}/pipeline_info/samplesheet.valid.tsv, which every airrflow run writes.
`SKIP_CLONAL_ANALYSIS`	—	Set when the run used --skip_clonal_analysis (params.json skip_clonal_analysis). Prunes every clonal_analysis/ derived DC.
`SKIP_REPORT`	—	Set when the run used --skip_report (params.json skip_report). Prunes the repertoire_comparison/ and parsed_logs/ derived DCs: the pRESTO log parser runs inside the report step.
`SKIP_THRESHOLD_REPORT`	—	Set when the run used --skip_report_threshold, or an explicit --clonal_threshold instead of 'auto'. Prunes the threshold summary DC.
`SKIP_MULTIQC`	—	Set when the run used --skip_multiqc (params.json skip_multiqc). Prunes the MultiQC DC, which drops the MultiQC tab; the other tabs are unaffected.
`ASSEMBLED_MODE`	—	Set when the run used --mode assembled (params.json mode). Prunes the pRESTO log DCs: assembled input skips read pre-processing entirely.
`GROUP_COL`	—	Samplesheet column holding the condition the dashboard filters and breaks down by (default: treatment). Must exist in pipeline_info/samplesheet.valid.tsv.
`GROUP_COL_DISPLAY`	—	Reader-facing label of GROUP_COL in filter titles and descriptions (default: Condition).

Data collections¶

20 data collections — 13 required 7 optional · 20 direct 0 derived.

Origin tells you whether a collection is real pipeline data or a reshape of it: direct = a pipeline output (scanned, or a recipe that reads raw files); derived = a recipe that reshapes one or more direct collections into the layout a visualization needs (no new measurement). Reads shows what produces it: a recipe .py transform, whose name links to its source on GitHub, or a raw file scanned off disk. (A direct collection can still have a recipe — one that merely parses/cleans the raw file; derived means the recipe reshapes another collection.)

Tag	Origin	Type	Reads	Status
`multiqc_data`	direct	MultiQC	file `(?:.*/)?multiqc(?:/[^/]+)?/multiqc_data/multiqc.parquet$`	required
`samplesheet`	direct	Table	file `{DATA_ROOT}/pipeline_info/samplesheet.valid.tsv`	required
`sequence_counts`	direct	Table	recipe `enchantr/sequence_counts.py`	required
`sequence_fates`	direct	Table	recipe `enchantr/sequence_fates.py`	required
`annotation_counts_assembled`	direct	Table	recipe `nf-core/airrflow/annotation_counts.py`	required
`annotation_counts_noclonal`	direct	Table	recipe `nf-core/airrflow/annotation_counts.py`	required
`repertoire_summary`	direct	Table	recipe `enchantr/repertoire_summary.py`	required
`repertoire_summary_noreport`	direct	Table	recipe `enchantr/repertoire_summary.py`	required
`clonal_diversity`	direct	Table	recipe `enchantr/clonal_diversity.py`	optional
`diversity_orders`	direct	Table	recipe `enchantr/diversity_orders.py`	optional
`clonal_abundance`	direct	Table	recipe `enchantr/clonal_abundance.py`	optional
`clone_sizes`	direct	Table	recipe `enchantr/clone_sizes.py`	required
`clone_sets`	direct	Table	recipe `enchantr/clone_sets.py`	required
`clonal_overlap`	direct	Table	recipe `enchantr/clonal_overlap.py`	optional
`v_gene_usage`	direct	Table	recipe `enchantr/v_gene_usage.py`	required
`v_gene_matrix`	direct	Table	recipe `enchantr/v_gene_matrix.py`	required
`v_gene_usage_noclonal`	direct	Table	recipe `enchantr/v_gene_usage.py`	required
`cdr3_spectratype`	direct	Table	recipe `nf-core/airrflow/cdr3_spectratype.py`	optional
`vj_usage_matrix`	direct	Table	recipe `nf-core/airrflow/vj_usage_matrix.py`	optional
`threshold_summary`	direct	Table	recipe `enchantr/threshold_summary.py`	optional

Conditional routes¶

Rows are data collections; columns are the variables you set or params.json flags auto-detected from the run. Each filled cell is the effect of setting that variable; an empty cell means that variable leaves the collection unchanged. (0 collections are unaffected by any variable — present on every run.)

+ included− removed⇄ repointed

Data collection	`SAMPLESHEET_FILE`	`SKIP_CLONAL_ANALYSIS`	`SKIP_REPORT`	`SKIP_THRESHOLD_REPORT`	`SKIP_MULTIQC`	`ASSEMBLED_MODE`
`multiqc_data`					−
`samplesheet`	⇄
`sequence_counts`			−			−
`sequence_fates`			−			−
`annotation_counts_assembled`			−			+
`annotation_counts_noclonal`		+	−
`repertoire_summary`		−
`repertoire_summary_noreport`		−	+
`clonal_diversity`		−
`diversity_orders`		−
`clonal_abundance`		−
`clone_sizes`		−
`clone_sets`		−
`clonal_overlap`		−
`v_gene_usage`			−
`v_gene_matrix`			−
`v_gene_usage_noclonal`		+	−
`cdr3_spectratype`		−
`vj_usage_matrix`		−
`threshold_summary`		−		−

Cross-DC links¶

24 links — selecting a value in the source collection filters the target. The join column is shown after the source.

Source · column		Target	Filters
`samplesheet` · sample_id	→	`multiqc_data`	Filter the MultiQC panels by samplesheet samples
`samplesheet` · sample_id	→	`sequence_counts`	Filter the sequence processing counts by samplesheet samples
`samplesheet` · sample_id	→	`sequence_fates`	Filter the attrition flow by samplesheet samples
`samplesheet` · sample_id	→	`annotation_counts_assembled`	Filter the assembled-route annotation counts by samplesheet samples
`samplesheet` · sample_id	→	`annotation_counts_noclonal`	Filter the no-clonal-route annotation counts by samplesheet samples
`samplesheet` · sample_id	→	`v_gene_usage_noclonal`	Filter the no-clonal-route V gene usage by samplesheet samples
`samplesheet` · sample_id	→	`repertoire_summary_noreport`	Filter the no-report-route repertoire summary by samplesheet samples
`samplesheet` · sample_id	→	`repertoire_summary`	Filter the repertoire summary by samplesheet samples
`samplesheet` · sample_id	→	`clonal_diversity`	Filter the diversity profiles by samplesheet samples
`samplesheet` · sample_id	→	`diversity_orders`	Filter the ranked diversity bars by samplesheet samples
`samplesheet` · sample_id	→	`clone_sizes`	Filter the clone size table by samplesheet samples
`samplesheet` · sample_id	→	`v_gene_usage`	Filter the V gene usage by samplesheet samples
`samplesheet` · sample_id	→	`cdr3_spectratype`	Filter the CDR3 spectratypes by samplesheet samples
`samplesheet` · subject_id	→	`vj_usage_matrix`	Filter the V-J pairing grid by samplesheet donors
`samplesheet` · sample_id	→	`clonal_overlap`	Filter the clonal overlap matrix by samplesheet samples
`samplesheet` · sample_id	→	`v_gene_matrix`	Filter the V gene matrix by samplesheet samples
`repertoire_summary` · sample_id	→	`clone_sizes`	Filter the clone size table by the repertoire summary selection
`repertoire_summary` · sample_id	→	`clonal_diversity`	Filter the diversity profiles by the repertoire summary selection
`repertoire_summary` · sample_id	→	`diversity_orders`	Filter the ranked diversity bars by the repertoire summary selection
`repertoire_summary` · sample_id	→	`clonal_overlap`	Filter the clonal overlap matrix by the repertoire summary selection
`repertoire_summary` · sample_id	→	`clonal_abundance`	Filter the rank abundance curves by the repertoire summary selection
`samplesheet` · sample_id	→	`clonal_abundance`	Filter the rank abundance curves by samplesheet samples
`samplesheet` · sample_id	→	`clone_sets`	Narrow the shared clone UpSet to the samplesheet samples
`samplesheet` · subject_id	→	`threshold_summary`	Filter the fitted clonal threshold by samplesheet subjects

Recipes¶

Each recipe reshapes raw pipeline output into a tidy table. The name links to its source; Output lists the validated OUTPUT_SCHEMA columns.

Recipe	Transforms	Output
`enchantr/clonal_abundance.py`	Rank-abundance curve per sample with its bootstrap confidence band.	`sample_id`, `subject_id`, `clone_id`, `rank`, `p`, `p_sd`, `lower`, `upper`
`enchantr/clonal_diversity.py`	Hill diversity profiles per sample, from the enchantR repertoire analysis report.	`sample_id`, `subject_id`, `q`, `d`, `d_sd`, `d_lower`, `d_upper`, `e`, `e_lower`, `e_upper`
`enchantr/clonal_overlap.py`	Sample by sample matrix of shared clones, for a clustered heatmap.	`sample_id`, `subject_id`
`enchantr/clone_sets.py`	Clone by sample presence matrix (0/1) for an UpSet plot of shared clones.	`clone`, `subject_id`, `n_samples`, `total_sequences`
`enchantr/clone_sizes.py`	Clone sizes with a within-sample rank and a clonal-homeostasis size class.	`sample_id`, `subject_id`, `clone_id`, `seq_count`, `seq_freq`, `rank`, `size_class`
`enchantr/diversity_orders.py`	Hill diversity at the three named orders, from the enchantR repertoire analysis report.	`sample_id`, `subject_id`, `order`, `q`, `d`, `d_lower`, `d_upper`
`enchantr/repertoire_summary.py`	One row per sample: clone counts, clone sizes and the three Hill numbers.	`sample_id`, `subject_id`, `sequences`, `number_of_clones`, `sequences_per_clone`, `clone_size_count_min`, `clone_size_count_median`, `clone_size_count_max`, `richness`, `richness_lower`, `richness_upper`, `shannon`, `shannon_lower`, `shannon_upper`, `simpson`, `simpson_lower`, `simpson_upper`, `evenness`, `evenness_lower`, `evenness_upper`
`enchantr/sequence_counts.py`	Sequences left after every pRESTO and Change-O step, one wide row per sample.	`sample_id`, `subject_id`, `sequences`, `retention`
`enchantr/sequence_fates.py`	Read fates through the airrflow pre-processing funnel, shaped for a Sankey.	`sample_id`, `subject_id`, `input`, `quality`, `paired`, `consensus`, `assembled`, `representative`, `reads`
`enchantr/threshold_summary.py`	Clonal distance threshold per subject, from enchantR's find-threshold report.	`subject_id`, `model`, `cutoff`, `loglk`, `threshold`, `sensitivity`, `specificity`, `pvalue`, `mean_threshold`
`enchantr/v_gene_matrix.py`	Sample by V gene usage matrix (sequence fractions) for a clustered heatmap.	`row_id`, `sample_id`, `subject_id`, `locus`
`enchantr/v_gene_usage.py`	V gene usage per sample, at family and gene resolution, in long format.	`sample_id`, `subject_id`, `locus`, `rank`, `taxon`, `v_family`, `seq_count`, `locus_count`, `seq_freq`
`nf-core/airrflow/annotation_counts.py`	Sequences left after each V(D)J annotation and QC step, one row per sample.	`sample_id`, `input`, `collapsed`
`nf-core/airrflow/cdr3_spectratype.py`	CDR3 length spectratype per sample, from the AIRR rearrangement table.	`sample_id`, `subject_id`, `locus`, `cdr3_aa_length`, `sequences`, `fraction`
`nf-core/airrflow/vj_usage_matrix.py`	V by J gene pairing matrix per subject, from the AIRR rearrangement table.	`row_id`, `subject_id`, `locus`, `v_gene`

Dashboard tabs¶

One dashboard: the Overview, then seven child tabs in three groups, read as a funnel from the reads to the clones that expand and are shared. Each tab below carries the same icon and colour the dashboard gives it, so the page and the app read alike.

Group	Tabs
Data & QC	MultiQC, Sequence Processing
Repertoire	V Gene Usage, CDR3 & Pairing
Clonality	Clonal Diversity, Clonal Expansion, Clone Sharing

Each child tab opens with a short intro and a strip of key numbers, then at most three open sections; tables and details follow, collapsed. The persistent Sample filters (the condition column, sample id, subject and sex) sit in the left panel and narrow every tab through the sample sheet's cross-DC links. The Sample sheet is pinned, collapsed, to the bottom of every child tab. Clones are defined within a subject, so the subject colours the per-sample lines and annotates the heatmaps.

Overview

MultiQC Sequence Processing V Gene Usage CDR3 & Pairing Clonal Diversity Clonal Expansion Clone Sharing

Immune receptor repertoires, from reads to the clones that expand and are shared.

A short hero links the run parameters. About this dashboard says how to move through the tabs, The run lists the samples and subjects, the library preparation, the clonal threshold setting and the input mode, and Pipeline walks six steps from trimming to the comparison of repertoires, each linked to its parameters and its tab. The Key figures are Samples (split by condition), Input reads and Clones (each summed, with the spread per sample) and Sequences per clone (each sample's mean clone size, the median over samples).

Filters and components

Filters · the left panel starts collapsed. Key figures has its own filter bar (the condition and the sample id), and so does Findings (the condition and the subject): each narrows its own section only.

Section	What it holds
Top	Hero, About this dashboard, The run, Pipeline
Key figures	4 headline cards: samples, input reads, clones, sequences per clone
Findings	Live result rows, then 4 figures: V family composition per sample, the Hill diversity profile, clones by number of samples holding them and the clonal homeostasis sunburst
How to read this dashboard	The tabs by group, each with its question

Every route keeps four Key figures: a card whose collection a route lacks gives its slot to an alternate. A --mode assembled run (ASSEMBLED_MODE) counts the input sequences instead of the input reads, and a --skip_report run (SKIP_REPORT) the sequences that entered clonal assignment. A --skip_clonal_analysis run (SKIP_CLONAL_ANALYSIS) shows unique sequences and V genes instead of clones and sequences per clone.

Data & QC · Did trimming and read quality hold for every sample?

MultiQC panels only: general statistics, fastp filtered reads and the FastQC sequence counts, then fastp's per-base quality beside the FastQC quality histograms. airrflow runs FastQC twice, on the raw reads and after assembly, and MultiQC labels the second run fastqc-1. Amplicons of one receptor locus show high duplication and a narrow GC band by design.

Filters and components

Filters · Sample ID, read from the MultiQC report.

Section	What it holds
QC overview	3 MultiQC panels
Base quality	2 MultiQC panels
QC details (collapsed)	6 MultiQC panels

Data & QC · How many reads survive each pRESTO and Change-O step?

Input reads, with the share kept through each pRESTO step, and the reads per sample with their spread. Then the read-fate Sankey, where losses peel off into a Lost lane, and per sample the sequences left at each step (log scale) beside the sequences each sample keeps per 1,000 input reads. UMI consensus and duplicate collapsing merge many reads into one sequence, so that ratio is not a share of reads kept.

Filters and components

Filters · Input reads and Sequences per input read ranges on sequence_counts, plus the sample filters.

Section	What it holds
Processing at a glance	2 cards
Where the reads go	Read fates through the pipeline
Per sample	Sequences left at each step, Sequences kept per 1,000 reads
Counts table (collapsed)	Sequence processing counts

Repertoire · Which V families and genes build each repertoire?

Distinct V genes, the annotated Sequences split by V family, the largest share one family takes in a sample and the median share of a V gene. Then the stacked composition per sample, by V family with gene resolution a switch away, and the sample by V gene heatmap, clustered and standardised per gene, so samples of one subject should sit together.

Filters and components

Filters · V family and Locus on v_gene_usage, plus the sample filters.

Section	What it holds
V genes at a glance	4 cards
Composition	1 advanced visualization
Clustered usage	1 advanced visualization

Repertoire · How long are CDR3 loops, and which V and J genes pair?

Productive sequences and the largest share one CDR3 length takes in a sample. Then the spectratype, one line per sample: a smooth bell is a diverse repertoire, and one length towering over the rest points to an expanded clone. The V by J heatmap pairs each subject's V genes with the J genes they joined. Both views read the AIRR rearrangement table of the productive sequences.

Filters and components

Filters · a CDR3 length (aa) range and Locus on cdr3_spectratype, plus the sample filters.

Section	What it holds
CDR3 at a glance	2 cards
Spectratype	CDR3 spectratype
V-J pairing	1 advanced visualization
Spectratype per sample (collapsed)	Spectratype per sample

Spectratype and V-J pairing need the AIRR table

Both read the *__repertoire-pass.tsv rearrangement table enchantR's repertoire analysis starts from, keeping only the handful of columns they need. The two collections are optional: a run with --skip_clonal_analysis, or a copy of the output without that table, drops the CDR3 & Pairing tab and keeps the rest of the dashboard.

Clonality · How many clones does each repertoire hold, and how diverse is it?

Clones and the median effective number of clones (Hill diversity at q = 1, on repertoires rarefied to a common depth). Then the Hill diversity profile, one curve per sample against the order q: the higher q, the more the largest clones weigh, so a steeply falling curve is a repertoire carried by a few expanded clones. The ranked diversity at one order, richness against evenness, and clones against sequencing depth follow; the SHazaM distance threshold that defined the clones sits in a collapsed section.

Filters and components

Filters · a Clones range on repertoire_summary, and the Diversity order on diversity_orders, which narrows the ranked bars only.

Section	What it holds
Diversity at a glance	2 cards
Diversity profile	1 advanced visualization
Samples compared	2 advanced visualizations
Clones and depth	Clones against depth
Clone definition (collapsed)	2 cards + Clonal distance threshold
Per-sample table (collapsed)	Repertoire summary

Clonality · How strongly do the largest clones dominate each repertoire?

Clones as a ring by size class, Sequences split by size class, the median share of a sample's largest clone, and the size of the largest clone with every clone's size as a histogram. Then the rank-abundance curves with their bootstrap bands, where a flat head is a polyclonal repertoire, and the clonal homeostasis sunburst by subject, sample and size class.

Filters and components

Filters · Size class on clone_sizes, plus the sample filters.

Section	What it holds
Expansion at a glance	4 cards
Rank abundance	1 advanced visualization
Clonal homeostasis	1 advanced visualization

Clonality · Which clones do the samples of one subject share?

Clones split by the number of samples holding them, and the sequences in shared clones. Then the clones by number of samples (log scale), the shared-clone heatmap for every pair of samples and the UpSet of exact sample sets. Only samples of the same subject can share a clone.

Filters and components

Filters · a Samples per clone range on clone_sets, plus the sample filters.

Section	What it holds
Sharing at a glance	2 cards
Clones by sharing	Clones by samples holding them
Pairs and sets	2 advanced visualizations
Overlap table (collapsed)	Clonal overlap matrix

Two things that look wrong and are not

The shared-clone heatmap's diagonal is written as 0, since a sample's overlap with itself dwarfs any real sharing and flattens the colour scale; cross-subject cells read zero too, because airrflow defines clones within a subject. A sample too shallow for enchantR to fit diversity numbers is dropped from clonal_diversity.tsv, so its diversity values are missing while its clone counts stand.

Tables and scatters select on their entity column: the sample sheet, the repertoire summary, the sequence counts and clonal overlap tables, the clones-against-depth figure, the richness-against-evenness scatter and the rank-abundance curve on sample_id, and the clonal threshold table on subject_id. A pick becomes a dashboard filter that the project links carry to every collection they reach.

Running the pipeline¶

Depictio reads the output of nf-core/airrflow, it does not run the pipeline. Run the pipeline first:

nextflow run nf-core/airrflow -r 5.1.1 \
  --input samplesheet.tsv \
  --mode fastq \
  --library_generation_method specific_pcr_umi \
  --cprimers CPrimers.fasta \
  --vprimers VPrimers.fasta \
  --umi_length 12 \
  --outdir results -profile docker

Then point Depictio at the results:

depictio ingest results/ \
  --template nf-core/airrflow/latest

See nf-co.re/airrflow/usage for full pipeline documentation.

Required data structure¶

Point depictio ingest at the directory holding the pipeline output. Depictio scans recursively and matches on file name, so the layout below only has to be present somewhere under the root. Nothing outside the run is needed.

<DATA_ROOT>/
├── pipeline_info/
│   ├── samplesheet.valid.tsv                       # hub DC, --var SAMPLESHEET_FILE
│   ├── params.json
│   └── software_versions.yml
├── multiqc/
│   └── multiqc_data/
│       └── multiqc.parquet                         # fastp + both FastQC runs
├── parsed_logs/
│   └── Table_sequences_process.tsv                 # pRESTO half of the funnel
├── repertoire_comparison/
│   ├── Sequence_numbers_summary/
│   │   └── Table_sequences_assembled.tsv           # Change-O half of the funnel
│   └── V_family/
│       ├── V_family_distribution_data.tsv
│       └── V_gene_distribution_by_sequence_data.tsv
└── clonal_analysis/
    ├── find_threshold/all_reps_dist_report/tables/
    │   └── all_reps_threshold-summary.tsv          # shazam threshold per subject
    └── repertoire_analysis/repertoire_analysis_report/
        ├── repertoires/
        │   └── *__repertoire-pass.tsv              # AIRR table: spectratype, V-J pairing
        └── tables/
            ├── clonal_abundance.tsv                # rank abundance with bootstrap band
            ├── clonal_diversity.tsv
            ├── clonal_overlap.tsv
            ├── clone_sizes_table.tsv
            └── num_clones_table.tsv

Validation runs¶

5.1.1 has no AWS megatest run: the bucket holds no prefix for the release. The 5.1.1 template was validated on an EMBL HPC run of the release's test profile (airrflow511), ingested on a local stack; its MultiQC 1.34 report already carries the parquet, so no reprocess step is needed. 5.1.1 publishes the 5.1.0 layout, so the megatest shown here is the 5.1.0 run. The repository ships download_test_data.sh next to the 5.1.0 template, which fetches the subset of that run the template needs:

bash depictio/projects/nf-core/airrflow/5.1.0/download_test_data.sh \
  /tmp/airrflow_test

The run is s3://nf-core-awsmegatests/airrflow/results-e69d49e3f23f11a3391755b5fb7aa4283c0a2471/ (the 5.1.0 release tag): a ten-sample, two-subject multiple sclerosis B cell study of cervical lymph node and brain lesion tissue, run in the default --mode fastq UMI route. The manifest fetches fifteen keys, about 330 MB, nearly all of it the AIRR rearrangement table behind the spectratype and the V-J grid. post_fetch_help in megatest.yaml prints the dry-run and full-run commands once the download finishes.

Then run Depictio against it:

depictio ingest /tmp/airrflow_test \
  --template nf-core/airrflow/latest

Additional resources¶

nf-co.re/airrflow: official pipeline documentation
nf-co.re/airrflow/5.1.0/results: AWS test results of the 5.1.0 release (5.1.1 has none)
Immcantation: enchantR, alakazam and shazam, the toolset behind the repertoire panels
Template System Reference: YAML format, variables, conditionals
Recipes: how to read, test, and write recipes

Authorship¶

Developers Wrote the template, its recipes and its dashboards.

weber8thomas

Reviewers Nobody has run it on their own data and signed it off yet, which is what keeps it Experimental. Open

Maintainers Keep it working as nf-core/airrflow releases.

weber8thomas

Reviewing a template on your own data, or taking over a role here, is a contribution in itself: the contributing guide says what each one involves.