Depictio CLI UsageΒΆ
Note about the CLI
The depictio-cli is a command line interface that allows you to interact with the Depictio backend. It is used to register projects information including workflow files metadata. The depictio-cli is currently in development and is not yet ready for production use.
π Table of ContentsΒΆ
- Installation
- Quick Reference
- Global Options
- π Commands
- π Run Command
- π³ Recipe Commands
- π Config Commands
- π Data Commands
- π Dashboard Commands
- πΎ Backup Commands
- π Migrate Commands
- π οΈ Common Use Cases
InstallationΒΆ
See the installation guide for instructions on how to install the depictio-cli.
Quick ReferenceΒΆ
| Command | Description | Access Level |
|---|---|---|
version |
Show CLI version | All users |
run |
Execute complete workflow | All users |
recipe list |
List all bundled recipes | All users |
recipe info <name> |
Show recipe sources and schema | All users |
recipe run <name> |
Execute a recipe locally with validation | All users |
config show-cli-config |
Display CLI configuration | All users |
config check-s3-storage |
Validate S3 storage setup | All users |
config check-server-accessibility |
Test server connection | All users |
config validate-project-config |
Validate project configuration | All users |
config sync-project-config-to-server |
Sync project config to server | All users |
data scan |
Scan project files | All users |
data process |
Process data collections | All users |
dashboard validate |
Validate dashboard YAML file locally | All users |
dashboard import |
Import dashboard YAML to server | All users |
dashboard export |
Export dashboard to YAML file | All users |
backup create |
Create system backup | Admin only |
backup list |
List available backups | Admin only |
backup validate |
Validate backup against models | Admin only |
backup restore |
Restore from backup | Admin only |
backup check-coverage |
Check validation coverage | Admin only |
migrate |
Migrate a project to another instance | Admin only |
Global OptionsΒΆ
| Option | Short | Type | Default | Description |
|---|---|---|---|---|
--verbose |
-v |
boolean |
false |
Enable verbose logging |
--verbose-level |
-vl |
string |
"INFO" |
Set verbose logging level |
π CommandsΒΆ
π Run CommandΒΆ
π¬ π₯οΈ `depictio-cli run` command example
Execute the complete Depictio workflow: validate β sync β scan β process
Quick Start:
Pipeline Steps:
- β Check server accessibility
- β Check S3 storage configuration
- β Validate project configuration
- β Sync project configuration to server
- β Scan data files
- β Process data collections
π Basic Configuration
| Parameter | Type | Default | Description |
|---|---|---|---|
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
CLI configuration file path |
--project-config-path |
string |
"" |
Pipeline configuration file path (mutually exclusive with --template) |
--workflow-name |
string |
null |
Specific workflow to process |
--data-collection-tag |
string |
null |
Data collection tag to process |
π³ Template Options
Use these flags to run from a pre-packaged template instead of a project YAML. --template and --project-config-path are mutually exclusive.
| Parameter | Type | Default | Description |
|---|---|---|---|
--template |
string |
null |
Template ID. Pin a version (nf-core/ampliseq/2.16.0), or use nf-core/ampliseq/latest: or just nf-core/ampliseq: to resolve the newest shipped version (v1.5.2+) |
--data-root |
path |
null |
Root directory substituted for {DATA_ROOT} in template. Required when --template is set. |
--project-name |
string |
null |
Custom project name (auto-generated from template if omitted) |
--dashboard-name |
string |
null |
Override the template's main dashboard title at import (the template file is left untouched). |
--dashboard |
path |
null |
Override default dashboard(s) to import. Repeatable. |
--skip-dashboard-import |
flag |
false |
Skip the automatic dashboard import step (Step 8) |
Since v1.6.0, resolving a template also picks up any recipe seed committed beside the data: a source: transformed data collection with a {DATA_ROOT}/{dc_tag}.tsv next to it is scanned from that file instead of re-running its recipe against raw pipeline inputs the bundled projects do not ship. Collections without a seed still run their recipes, and no flags change.
See Templates for full documentation.
π Provisioning Options (v1.1.3+)
Run a pipeline on behalf of a user who has no account yet. The CLI creates-or-gets the account, runs the whole pipeline as them, and prints a passwordless login link straight to the dashboard it imported.
| Parameter | Type | Default | Description |
|---|---|---|---|
--user |
string |
null |
Email of the user to provision and run as. Requires --provisioning-key. |
--provisioning-key |
string |
null |
Shared provisioning secret. Reads DEPICTIO_AUTH_PROVISIONING_API_KEY from the environment if the flag is omitted. |
depictio-cli run \
--template nf-core/ampliseq/latest \
--data-root /data/ampliseq-run-42 \
--user alice@example.org \
--provisioning-key "$DEPICTIO_AUTH_PROVISIONING_API_KEY"
The key is a server-side secret: the instance must be started with the
matching DEPICTIO_AUTH_PROVISIONING_API_KEY, otherwise the provisioning
endpoints stay disabled and the run is rejected. The CLI redacts the value
from its own logs. See
Pipeline provisioning.
βοΈ Flow Control Options
| Parameter | Type | Default | Description |
|---|---|---|---|
--skip-server-check |
boolean |
false |
Skip server accessibility check |
--skip-s3-check |
boolean |
false |
Skip S3 storage validation |
--skip-sync |
boolean |
false |
Skip config sync to server |
--skip-scan |
boolean |
false |
Skip data scanning step |
--skip-process |
boolean |
false |
Skip data processing step |
π Sync & Scan Options
| Parameter | Type | Default | Description |
|---|---|---|---|
--update-config |
boolean |
false |
Update project configuration on server |
--rescan-folders |
boolean |
false |
Reprocess all runs for data collection |
--sync-files |
boolean |
false |
Update files for data collection |
--overwrite |
boolean |
false |
Overwrite workflow if it already exists |
π₯οΈ Output & Control
| Parameter | Type | Default | Description |
|---|---|---|---|
--rich-tables |
boolean |
false |
Show detailed execution summary |
--continue-on-error |
boolean |
false |
Continue execution on step failure |
--dry-run |
boolean |
false |
Show execution plan without running |
β‘ Ingestion Performance (v1.3.0+)
Both are opt-in and default to the previous behaviour.
| Parameter | Type | Default | Description |
|---|---|---|---|
--streaming |
boolean |
false |
Stream the Delta write instead of materialising the whole table in memory. Lowers peak RSS on large ingests. Experimental, falls back to the standard write on any failure. |
| Variable | Default | Description |
|---|---|---|
DEPICTIO_INGEST_STREAMING_WRITE |
unset | Same as --streaming, as an environment toggle. |
DEPICTIO_INGEST_DC_WORKERS |
1 |
Ingest this many data collections concurrently. Clamped to 4 and to the number of data collections, so an over-large value degrades to the cap. An unparseable value logs a warning and falls back to sequential. |
Examples:
# Set up a complete ampliseq project from raw pipeline output
depictio-cli run \
--template nf-core/ampliseq/latest \
--data-root /data/my_ampliseq_run
# With detailed output and error handling
depictio-cli run \
--project-config-path ./config.yaml \
--continue-on-error \
--rich-tables
# Preview execution without running
depictio-cli run \
--project-config-path ./config.yaml \
--dry-run
# Skip server checks for faster execution
depictio-cli run \
--project-config-path ./config.yaml \
--skip-server-check \
--skip-s3-check
π§βπ³ Recipe CommandsΒΆ
Command Group: depictio-cli recipe
All commands in this section are part of the recipe command family. Use them to discover, inspect, and locally test data transformation recipes before running them in a project. For full recipe documentation, see Recipes.
Discover and execute data transformation recipes locally.
recipe listΒΆ
List all bundled recipes.
Output:
Available recipes (5):
nf-core/ampliseq/alpha_diversity.py
nf-core/ampliseq/alpha_rarefaction.py
nf-core/ampliseq/ancombc.py
nf-core/ampliseq/taxonomy_composition.py
nf-core/ampliseq/taxonomy_rel_abundance.py
recipe info <name>ΒΆ
Show recipe details: description, sources, and expected output schema.
| Parameter | Type | Default | Description |
|---|---|---|---|
recipe_name |
string |
required | Recipe name (e.g. nf-core/ampliseq/alpha_diversity.py) |
Output:
Recipe: nf-core/ampliseq/alpha_diversity.py
Description: Transform QIIME2 alpha diversity vector to per-sample Faith PD table.
Sources (1):
faith_pd: qiime2/diversity/alpha_diversity/faith_pd_vector/metadata.tsv (TSV)
Expected output schema (3 columns):
sample: Utf8
habitat: Utf8
faith_pd: Float64
recipe run <name>ΒΆ
Execute a recipe against a local data directory with all 4 validation checkpoints.
| Parameter | Short | Default | Description |
|---|---|---|---|
recipe_name |
β | required | Recipe name (e.g. nf-core/ampliseq/alpha_diversity.py) |
--data-dir |
-d |
required | Root directory with workflow output files |
--output |
-o |
null |
Save result to .parquet or .csv file |
--head |
-n |
20 |
Number of rows to display |
# Run with validation output
depictio-cli recipe run nf-core/ampliseq/alpha_diversity.py \
--data-dir /data/ampliseq_results
# Save output and show first 5 rows
depictio-cli recipe run nf-core/ampliseq/alpha_diversity.py \
--data-dir /data/ampliseq_results \
--output alpha_diversity.parquet \
--head 5
π Config CommandsΒΆ
Command Group: depictio-cli config
All commands in this section are part of the config command family. Use them to manage Depictio configurations and validate connections.
Manage Depictio configurations and validate connections.
config show-cli-configΒΆ
Display the current CLI configuration.
| Parameter | Type | Default | Description |
|---|---|---|---|
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
CLI configuration file path |
config check-s3-storageΒΆ
Validate S3 storage configuration.
| Parameter | Type | Default | Description |
|---|---|---|---|
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
CLI configuration file path |
config check-server-accessibilityΒΆ
Test connection to Depictio server.
| Parameter | Type | Default | Description |
|---|---|---|---|
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
CLI configuration file path |
config show-depictio-project-metadata-on-serverΒΆ
Display metadata for registered projects.
| Parameter | Type | Default | Description |
|---|---|---|---|
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
CLI configuration file path |
--project-name |
string |
"" |
Specific project name |
config validate-project-configΒΆ
Validate project configuration file.
| Parameter | Type | Default | Description |
|---|---|---|---|
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
CLI configuration file path |
--project-config-path |
string |
"" |
Pipeline configuration file path |
config sync-project-config-to-serverΒΆ
Sync project configuration to server.
| Parameter | Type | Default | Description |
|---|---|---|---|
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
CLI configuration file path |
--project-config-path |
string |
"" |
Pipeline configuration file path |
--update |
boolean |
false |
Update existing project configuration |
π Data CommandsΒΆ
Command Group: depictio-cli data
All commands in this section are part of the data command family. Use them to manage data scanning and processing operations.
Manage data scanning and processing operations.
data scanΒΆ
Scan project files for data collections.
| Parameter | Type | Default | Description |
|---|---|---|---|
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
CLI configuration file path |
--project-config-path |
string |
"" |
Pipeline configuration file path |
--workflow-name |
string |
null |
Specific workflow to scan |
--data-collection-tag |
string |
null |
Data collection tag to scan |
--rescan-folders |
boolean |
false |
Reprocess all runs for data collection |
--sync-files |
boolean |
false |
Update files for data collection |
data processΒΆ
Process data collections for workflow execution.
| Parameter | Type | Default | Description |
|---|---|---|---|
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
CLI configuration file path |
--project-config-path |
string |
"" |
Pipeline configuration file path |
--overwrite |
boolean |
false |
Overwrite existing workflow |
π Dashboard CommandsΒΆ
Command Group: depictio-cli dashboard
All commands in this section are part of the dashboard command family. Use them to manage dashboard YAML files - validate, import to server, and export from server.
Manage dashboard YAML files for the Dashboard YAML Management feature.
| Command | Description | Server Required |
|---|---|---|
validate |
Validate YAML schema locally | No |
import |
Import YAML to server | Yes (unless --dry-run) |
export |
Export dashboard to YAML | Yes |
dashboard validateΒΆ
Validate a dashboard YAML file. Runs in two passes: schema + domain constraints (always), then server schema validation (when --config is provided).
| Parameter | Type | Default | Description |
|---|---|---|---|
yaml_file |
path |
required | Path to YAML dashboard file |
--config / -c |
string |
null |
CLI config file β enables server schema validation (Pass 2) |
--offline |
boolean |
false |
Skip server schema check even when --config is provided |
--verbose / -v |
boolean |
false |
Show detailed validation output |
--api |
string |
from config | API base URL |
# Schema + domain only (no server needed)
depictio-cli dashboard validate my_dashboard.yaml
# Full validation including server column check
depictio-cli dashboard validate my_dashboard.yaml --config ~/.depictio/admin_config.yaml
# Force offline even when config is provided
depictio-cli dashboard validate my_dashboard.yaml --config ~/.depictio/admin_config.yaml --offline
Example Output (Success):
Validating: my_dashboard.yaml Pass 1: schema + domain constraints β Schema + domain OK Pass 2: server schema validation β Server schema OK β Validation passed
Example Output (Failure β per-component error table):
Validating: my_dashboard.yaml Pass 1: schema + domain constraints β Schema/domain validation failed Validation Errors βββββββββββββββββ³ββββββββββββ³βββββββββββββββββββββββββββββββββββββββββββββββββββββ β Component β Field β Message β β‘βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ© β pie-chart β - β Invalid visu_type 'pie' for mode='ui'. β β β β Valid values: scatter, line, bar, box, histogram β βββββββββββββββββ΄ββββββββββββ΄βββββββββββββββββββββββββββββββββββββββββββββββββββββ
dashboard importΒΆ
Import a dashboard YAML file to the server. The project is determined from the YAML project_tag field or the --project option.
| Parameter | Type | Default | Description |
|---|---|---|---|
yaml_file |
path |
required | Path to YAML dashboard file |
--config / -c |
string |
null |
Path to CLI config file (required unless --dry-run) |
--project / -p |
string |
null |
Project ID (overrides project_tag in YAML) |
--overwrite |
boolean |
false |
Update existing dashboard with same title |
--dry-run |
boolean |
false |
Validate schema + domain only, don't import (no server needed) |
--offline |
boolean |
false |
Skip server schema check (column names not verified) |
--api |
string |
from config | API base URL |
# Validate locally without server (no config needed)
depictio-cli dashboard import dashboard.yaml --dry-run
# Import to server
depictio-cli dashboard import dashboard.yaml --config ~/.depictio/admin_config.yaml
# Update existing dashboard with same title
depictio-cli dashboard import dashboard.yaml --config ~/.depictio/admin_config.yaml --overwrite
# Override project from YAML
depictio-cli dashboard import dashboard.yaml --config ~/.depictio/admin_config.yaml --project 646b0f3c1e4a2d7f8e5b8c9a
Example Output:
Validating: dashboard.yaml β Validation passed Title: Iris Dashboard Demo Components: 7 Project: Iris_Dataset_Project (from YAML project_tag) Loading CLI configuration... β Configuration loaded API URL: localhost:8058 Importing dashboard (project: Iris_Dataset_Project)... β Dashboard imported successfully! Dashboard ID: 6824cb3b89d2b72169309737 Title: Iris Dashboard Demo Project ID: 650a1b2c3d4e5f6a7b8c9d0e View at: localhost:8058/dashboard/6824cb3b89d2b72169309737
dashboard exportΒΆ
Export a dashboard from the server to a YAML file.
| Parameter | Type | Default | Description |
|---|---|---|---|
dashboard_id |
string |
required | Dashboard ID to export |
--config / -c |
string |
required | Path to CLI config file |
--output / -o |
path |
dashboard.yaml |
Output file path |
--api |
string |
from config | API base URL |
# Export to default file
depictio-cli dashboard export 6824cb3b89d2b72169309737 --config ~/.depictio/admin_config.yaml
# Export to specific file
depictio-cli dashboard export 6824cb3b89d2b72169309737 --config ~/.depictio/admin_config.yaml -o iris_dashboard.yaml
Example Output:
Loading CLI configuration... Exporting dashboard 6824cb3b89d2b72169309737... β Dashboard exported to: iris_dashboard.yaml
For more information about dashboard YAML format and workflows, see Dashboard YAML Management.
ποΈ Catalog CommandsΒΆ
Command Group: depictio-cli catalog
Browse the bioinformatics toolβviz catalog by rendering its components on their bundled fixture data. Handy for previewing what a module produces before wiring it into a dashboard. For the live, hosted version see the Depictio Modules catalog.
catalog previewΒΆ
Render a single catalog output on its fixture and serve it on a throwaway localhost server (nothing is written to disk). Pass --out to export a portable, self-contained HTML file instead.
# Serve an ephemeral preview (Ctrl-C to stop)
depictio-cli catalog preview <output-id>
# Export a standalone HTML file instead of serving
depictio-cli catalog preview <output-id> --out preview.html
catalog galleryΒΆ
Same as preview, but renders the whole catalog as a browsable gallery.
| Parameter | Type | Default | Description |
|---|---|---|---|
--out, -o |
path |
null |
Export the self-contained HTML here instead of serving it |
--port |
int |
0 |
Port for the ephemeral server (0 = auto) |
--theme, -t |
string |
light |
Theme: light or dark |
--no-open |
flag |
false |
Do not open a browser tab automatically |
πΎ Backup CommandsΒΆ
Command Group: depictio-cli backup
All commands in this section are part of the backup command family. Use them to backup and restore MongoDB database and S3 data. Admin access required.
Backup and restore system data and configurations.
backup createΒΆ
Create a backup of the MongoDB database.
| Parameter | Type | Default | Description |
|---|---|---|---|
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
CLI configuration file path |
--dry-run |
boolean |
false |
Validate without creating backup |
--include-s3-data |
boolean |
false |
Include S3 deltatable data in backup |
--s3-backup-prefix |
string |
backup |
Prefix for S3 backup location |
backup listΒΆ
List available backup files on the server.
| Parameter | Type | Default | Description |
|---|---|---|---|
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
CLI configuration file path |
backup validateΒΆ
Validate a backup file against Pydantic models.
| Parameter | Type | Default | Description |
|---|---|---|---|
backup_id |
string |
required | Backup ID to validate (YYYYMMDD_HHMMSS) |
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
CLI configuration file path |
backup restoreΒΆ
Restore data from a backup file. Warning: destructive operation.
| Parameter | Type | Default | Description |
|---|---|---|---|
backup_id |
string |
required | Backup ID to restore (YYYYMMDD_HHMMSS) |
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
CLI configuration file path |
--dry-run |
boolean |
false |
Simulate restore without making changes |
--collections |
string |
null |
Comma-separated list of collections |
--force |
boolean |
false |
Skip confirmation prompt |
backup check-coverageΒΆ
Check validation coverage for all MongoDB collections.
| Parameter | Type | Default | Description |
|---|---|---|---|
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
CLI configuration file path |
π Migrate CommandsΒΆ
Admin Access Required
All migrate operations require administrator privileges on both the source and target instances.
migrateΒΆ
Migrate a project from one Depictio instance to another β non-destructive upsert, never wipes existing data on the target.
| Parameter | Type | Default | Description |
|---|---|---|---|
--project |
string |
required | Project name to migrate |
--CLI-config-path |
string |
~/.depictio/CLI.yaml |
Source instance CLI config (credentials + API URL) |
--target-config |
string |
~/.depictio/CLI_remote.yaml |
Target instance CLI config |
--mode |
string |
all |
Migration scope: all, metadata, dashboard, files |
--dry-run |
flag |
False |
Preview changes without writing anything |
Migration modes:
| Mode | What is migrated | When to use |
|---|---|---|
all |
MongoDB documents + S3 files | First-time full migration |
metadata |
MongoDB documents only | Both instances share the same S3 storage |
dashboard |
Dashboard documents only | Project already exists on remote, updating layouts |
files |
S3 files only | MongoDB already migrated, syncing data files |
# Dry-run first (recommended before any migration)
depictio-cli migrate \
--project "My Project" \
--CLI-config-path ~/.depictio/CLI_local.yaml \
--target-config ~/.depictio/CLI_remote.yaml \
--dry-run
# Full migration (MongoDB + S3)
depictio-cli migrate \
--project "My Project" \
--CLI-config-path ~/.depictio/CLI_local.yaml \
--target-config ~/.depictio/CLI_remote.yaml
# Metadata-only (shared S3)
depictio-cli migrate \
--project "My Project" \
--CLI-config-path ~/.depictio/CLI_local.yaml \
--target-config ~/.depictio/CLI_remote.yaml \
--mode metadata
# Dashboard-only update
depictio-cli migrate \
--project "My Project" \
--CLI-config-path ~/.depictio/CLI_local.yaml \
--target-config ~/.depictio/CLI_remote.yaml \
--mode dashboard
π οΈ Common Use CasesΒΆ
π Quick StartΒΆ
# 1. Validate your project configuration
depictio-cli config validate-project-config --project-config-path ./config.yaml
# 2. Run the complete workflow
depictio-cli run --project-config-path ./config.yaml
# 1. Check system status
depictio-cli config check-server-accessibility
depictio-cli config check-s3-storage
# 2. Validate and sync configuration
depictio-cli config validate-project-config --project-config-path ./config.yaml
depictio-cli config sync-project-config-to-server --project-config-path ./config.yaml
# 3. Scan and process data
depictio-cli data scan --project-config-path ./config.yaml
depictio-cli data process --project-config-path ./config.yaml
π§ Development WorkflowΒΆ
# Preview changes without execution
depictio-cli run --project-config-path ./config.yaml --dry-run
# Test with specific workflow
depictio-cli run --project-config-path ./config.yaml --workflow-name test-workflow
# Enable verbose logging
depictio-cli run --project-config-path ./config.yaml --verbose --continue-on-error
# Skip problematic steps
depictio-cli run --project-config-path ./config.yaml --skip-server-check --skip-s3-check
πΎ Backup OperationsΒΆ
Admin Access Required
All backup operations require administrator privileges.
# Create database backup
depictio-cli backup create
# Include S3 deltatable data
depictio-cli backup create --include-s3-data
# List available backups
depictio-cli backup list
# Preview restore
depictio-cli backup restore 20240115_143052 --dry-run
# Restore specific backup
depictio-cli backup restore 20240115_143052 --force
π Data ManagementΒΆ
# Rescan all folders
depictio-cli data scan --project-config-path ./config.yaml --rescan-folders
# Sync file updates
depictio-cli data scan --project-config-path ./config.yaml --sync-files
# Overwrite existing workflow
depictio-cli data process --project-config-path ./config.yaml --overwrite
# Update and reprocess
depictio-cli run --project-config-path ./config.yaml --update-config --overwrite
π Project MigrationΒΆ
Admin Access Required
Migration requires administrator privileges on both source and target instances.
# 1. Dry-run to preview what will be migrated
depictio-cli migrate \
--project "My Project" \
--CLI-config-path ~/.depictio/CLI_local.yaml \
--target-config ~/.depictio/CLI_remote.yaml \
--dry-run
# 2. Run full migration (MongoDB docs + S3 files)
depictio-cli migrate \
--project "My Project" \
--CLI-config-path ~/.depictio/CLI_local.yaml \
--target-config ~/.depictio/CLI_remote.yaml
# When source and target share the same S3, only migrate MongoDB docs
depictio-cli migrate \
--project "My Project" \
--CLI-config-path ~/.depictio/CLI_local.yaml \
--target-config ~/.depictio/CLI_remote.yaml \
--mode metadata
# Push updated dashboard layouts to a remote instance
# (project and data already exist on remote)
depictio-cli migrate \
--project "My Project" \
--CLI-config-path ~/.depictio/CLI_local.yaml \
--target-config ~/.depictio/CLI_remote.yaml \
--mode dashboard