Reference and example data
PIASO provides a built-in data management system for downloading and caching tutorial datasets and genome reference files. This notebook demonstrates how to use the piaso.data module.
All data is hosted on Zenodo and GitHub.
import piaso1. Browsing Available Datasets
Use piaso.data.list_datasets() to see all available tutorial datasets with their sizes and descriptions.
datasets = piaso.data.list_datasets()Name Title Size------------------------------------------------------------------------------------------------- sea_ad_mtg_20k SEA-AD MTG 20K subsample (raw) 1922.1 MB adult_cortex_multiome_rna Adult Mouse Cortex Multiome RNA (P57) 2663.4 MB mouse_brain_10k_gemx 10K Mouse Brain GEM-X v4 (10x Genomics) 68.7 MB e18_v3_cell E18 Mouse Brain Neurons 10K v3 (10x Genomics) 47.6 MB e18_v3_nuclei E18 Mouse Brain Nuclei 5K v3.1 (10x Genomics) 20.3 MB e18_v4_cell E18 Mouse Neurons 10K GEM-X v4 (10x Genomics) 67.6 MB pbmc_multiome_san1 Human PBMC snMultiome SAN1 (De Rop 2024) 76.7 MB pbmc_multiome_san2 Human PBMC snMultiome SAN2 (De Rop 2024) 87.8 MB piaso_markerdb_allen_immune PIASOmarkerDB Allen Human Immune Health Atlas L2 0.1 MBTo get detailed metadata for a specific dataset:
info = piaso.data.dataset_info("sea_ad_mtg_20k")print(f"Title: {info['title']}")print(f"Species: {info['species']}")print(f"Cells: {info.get('cells', 'N/A'):,}")print(f"Features: {info.get('features', 'N/A'):,}")print(f"Format: {info['format']}")print(f"Size: {info['size_bytes'] / 1e9:.1f} GB")print(f"Reference: {info.get('reference', '')}")print(f"Tutorials: {', '.join(info.get('tutorials', []))}")Title: SEA-AD MTG 20K subsample (raw)Species: humanCells: 20,000Features: 36,601Format: h5adSize: 1.9 GBReference: Gabitto et al. Nat Neurosci 27, 2366-2383 (2024)Tutorials: PIASO_tutorial, 01_showThePlotFunction, colorPalettes, SCALAR2. Downloading and Loading Datasets
Option A: Download and load in one step
load_dataset() downloads the file (if not already cached), verifies the MD5 checksum, and returns an AnnData object. This is the recommended approach for most users.
# Load an h5ad dataset (returns AnnData)# Note: This downloads ~1.8 GB on first run# adata = piaso.data.load_dataset("sea_ad_mtg_20k")# adata# Load a 10x h5 dataset (uses scanpy.read_10x_h5 internally)# Note: Downloads ~65 MB on first run# adata_10x = piaso.data.load_dataset("mouse_brain_10k_gemx")# adata_10x# Load a CSV dataset (returns pandas DataFrame) — small, ~115 KBmarkerdb = piaso.data.load_dataset("piaso_markerdb_allen_immune")markerdb.head() Cell_Type Gene Specificity_Score ... Species Tissue Condition0 Platelet GP9 16.455748 ... Human blood normal1 Erythrocyte ALAS2 16.422475 ... Human blood normal2 Platelet CMTM5 16.400534 ... Human blood normal3 Platelet TMEM40 16.373334 ... Human blood normal4 Erythrocyte AHSP 16.338337 ... Human blood normal
[5 rows x 7 columns]Option B: Download only (returns file path)
Use fetch_dataset() when you want to control how the file is loaded, or when working with very large files.
# fetch_dataset returns the local path without loadingpath = piaso.data.fetch_dataset("piaso_markerdb_allen_immune")print(f"File cached at: {path}")print(f"File size: {path.stat().st_size / 1e3:.1f} KB")File cached at: .../PIASOmarkerDB_AllenHumanImmuneHealthAtlas_L2_251219.csvFile size: 114.6 KBRe-downloading
Files are cached in ~/.piaso/data/datasets/. To force re-download (e.g., if the file is corrupted):
# Force re-download# path = piaso.data.fetch_dataset("sea_ad_mtg_20k", force=True)3. Genome Reference Files
PIASO also manages genome reference files (gene bodies, promoters, CTCF sites, chromosome sizes, TSS positions) needed for scATAC-seq analysis.
# List supported genomesprint("Supported genomes:", piaso.data.list_available_genomes())
# List locally downloaded genomesprint("Downloaded genomes:", piaso.data.list_downloaded_genomes())Supported genomes: ['hg38', 'mm10']Downloaded genomes: ['mm10', 'hg38']# Download genome reference files (requires PIASO-data GitHub release)# piaso.data.fetch_genome("hg38")# piaso.data.fetch_genome("mm10")# Resolve file paths for a genome (after download)# paths = piaso.data.resolve_genome_files("hg38")# for key, path in paths.items():# print(f" {key}: {path}")These genome files are used automatically by the functions that need a genome — motif scanning against promoter sequences, for instance, or any step that needs a TSS annotation.
You can also install genome files from a local directory:
# Install from a local directory# piaso.data.fetch_genome("hg38", source_dir="/path/to/genome/files")4. Refreshing the Dataset Registry
The dataset registry is fetched from PIASO-data on GitHub and cached locally. If new datasets have been added, refresh the registry:
piaso.data.refresh_registry()Dataset registry refreshed.5. Using Datasets in PIASO Tutorials
Here is a quick example showing how a downloaded dataset feeds into a PIASO analysis:
# Example: load a dataset and plot# (uncomment after downloading sea_ad_mtg_20k)## import scanpy as sc# adata = piaso.data.load_dataset("sea_ad_mtg_20k")# sc.pp.filter_cells(adata, min_genes=200)# sc.pp.filter_genes(adata, min_cells=3)# piaso.settings.set_figure_params()# piaso.pl.embedding(adata, basis="X_umap", color="subclass", title="SEA-AD MTG 20K")Summary
| Function | Purpose |
|---|---|
piaso.data.list_datasets() | Browse available tutorial datasets |
piaso.data.dataset_info(name) | Get metadata for a specific dataset |
piaso.data.fetch_dataset(name) | Download dataset, return local path |
piaso.data.load_dataset(name) | Download and load as AnnData/DataFrame |
piaso.data.refresh_registry() | Update dataset registry from GitHub |
piaso.data.list_available_genomes() | List supported genome references |
piaso.data.list_downloaded_genomes() | List locally cached genomes |
piaso.data.fetch_genome(genome) | Download genome reference files |
piaso.data.resolve_genome_files(genome) | Get paths to genome BED files |