piaso.data — datasets and references

Example datasets, genome references and motif databases.

Function	What it does
build_tf_motif_map	Group PWMs by TF symbol, optionally restricting to known TFs and the RNA universe.
buildTFMotifMap	Group PWMs by TF symbol, optionally restricting to known TFs and the RNA universe.
ccres_near_tss	Return [(start, end)] cCREs on chrom within ±window of tss (binary search on sorted starts).
chembl_targets_to_dict	Collapse filtered activities into {'CHEMBL_ID|NAME': [genes]}.
chrom_sizes	Chromosome sizes of a genome, {chrom: size} in canonical order.
dataset_info	Get metadata for a specific dataset.
extract_sequences	Extract sequences for intervals = [(chrom, start, end, strand), ...].
fetch_2bit	Download the UCSC .2bit for genome (OPT-IN, ~700-800 MB).
fetch_animaltfdb_tf_list	Removed in 1.2.4 — AnimalTFDB cannot be downloaded from Python.
fetch_chembl	Download the merged ChEMBL 30 activity table (OPT-IN, ~2.7 GB).
fetch_cisbp	Download CIS-BP motifs (MEME format) from the MEME Suite bundle (OPT-IN).
fetch_cistarget_motifs	Download the aertslab cisTarget motif collection (PWMs + motif2TF) — OPT-IN.
fetch_codebook	Download the Codebook representative PWM archive (OPT-IN, ~1 MB).
fetch_dataset	Download a tutorial dataset into the PIASO data root.
fetch_genome	Download or copy genome reference files.
fetch_jaspar	Download a JASPAR MEME motif file (OPT-IN). Returns the cached path.
fetch_lr_database	Download the CellChatDB ligand-receptor table for species.
fetch_screen	Download the SCREEN cCRE BED for genome (OPT-IN). Returns the path.
fetch_tf_list	Download a TF symbol list for species and cache it locally.
fetchChEMBL	Download the merged ChEMBL 30 activity table (OPT-IN, ~2.7 GB).
fetchCISBP	Download CIS-BP motifs (MEME format) from the MEME Suite bundle (OPT-IN).
fetchGenomeFasta	Download the UCSC .2bit for genome (OPT-IN, ~700-800 MB).
fetchJASPAR	Download a JASPAR MEME motif file (OPT-IN). Returns the cached path.
fetchLRDatabase	Download the CellChatDB ligand-receptor table for species.
fetchSCREEN	Download the SCREEN cCRE BED for genome (OPT-IN). Returns the path.
fetchTFList	Download a TF symbol list for species and cache it locally.
filter_chembl_activities	Filter raw ChEMBL activities down to confident drug-target pairs.
list_available_genomes	Return list of supported genome names.
list_available_gtf_sources	Return {source: [release, ...]} for the given genome.
list_datasets	List available tutorial datasets.
list_downloaded_genomes	Return list of genome names present under the PIASO data root.
load_chembl_targets	Build drug-target gene sets from ChEMBL, ready for :func:piaso.tl.score.
load_cisbp	Load motifs from a CIS-BP PWM directory.
load_cisbp_meme	Parse a CIS-BP MEME file (MEME Suite bundle) → list[PWM] (source='cisbp').
load_cistarget_motifs	Load the cisTarget collection -> list[PWM] with tf_name from the motif2TF table.
load_codebook	Parse the Codebook PWM archive -> list[PWM] (source='codebook').
load_dataset	Download and load a tutorial dataset as AnnData or a cytome Dataset.
load_jaspar_meme	Parse a JASPAR MEME-format file → list[PWM] (source='jaspar').
load_lr_database	Load the CellChatDB table, fetching it first if needed.
load_meme	Parse a MEME-format motif file (JASPAR, CIS-BP-from-MEME, …) → list[PWM].
load_screen_ccres	Load cCREs into {chrom: {'starts': sorted ndarray, 'ends': ndarray}}.
load_tf_list	Return a set of TF gene symbols.
loadChEMBLTargets	Build drug-target gene sets from ChEMBL, ready for :func:piaso.tl.score.
loadLRDatabase	Load the CellChatDB table, fetching it first if needed.
loadMotifs	Parse a JASPAR MEME-format file → list[PWM] (source='jaspar').
loadTFList	Return a set of TF gene symbols.
makeGenomeFiles	Write <genome>.chrom.sizes and three BED files from a reference.
PWM	A single transcription-factor motif.
refresh_registry	Force re-download the dataset registry from GitHub.
resolve_2bit_path	Return a local .2bit path for genome if one exists, else None.
resolve_chembl_path	Local path to a cached ChEMBL table, or None if it is not there.
resolve_cisbp_meme_path	Return a usable CIS-BP MEME path (explicit, or cached), else None.
resolve_cistarget_paths	Return (cb_singletons_dir, motif2tf_tbl_path) if cached, else (None, None).
resolve_genome_files	Return dict with resolved paths to BED files for a genome.
resolve_jaspar_path	Return a usable JASPAR MEME path (explicit jaspar_path or cached), else None.
resolve_lr_path	Local path for a cached LR database, or None if it is not there yet.
resolve_screen_path	No description.
revcomp	Reverse-complement a DNA string (IUPAC ACGTN; case preserved).
write_meme	Write a list[PWM] to a minimal MEME-format file (MOTIF <id>_<tf> <tf> headers, so :func:load_meme recovers tf_name from the 3rd token). Lets any PWM collection (e.g. cisTarget) drive inferGRN’s existing jaspar_path= (MEME) interface.

build_tf_motif_map

build_tf_motif_map(
    pwms: 'list[PWM]',
    tf_list: 'Optional[set[str]]' = None,
    gene_universe: 'Optional[list[str]]' = None,
)

Signature defaults

pwms, tf_list=None, gene_universe=None

Group PWMs by TF symbol, optionally restricting to known TFs and the RNA universe.

Parameters

pwms — list[PWM]

Source PWMs (from :func:load_jaspar_meme or :func:load_cisbp).

tf_list — set[str] or None

If given, only PWMs whose tf_name matches a symbol in tf_list (case-insensitive) are retained.

gene_universe — list[str] or None

The RNA var_names (exact-case gene symbols in the expression matrix). When provided, further restricts to TFs present in the universe and remaps each retained PWM’s tf_name to the exact-case symbol used in the expression matrix. Handles human (ALL-CAPS) and mouse (Title-case) convention differences via case-insensitive matching.

Returns

dict[str, list[PWM]] {tf_symbol: [PWM, ...]} where tf_symbol is the exact-case key from gene_universe (when provided) or the original tf_name / the tf_list symbol.

Notes

PWMs with tf_name matching multiple gene_universe entries (e.g. a DB uses "STAT3" but the universe has both "Stat3" and "STAT3") are mapped to the first (alphabetically sorted) match to remain deterministic.

buildTFMotifMap

build_tf_motif_map(
    pwms: 'list[PWM]',
    tf_list: 'Optional[set[str]]' = None,
    gene_universe: 'Optional[list[str]]' = None,
)

Signature defaults

pwms, tf_list=None, gene_universe=None

Group PWMs by TF symbol, optionally restricting to known TFs and the RNA universe.

Parameters

pwms — list[PWM]

Source PWMs (from :func:load_jaspar_meme or :func:load_cisbp).

tf_list — set[str] or None

If given, only PWMs whose tf_name matches a symbol in tf_list (case-insensitive) are retained.

gene_universe — list[str] or None

The RNA var_names (exact-case gene symbols in the expression matrix). When provided, further restricts to TFs present in the universe and remaps each retained PWM’s tf_name to the exact-case symbol used in the expression matrix. Handles human (ALL-CAPS) and mouse (Title-case) convention differences via case-insensitive matching.

Returns

dict[str, list[PWM]] {tf_symbol: [PWM, ...]} where tf_symbol is the exact-case key from gene_universe (when provided) or the original tf_name / the tf_list symbol.

Notes

PWMs with tf_name matching multiple gene_universe entries (e.g. a DB uses "STAT3" but the universe has both "Stat3" and "STAT3") are mapped to the first (alphabetically sorted) match to remain deterministic.

ccres_near_tss

ccres_near_tss(
    ccres: 'Dict[str, dict]',
    chrom: 'str',
    tss: 'int',
    window: 'int',
)

Signature defaults

ccres, chrom, tss, window

Return [(start, end)] cCREs on chrom within ±window of tss (binary search on sorted starts).

chembl_targets_to_dict

chembl_targets_to_dict(dataframe, sep: 'str' = '|')

Signature defaults

dataframe, sep='|'

Collapse filtered activities into {'CHEMBL_ID|NAME': [genes]}.

The gene column holds pipe-separated synonyms for one protein, so it is split and de-duplicated per compound.

chrom_sizes

chrom_sizes(genome: str)

Signature defaults

genome

Chromosome sizes of a genome, {chrom: size} in canonical order.

Parameters

genome

A name PIASO knows (hg38, mm10, Mmul_10, mCalJac1, hg19, mm39) or the path to a chrom.sizes / .fai file.

Returns

dict

Primary chromosomes only, numbered first, then X and Y. For a name, exactly the rows the fragment importer builds its tile grid from and the peak caller walks; for a file, the same reduction those tools apply.

Examples

>>> piaso.data.chrom_sizes("Mmul_10")["chr1"]
223616942
>>> len(piaso.data.chrom_sizes("/path/to/panTro6.chrom.sizes"))

dataset_info

dataset_info(name: str)

Signature defaults

name

Get metadata for a specific dataset.

Parameters

name — str

Dataset name (e.g., ‘sea_ad_mtg_20k’).

Returns

info — dict

Dataset metadata including title, description, URL, size, etc.

extract_sequences

extract_sequences(
    twobit_path: 'str',
    intervals: 'Sequence[Tuple[str, int, int, str]]',
    uppercase: 'bool' = True,
)

Signature defaults

twobit_path, intervals, uppercase=True

Extract sequences for intervals = [(chrom, start, end, strand), ...].

0-based half-open coordinates. strand == '-' returns the reverse complement. Out-of-range / missing-chrom intervals yield "" (the caller can drop them). Opens the .2bit once and reuses the handle (RAM = one sequence at a time).

fetch_2bit

fetch_2bit(
    genome: 'str',
    dest_dir: 'Optional[str]' = None,
    force: 'bool' = False,
)

Signature defaults

genome, dest_dir=None, force=False

Download the UCSC .2bit for genome (OPT-IN, ~700-800 MB).

Returns the local path. No-op if already present (unless force).

fetch_animaltfdb_tf_list

fetch_animaltfdb_tf_list(species: 'str', dest_dir: 'Optional[str]' = None)

Signature defaults

species, dest_dir=None

Removed in 1.2.4 — AnimalTFDB cannot be downloaded from Python.

.. deprecated:: 1.2.4 Raises immediately, without a network call. AnimalTFDB’s only host is behind a WAF that answers every non-browser client with a captcha page, so this function could never return a TF list; the page it did return parsed into a dozen junk “symbols”. Save the table from a browser and read it with load_tf_list(path=...), or use fetch_tf_list(species), which defaults to cisTarget.

This shim exists because the function shipped in 1.2.3. It will be deleted in 1.3.

Raises

RuntimeError

Always.

fetch_chembl

fetch_chembl(dest_dir: 'Optional[str]' = None, force: 'bool' = False)

Signature defaults

dest_dir=None, force=False

Download the merged ChEMBL 30 activity table (OPT-IN, ~2.7 GB).

This is much larger than the other references PIASO fetches, and reading it needs roughly 10 GB of RAM. It is not downloaded implicitly by anything; call this when you want it.

Returns

str

Local path to the .pkl.

fetch_cisbp

fetch_cisbp(
    species='Homo_sapiens',
    version='2.00',
    dest_dir=None,
    force=False,
    genome=None,
)

Signature defaults

species='Homo_sapiens', version='2.00', dest_dir=None, force=False, genome=None

Download CIS-BP motifs (MEME format) from the MEME Suite bundle (OPT-IN).

The MEME Suite motif-database bundle ships per-species single-MEME CIS-BP files (CIS-BP_<version>/<Species>.meme; one motif per TF, redundancy-reduced). This streams the .tgz and extracts only the requested file (the CIS-BP entries sit near the start of the archive, so the whole bundle is not read).

Parameters

species — str

'Homo_sapiens' / 'Mus_musculus' (MEME naming). Or pass genome ('hg38' → Homo_sapiens, 'mm10' → Mus_musculus).

version — str

CIS-BP version inside the bundle (default '2.00'; '1.02' available).

fetch_cistarget_motifs

fetch_cistarget_motifs(
    species='Homo_sapiens',
    version='v10nr_clust',
    dest_dir=None,
    force=False,
    genome=None,
)

Signature defaults

species='Homo_sapiens', version='v10nr_clust', dest_dir=None, force=False, genome=None

Download the aertslab cisTarget motif collection (PWMs + motif2TF) — OPT-IN.

Downloads the <version>_public.zip cluster-buster singletons (~85 MB) and the species motif->TF annotation table into ~/.piaso/data/motifs/cistarget. Returns (cb_singletons_dir, motif2tf_tbl_path).

fetch_codebook

fetch_codebook(dest_dir=None, force=False)

Signature defaults

dest_dir=None, force=False

Download the Codebook representative PWM archive (OPT-IN, ~1 MB).

Supplementary Data 1 of Jolma, Laverty, Fathi et al., Nature 657, 275-283 (2026) — representative motifs for 1,421 human TFs, served by the publisher under the article’s CC-BY-4.0 licence.

Parameters

dest_dir — str, optional

Cache directory. Defaults to ~/.piaso/data/motifs.

force — bool, default False

Re-download even if the archive is already cached.

Returns

str

Path to the cached .zip. Pass it to :func:load_codebook.

See Also

load_codebook : parse the archive into PWM objects.

Notes

Added in 1.2.4. The companion Zenodo record (10.5281/zenodo.15667805) holds the full GRECO-BIT/Codebook Motif Explorer output — tens of gigabytes of per-experiment results. This is the small representative set, which is what a scan needs.

fetch_dataset

fetch_dataset(name: str, force: bool = False, data_dir=None)

Signature defaults

name, force=False, data_dir=None

Download a tutorial dataset into the PIASO data root.

Downloads the file if it is not already there. The download goes to a temporary name, its MD5 checksum is verified, and only then is it renamed into place, so a file under its final name is always complete. An existing file is returned as it is, without being read again: a downloaded .cytome is a working file that analyses write into, and it is never replaced unless force=True.

Parameters

name — str

Dataset name (e.g., ‘sea_ad_mtg_20k’, ‘mouse_brain_10k_gemx’).

force — bool

If True, download again and replace the existing file (only once the new download has passed its checksum).

data_dir — str or Path, optional

Store this dataset under <data_dir>/datasets/ instead of the default root. Resolution order, most specific wins: this argument, then piaso.settings.data_dir, then the PIASO_DATA_DIR environment variable, then ~/.piaso/data.

Returns

path — Path

Local path to the downloaded file.

Examples

>>> path = piaso.data.fetch_dataset("sea_ad_mtg_20k")
>>> adata = anndata.read_h5ad(path)

fetch_genome

fetch_genome(
    genome: str,
    source_dir: Optional[str] = None,
    dest_dir: Optional[str] = None,
    force: bool = False,
    download_gtf: bool = True,
    source: str = 'gencode',
    release: Optional[str] = None,
)

Signature defaults

genome, source_dir=None, dest_dir=None, force=False, download_gtf=True, source='gencode', release=None

Download or copy genome reference files.

Parameters

genome — str

Genome name (e.g., ‘hg38’, ‘mm10’).

source_dir — str, optional

Local directory containing the genome files. If provided, copies from this directory instead of downloading.

dest_dir — str, optional

Destination directory for the genome files. If None (default), uses ~/.piaso/data/{genome}/. Files always end up in a {genome}/ subdirectory under whichever destination is used.

force — bool

If True, re-download even if files exist.

download_gtf — bool, default True

Whether to fetch the GTF file. The GTF is large (~30-50 MB) relative to the BED files, so users can opt out if they don’t need the gene-structure overlay in plotCoverage / plotBigWig.

source — str, default 'gencode'

GTF upstream source. One of 'gencode', 'refseq', 'ensembl'. RefSeq + Ensembl are auto-remapped to UCSC chr1-style chrom names; GENCODE already uses them.

release — str, optional

GTF release identifier. None picks the per-source default from :data:DEFAULT_GTF_RELEASE. Use :func:list_available_gtf_sources to enumerate.

Returns

genome_dir — str

Path to the genome data directory (as a string, so it drops straight into open() / os.path.join / f-strings; wrap in pathlib.Path(...) if you need Path methods). Matches the string paths returned by :func:resolve_genome_files.

fetch_jaspar

fetch_jaspar(
    release='JASPAR2024',
    collection='CORE',
    taxon='vertebrates',
    dest_dir=None,
    force=False,
)

Signature defaults

release='JASPAR2024', collection='CORE', taxon='vertebrates', dest_dir=None, force=False

Download a JASPAR MEME motif file (OPT-IN). Returns the cached path.

Parameters

release — str

e.g. 'JASPAR2024' (the year 2024 is parsed from the trailing digits).

collection — str

e.g. 'CORE', 'UNVALIDATED'.

taxon — str

e.g. 'vertebrates', 'plants', 'insects', 'fungi', 'nematodes', 'urochordates'.

fetch_lr_database

fetch_lr_database(
    species: 'str' = 'human',
    dest_dir: 'Optional[str]' = None,
    force: 'bool' = False,
)

Signature defaults

species='human', dest_dir=None, force=False

Download the CellChatDB ligand-receptor table for species.

Parameters

species

'human' or 'mouse'.

dest_dir

Where to cache it. Defaults to ~/.piaso/data.

force

Re-download even if it is already cached.

Returns

str

Local path to the CSV.

fetch_screen

fetch_screen(genome, dest_dir=None, force=False)

Signature defaults

genome, dest_dir=None, force=False

Download the SCREEN cCRE BED for genome (OPT-IN). Returns the path.

fetch_tf_list

fetch_tf_list(
    species: 'str',
    source: 'str' = 'cistarget',
    dest_dir: 'Optional[str]' = None,
)

Signature defaults

species, source='cistarget', dest_dir=None

Download a TF symbol list for species and cache it locally.

Parameters

species — str

"human" or "mouse".

source — str

"cistarget" (default), the only fetchable catalogue. See Notes.

dest_dir — str or None

Directory to save the file. Defaults to ~/.piaso/grn/.

Returns

str

Absolute path to the cached file.

Notes

This function makes a network request. It is deliberately not called automatically by :func:load_tf_list, so that a pipeline is network-free and reproducible after the first run.

For regulon inference you usually need neither. inferRegulon with tf_list=None already restricts to TFs that (a) are named in the motif database and (b) are expressed in the data, which is the only set that can produce an edge.

Changed in 1.2.4: source="animaltfdb" was removed. Its only host (guolab.wchscu.cn) is behind a WAF that answers every non-browser client — urllib, requests and curl alike, under any User-Agent — with a captcha page or 405, and the old bioinfo.life.hust.edu.cn mirror no longer resolves. Save the table from a browser and read it with load_tf_list(path=...).

The downloaded file is checked before it is kept: a page that arrives with HTTP 200 but is not a TF list is deleted rather than cached.

fetchChEMBL

fetch_chembl(dest_dir: 'Optional[str]' = None, force: 'bool' = False)

Signature defaults

dest_dir=None, force=False

Download the merged ChEMBL 30 activity table (OPT-IN, ~2.7 GB).

This is much larger than the other references PIASO fetches, and reading it needs roughly 10 GB of RAM. It is not downloaded implicitly by anything; call this when you want it.

Returns

str

Local path to the .pkl.

fetchCISBP

fetch_cisbp(
    species='Homo_sapiens',
    version='2.00',
    dest_dir=None,
    force=False,
    genome=None,
)

Signature defaults

species='Homo_sapiens', version='2.00', dest_dir=None, force=False, genome=None

Download CIS-BP motifs (MEME format) from the MEME Suite bundle (OPT-IN).

The MEME Suite motif-database bundle ships per-species single-MEME CIS-BP files (CIS-BP_<version>/<Species>.meme; one motif per TF, redundancy-reduced). This streams the .tgz and extracts only the requested file (the CIS-BP entries sit near the start of the archive, so the whole bundle is not read).

Parameters

species — str

'Homo_sapiens' / 'Mus_musculus' (MEME naming). Or pass genome ('hg38' → Homo_sapiens, 'mm10' → Mus_musculus).

version — str

CIS-BP version inside the bundle (default '2.00'; '1.02' available).

fetchGenomeFasta

fetch_2bit(
    genome: 'str',
    dest_dir: 'Optional[str]' = None,
    force: 'bool' = False,
)

Signature defaults

genome, dest_dir=None, force=False

Download the UCSC .2bit for genome (OPT-IN, ~700-800 MB).

Returns the local path. No-op if already present (unless force).

fetchJASPAR

fetch_jaspar(
    release='JASPAR2024',
    collection='CORE',
    taxon='vertebrates',
    dest_dir=None,
    force=False,
)

Signature defaults

release='JASPAR2024', collection='CORE', taxon='vertebrates', dest_dir=None, force=False

Download a JASPAR MEME motif file (OPT-IN). Returns the cached path.

Parameters

release — str

e.g. 'JASPAR2024' (the year 2024 is parsed from the trailing digits).

collection — str

e.g. 'CORE', 'UNVALIDATED'.

taxon — str

e.g. 'vertebrates', 'plants', 'insects', 'fungi', 'nematodes', 'urochordates'.

fetchLRDatabase

fetch_lr_database(
    species: 'str' = 'human',
    dest_dir: 'Optional[str]' = None,
    force: 'bool' = False,
)

Signature defaults

species='human', dest_dir=None, force=False

Download the CellChatDB ligand-receptor table for species.

Parameters

species

'human' or 'mouse'.

dest_dir

Where to cache it. Defaults to ~/.piaso/data.

force

Re-download even if it is already cached.

Returns

str

Local path to the CSV.

fetchSCREEN

fetch_screen(genome, dest_dir=None, force=False)

Signature defaults

genome, dest_dir=None, force=False

Download the SCREEN cCRE BED for genome (OPT-IN). Returns the path.

fetchTFList

fetch_tf_list(
    species: 'str',
    source: 'str' = 'cistarget',
    dest_dir: 'Optional[str]' = None,
)

Signature defaults

species, source='cistarget', dest_dir=None

Download a TF symbol list for species and cache it locally.

Parameters

species — str

"human" or "mouse".

source — str

"cistarget" (default), the only fetchable catalogue. See Notes.

dest_dir — str or None

Directory to save the file. Defaults to ~/.piaso/grn/.

Returns

str

Absolute path to the cached file.

Notes

This function makes a network request. It is deliberately not called automatically by :func:load_tf_list, so that a pipeline is network-free and reproducible after the first run.

For regulon inference you usually need neither. inferRegulon with tf_list=None already restricts to TFs that (a) are named in the motif database and (b) are expressed in the data, which is the only set that can produce an edge.

Changed in 1.2.4: source="animaltfdb" was removed. Its only host (guolab.wchscu.cn) is behind a WAF that answers every non-browser client — urllib, requests and curl alike, under any User-Agent — with a captcha page or 405, and the old bioinfo.life.hust.edu.cn mirror no longer resolves. Save the table from a browser and read it with load_tf_list(path=...).

The downloaded file is checked before it is kept: a page that arrives with HTTP 200 but is not a TF list is deleted rather than cached.

filter_chembl_activities

filter_chembl_activities(
    dataframe,
    drug_max_phase: 'Optional[Union[int, Sequence[int]]]' = 4,
    assay_type: 'Optional[Union[str, Sequence[str]]]' = 'F',
    add_drug_mechanism: 'bool' = True,
    remove_inactive: 'bool' = True,
    include_active: 'bool' = True,
    pchembl_target_column: 'Optional[str]' = 'target_class',
    pchembl_threshold: 'Optional[Union[float, Dict[str, float]]]' = None,
    default_pchembl: 'float' = 6.0,
    verbose: 'bool' = True,
)

Signature defaults

dataframe, drug_max_phase=4, assay_type='F', add_drug_mechanism=True, remove_inactive=True, include_active=True, pchembl_target_column='target_class', pchembl_threshold=None, default_pchembl=6.0, verbose=True

Filter raw ChEMBL activities down to confident drug-target pairs.

The filters are applied in the order of the arguments, and two of them (add_drug_mechanism, include_active) do not remove rows — they mark rows as protected so later filters cannot drop them. A curated mechanism of action is better evidence than any activity measurement, so a row carrying one survives an assay-type or potency cut it would otherwise fail.

Parameters

dataframe

The merged ChEMBL table from :func:fetch_chembl.

drug_max_phase

Clinical stage to keep. 4 is approved drugs; pass a list for several, or None for no filter.

assay_type

ChEMBL assay type. 'F' (functional) measures a biological effect; 'B' (binding) measures affinity.

add_drug_mechanism

Protect rows that have a curated drug mechanism.

remove_inactive

Drop rows whose activity comment says the compound was inactive.

include_active

Protect rows whose activity comment says it was active.

pchembl_target_column

Column holding the target class, used to look up per-class thresholds.

pchembl_threshold

A single pChEMBL minimum, or a dict of {target_class: minimum}. Defaults to :data:PCHEMBL_THRESHOLDS.

default_pchembl

Threshold applied to target classes absent from the dict, so tables whose class vocabulary differs from :data:PCHEMBL_THRESHOLDS still filter instead of raising; verbose reports which classes took the fallback.

verbose

Report how many rows each step removed.

Returns

pandas.DataFrame

The surviving rows, with the helper columns keep and pchembl_active retained so the decisions stay inspectable.

list_available_genomes

list_available_genomes()

Return list of supported genome names.

list_available_gtf_sources

list_available_gtf_sources(genome: str)

Signature defaults

genome

Return {source: [release, ...]} for the given genome.

Example: list_available_gtf_sources('mm10') → {'gencode': ['vM25'], 'refseq': ['106'], 'ensembl': ['102']}.

list_datasets

list_datasets()

List available tutorial datasets.

Returns

datasets — dict

Dictionary mapping dataset names to their metadata.

Examples

>>> piaso.data.list_datasets()

list_downloaded_genomes

list_downloaded_genomes()

Return list of genome names present under the PIASO data root.

load_chembl_targets

load_chembl_targets(
    chembl_pkl: 'Optional[str]' = None,
    data_dir: 'Optional[str]' = None,
    min_genes: 'int' = 1,
    return_table: 'bool' = False,
    **filter_kwargs,
)

Signature defaults

chembl_pkl=None, data_dir=None, min_genes=1, return_table=False, **filter_kwargs

Build drug-target gene sets from ChEMBL, ready for :func:piaso.tl.score.

Fetches the table on first use (~2.7 GB, see :func:fetch_chembl), applies :func:filter_chembl_activities, and returns {'CHEMBL_ID|DRUG NAME': [target gene symbols]}.

Parameters

chembl_pkl

Use this file instead of the cached one.

data_dir

Look here before the default cache.

min_genes

Drop drugs with fewer than this many targets. A one-gene set is a legitimate result for a selective drug, so the default keeps them.

return_table

Also return the filtered DataFrame.

**filter_kwargs

Passed to :func:filter_chembl_activities — drug_max_phase, assay_type, pchembl_threshold and so on.

Returns

dict, or (dict, DataFrame) when return_table=True

load_cisbp

load_cisbp(pwm_dir: 'str', tf_info_path: 'Optional[str]' = None)

Signature defaults

pwm_dir, tf_info_path=None

Load motifs from a CIS-BP PWM directory.

Parameters

pwm_dir — str

Directory containing per-motif files named <motif_id>.txt. Each file has a tab-separated header row Pos\tA\tC\tG\tT followed by one row per position.

tf_info_path — str or None

Optional path to TF_Information.txt (tab-separated, columns include Motif_ID and TF_Name). When None or the file is missing, the motif filename stem is used as tf_name.

Returns

list[PWM] One PWM per successfully parsed file. source is "cisbp".

Raises

FileNotFoundError

If pwm_dir does not exist.

load_cisbp_meme

load_cisbp_meme(path: 'str')

Signature defaults

path

Parse a CIS-BP MEME file (MEME Suite bundle) → list[PWM] (source='cisbp').

For the MEME-format CIS-BP file produced by :func:fetch_cisbp (e.g. CIS-BP_2.00/Homo_sapiens.meme). The raw per-motif CIS-BP layout (<motif_id>.txt + TF_Information.txt) is handled by :func:load_cisbp.

load_cistarget_motifs

load_cistarget_motifs(
    cb_dir,
    motif2tf_path,
    tf_universe=None,
    max_per_tf=None,
    direct_only=False,
)

Signature defaults

cb_dir, motif2tf_path, tf_universe=None, max_per_tf=None, direct_only=False

Load the cisTarget collection -> list[PWM] with tf_name from the motif2TF table.

Parameters

cb_dir — str

Directory of cluster-buster <motif_id>.cb singletons.

motif2tf_path — str

Path to a motifs-...-nr.<species>-m...-o....tbl annotation (maps motif_id -> gene_name/TF, with similarity q-value + a ‘directly annotated’ vs orthology note).

tf_universe — set[str] or None

Restrict to these TF symbols (upper-cased) — pass the dataset’s expressed TFs to keep the scan small.

max_per_tf — int or None

Keep at most this many motifs per TF (best motif-similarity q-value first). 1 gives one representative motif per TF (comparable in size to JASPAR/CIS-BP).

direct_only — bool

Keep only motifs directly annotated to the TF (drop orthology-extended rows).

load_codebook

load_codebook(path=None, keep_construct_variants=False)

Signature defaults

path=None, keep_construct_variants=False

Parse the Codebook PWM archive -> list[PWM] (source='codebook').

Parameters

path — str, optional

The .zip from :func:fetch_codebook, or a directory holding the unpacked SupplementaryData1/*.txt files. None fetches (and caches) the archive.

keep_construct_variants — bool, default False

Some files name the construct assayed rather than the gene — CASZ1.FL (full length), ZNF729.DBD3 (a single DNA-binding domain). By default the suffix is stripped so the motif is filed under the gene symbol, which is what build_tf_motif_map matches against. Set True to keep the names verbatim.

Returns

list[PWM] One per motif file; TFs with two representative motifs contribute two, and the cistrome step pools them per TF as it does for JASPAR.

Notes

Added in 1.2.4. Human only — there is no mouse Codebook. On mouse data it behaves like JASPAR’s vertebrate collection: :func:build_tf_motif_map matches case-insensitively and remaps to the symbols in var_names, so ALL-CAPS motif names still find Sox9.

Three clone identifiers (AC023509.3 and two others) are dropped: they are not gene symbols and cannot match an expression matrix.

Against JASPAR 2024 CORE vertebrates (754 TFs) this is 1,421 TFs, covering all but eleven of JASPAR’s. The extra 678 are dominated by C2H2 zinc-finger proteins (351), whose motifs come largely from ChIP-exo on transposable elements and run up to 41 bp — worth knowing before scanning a 1.5 kb promoter window with them.

References

Jolma A., Laverty K.U., Fathi A. et al. An expanded codebook of human transcription factor DNA-binding specificity. Nature 657, 275-283 (2026). doi:10.1038/s41586-026-10798-9. Motif provenance per TF is in the paper’s Supplementary Table 10.

load_dataset

load_dataset(
    name: str,
    return_type: str = 'anndata',
    modality: str = 'RNA',
    cytome_path=None,
    backed: bool = True,
    data_dir=None,
    **kwargs,
)

Signature defaults

name, return_type='anndata', modality='RNA', cytome_path=None, backed=True, data_dir=None, **kwargs

Download and load a tutorial dataset as AnnData or a cytome Dataset.

Parameters

name — str

Dataset name (e.g., ‘sea_ad_mtg_20k’).

return_type — str, default ‘anndata’

'anndata' returns an AnnData (default). 'cytome' converts the downloaded data to a cytome and returns an opened cytome.Dataset (the caller owns it — call .close() when done). The cytome is built once and cached, so repeat calls reopen it.

modality — str, default ‘RNA’

Modality used when converting to a cytome (return_type='cytome').

cytome_path — str, optional

Where to write/read the cytome. Defaults to a .cytome file beside the cached download (<name>.cytome).

backed — bool, default True

For h5ad → cytome, use the streaming (bounded-RAM) from_h5ad path.

data_dir — str or Path, optional

Store the download under <data_dir>/datasets/ (see :func:fetch_dataset for the full resolution order).

**kwargs

Passed to anndata.read_h5ad() / :func:piaso.pp.read_10x_h5 (AnnData path, or the 10x/csv → cytome conversion).

Returns

AnnData or cytome.Dataset

Examples

>>> adata = piaso.data.load_dataset("sea_ad_mtg_20k")
>>> ds = piaso.data.load_dataset("sea_ad_mtg_20k", return_type="cytome")

load_jaspar_meme

load_jaspar_meme(path: 'str')

Signature defaults

path

Parse a JASPAR MEME-format file → list[PWM] (source='jaspar').

Thin wrapper over :func:load_meme. Pair with :func:fetch_jaspar.

load_lr_database

load_lr_database(
    species: 'str' = 'human',
    annotation: 'Optional[str]' = None,
    lr_csv: 'Optional[str]' = None,
    data_dir: 'Optional[str]' = None,
)

Signature defaults

species='human', annotation=None, lr_csv=None, data_dir=None

Load the CellChatDB table, fetching it first if needed.

Parameters

species

'human' or 'mouse'. Ignored when lr_csv is given.

annotation

Keep only one interaction class — one of :data:ANNOTATION_CLASSES. Secreted signalling and direct contact are different mechanisms; pooling them is a choice worth making explicitly.

lr_csv

Use this file instead of the cached one.

data_dir

Look here before the default cache.

Returns

pandas.DataFrame

Columns include ligand, receptor, pathway_name and annotation, plus CellChatDB’s own metadata.

load_meme

load_meme(path: 'str', source: 'str' = 'meme')

Signature defaults

path, source='meme'

Parse a MEME-format motif file (JASPAR, CIS-BP-from-MEME, …) → list[PWM].

Pure-Python parser (NumPy + stdlib only — no MEME package required). Used by both :func:load_jaspar_meme (source='jaspar') and :func:load_cisbp_meme (source='cisbp').

Parameters

path — str

Path to a MEME 4 file (e.g. JASPAR2024_CORE_vertebrates.meme or CIS-BP_2.00/Homo_sapiens.meme from the MEME Suite bundle).

source — str

Provenance label stored on each returned PWM ('jaspar' / 'cisbp').

Returns

list[PWM] One PWM per MOTIF block. The parser is robust to blank lines, URL lines and varying whitespace; MOTIF <id> with only two tokens sets tf_name = motif_id, MOTIF <id> <name> uses the name.

load_screen_ccres

load_screen_ccres(
    path: 'str',
    classes: 'Optional[Sequence[str]]' = ('PLS', 'pELS', 'dELS'),
)

Signature defaults

path, classes=('PLS', 'pELS', 'dELS')

Load cCREs into {chrom: {'starts': sorted ndarray, 'ends': ndarray}}.

classes filters the 6th-column class (substring match, e.g. 'pELS' matches; None keeps all).

load_tf_list

load_tf_list(
    species: 'str' = 'human',
    source: 'str' = 'cistarget',
    path: 'Optional[str]' = None,
)

Signature defaults

species='human', source='cistarget', path=None

Return a set of TF gene symbols.

Parameters

species — str

Species identifier ("human" or "mouse"). Ignored when path is given.

source — str

"cistarget" (default) — the cisTarget / SCENIC+ TF universe (~1,890 human, ~1,860 mouse symbols), and the only fetchable catalogue. "motifdb" — sentinel meaning “use the motif DB’s own TF names”; returns None and build_tf_motif_map interprets that as no restriction. This is the right choice for regulon inference: a TF with no motif in the database cannot be scanned, so restricting to a catalogue on top of it can only remove TFs that were usable.

path — str or None

Path to a local TF list. Accepts either a one-symbol-per-line plain text file OR a TSV with a Symbol / TF / gene header column. When provided, species and source are ignored. This is the route for any catalogue PIASO cannot download — an AnimalTFDB *_TF table saved from a browser is read here as-is.

Returns

set[str] or None Set of gene symbols, or None when source="motifdb".

Raises

ValueError

For an unsupported source.

FileNotFoundError

If a local path is specified but does not exist.

RuntimeError

When no path is given and no cached file exists — instructs the caller to run :func:fetch_tf_list once.

Notes

Changed in 1.2.4: source="animaltfdb" was removed. Its host serves a WAF captcha page to every non-browser client, so the download never produced a TF list; the page used to parse into a dozen junk “symbols”. Files read here are now checked for that failure mode (see :func:_read_tf_list_file), and an AnimalTFDB table saved from a browser is still read via path=.

loadChEMBLTargets

load_chembl_targets(
    chembl_pkl: 'Optional[str]' = None,
    data_dir: 'Optional[str]' = None,
    min_genes: 'int' = 1,
    return_table: 'bool' = False,
    **filter_kwargs,
)

Signature defaults

chembl_pkl=None, data_dir=None, min_genes=1, return_table=False, **filter_kwargs

Build drug-target gene sets from ChEMBL, ready for :func:piaso.tl.score.

Fetches the table on first use (~2.7 GB, see :func:fetch_chembl), applies :func:filter_chembl_activities, and returns {'CHEMBL_ID|DRUG NAME': [target gene symbols]}.

Parameters

chembl_pkl

Use this file instead of the cached one.

data_dir

Look here before the default cache.

min_genes

Drop drugs with fewer than this many targets. A one-gene set is a legitimate result for a selective drug, so the default keeps them.

return_table

Also return the filtered DataFrame.

**filter_kwargs

Passed to :func:filter_chembl_activities — drug_max_phase, assay_type, pchembl_threshold and so on.

Returns

dict, or (dict, DataFrame) when return_table=True

loadLRDatabase

load_lr_database(
    species: 'str' = 'human',
    annotation: 'Optional[str]' = None,
    lr_csv: 'Optional[str]' = None,
    data_dir: 'Optional[str]' = None,
)

Signature defaults

species='human', annotation=None, lr_csv=None, data_dir=None

Load the CellChatDB table, fetching it first if needed.

Parameters

species

'human' or 'mouse'. Ignored when lr_csv is given.

annotation

Keep only one interaction class — one of :data:ANNOTATION_CLASSES. Secreted signalling and direct contact are different mechanisms; pooling them is a choice worth making explicitly.

lr_csv

Use this file instead of the cached one.

data_dir

Look here before the default cache.

Returns

pandas.DataFrame

Columns include ligand, receptor, pathway_name and annotation, plus CellChatDB’s own metadata.

loadMotifs

load_jaspar_meme(path: 'str')

Signature defaults

path

Parse a JASPAR MEME-format file → list[PWM] (source='jaspar').

Thin wrapper over :func:load_meme. Pair with :func:fetch_jaspar.

loadTFList

load_tf_list(
    species: 'str' = 'human',
    source: 'str' = 'cistarget',
    path: 'Optional[str]' = None,
)

Signature defaults

species='human', source='cistarget', path=None

Return a set of TF gene symbols.

Parameters

species — str

Species identifier ("human" or "mouse"). Ignored when path is given.

source — str

"cistarget" (default) — the cisTarget / SCENIC+ TF universe (~1,890 human, ~1,860 mouse symbols), and the only fetchable catalogue. "motifdb" — sentinel meaning “use the motif DB’s own TF names”; returns None and build_tf_motif_map interprets that as no restriction. This is the right choice for regulon inference: a TF with no motif in the database cannot be scanned, so restricting to a catalogue on top of it can only remove TFs that were usable.

path — str or None

Path to a local TF list. Accepts either a one-symbol-per-line plain text file OR a TSV with a Symbol / TF / gene header column. When provided, species and source are ignored. This is the route for any catalogue PIASO cannot download — an AnimalTFDB *_TF table saved from a browser is read here as-is.

Returns

set[str] or None Set of gene symbols, or None when source="motifdb".

Raises

ValueError

For an unsupported source.

FileNotFoundError

If a local path is specified but does not exist.

RuntimeError

When no path is given and no cached file exists — instructs the caller to run :func:fetch_tf_list once.

Notes

Changed in 1.2.4: source="animaltfdb" was removed. Its host serves a WAF captcha page to every non-browser client, so the download never produced a TF list; the page used to parse into a dozen junk “symbols”. Files read here are now checked for that failure mode (see :func:_read_tf_list_file), and an AnimalTFDB table saved from a browser is still read via path=.

makeGenomeFiles

makeGenomeFiles(
    reference_dir: 'str',
    genome: 'str',
    outdir: 'str',
    chromosomes: 'Optional[List[str]]' = None,
    verbose: 'bool' = True,
)

Signature defaults

reference_dir, genome, outdir, chromosomes=None, verbose=True

Write <genome>.chrom.sizes and three BED files from a reference.

Parameters

reference_dir — str

A cellranger or cellranger-arc reference directory, holding fasta/genome.fa.fai and genes/genes.gtf.gz.

genome — str

Name to write the files under, e.g. 'Mmul_10'. Use the name of the reference folder, so the provenance of the files is visible.

outdir — str

Directory to write into, normally PIASO-data/<genome>/.

chromosomes — list of str, optional

Restrict to these sequence names. The default keeps everything, including unplaced scaffolds, which matches what the shipped genomes do and keeps the sizes file a faithful description of the reference.

verbose — bool, default True

Report what was written.

Returns

dict

{kind: path} for the four files written.

Notes

No cCRE file is produced: SCREEN publishes candidate cis-regulatory elements for human and mouse only, so splitby='ccre' is unavailable for other species. That is a gap in the source data, not in this function.

PWM

PWM(
    motif_id: 'str',
    tf_name: 'str',
    probs: 'np.ndarray',
    source: 'str' = 'jaspar',
    meta: 'dict' = <factory>,
)

Signature defaults

motif_id, tf_name, probs, source='jaspar', meta=<factory>

A single transcription-factor motif.

Attributes

motif_id

Stable DB identifier (e.g. JASPAR MA0004.1, CIS-BP M00123).

tf_name

TF gene symbol the motif is associated with (e.g. Arnt). One TF may own several PWMs; the cistrome step aggregates per TF (N2: union).

probs

(4, w) float32 probability matrix (columns sum to 1), rows = A,C,G,T.

source

"jaspar" / "cisbp" — provenance.

refresh_registry

refresh_registry()

Force re-download the dataset registry from GitHub.

Use this if new datasets have been added to PIASO-data.

resolve_2bit_path

resolve_2bit_path(
    genome: 'str',
    twobit_path: 'Optional[str]' = None,
    data_dir: 'Optional[str]' = None,
)

Signature defaults

genome, twobit_path=None, data_dir=None

Return a local .2bit path for genome if one exists, else None.

Search order: explicit twobit_path → <data_dir>/<genome>.2bit → ~/.piaso/data/<genome>/<genome>.2bit → ~/.piaso/data/<genome>.2bit. Never downloads (use :func:fetch_2bit for that).

resolve_chembl_path

resolve_chembl_path(
    chembl_pkl: 'Optional[str]' = None,
    data_dir: 'Optional[str]' = None,
)

Signature defaults

chembl_pkl=None, data_dir=None

Local path to a cached ChEMBL table, or None if it is not there.

resolve_cisbp_meme_path

resolve_cisbp_meme_path(
    species='Homo_sapiens',
    version='2.00',
    cisbp_meme_path=None,
    dest_dir=None,
    genome=None,
)

Signature defaults

species='Homo_sapiens', version='2.00', cisbp_meme_path=None, dest_dir=None, genome=None

Return a usable CIS-BP MEME path (explicit, or cached), else None.

resolve_cistarget_paths

resolve_cistarget_paths(
    species='Homo_sapiens',
    version='v10nr_clust',
    dest_dir=None,
    genome=None,
)

Signature defaults

species='Homo_sapiens', version='v10nr_clust', dest_dir=None, genome=None

Return (cb_singletons_dir, motif2tf_tbl_path) if cached, else (None, None).

resolve_genome_files

resolve_genome_files(
    genome: str,
    source: str = 'gencode',
    release: Optional[str] = None,
)

Signature defaults

genome, source='gencode', release=None

Return dict with resolved paths to BED files for a genome.

Parameters

genome — str

Genome name (e.g., ‘hg38’, ‘mm10’).

source — str, default 'gencode'

GTF source. One of 'gencode', 'refseq', 'ensembl'. Selects which cached GTF to return under the 'gtf' key.

release — str, optional

GTF release (e.g. 'vM25', 'v44', '110'). None picks the per-source default from :data:DEFAULT_GTF_RELEASE.

Returns

paths — dict

Keys: ‘gene_boundary’, ‘promoter’, ‘ctcf’, ‘chrom_sizes’, ‘tss_bed’, and optionally ‘gtf’ (if the requested GTF was downloaded).

Raises

ValueError

If genome / source / release is not supported.

FileNotFoundError

If required files are not downloaded yet.

resolve_jaspar_path

resolve_jaspar_path(
    release='JASPAR2024',
    collection='CORE',
    taxon='vertebrates',
    jaspar_path=None,
    dest_dir=None,
)

Signature defaults

release='JASPAR2024', collection='CORE', taxon='vertebrates', jaspar_path=None, dest_dir=None

Return a usable JASPAR MEME path (explicit jaspar_path or cached), else None.

resolve_lr_path

resolve_lr_path(
    species: 'str',
    lr_csv: 'Optional[str]' = None,
    data_dir: 'Optional[str]' = None,
)

Signature defaults

species, lr_csv=None, data_dir=None

Local path for a cached LR database, or None if it is not there yet.

resolve_screen_path

resolve_screen_path(genome, screen_bed=None, data_dir=None)

Signature defaults

genome, screen_bed=None, data_dir=None

No description.

revcomp

revcomp(seq: 'str')

Signature defaults

seq

Reverse-complement a DNA string (IUPAC ACGTN; case preserved).

write_meme

write_meme(pwms, path: 'str', bg=(0.25, 0.25, 0.25, 0.25))

Signature defaults

pwms, path, bg=(0.25, 0.25, 0.25, 0.25)

Write a list[PWM] to a minimal MEME-format file (MOTIF <id>_<tf> <tf> headers, so :func:load_meme recovers tf_name from the 3rd token). Lets any PWM collection (e.g. cisTarget) drive inferGRN’s existing jaspar_path= (MEME) interface.