Skip to content

Release notes

v1.2.8

Terminal window
pip install -U piaso-tools

This release fixes what a review of the whole package found in October 2026: three plots that showed wrong data, functions that gave another answer on a cytome than on an AnnData, and failures that passed without a word. A default call on an AnnData gives the same result as in 1.2.7 in most functions; every change in output is listed below, with what to do about it.

What to redo after upgrading:

  • Redraw vertical plotLigandReceptorLollipop plots (every row but the middle one named another interaction), plotLigandReceptorInteraction heatmaps drawn with one colormap (the ligand and receptor rows were swapped) and plot_dendrogram on a cytome (it clustered the groups on cell index and fragment counts).
  • Recompute ATAC FRiP from piaso.pp.calculateCellMetrics: on a matrix that counts insertions, two per fragment (the default), it was twice the fraction of fragments in peaks (up to 2.0), so halve a threshold chosen on 1.2.7’s values.
  • Name or score a cytome again if you used kamon (predictCellTypeByMarker) or calculateScoreParallel on one with the default arguments: a store’s raw counts were scored where an AnnData’s INFOG layer is.
  • Batch-aware runGDR on an AnnData (batch_key=... with the default layer='infog') found each batch’s markers on the raw counts; it now uses INFOG, as a cytome and a single batch do, so X_gdr changes.
  • runGDR with batch_key and groupby, on an AnnData or a cytome: a batch that lacks one of the groups no longer adds a dimension of unrelated genes for it, so X_gdr has one column fewer per such batch-group pair.
  • predictCellTypeByGDR labels: with COSG 1.2.0 or earlier, each reference type and each query cluster added a dimension of unrelated genes for the batch that lacks it, about half of the joint embedding. Labels transferred with it may change; re-run if affected.
  • projectGDR novelty for a reference built with batch_key reported a shift for every query, even for cells of the reference’s own distribution.
  • Scrublet scores all move a little, and a cell that cannot be scored now gets a missing call, not “not a doublet”.
  • Motif scans: a motif whose best word cannot reach the scan’s p-value is no longer scanned; it was called at its best word’s score, and numpy and Rust disagreed there.

Motif occupancy from the scanner: scan_motifs(..., occupancy=True)

  • New: piaso.pp.scan_motifs(..., occupancy=True) also returns, per motif and sequence, the sum over the sequence’s windows (both strands) of each window’s likelihood ratio, motif against background. It is the evidence for one site at an unknown position integrated over the positions, so every good site counts, not only the best.
  • It is defined for every motif, including one too short to reach the p-value at any single site (a 6 bp core, whose best word occurs by chance about once in 4 kb, has no hits at p 1e-4 but an occupancy that counts its copies). Region-level calls built on it are in cytorete (motif_calls="occupancy").
  • The numpy and Rust backends agree to the bit. With occupancy the scan takes about twice as long.

Naming cells from marker genes is piaso.tl.kamon

The marker-based naming method has a name, KAMON, and its function is piaso.tl.kamon. piaso.tl.predictCellTypeByMarker is the same function under its earlier name and keeps working unchanged, so existing scripts and tutorials need no edit.

The API reference now documents each function once and lists its other names beside it (for example align_spatial_coordinates beside alignSpatialCoordinates), instead of repeating the whole entry.

Naming and scoring a cytome score what an AnnData scores

  • On a cytome, kamon and calculateScoreParallel score score_layer ('infog' by default, normalized on the fly from the store’s INFOG parameters), as on an AnnData. They scored the store’s raw counts (cytome_layer='counts') whatever score_layer said. cytome_layer now defaults to None and still decides when given, and modality defaults to 'RNA'. A store without INFOG parameters raises an error that names the call computing them (piaso.tl.infog(ds, modality='RNA')). To reproduce a 1.2.7 call on a cytome, pass cytome_layer='counts'.
  • On a cytome, kamon writes the columns an AnnData gets in .obs to the cells table: the call, its score or -log10 p-value, and with smoothing the call before smoothing and the confidence.
  • Ties between p-values go to the higher score (use_score=False). With 100 control sets the -log10 p-value tops out at log10(101), so cell types that share markers tied and the order of the marker table decided: on a 10,003-nucleus human cortex dataset with its own markers, the p-value route went from 0.698 to 0.976 accuracy, the score route’s. n_ctrl_set is a parameter (100 by default), and score_method='scanpy' with use_score=False raises at once instead of failing later.
  • predictCellTypeByGDR joins the reference and the query as two batches of one runGDR. With COSG 1.2.0 or earlier, each reference type and each query cluster added a dimension of unrelated genes for the batch that lacks it, about half of the joint embedding, which Harmony then aligned and the label transfer used. Labels transferred with it may change; re-run if affected. This needs COSG 1.2.1, which scores only the groups that have cells.
  • predictCellTypeByGDR with default arguments works: a missing log1p layer is built from integer counts in .X, never on your object, and a query and a reference that share barcodes no longer fail.
  • smoothCellTypePrediction on a graph pools the cell itself and its k - 1 strongest neighbors, as documented (it left the cell out); a missing label casts no vote instead of failing.
  • On an AnnData, the default call (use_score=True) names cells as in 1.2.7.

score() and INFOG

  • cell_mask on an AnnData: the results land on your object, NaN outside the mask. They were written to a copy and lost. Integer cell_mask indices select those cells, as documented (they were read as a boolean mask).
  • Inputs that failed: a dense layer scores (it raised), and a matrix of more than 2^31 non-zero values, about a million RNA cells, is scored in row blocks (the compiled kernel’s row pointers wrapped).
  • Gene sets: a set with no gene in the data is named in one warning per call (it was dropped without a word, so its cell type could never be called), and a single set with no gene raises. A gene name that an AnnData holds twice is scored over both columns, as on a cytome, with the same warning. Gene weights of another length than their set raise instead of being cut short.
  • INFOG refuses counts that are NaN or infinite (one such value made every value NaN), and the in-memory and streaming paths order genes of equal variance the same way.
  • ensure_infog_params(ds, modality) computes that modality’s parameters only; it computed RNA’s whatever was asked, and rewrote the store’s highly_variable column on every call.

filter_cells: missing values count the same on a cytome and on an AnnData

  • A condition about a label keeps the cells that have none, on both inputs (pandas’ rule). On a cytome, cell_type != 'Doublet' dropped every unlabelled cell, and with inplace=True they were deleted for good, while an AnnData kept them. != and not in are true for a missing value, every other comparison false, and not / ~ negate that. This also holds on pandas’ nullable columns, where AnnData queries failed with a TypeError.
  • In a dict mask, None means “the value is missing” on both inputs (it matched nothing on a cytome).
  • Integer indices outside the cells raise (-1 selected the last cell), and a query whose value is a number ("flag + 1") is refused.
  • layer= chooses the matrix the QC thresholds read (adata.layers[layer], or {modality}_{layer} on a cytome).

Scrublet

  • Every score moves a little: each cell’s neighbor graph now leaves the cell itself out, as scrublet does (each simulated doublet counted itself as a doublet neighbor). Against scrublet 0.2.3 on 4,000 nuclei, 99.7 % of scores are now identical, against 54.4 % before.
  • A cell that cannot be scored gets a missing value: a score of NaN and a missing doublet call, where it got 0 and “not a doublet” (a library of fewer than 50 cells, or with too few variable genes). On an AnnData the call is then a nullable boolean, so adata[~adata.obs['is_doublet']] raises until the missing calls are handled; piaso.pp.filter_cells(adata, mask="~is_doublet") keeps those cells.
  • library_key naming an empty column (no cell had a library) scored nothing and called every cell not a doublet; a key that is not a column pooled every cell. Both raise now.
  • layer= names the raw counts; a matrix whose values are not integers is refused instead of being scored as counts.

runSVD: the same embedding on any number of threads

  • The dense steps run on row blocks of a fixed height, so an embedding no longer depends on the number of cores (OpenBLAS split them by core count, which moved the last digits and could tip a kNN tie). Embeddings agree with 1.2.7’s to 1e-11, relative, and take about as long.
  • Each component’s sign follows one rule: the cell coordinate of largest magnitude is positive. A stored embedding’s components may come out negated against 1.2.7’s once; distances, neighbors, Leiden and UMAP do not depend on signs.
  • Fixed on a cytome:
    • without measurement, the number of features is the modality’s own (an ATAC-only store with use_highly_variable=False was refused as having no feature selected);
    • with tfidf_params and a feature selection but no col_mask, the TF-IDF took the IDF of the first features instead of the selected ones;
    • with cell_mask, the TF-IDF computed in Python took the sequencing depths of the first cells instead of the masked ones;
    • cached TF-IDF statistics are checked against their matrix and recomputed when the matrix changed (after the matrix was written again they gave stale depths, an all-NaN IDF and “Eigenvalues did not converge”).
  • TF-IDF statistics that are NaN, infinite or negative are refused, and a fallback from the compiled engine to Python warns.

runGDR and projectGDR

  • Batch-aware runGDR on an AnnData found each batch’s markers on the raw counts when layer='infog' (the default), while a single batch and a cytome use the INFOG layer; the markers and X_gdr change, and an AnnData and the same data in a cytome now agree.
  • runGDR with batch_key and groupby: a batch that lacks one of the groups no longer adds a marker set of unrelated genes for it (one noise dimension of X_gdr per missing batch-group pair). This needs COSG 1.2.1, which scores only the groups that have cells.
  • A marker set none of whose genes could be mapped shifted every block after it (AnnData) or failed (cytome); the blocks now stay aligned. Repeated obs_names no longer fail.
  • On a cytome: a batch-aware run holds one batch’s scores at a time; save_reference=False works (NameError); a de novo run (groupby=None) clusters on INFOG’s 5,000 most variable genes, as on an AnnData (n_highly_variable_genes and use_highly_variable were ignored), keeps its labels as gdr_local and leaves no temporary embedding or graph.
  • On an AnnData, a de novo run no longer overwrites your INFOG layer and highly variable genes, and leaves no temporary graph.
  • projectGDR’s novelty for a reference built with batch_key is calibrated on the reference’s own cells projected as a query is; it was calibrated in another space, so held-out cells of the reference’s own distribution came out novel. For a single-batch reference nothing changes.
  • projectGDR’s control genes: frozen control genes are matched to the query by gene name (a query with the same genes in another order drew other controls while reporting them frozen), and a cytome reference freezes them too, so a projection no longer depends on the size of the query. A read-only reference works on its second use.

stitchSpace on a cytome integrates

On a cytome, stitchSpace did no integration and reported success: it looked for an embedding name PIASO never writes, ran COSG on a placeholder matrix, and so pruned every link between batches. It now reads the store’s embedding (modality=), runs COSG per batch on the store’s expression, and writes the integrated embedding to the store; the result equals the same data’s AnnData result. filter_bbknn_computation and filter_bbknn_metric reach BBKNN (annoy was always used), and a failure raises instead of printing a message and returning as success does.

Clustering and UMAP

  • leiden and neighbors take cell_mask as indices, as documented (they read indices as a boolean mask), and refuse repeated or out-of-range ones.
  • leiden_local puts each group’s labels back by position (cells sharing a barcode took the label of the last of them) and takes svd_drop_first_component for each group’s embedding.
  • leiden and umap warn when they fall back to igraph or umap-learn because the compiled extension lacks an entry point.
  • The compiled leiden_csr defaults to 10 iterations, as tl.leiden does (it was 2 when called directly); tl.leiden’s output does not change.

The motif scanner and the motif-to-TF tables

  • A motif whose best word cannot reach the p-value is not scanned, and one warning names such motifs with their best word’s p-value. pvalue_to_threshold returns +inf for them; it returned the best word’s score, where numpy and Rust disagreed (one motif matched 0 % of regions on Rust and 28 % on numpy). At p = 1e-4 this is about a hundred of JASPAR 2024’s 879 vertebrate motifs, depending on the background; CIS-BP has few.
  • Short motifs’ thresholds are exact: the score distribution lost its top when rounded, so some thresholds let through up to 24 times the target rate (now under 2.6 times on 800 random motifs).
  • numpy and Rust now score windows the same way and agree to the bit, and a minus-strand hit is reported at the forward start of its window.
  • build_tf_motif_map gives a JASPAR heterodimer (FOS::JUN) to each partner, as cisTarget’s tables do; as one name it matched no gene, so 76 of JASPAR 2024’s 879 vertebrate motifs were lost. load_cistarget_motifs(max_per_tf=k) keeps each TF’s k motifs with the best similarity q-values (it kept the first k in the table’s order).

Specificity matrices and SCALAR

  • specificity_matrix on a cytome: a cached matrix is used only while the labels, the cells and the matrices it came from are unchanged. A relabelled or subset store returned the old matrix, and cytorete’s inferRegulon reads this cache by default. The cache holds no pickle; an older entry is recomputed once.
  • specificity_matrix on an AnnData: cosg_layer names the matrix COSG scores (every layer gave .X’s result), and uns['cosg'] is left alone.
  • runSCALAR: call_breadth divides by the groups where a pair was tested (by every group, so a pair the detection floor kept out of most cell types looked specific); a misspelt layer raises (it fell back to .X); on a cytome, layer=None reads the matrix that was the AnnData’s .X (else the counts), so a store and its AnnData give the same p-values; a pair listed twice is tested once, with a warning; verbose=False silences the messages and progress bars.

The PIASOmarkerDB client

  • Requests are retried with a growing pause on a failed connection or a busy server (retries=, 3 by default), and an answer that is not JSON raises an error naming the status and the start of the answer.
  • cache_dir is an opt-in response cache, kept for cache_ttl seconds (a day by default). It defaulted to ~/.piaso/markers, created by every client although nothing was cached, so a read-only home failed every call.
  • Large requests are read in pages within the server’s limit (a larger page came back short and was read as the last); a cell type whose name holds a comma is matched by its exact name; offset= without limit= works.
  • as_dict=True lists a gene once per cell type (it repeated it once per study or tissue, and score() counted it each time) and warns when one cell type name gathers several contexts. loadCustomMarkers drops repeated genes too, getRecommendedStudy returns the studies’ current names, and analyzeGenes with several gene sets asks for all their genes in batches of 200 instead of once per set.

calculateCellMetrics: FRiP in one unit

ATAC FRiP divided the insertions in peaks (two per fragment) by the fragments, so a cell with every fragment in peaks got 2.0. It is now the insertions in peaks over two cut sites per fragment, capped at 1. A matrix that counts each fragment once is divided by the fragments (counting='overlaps'); a matrix that records no counting, such as one converted from AnnData, is taken as insertions, with a warning; a presence matrix gets no FRiP. A failure in this step raises instead of leaving the column out silently.

Downloads in piaso.data

  • Every download goes to a temporary file, is checked (length, size or checksum) and only then renamed, so an interrupted download no longer counts as complete; connections time out after 60 s, and network errors are retried three times.
  • The newest dataset registry wins: a registry cached by an older PIASO hid the datasets added since.
  • fetch_dataset(force=True) removes the old store’s -wal and -shm files, which SQLite would replay onto the new one, and a downloaded file takes your usual file permissions.

Plotting: three plots that showed wrong data, and one rule for colours and figures

Redraw these. Three plots drew wrong data under their defaults:

  • plotLigandReceptorLollipop, vertical layout (the default): every row but the middle one was labelled with another interaction, since SCALAR was added. The rows were reversed for display and their labels reversed a second time; the label colours went with the labels, so text and colour agreed with each other but not with the row. Every vertical lollipop drawn with PIASO up to this release is affected, including the old SCALAR tutorial’s (vertical_layout=True); the horizontal layout was right.
  • plotLigandReceptorInteraction, one colormap (the default), horizontal layout: the ligand specificities were drawn in the bottom row, under labels that read “Ligand” above “Receptor”. The two-colormap and vertical layouts were right. The labels now sit at each row’s centre.
  • plot_dendrogram on a cytome: use_rep='auto' found no embedding (it looked for AnnData’s X_ names only) and clustered the groups on the cells table’s numeric columns, cell index and fragment counts, without a warning. It now uses the store’s embedding (RNA_pca, ATAC_svd, …); a store with neither an embedding nor highly variable genes raises, naming what to pass.

Colours follow the category, by name, in every plot. One resolver now gives each category its colour over the whole column, from the stored colours (adata.uns, a cytome’s set_categories, or the colours an AnnData carried into a store) or PIASO’s palette in the column’s order. Colours used to be matched to the categories a plot happened to draw, so these changed colour, and now keep it:

  • embedding(cell_mask=...), which drew a category in the colour of one the mask left out;
  • a cytome converted from AnnData, whose categorical order was ignored (now its categories follow the AnnData’s order and colours);
  • sankey(color_by='right') (offset by the number of left categories) and left_order='auto';
  • stacked_barplot(sort_groups=True); a violin with an empty group; the dendrogram’s leaves; the cell-level heatmap’s sidebar; plotGroupMetrics.

A store’s stored order and colours now reach the violin, the stacked bar, the Sankey, the dendrogram and the heatmap, and labels without a stored order sort naturally (0, 1, 2, … 10) in the stacked bar, as in the embedding. A qualitative palette named by string ('tab10', 'Set2') gives its colours in order instead of an even sample across them, and a stored colour list shorter than its categories is ignored with a warning instead of wrapped.

Missing values are drawn and named. A missing label (NaN, None, or the text 'nan' a cytome stores for one) is drawn first, in na_color, and listed as “NA”, instead of becoming a 'nan' category (embedding, split_embedding, scatter, which dropped those points). Continuous NaN values are drawn in na_color; they were transparent.

Figures. show defaults to showing unless ax= is given; a function never closes a figure passed in through ax= (six of them did, so a later plt.savefig wrote a blank page); a figure that is not shown is returned, and one a function made is closed only once it is saved, so show=False followed by plt.savefig() keeps working. Layout acts on the figure drawn, not the current one, and leaves a figure with constrained layout alone. PDFs and EPS files saved with save= embed TrueType fonts, as journals ask. In a notebook, a show=False call left as a cell’s last line is now displayed twice (once as the returned value); assign it (fig = ...) or end the line with ;.

Other changes in the plots’ output:

  • dotplot: expression_cutoff is applied (it was ignored, and a fraction was always of values above 0), and mean_only_expressed averages the values above it; the dot-size key draws each fraction at its plotted size (it was capped, so the 80 % and 100 % dots were alike and smaller than a 100 % data dot).
  • scatter: a cytome’s text column (leiden) and a boolean column are categories, as in the embedding (the first was drawn on a colorbar). With on=, a text column’s missing values are drawn as “NA” (they were a category named “None”), and on= takes the store’s feature tables (genes, peaks, tiles) and samples, read through cytome’s accessors; an internal table is no longer accepted.
  • split_embedding: a boolean colour is categorical, as in embedding.
  • plotLigandReceptorLollipop: the circle-size key draws the plotted sizes (its dots were 1.13x wider); score_range_min and score_range_max each clip both ends of the stems (one alone clipped one end); circle_size_scale never had an effect and now warns. plotLigandReceptorInteraction: with shared_legend=True and heatmap_vmax the colorbar is labelled (its ticks were at -inf), and with preserve_input_order=True the shared scale covers the rows drawn.
  • heatmap(cell_level=True) works (it raised NameError); with standard_scale the colorbar reads “z-score”.
  • plotConfusionMatrix: the reference colour bar is on top, beside its labels.
  • New options: sort_order= ('size' draws rare categories on top) and vcenter= in the embedding plots; a basis matching both RNA_umap and ATAC_umap warns, and modality= chooses.
  • import piaso no longer reseeds numpy’s global random stream; the shuffled palettes are unchanged.

PIASOmarkerDB’s marker regions in PIASO: getMarkerRegions

  • New: piaso.tl.getMarkerRegions queries PIASOmarkerDB’s marker regions, the cell type-specific 500 bp tiles of single-nucleus chromatin atlases, up to 1,000 per cell type ranked by COSG. as_dict=True returns {cell_type: [tile IDs]} named as a PIASO store names its tiles ('chr1:3000500-3001000'), the form predictCellTypeByMarker takes, so nuclei can be named from their tile counts. top_n keeps each cell type’s best regions, and tile_size maps them onto another grid.
  • The genome build is checked: each region study is on one build (hg38 for human, mm10 for mouse). Pass genome= with your data’s build, and a study on the other build raises an error instead of matching nothing; regions are not lifted over between builds or species.
  • getMarkerRegions(list_studies=True) lists the region studies; the client class PIASOmarkerDB has getMarkerRegions and listRegionStudies.

Installation and packaging

  • PIASO needs Python 3.10 or later (requires-python = ">=3.10"). Pre-compiled wheels are built for Linux on x86_64, CPython 3.10 to 3.12; elsewhere pip builds PIASO from source, which needs Rust and a C compiler. The README promised macOS and Windows wheels, which were never built.
  • Requires cytome 0.3.9 and COSG 1.2.1; pip install -U piaso-tools installs them. COSG 1.2.1 scores only the groups that have cells (the runGDR and predictCellTypeByGDR fixes above), works under pandas 3 and reads a cytome’s column-major matrices. matplotlib’s floor is 3.6 (on 3.5 a colormap you named was silently ignored).
  • The source distribution ships Cargo.lock and is built with it, so a build from source uses the Rust crates the tests ran with, and every wheel is installed and tested before it is published.
  • The tests carry markers (slow, data, network), so pytest -m "not slow" is a fast subset.

Also in this release

  • Cell Ranger feature names made unique never repeat a name (['A', 'A', 'A-1'] gave two 'A-1'); a boolean mask with missing entries leaves them out; a mistyped cytome path raises FileNotFoundError instead of leaving an empty file behind.

v1.2.7

Terminal window
pip install -U piaso-tools

runSVD stops when its singular vectors have converged, and ATAC embeddings come at unit length

Two defaults of piaso.tl.runSVD change, and with them the embedding a fixed random_state gives:

  • The stopping rule: stop_on='residuals', the new default, stops when every component’s relative residual is below tol, so the singular vectors themselves have converged. 1.2.6 stopped when the leading singular values settled, which can happen while the vectors still depend on the random start. A run may take a few more passes.
  • Unit length for ATAC: with unit_length unset, the TF-IDF of ATAC peaks or tiles is embedded at unit length, so neighbors, leiden and umap work with the angle between cells rather than a length that follows sequencing depth. Each cell’s length is kept (adata.obs['X_svd_length']; in a cytome, the cells table’s <embedding>_length column). RNA embeddings are not scaled.

What to know when upgrading:

  • Embeddings change. To reproduce an embedding made with 1.2.6, pass both:

    piaso.tl.runSVD(data, ..., stop_on="singular_values", unit_length=False)
  • So does what embeds through runSVD. leiden_local passes both to each group’s SVD (svd_stop_on, svd_unit_length), and runGDR’s own SVD stops on the residuals.

  • An AnnData whose matrix already holds TF-IDF values carries nothing that says so: pass modality='ATAC' or unit_length=True.

runSVD can leave out the first component, and runs the same engine on an AnnData as on a cytome

  • drop_first_component=True computes n_components + 1 components and keeps components 2 to n_components + 1, so the embedding still has n_components columns, and S and Vt hold the same components. For non-negative, uncentered inputs such as the TF-IDF of peak counts, the first singular vector is the axis of the mean profile and follows sequencing depth more than cell type. The choice and the left-out singular value are recorded with the embedding. It is not meant for RNA inputs.
  • An AnnData gets the solver and the Rust pass engine a cytome gets, with the engine reading the matrix in memory: method, tol and n_iter are honored, every allowed core is used, and an AnnData and the same matrix in a cytome give the same embedding, S and Vt to the bit. auto_tfidf=True, cell_mask, scale_data, return_svd, tfidf_params and infog_params now work on an AnnData (the in-memory path ignored the last two).
  • What to know: an AnnData’s embedding changes once, since it came from scikit-learn’s randomized SVD (seven fixed rounds on one core). auto_tfidf=True on an AnnData whose values are not counts raises.

piaso.tl.umap is PIASO’s own: parallel, and the same picture on any number of threads

piaso.tl.umap now lays out the graph with PIASO’s extension by default (backend='auto'), whenever a kNN is stored (after piaso.tl.neighbors) or passed. It computes umap-learn’s algorithm: the same fuzzy graph, the same curve, the same spectral start and the same schedule of attractions and repulsions. What differs is the order of the work. umap-learn makes a seeded run single-threaded, because its updates share one stream of random numbers. PIASO’s layout cuts the nodes into 128 blocks by a hash of their names, and an epoch visits every pair of blocks once; the pairs that run at the same time share no node, so the threads together compute one sequential pass in a fixed order. One graph and one random_state give the same coordinates on 1 thread or 20.

  • Quality. Its pictures keep each cell’s graph neighbors as umap-learn’s do, within the variation between umap-learn’s own seeds.
  • Speed. The layout uses every core the process may use, so a large dataset takes a fraction of umap-learn’s single-threaded time.
  • The start does not depend on the seed, as umap-learn’s does not, and each axis’s sign is fixed, so two seeds no longer give mirrored pictures.
  • Memory. The layout keeps 12 bytes per edge of the graph and about 40 per cell, the same on any number of threads.

What to know when upgrading:

  • The coordinates change. They differ from umap-learn’s as two umap-learn seeds differ. To reproduce an embedding made with 1.2.6:

    piaso.tl.umap(data, ..., backend="umap-learn")
  • Threads. n_threads=0 (the default) uses the cores the process may use; the picture does not depend on it.

  • The whole stored kNN is used. umap-learn takes a stored kNN at its own n_neighbors (15): it cuts a larger one to its first 15 columns and sets a smaller one aside for neighbors it computes itself. PIASO’s layout uses every column. With neighbors’ default of 15 the two agree.

  • umap-learn stays a dependency; without a stored kNN, tl.umap still calls umap-learn, which computes its own neighbors, with a warning.

score() leaves a gene set’s own genes out of its controls

Every path now draws control genes the same way: in numpy, a fresh random state per gene set seeded by random_seed, gene by gene, with the genes’ means and variances summed in float64. The same seed gives the same control sets and the same scores on an AnnData with or without the compiled extension, in the single-set mode and on a cytome.

The new default, exclude_set_genes=True (in score, calculateScoreParallel and predictCellTypeByMarker), draws each gene’s controls from its nearest genes outside its own set. A member among a gene’s nearest genes gives its place to the next nearest non-member, so only the draws that would have been a member change.

What to know when upgrading:

  • Scores change where a set’s genes are among each other’s nearest genes, which is common for co-expressed marker sets. exclude_set_genes=False keeps the earlier draws of the numpy path; the compiled path, which drew its controls with another generator, now draws as numpy does, so its scores change once either way.
  • A panel with fewer features than the neighbors asked for (twice n_nearest_neighbors with the exclusion) still scores, and one warning per call names the genes left with fewer candidate controls.
  • The nearest-gene table is int32, so the wider table the exclusion needs takes the memory the int64 table took before.

predictCellTypeByMarker(smooth_by='scores')

smooth_by='scores' averages each cell’s scores (or -log10 p-values) over its smoothing neighbors before taking the best cell type, instead of voting the neighbors’ own best types (smooth_by='labels', the default, unchanged). The neighbors are the vote’s, the averaged matrix is kept in obsm['{key_added}_score_smoothed'] (or '_nlog10pvals_smoothed'), and the confidence is the share of the neighbors whose own best type is the one chosen. On an AnnData and on a cytome.

Fixed: runSVD(ds, modality='ATAC') reads the peak counts

On a cytome, runSVD with no measurement read an INFOG layer whatever the modality, so the plain call on a store holding ATAC peak or tile counts stopped with KeyError: no ATAC_infog matrix. Left unnamed, the layer now follows the modality: the counts for 'ATAC' and 'tiles', which the default TF-IDF transforms, and INFOG for RNA. The call gives what measurement='counts' gave, value for value.

Fixed: after runGDR on labels with ”/”, the AnnData can be written

anndata 0.13 refuses a ”/” in a key of an h5ad file by default (anndata.settings.disallow_forward_slash_in_h5ad), and runGDR kept its marker sets in adata.uns['gdr_reference'] as a table whose columns are the cell types, so on labels such as “L2/3 IT” adata.write_h5ad failed with “Forward slashes are not allowed in keys”. Such names are now stored with ”/” written as “%2F”, and the exact names beside them (marker_gene_names); projectGDR reads them back unchanged. Labels without ”/” are stored as before, and a reference written by 1.2.6 still reads.

Also in this release

  • Small inputs. neighbors caps the number of neighbors at the number of cells, and replaces a graph in which the approximate search left a slot unfilled by the exact graph (a 10-cell batch failed with “negative axis 1 index: -1”). runGDR(batch_key=...) gives a batch of fewer than 16 cells no markers of its own, with a warning, instead of failing, and still embeds its cells.
  • leiden_local(tmp_dir=) places the groups’ temporary stores, and when the disk is short writes them in batches that fit, one scan of the matrix each; the labels are those of a single split.
  • filter_cells(check_integrity=True) on a cytome asks for SQLite’s integrity check of the filtered store; cytome 0.3.7 otherwise runs only its own consistency checks.
  • makeGenomeFiles reads the GTF with cytome.read_gtf: one gene per gene_id, so genes that share a name are no longer merged into one span (in Cell Ranger ARC’s Mmul_10 reference, the copies of TRNAN-GUU spanned 118.6 Mb of chr1).
  • specificity_matrix(cosg_remove_lowly_expressed=False) builds runSCALAR’s input: without COSG’s detection filter the matrix has no negative entries.
  • A renamed PIASOmarkerDB study’s former name still works in analyzeGenes and analyzeMarkers: studies= and exclude_studies= are resolved through the server’s map of former names (studyAliases()), in any case, with one warning per renamed name. A server without the map leaves the names as given.
  • Requires cytome 0.3.7 (for cytome.read_gtf); pip install -U piaso-tools installs it.

v1.2.6

Terminal window
pip install -U piaso-tools

piaso.tl.leiden is PIASO’s own: parallel, and the same clusters on any number of threads

Leiden now runs in PIASO’s Rust extension by default. It is the same algorithm (Traag, Waltman and van Eck 2019), with the same objective and the same randomised refinement as igraph’s, and it reaches the same modularity. What differs is how it is run:

10 iterationscellsPIASOigraph
mouse developing visual cortex200,0613.4 s, 0.11 GB21.8 s, 0.32 GB
whole mouse brain2,341,35026.3 s, 1.3 GB553.7 s, 3.7 GB

Measured on a 10-core workstation (20 threads); memory is what the call adds to the process.

One graph and one random_state give one partition. The number of threads, the cores of the machine and the layout of the graph in memory do not enter the result: the labels are identical at 1, 2, 4, 10 and 20 threads. Parallel clustering usually gives that up, because threads see each other’s moves in an order that changes from run to run. Here a round of moves reads the state the round began with, every random number is a hash of the seed and the cell, and edge weights are summed as integers, so nothing depends on which thread did what first.

piaso.tl.leiden(adata, resolution=1.0) # all the cores this process may use
piaso.tl.leiden(adata, resolution=1.0, n_threads=4) # the same labels
piaso.tl.leiden(adata, resolution=1.0, backend="igraph") # the partitions of 1.2.5

What to know when upgrading:

  • Clusters change. The partitions differ from those of 1.2.5 by about as much as another seed would change them (ARI 0.82 to 0.88 between the two on the datasets above, where two seeds of igraph agree at 0.80 to 0.89). Pass backend="igraph" to reproduce an earlier result.
  • So does everything that clusters. leiden_local and runGDR call piaso.tl.leiden, so their results follow. leiden_local takes backend= and n_threads= and passes them on.
  • The graph must be symmetric. A directed graph is refused, with the pair of nodes whose two directions differ. igraph’s path took the upper triangle and clustered it without a word.
  • n_iterations stays at 10. Two runs give 96 to 99 % of the partition of ten in under half the time, which is worth knowing at tens of millions of cells; ten make two seeds agree a little more.
  • Ctrl-C stops a run. A graph of fewer than 32,768 nodes is clustered on one thread, which is faster there.

The Leiden at scale tutorial runs it on 200,000 cells and shows the clusters beside igraph’s.

Reading a store you cannot write

The plotting functions open a .cytome read-only, so plotting from a downloaded or shared store leaves the file exactly as it was. Statistics that are normally cached in the store (cell depth, INFOG and TF-IDF parameters) are computed for the call instead when it cannot be written. Functions that compute and then write (leiden, leiden_local, neighbors, umap, runSVD, runGDR, infog, run_TFIDF) check first, and a read-only store fails in a second with a message saying why, not at the end of the run.

The motif scanner: twice as fast, and safe after fork

piaso.pp.scan_motifs(backend="rust") scores a window by walking it and the motif’s columns together, with no index arithmetic in the inner loop: 300 motifs against 100,000 sequences of 300 bp take 5.7 s instead of 10.4 s, and peak memory is 1.03 GB instead of 1.26 GB. The scores are identical to the bit.

It also hung for ever in a process forked after it had run, which is Python’s multiprocessing on Linux. It now runs on a thread pool of its own, releases Python’s lock while it works (other Python threads keep running) and stops on Ctrl-C. Every parallel function of the extension now builds its pool the same way, and n_threads=0 means the same in each: RAYON_NUM_THREADS if set, else the cores the process may use.

runSVD on a cytome no longer opens the file a second time

The SVD engine used to open the store with its own copy of SQLite while Python’s connection held it. The two could not see each other’s locks, and the engine could reset the shared index of the write-ahead log that Python had mapped; the next large write then ended the process with a bus error (SIGBUS). The engine now computes on the compressed chunks the open dataset hands it (ds.chunk_source), so no second SQLite touches the file. The result is identical to the bit, and cache_chunks=True now works with the engine, keeping the compressed chunks within half the memory available.

Also in this release

  • leiden_local on a cytome makes every group’s store in one scan of the matrix (ds.split), instead of one subset per group. The labels are unchanged.
  • piaso.data.chrom_sizes(genome) gives the chromosome sizes of hg38, mm10, Mmul_10, mCalJac1, hg19 and mm39 by name, or of any chrom.sizes or .fai file, reduced to the primary chromosomes.
  • piaso.data.fetch_dataset checks a download before it takes its final name, and never replaces a file that is already there: a downloaded .cytome is a working file, and checking it on every call re-downloaded it over the results written into it.
  • color= given a NumPy array of gene names draws one panel per name. It was read as one value per cell.
  • Fixed: plot_embeddings_split without groups= could give categories each other’s colours. Each panel’s palette was built in alphabetical order but applied in the column’s category order, so wherever the two differed (a categorical order, or one stored with ds.set_categories) the colours were swapped, with no warning. Every panel now colours each category as plotEmbedding does. Figures made with 1.2.5 or earlier without groups= are worth a second look.

Requires cytome>=0.3.6.

v1.2.5

Terminal window
pip install -U piaso-tools

runSVD on a cytome runs its passes in Rust, and converges by default

The streaming SVD’s cost is the pass over the store: one sparse product per pass, twenty-odd passes per solve. Those passes now run in a Rust engine that reads the chunks straight out of the file, in whichever layout the matrix is stored (row-major or column-major, any integer width or float, and the zlib blobs older files carry under the zstd label), blocks each chunk against the dense operand so every random access stays in cache, and lets each thread own its slice of the output. Every entry of every product is one sum in a fixed order, so the result does not depend on the thread count and reruns are bit-identical. Measured on a 37,609-cell, 317,726-peak matrix, one round of the solver went from 27 s to 9.5 s; on a million-cell tile matrix, from 199 s to 135 s. n_threads=0 (the default) uses every core the process is allowed; PIASO_SVD_PROFILE=1 prints, per pass, how long was spent reading, bucketing and multiplying.

The solver changed with it. method="auto", now the default, runs block Lanczos with a stopping rule (tol=1e-3) and reports what it did:

Solver: block Lanczos (at most 12 rounds = 26 passes)
Solver done: block Lanczos, 9 rounds, 20 passes (converged)
Solver time: 43.0 s, of which 41.6 s in 20 passes over the store and 1.4 s in numpy between them

The old power iteration at its default of seven rounds was not converged on a peak matrix; it needs about twenty-two. method="power" with n_iter still runs exactly that many rounds when you want a fixed budget, and method="krylov" with n_iter does the same for Lanczos.

Two more defaults for peak and tile matrices on a cytome. TF-IDF is applied on the fly to raw ATAC or tile counts unless auto_tfidf=False; the pass engine weights each entry as it reads it, so no normalised layer is written. The guard that refuses to normalise an already-normalised layer runs first: inferred TF-IDF steps aside with a warning on a layer that does not look like raw counts, and an explicit auto_tfidf=True raises. And the reads take no file locks: once the write-ahead log has been checkpointed, the engine opens the store as an immutable snapshot, which is what lets it run at full speed from a network file system. On a cluster node reading from a network mount, a twenty-pass solve went from 5 min 13 s to 44.9 s; from the node’s local disk the same solve is 29 s. The installation page has a section on running on a cluster.

leiden_local re-embeds every group with runSVD

dr_method="X_svd" is the new default: each group is embedded by runSVD on its own selected features and clustered from there. On a cytome the per-group work streams from disk for any modality, with the TF-IDF runSVD applies to peaks or tiles and the store’s highly_variable selection, and each group’s subset carries only the one matrix it needs. "X_svd_full" uses every feature rather than the selected ones; "X_pca" keeps the INFOG route. min_cells, svd_method, n_iter and svd_tol pass through, and the function no longer touches scanpy. infog_svd likewise defaults to the infog layer, and where there is no highly_variable column it uses every gene and says so.

Feature selection by name, and two primate genomes

piaso.pp.selectFeatures(ds, features) flags the named features (peak ids, gene symbols, Ensembl ids, whatever the cytome stores) in one call, writing the boolean column runSVD and leiden_local read by default. match="all" and min_matched= turn a partial match into an error.

piaso.data.fetch_genome knows rhesus macaque (Mmul_10) and common marmoset (mCalJac1), with chromosome names as the Cell Ranger ARC references carry them; piaso.data.makeGenomeFiles builds the same files for any assembly from a GTF and a chromosome-sizes file. A genome without a GTF source no longer announces one.

Every reader takes a column-major matrix

Peak and tile matrices stored column-major (the layout the SVD engine prefers; cytome 0.3.3 reads and subsets it as such) are read that way throughout: score, calculateCellMetrics, calculateFeatureMetrics, normalize_log1p, the ligand-receptor readers, the TF-IDF raw-count guard and the feature reads behind plotting all take the columns when that is what the store holds, instead of asking for row chunks a column-major matrix does not have.

A breaking change: getMarkers(as_dict=True) returns the dictionary

It used to return (DataFrame, dict). Almost every use of it wanted the dict — it is the form predictCellTypeByMarker takes — and paid for that by unpacking a pair and discarding the first element. Asking for a dict now gets a dict:

marker_db = piaso.tl.getMarkers(study="AllenWholeMouseBrain_isocortex",
as_dict=True)
piaso.tl.predictCellTypeByMarker(adata, marker_gene_set=marker_db)

as_dict='both' returns the old pair. It is deprecated and will be removed in the next release. as_dict=False, the default, is unchanged.

The change announces itself once per session on the first as_dict=True, because a caller that keeps unpacking does not always fail loudly: with exactly two cell types, df, d = getMarkers(as_dict=True) succeeds and binds two cell-type names. Search your code for , ... = piaso.tl.getMarkers( — the tutorials are updated.

Dot plots: the same numbers, two orders of magnitude faster

piaso.pl.dotplot computed each group’s mean and expressed fraction with one Python comparison per (feature, group) pair, each scanning every cell. It now builds a one-hot group indicator once and takes two sparse products. The numbers are identical to 5e-16. A 100,000-cell, 60-feature, 30-group plot goes from 68 s to under half a second; at 20,000 cells it is 9.1 s to 0.05 s.

A missing groupby column is an error, not a wrong answer

SQLite has a compatibility misfeature: a double-quoted token that does not resolve to a column is treated as a string literal rather than raising. So SELECT "Leiden" FROM cells on a cytome with no Leiden column returns the string 'Leiden' for every row — and the caller sees one group, named after the typo, with a marker table that looks entirely plausible. That is how a COSG run reported 1 groups for a column that was never there.

Every place a user-supplied column name reaches such a query now looks it up first: cosg.run_cosg_cytome, piaso.tl.runSCALAR, and the new fragment-length functions. The error lists the columns that do exist. A case variant is still accepted, because SQLite column names are case-insensitive and leiden really does name Leiden. piaso.utils.require_cells_column is the shared check.

groups= draws a legend of what is on the plot

plotEmbedding(color='CellTypes', groups=['A','B']) draws A and B in their palette colours and everything else as one grey cloud. The legend was built from the full category list, so on a forty-cluster column it listed forty entries in forty colours, thirty-eight of which appeared nowhere on the figure — while the grey covering most of the points had no entry at all.

It now lists the highlighted groups and one final entry for the grey, labelled by the new na_label (default 'other') and shown only when something was actually left out. Highlighted groups keep the colour they have on an unfiltered plot.

color= can be the values, not only a column name

Anything computed on the fly — a regulon activity vector, a per-cell score never written back, a mask — had to be assigned into obs first. Passing the array reached color in adata.obs.columns and came back as TypeError: unhashable type: 'numpy.ndarray'.

plotEmbedding and plot_embeddings_split now accept a numpy array, a pandas Series or a Categorical as one value per cell. Numeric values get a colorbar, everything else a category legend; a Categorical keeps its own order, and a named Series titles the panel. A wrong length names both numbers. A plain list/tuple still means one panel per colour.

A Series is aligned by name when its index resolves to the cell names — anything that came out of a groupby, a sort or a .loc colours the right cells instead of being silently permuted. A RangeIndex, or an index sharing no labels with the cells, is used positionally; an index that resolves some of its labels and not others is refused, naming the count, because neither reading is safe. Duplicate cell names make alignment ambiguous, so those fall back to position with a warning.

predictCellTypeByMarker can weight its markers

A marker list treats a textbook marker and a weak, broadly expressed one identically. marker_gene_weights= passes per-gene weights through to piaso.tl.score(gene_weights=): a dict keyed by cell type, a DataFrame read column by column, or a sequence in set order.

The weights are used as given, with no normalisation per cell type. score() divides by median(weights) * n_genes and builds its control sets from the same weights, so rescaling one cell type’s weights cannot change which cell type wins — measured at 9e-16 across a 1000× rescale. Normalising would be a step that provably changes nothing.

What a rescale cannot survive is a median of zero: the divisor becomes zero and every score becomes NaN without a word. score() now rejects that, and non-finite weights, naming the gene set. Mismatched lengths, a missing cell type and an unknown one are each refused with both numbers.

Smaller things

  • predictCellTypeByGDR fits on the shared label column. With a query that lacks the reference’s label column, the classifier was fitted on a column that only the reference rows carried and failed with a KeyError; it now fits on the combined column both sides share.

  • Twenty-two public functions have complete docstrings, checked by a test against their signatures: every parameter present, every default the one the code uses, the cytome-only parameters and the deprecated aliases documented where they live. The remaining functions are on the same test’s backlog.

  • leiden_local stops warning about its own work. It called neighbors without passing the modality on, so an ATAC run warned on every call that it had found the embedding PIASO itself had just written under another modality. The argument is forwarded; graphs and labels are unchanged, and a genuine cross-modality guess still warns.

  • A per-cell column will not land on someone else’s. SQLite column names are case-insensitive, and cytome resolves a new spelling to an existing column with only a warning — so writing tss_score into a cytome already carrying a TSS_score column from another tool replaced it. piaso.utils.write_cells_column is the shared guard: the same spelling overwrites (re-running a QC step replaces its own column), a spelling that differs only in case is refused unless overwrite=True.

  • The TF-IDF cache no longer grows with the dataset. compute_tfidf_stats kept its per-cell depth and per-feature IDF vectors in ds.metadata, which is a JSON document rewritten whole on every save — large enough on a real ATAC cytome for cytome to warn about its size. The depth is now a cells column, the IDF stays the feature column it was already written to, and metadata keeps the scale factor and the two column names. At 20K cells by 30K peaks that is 98 bytes instead of 1.0 MB. Cytomes written the old way are read unchanged.

v1.2.4

Terminal window
pip install -U piaso-tools

SCALAR: a detection floor, applied in the test

piaso.tl.runSCALAR gains expressed_pct (default 0.1) and groupby. An interaction is tested only if the ligand is detected in at least that fraction of the sender’s cells and the receptor in the receiver’s, and each gene’s matched control genes are drawn from genes that clear the same floor in the same cell type. The null therefore never contains a control that is not expressed where it is used, and the k × k support stays full. Rows that fail the floor are returned with testable = False, p_value = 1 and are excluded from FDR; two new columns, ligand_expressed_pct and receptor_expressed_pct, carry the fractions the floor was applied to. groupby names the cell-type column and is required when the floor is on; expressed_pct=None reproduces the previous behaviour exactly.

Why in the test and not in the matrix: zeroing lowly-detected genes in the specificity matrix leaves the observed pair conditioned on passing the floor while its controls are zeroed wherever they fall below it, so most of the null becomes zeros and the p-value drifts toward “detected in more cells than its peers”. Measured on the SCALAR tutorial data, a quarter of the calls that approach produces are significant only because of the zero padding.

runSCALAR now validates its input. NaN specificity scores raise (they are what cosg.indexByGene leaves for genes outside a top-N list unless set_nan_to_zero=True, and a NaN observation would otherwise score at the p-value floor). Negative scores raise (COSG marks genes failing its own expressed_pct filter with −1, and two such sentinels multiply to +1, the largest product in the table). Existing callers that pass a clean matrix and no groupby will now see an error asking for one; add groupby= or expressed_pct=None.

The SCALAR tutorial’s recipe changes with it: COSG at mu=10 over all genes, raw scores, no COSG-side filter, and the floor applied by runSCALAR. Any per-cell-type linear rescaling of the specificity matrix (for instance dividing by an IQR) leaves every p-value unchanged, because the observed product and its null scale together; cosg.iqrLogNormalize applies a log1p on top and is not recommended as SCALAR input.

SCALAR: a readable score, and a magnitude that compares across pairs

The interaction score is a product of two specificities, each below 1, so a pair of two 0.02 specificities scored 0.0004 — a number no one can hold, and one that is not comparable across sender–receiver pairs because specificity is calibrated within a cell type. Four columns fix both without touching a single p-value:

  • ligand_specificity and receptor_specificity, the two inputs.
  • specificity_geomean, their geometric mean: a monotone function of the product, so it ranks exactly as the score does, on the specificity scale. The tutorial’s plots are sized by it.
  • null_mean and log2_enrichment: the mean of the interaction’s own matched null and the log2 ratio of the observation to it (a 10⁻¹² guard against division by zero, nothing more), so every pair is referenced to what expressed, expression-matched gene pairs achieve between the same two cell types. This is the number to compare across pairs. On the tutorial data the largest raw score in the table (PTPRC→CD22) is ten times its null; a somatostatin pair scoring 0.0008 is six thousand times its null.

statistic="min" is a second test statistic: the smaller of the two specificities, so both sides must beat the controls’ weaker sides — the “are both specific” question. Its exact null factorises into two one-sided counts. The default remains the product.

piaso.tl.umap and piaso.tl.leiden no longer return a value

Breaking. Both wrote their result and returned it on AnnData, while returning None on a cytome — one function with two contracts, and a 161,027-row array echoed into every notebook cell that called it. They now return None on both backends; read the result from adata.obsm[key_added] or adata.obs[key_added], as with every other piaso.tl function.

The two calls that have nowhere to write still return their result: data=None with a knn_result (in-memory), and leiden(..., cell_mask=...), whose labels cover the masked cells only and would mis-align if written full-length.

If you have emb = piaso.tl.umap(adata) in a script, it now binds None.

TF lists: one source that works, and a file that is checked

piaso.data.fetch_tf_list(species, source=...) replaces fetch_animaltfdb_tf_list as the entry point, defaulting to the cisTarget/SCENIC+ TF universe (~1,890 human, ~1,860 mouse).

AnimalTFDB was removed as a source. Its only host (guolab.wchscu.cn) sits behind a WAF that answers every non-browser client — urllib, requests and curl alike, under any User-Agent — with a captcha page or 405, and the old bioinfo.life.hust.edu.cn mirror no longer resolves. source="animaltfdb" now raises and says so; fetch_animaltfdb_tf_list, which shipped in 1.2.3, raises without making a request and will be deleted in 1.3. An AnimalTFDB table saved from a browser still works: pass it to load_tf_list(path=...), whose TSV parser reads its Symbol column.

Any TF-list file is now validated. The captcha page arrived as HTTP 200 and used to parse into a dozen plausible-looking “symbols” that silently shrank the TF universe. A file that starts with markup, or that yields tokens no gene symbol could be, is refused; a downloaded catalogue must also yield at least 50 symbols, and one that does not is deleted rather than cached. No minimum applies to a file you pass yourself — a ten-TF list is a legitimate thing to hand in.

For regulon inference you need neither: cytorete.tl.inferRegulon with tf_list=None already restricts to TFs that have a motif and are expressed in the data, which is the only set that can produce an edge.

Fixed: the source distribution was too large for PyPI

maturin sdist builds from the cargo package, which swept in docs/ — the rendered tutorials and their images. The archive came to 407 MB against 2.1 MB of actual source, and PyPI refuses any file over 100 MiB.

The three wheels upload first and succeed, so the effect was a red publish step at the end of every release since 1.2.0 and no sdist on PyPI for any version. Cargo.toml now excludes the documentation and the other non-source trees; the archive is 590 kB, installs from source, and passes twine check.

Fixed: a dict palette set the colours and lost the order

palette={"Neuron": ..., "Glia": ...} coloured the categories correctly and then re-sorted them alphabetically, so the sequence written at the call site was dropped without a word. A dict states an order — Python has preserved insertion order since 3.7 — and it now becomes the category order, ahead of a stored categorical order, because the call site is the more specific of the two. A partial dict leads and the rest follow alphabetically; a list palette is unaffected, since a list cannot state an order.

plot_embeddings_split had a second half of the same bug: it resolved one order across all panels and then did not pass it to the renderer, which re-sorted each panel on its own.

Fixed: legend_loc="none" left the colour bar behind

For a continuous colour the colour bar is the legend, and legend_loc never reached it — plotEmbedding drew one anyway, and plot_embeddings_split drew one per panel. Both now honour "none". Categorical legends were already suppressed correctly.

umap-learn must be 0.5.8 or newer

Earlier versions call scikit-learn’s check_array(force_all_finite=...), which scikit-learn renamed in 1.6 and removed in 1.8. The combination raises a TypeError from three frames inside scikit-learn with no mention of umap, which reads like a PIASO bug; it is an upstream one, fixed upstream in umap-learn 0.5.8. The requirement is now umap-learn>=0.5.8, and piaso.tl.umap catches that specific failure and names the package to upgrade, for environments that already have the bad pair.

pandas 3: supported, not excluded

pandas 3 turns on future.infer_string, so a string column is StringDtype rather than object. Two checks had been asking numpy about that dtype:

  • np.issubdtype(StringDtype, np.number) raises rather than returning False, so the numeric-versus-categorical dispatch crashed and took every categorical embedding with it. It now asks pandas, which understands both its own extension dtypes and numpy’s. Booleans stay categorical — the one place the two libraries disagree, pandas calling bool numeric.
  • A string cluster label could return from a .cytome as an integer. That was in cytome’s entity writer, which reached TEXT only through is_object_dtype and otherwise defaulted to INTEGER; fixed in cytome 0.3.2, which this release now requires. Nothing raised at the time, because SQLite stores what it is handed and only the read is typed — the damage appeared later, wherever a label was compared to a string or looked up in a stored category order.

There is no upper bound on pandas. An earlier build of this release capped it at <3; that was wrong. The problem was ours, it is fixed, and the suite passes on 3.x. A cap would have made PIASO uninstallable beside anything that requires pandas 3, which in this ecosystem is increasingly everything.

Both behaviours are covered by tests that run on pandas 2 as well, through the future.infer_string option that has existed since 2.1 — so this stays tested wherever it is installed rather than only where pandas 3 happens to be.

Fixed: piaso.tl.umap(data=None, knn_result=...) crashed

The in-memory path — documented, and advertised in this release as one of the two calls that still return their result — asked None for .obsm and died with AttributeError. It had been invisible because the test covering it skips wherever the installed umap-learn and scikit-learn disagree, which is most machines. Calling it with no knn_result now explains itself instead of raising from three frames down.

Fixed: importCellRanger reported a broken install

It imported the ATAC fragment leg unconditionally, so in the released package every call failed with ModuleNotFoundError, including modality='rna', which needs nothing from it. RNA-only now works; asking for fragments says which method is unavailable and why.

Fixed: embedding dots were drawn as rings, at 3x the size requested

plotEmbedding, plot_embeddings_split and scatter never passed linewidths to ax.scatter, so matplotlib stroked every marker at rcParams['patch.linewidth'] — 1.0 pt, in edgecolors='face'.

On a cell-sized point that stroke is most of the dot. point_size=0.2 is a disc 0.45 pt across, so it drew 1.45 pt wide: 3.2x the size asked for. With alpha < 1 the stroke and the fill composite twice around the rim, so each point rendered as an annulus with a washed-out core whose colour was neither the palette colour nor the background. Measured on a 600 dpi render, ink peaked two pixels off-centre.

All three scatter calls now pass linewidths=0, edgecolors='none', and both are exposed as parameters for anyone who wants a deliberate outline. The same fix is applied in piaso.pl.scatter and the plotByCluster jitter points. No visible change on large markers; the workaround plt.rc_context({"patch.linewidth": 0}) is no longer needed.

Fixed: plot_embeddings_split silently ignored extra keywords

It accepted **kwargs and never forwarded them, so styling a split panel had no effect and no error. Unrecognised keywords now raise TypeError, as plotEmbedding does. show is accepted as an alias for show_figure — it was among the dropped keywords, which meant show=False displayed the figure anyway.

Codebook motifs: about twice the TF universe, opt-in

piaso.data.fetch_codebook() and load_codebook() add the representative PWM set of Jolma, Laverty, Fathi et al., Nature 657, 275-283 (2026) — 1,421 human TFs against JASPAR 2024 CORE vertebrates’ 754, covering all but eleven of them. On SEA-AD middle temporal gyrus that lifts the count of TFs with a motif and expression from 483 to 1,023. The archive is ~1 MB, downloaded from the publisher under the article’s CC-BY-4.0 licence on first use and cached; nothing is redistributed with PIASO.

cytorete.tl.inferRegulon(..., motif_db="codebook") uses it. JASPAR remains the default, and that is now measured rather than assumed. Both were run end-to-end on the same 20,000 SEA-AD nuclei:

JASPAR 2024Codebook
TFs with motifs in the data7401,408
regulons415982 (412 of JASPAR’s 415 among them)
zinc-finger regulons65 (16%)385 (39%)
agreement with the TF’s own specificity26.7%12.7%
runtime13 min56 min

The result is mixed rather than bad. Codebook finds more agreeing regulons in absolute terms (125 against 111) and fixes cases JASPAR gets wrong — OLIG2 now tops OPC rather than oligodendrocytes, and SOX10, absent from JASPAR, tops oligodendrocytes. But it dilutes: agreement halves, the zinc-finger share more than doubles, and neuronal factors such as RORB and LHX6 fall in rank. Reach for it when the question is glial, zinc-finger, or coverage-limited; keep JASPAR when it is a survey.

The loader repairs what the archive carries: construct names (CASZ1.FL, ZNF729.DBD3) are filed under the gene symbol, three clone identifiers are dropped, and five CIS-BP conversions with all-zero positions are cleaned and renormalised.

Removed: cytorete.tl.markerMotifEnrichment

Never released, and removed after measurement rather than for tidiness. Two cutoff-free designs were tried on SEA-AD: a Fisher test on top-N markers against a GC- and length-matched background returned 0, 1 and 5 significant pairs at 50, 100 and 200 markers, with Jaccard 0.00 between the first two; a PIASOscore of each TF’s motif gene set against matched controls agreed with the TF’s own expression in 3-4% of cases.

The same agreement measure on inferRegulon’s 415 regulons, same cells and same motifs, is 26.7%. A PWM is a family address, not a TF address, so a motif-only statistic cannot say which cell type a TF acts in — the TF’s expression profile is what separates SPI1 from ELF1. §10 of the motif analysis tutorial now reports both measurements and hands over to inferRegulon, which adds exactly that step. cytorete.pp.build_cistrome still returns the TF × gene matrix for anyone who wants to run their own test.

Sankey: node heights, and one size key instead of two

piaso.pl.sankey gains min_node_size (height of the shortest category block, in points) and node_scale ('linear', 'sqrt', 'log'), the node-level parallel of min_flow_width and flow_scale. Both are solved the same affine way, so blocks stay monotone in group size and nothing is tied.

This fixes a distortion the default layout has always had: a node is exactly as tall as the ribbons landing on it, so under min_flow_width its height is m * size + b * fan_out, and two equally large categories are drawn at different heights when one splits more ways than the other. Measured on a two-category example at a 20 pt baseline, the four-way split draws twice as tall as the equal one-way split.

What it costs is stated in the docstring and enforced in the key: each node’s ribbons are renormalised to fill its height, so the count-to-thickness map becomes per-node and min_flow_width stops being exact.

show_width_legend / width_legend_loc are now show_size_legend / size_legend_loc, and draw one key rather than two. Only one map can be global at a time, so only one key can be true: the key describes flows when the flow map is non-proportional, and switches to describing nodes — suppressing the flow key — as soon as the nodes are sized.

Confusion matrices: hierarchical ordering

order_query / order_reference accept 'hclust' (average linkage on that axis’s own profiles) and 'hclust_symmetric' (one clustered order on both axes, for a square table over one label set). 'auto' now prefers the symmetric order when the two axes carry the same labels — two independent orders moved a real diagonal off the diagonal — and is unchanged otherwise.

Fixed

  • The GDR SettingWithCopyWarning. Each batch was scored by slicing an anndata view, whose constructor rewrites categorical obs columns through _remove_unused_categories — an anndata-internal pandas idiom, and pure overhead here since the scorer reads only the matrix and var_names. The batch object is now built directly. This also removes two warnings.catch_warnings blocks, one of them inside a ThreadPoolExecutor worker, where mutating the process-global filter list races between threads.

  • plotLigandReceptorLollipop tick labels were formatted to one decimal, so an axis spanning a few 10⁻⁴ — what neuronal ligand–receptor scores look like — read 0.0 at every tick. The precision now follows the axis range.

v1.2.3

Terminal window
pip install -U piaso-tools

Fixed

  • piaso.tl.predictCellTypeByGDR discarded its own result. The function copies the query internally to avoid mutating .X, and the copy rebound the local name, so the prediction was written to the copy. It printed “All finished. The predicted cell types are saved as CellTypes_gdr in adata.obs” and the caller’s AnnData came back without that column. The cytome path was unaffected — it writes through a separate handle, which is why this survived. If you called it on an AnnData and found no prediction, this was why.

  • plot_embeddings_split(vmin_pct=…, vmax_pct=…) was accepted and ignored. The parameters arrived through **kwargs and were dropped, so a request for 10/90 silently produced the full range. Nothing raised and the figure still rendered — only the contrast was wrong. Now implemented to match plotEmbedding, with explicit vmin/vmax still taking precedence.

Added

  • piaso.pp.alignSpatialCoordinates. Sections are placed on their chips independently, so a split plot renders each panel at its own offset; this centres each sample on its own centroid, and with_std=True also equalises apparent section size. Works on AnnData, an open cytome, or a path, and takes groupby= or batch_key= for the same column.

    This was described in the v1.2.2 notes but did not make the 1.2.2 wheel. It ships here.

  • palette= accepts a colormap name for a categorical column, sampled across the categories in their resolved order. On ordered categories — ages, timepoints, stages — that turns the legend into a ramp instead of a set of unrelated hues.

v1.2.2

Terminal window
pip install -U piaso-tools

Spatial

  • Tissue images and regions of interest. piaso.pl.plotEmbedding and plot_embeddings_split take image=True to draw a registered morphology image under a spatial embedding, with orientation and units handled: the image is placed in coordinate space, so nothing needs flipping or scaling. Needs cytome ≥ 0.2.6, which stores the image in the file.
  • cell_mask= plots a subset — a boolean mask or integer indices — without writing a subset file. It pairs with cytome’s cells_in_region, so one rectangle selects its cells and, via spatial_images.crop, its pixels.
  • piaso.pp.rotateSpatialCoordinates works directly on a cytome, not only on an AnnData, and rebuilds the coordinate index in the same call so region queries keep agreeing with what you plot.

Plotting

  • Continuous colour now depends on where the value came from. A feature keeps PIASO’s sequential color_1; a numeric cell column — regulon activity, a QC metric, pseudotime — gets Spectral_r. An explicit cmap= still wins. Both are continuous, but drawing metadata on the expression ramp made a metadata panel read as expression.
  • Fixed: plot_embeddings_split(fix_coordinate_ratio=True) is documented as the default but only the False branch acted, so panels were stretched to their subplot box and spatial tissue rendered squashed.
  • Fixed: basis='spatial' resolved to the last matching embedding, so writing an aligned or backup copy beside it silently redirected existing calls. Candidates are ranked now — the name that is the key beats one that merely contains it.

v1.2.1

Terminal window
pip install -U piaso-tools
  • piaso.tl.cospecificity_trans is public; it is the trans co-specificity used by cytorete to decide regulon edges.
  • The GRN entry points that used to live in piaso.tl remain as lazy forwarders to cytorete, raising an actionable ImportError when it is not installed.
  • Motif scanning (piaso.pp.scan_motifs), the PWM loaders and the SCREEN cCRE registry are part of the public API.

v1.2.0

Terminal window
pip install -U piaso-tools

cytome datasets and streaming

PIASO works directly on cytome files. Anywhere a function took an AnnData, it also takes a path to a .cytome file or an open dataset:

import piaso, cytome
ds = cytome.open("atlas.cytome")
piaso.tl.runGDR(ds, groupby="cell_type", layer="infog")

The matrix is read in chunks rather than loaded, so peak memory is set by the batch size instead of by the number of cells. cytome is installed automatically — there is no extra to remember.

A self-contained analysis workflow

From raw UMI counts to clusters, an embedding and marker genes:

import piaso, cosg
# Input must be RAW UMI counts. This reads adata.X by default; if your raw
# counts live in a layer, pass it explicitly:
# piaso.tl.infog(adata, layer="counts", n_top_genes=3000)
piaso.tl.infog(adata, n_top_genes=3000)
# Pass layer="infog" — runSVD defaults to adata.X, which would silently
# ignore the normalization above.
piaso.tl.runSVD(adata, layer="infog", n_components=50, key_added="X_svd")
piaso.tl.neighbors(adata, use_rep="X_svd", n_neighbors=15)
piaso.tl.leiden(adata, resolution=1.0, key_added="leiden")
piaso.tl.umap(adata, use_rep="X_svd")
cosg.cosg(adata, groupby="leiden", key_added="cosg")
piaso.pl.embedding(adata, basis="X_umap", color="leiden")

Every step runs on a plain pip install piaso-tools — no scanpy required. scanpy remains an optional extra (pip install 'piaso-tools[scanpy]') for interoperability.

One key to watch when moving older code over: piaso.tl.leiden writes its labels to adata.obs['leiden'] by default, whereas the scanpy-based tutorials that preceded this release passed key_added='Leiden' explicitly. Downstream calls that name the column need the same spelling.

Motif scanning

piaso.pp.scan_motifs is a pure-numpy PWM scanner; scan_motifs_rust is a rayon-parallel implementation with the same contract. The motif-database loaders and .2bit sequence access live alongside it in piaso.data, so the whole workflow — fetch a genome, extract sequences, load PWMs, scan — is available from one install.

Reference data

piaso.data fetches and caches what analyses need: genome sequence and annotation, example datasets, motif databases (JASPAR, CIS-BP, cisTarget), and the SCREEN cCRE registry.

Requirements

Requires cosg>=1.1.0. pip install -U piaso-tools pulls it in.

Security

pyo3 upgraded to 0.29.2, closing GHSA-36hh-v3qg-5jq4 and GHSA-chgr-c6px-7xpp. PIASO called neither affected API, so exposure was nil, but the crate linked the code.

v1.1.0

Marker-gene-guided dimensionality reduction (GDR), INFOG normalization, gene-set scoring, cell-type annotation and label transfer, and the plotting suite. See the GitHub releases for the full history.