Skip to content

UMAP at scale: the same coordinates on any number of threads

piaso.tl.umap embeds the neighbour graph with PIASO’s parallel deterministic Rust UMAP engine (backend="rust", the default since v1.2.7). It optimizes the same fuzzy simplicial set cross-entropy objective as McInnes, Healy and Melville (2018), starting from the same spectral initialization and using the same (a, b) curve parameters fitted from min_dist and spread.

Two things are different from umap-learn:

  1. You do not have to choose between a seed and multiple cores. In umap-learn, passing an integer random_state disables multi-threaded SGD and forces execution onto a single thread, because its parallel mode relies on unsynchronized Hogwild! updates to shared vertex coordinates.
  2. The coordinates do not depend on n_threads. One graph and one random_state produce bit-for-bit identical float32 coordinates on 1 thread, 4 threads, or 20 threads.

This page embeds the 82,031-cell Parse Evercode v4 human PBMC atlas (1,912,562 kNN edges) from the Parse Evercode v4 tutorial, compares the embedding beside umap-learn’s, and measures runtime, thread scaling, and local neighbourhood preservation on a 10-core workstation (20 threads).

Embed

import anndata as ad
import numpy as np
import piaso
adata = ad.read_h5ad("parse_evercode_pbmc_82k.h5ad")
piaso.tl.umap(
adata,
neighbors_key="neighbors_gdr",
min_dist=0.5,
spread=1.0,
random_state=42,
key_added="X_umap",
)

The call is identical on an AnnData or a .cytome path. For this 82,031-cell graph (1,912,562 edges), piaso.tl.umap finishes in 7.30 s on 20 threads.

The number of threads changes the time, not the coordinates

import hashlib
coords = {}
for t in (1, 2, 4, 10, 20):
piaso.tl.umap(
adata,
neighbors_key="neighbors_gdr",
random_state=42,
n_threads=t,
key_added=f"X_umap_t{t}",
)
coords[t] = adata.obsm[f"X_umap_t{t}"].copy()
all(np.array_equal(coords[1], coords[t]) for t in coords)
True
{t: hashlib.sha256(coords[t].tobytes()).hexdigest()[:16] for t in coords}
{1: '2540b5a4b31254ff',
2: '2540b5a4b31254ff',
4: '2540b5a4b31254ff',
10: '2540b5a4b31254ff',
20: '2540b5a4b31254ff'}
Threads (n_threads)1241020
Time (piaso.tl.umap)57.42 s35.95 s17.05 s8.17 s7.30 s
**`maxX_t - X_1`**0.00.00.0
SHA-256 prefix2540b5a4b31254ff2540b5a4b31254ff2540b5a4b31254ff2540b5a4b31254ff2540b5a4b31254ff
Thread scaling and bit-identical verification on 82,031 human PBMCs

Why parallel SGD is usually non-deterministic — and why this one is not

In standard parallel UMAP (umap-learn with random_state=None), worker threads pull edges from a shared queue and update the 2D coordinates of the endpoints head[e] and tail[e] in place without locks. Whenever two threads process edges incident to the same cell at the same time, their reads and writes interleave in an order decided by OS thread scheduling. Even with a fixed initial layout, two runs on the same machine drift apart (max |Δ| = 11.41 coordinate units on this dataset).

PIASO’s Rust UMAP engine eliminates both sources of thread dependence:

  1. Conflict-free edge coloring. Before SGD begins, the directed kNN edge list is partitioned into independent color classes (batches) such that no two edges in the same batch share an endpoint (head or tail) or a negative-sampling target. Within each batch, Rayon threads update disjoint sets of vertices in parallel without atomic contention or race conditions; a lightweight barrier separates batches.
  2. Per-edge counter-based PRNG. Instead of drawing negative samples from a thread-local stateful RNG (whose sequence depends on which thread claims which edge), each scheduled edge event derives its negative samples from a stateless SplitMix64 hash of (seed, epoch, edge_index). Edge e at epoch t therefore draws the exact same negative vertices whether the job runs on 1 core on a laptop or 20 threads on a cluster node.

What it looks like, beside umap-learn

To run umap-learn through the same interface, pass backend="umap-learn":

piaso.tl.umap(
adata,
neighbors_key="neighbors_gdr",
random_state=42,
backend="umap-learn",
key_added="X_umap_umaplearn",
)
import matplotlib.pyplot as plt
piaso.settings.set_figure_params(style="cell")
fig, axes = plt.subplots(1, 3, figsize=(18, 5.5))
for ax, basis, title in zip(
axes,
("X_umap_t20", "X_umap_t1", "X_umap_umaplearn"),
(
"piaso.tl.umap (20 threads, 7.3 s)",
"piaso.tl.umap (1 thread, 57.4 s, max |Δ| = 0.0)",
"umap-learn (seeded, 1 thread, 70.3 s)",
),
):
piaso.pl.embedding(
adata,
basis=basis,
color="cell_type",
ax=ax,
show=False,
legend_loc="none",
frameon=False,
)
ax.set_title(title)
Side-by-side comparison of piaso.tl.umap at 20 threads, 1 thread, and seeded umap-learn

Same neighbourhood quality, 9.6× faster than seeded umap-learn

We evaluate embedding fidelity on the 82,031-cell PBMC dataset using three standard metrics:

  • Trustworthiness (k = 15) — fraction of 2D nearest neighbours that are genuine neighbours in the 30-dimensional GDR space ([0, 1], higher is better).
  • kNN recall (k = 15) — exact overlap of the 15-nearest-neighbour sets between high-dimensional space and the 2D UMAP ([0, 1], higher is better).
  • Cell-type silhouette — separation of the 27 annotated immune populations in 2D UMAP space.
MethodWall timeSpeedupDeterministic across runs?Thread-invariant?Trustworthiness (k = 15)kNN recall (k = 15)Cell-type silhouette
piaso.tl.umap (backend="rust", 20 threads)7.30 s9.63×Yes (Δ = 0.0)Yes (1–20 threads)0.9754 (5-seed median 0.9763)0.18520.2713
piaso.tl.umap (backend="rust", 1 thread)57.42 s1.22×Yes (Δ = 0.0)Yes0.97540.18520.2713
umap-learn (random_state=42, forced 1 thread)70.31 s1.00×Yes (1 thread only)N/A (single-threaded)0.97580.18610.2676
umap-learn (random_state=None, 20 threads)37.54 s1.87×**No (`maxΔ= 11.41`)**No0.9756

piaso.tl.umap matches umap-learn’s local trustworthiness (0.9754 vs 0.9758, and 0.9763 median across 5 seeds) and slightly improves cell-type silhouette (0.2713 vs 0.2676), while running 9.63× faster than reproducible (random_state=42) umap-learn and 5.14× faster than unseeded multi-threaded umap-learn.

Reproducing an embedding from PIASO 1.2.6 or earlier

Before v1.2.7 piaso.tl.umap delegated to umap-learn. That backend remains available:

piaso.tl.umap(adata, backend="umap-learn", random_state=42)

Good to know

  • n_threads=None (or 0), the default, uses all CPU cores available to the process. Because the coordinates do not depend on n_threads, changing core counts between interactive exploration and batch pipelines never alters your UMAP figures.
  • n_components=2 and n_components=3 use dedicated stack-allocated 2D and 3D SIMD SGD kernels; arbitrary dimensions ($D > 3$) are also supported.
  • init_pos="spectral" computes the normalized graph Laplacian eigenvector initialization directly in Rust (Lanczos eigensolver with connected-component anchoring) before running the conflict-free SGD epochs. You can also pass an existing obsm key or an (n_cells, n_components) NumPy array to init_pos to warm-start from custom coordinates.