Skip to content
PIASO

cytome in R

The R package reads the same .cytome files the Python package writes, with no Python runtime, no reticulate, and no HDF5 intermediate. A .cytome is a plain SQLite database, so R reads the tables through DBI and decodes the compressed count matrices through a small Rcpp layer.

install.packages("cytome", repos = "https://genecell.r-universe.dev")

That gives compiled binaries on Windows and macOS. Building from source needs lz4, zstd and zlib:

Debian/Ubuntu  apt-get install liblz4-dev libzstd-dev zlib1g-dev
Fedora/RHEL    dnf install lz4-devel libzstd-devel zlib-devel
macOS          brew install lz4 zstd
conda          conda install -c conda-forge lz4-c zstd zlib

Reading

One entry point, and as chooses what comes back.

library(cytome)


sce <- read_cytome("data.cytome")                       # SingleCellExperiment
so  <- read_cytome("data.cytome", as = "Seurat")        # Seurat
x   <- read_cytome("data.cytome", as = "cytome")        # the open handle

Alternative modalities travel with the object. A cytome holding RNA and ATAC gives an SCE with an ATAC altExp, or a Seurat with an ATAC assay. Embeddings arrive as reducedDims or DimReducs, and any KNN graph in the file arrives as a colPair or in the graphs slot.

Looking before loading

The handle answers questions about the file without reading the matrix, which matters when the matrix is the thing you are trying to avoid reading.

x <- read_cytome("data.cytome", as = "cytome")
x                                  # cells, matrices, embeddings
cytome_matrices(x)                 # what is in there
cytome_obs(x, "cells")             # the cell table
cytome_var(x, "RNA_counts")        # the feature table
cytome_embeddings(x)               # available embeddings
cytome_graphs(x)                   # available graphs
cytome_close(x)

Writing

write_cytome() is a generic. It dispatches on the object, so the same call takes a Seurat or a SingleCellExperiment and you do not have to remember which function belongs to which class.

write_cytome(so,  "out.cytome")
write_cytome(sce, "out.cytome")
write_cytome(sce, "out.cytome", layers = TRUE)     # carry normalized assays
write_cytome(so,  "out.cytome", graphs = FALSE)    # leave the KNN graph out

Normalized layers are off by default. For an archive they are recomputable from the counts and cost more space than the counts do. For a handoff they are often the point, so layers = TRUE carries them and they come back as extra assays. See Converting: AnnData, Seurat, SCE for what does and does not travel, measured rather than asserted.

Data too large to load

Two ways, and they answer different questions.

Out-of-core, for code you did not write. delayed = TRUE backs the assay with a DelayedArray that reads from the file on demand, so the Bioconductor block-processing machinery works on a matrix that never has to fit in memory:

sce <- read_cytome("big.cytome", delayed = TRUE)
SummarizedExperiment::assay(sce)              # a DelayedArray
scuttle::logNormCounts(sce)                   # never holds the matrix whole
DelayedArray::colSums(SummarizedExperiment::assay(sce))

Extraction is chunk-aligned: only the storage chunks overlapping the requested cells are read, and chunkdim() reports the storage geometry so the block grid lands on chunk boundaries rather than straddling them. Peak memory is set by the block size, not by the size of the matrix. On data that fits, leave delayed = FALSE, which is faster.

Chunk streaming, for code you are writing. cytome_stream() applies a function chunk by chunk and returns one result per chunk:

x <- read_cytome("big.cytome", as = "cytome")
totals   <- cytome_stream(x, "RNA_counts",
                          function(chunk, i0, i1, k) Matrix::rowSums(chunk))
per_gene <- Reduce(`+`, totals)
cytome_close(x)

The difference is who writes the loop. With cytome_stream() you do, and anything expressible as per-chunk work plus a reduction is available. With DelayedArray the loop is inside scran, scuttle or DelayedMatrixStats, which is the only route to running code you did not write on data that does not fit.

What the R package does not do

It is a reader, writer and streaming interface for the format, not an analysis package. Building a cytome from Cell Ranger output, merging, subsetting and filtering all live on the Python side. Once your data is a Seurat object or a SingleCellExperiment, use the tools you already use. The package README carries a per-capability table.

Is it really the same file?

That claim is a test, not a promise. The R package ships a reference .cytome written by the Python implementation together with the expected values, and its test suite asserts it reads them bit-for-bit, exercising both codecs (zstd for RNA, lz4 for ATAC). CI runs the reverse as well: a file written by R, read back by Python, compared against the same expectations.

For two implementations of one format, testing a single direction leaves the other free to rot. The reverse check is what caught alternative assays being dropped on write, a bug no amount of read-direction testing would have found.