Cell segmentation in spatial transcriptomics gets a cross-platform test: SAM tops the raw score, SCS leads once coverage counts

Segmentation is the least standardized step in spatial transcriptomics: cell boundaries decide which pixels and transcripts belong to which cell, and errors propagate into cell typing, spatial patterns and ligand–receptor inference. Earlier segmentation benchmarks relied mainly on image-based data and lacked assessments that incorporate the transcriptomic context produced by sequencing-based spatial methods, as well as biologically relevant metrics. The evaluation spans 15 datasets from 5 spatial transcriptomics technologies, 12 human and mouse tissue types and about 2.3 million cells, with individual datasets ranging from roughly 700 cells to nearly 800,000.

Methods trade strengths across datasets and metrics, and no single method leads on every dimension. On the average of 15 metrics, SAM scores 0.58, SAM2 0.53, BIDCell 0.52, SCS 0.48, Baysor and mSAM both 0.45, Cellpose 0.42, CelloType 0.37 and Watershed 0.27. SAM reaches high transcriptomic richness (0.71 transcripts per cell, 0.73 genes per cell) but limited segmentation completeness (0.28), while BIDCell combines segmentation completeness of 0.62 with expression concordance against the reference of 0.89.

Reading segmentation output together with coverage changes the ranking, so a quality score alone favors methods that segment only a small fraction of cells. After weighting by the ratio of segmented to reference cells, SCS leads at 0.34, with BIDCell and Baysor both at 0.33; BIDCell and Baysor accumulate the most dataset wins, 6 and 5 respectively. The 15 metrics are grouped into 8 biological categories, and expression purity shows no correlation with other categories across datasets, indicating information the other metrics do not capture.

Segmentation choice changes downstream biological conclusions, and so does platform choice. Across three mouse brain datasets, MERFISH shows higher background and nonspecific contamination while STARmap PLUS and Stereo-seq give more cell-type-specific expression; in some conditions negative-marker expression corresponds to contamination affecting up to about 10–15% of cells. In ligand–receptor analysis, STARmap PLUS and Stereo-seq place Pdgfc–Pdgfra, Prok2–Prokr2 and Vip–Vipr2 in the expected cells, whereas MERFISH disperses these interactions across incorrect cell populations.

The demonstrated scope is 15 public datasets from 5 spatial transcriptomics technologies, covering 12 human and mouse tissue types and about 2.3 million cells, together with an open Nextflow pipeline for running and evaluating segmentation methods.

The benchmark does not exhaustively cover all spatial transcriptomics technologies or all parameter settings of each method; differences in platform design, image quality, gene panels and tissue composition make algorithmic performance hard to fully separate from technology effects; the gene-set sensitivity analysis was performed on a single dataset and is a preliminary assessment.