Spatial transcriptomics stops swapping only cell-level states: SpaAxFormer keeps each gene's state and beats Impeller on DLPFC imputation L1
Synopsis
Spatial transcriptomics models usually exchange representations only between cells or spots, with gene information arriving through the attached expression profile. SpaAxFormer keeps every cell–gene entry as its own state, first letting genes within a cell reference each other, then exchanging the same gene's state across spatial neighbors, with a separate cell-level summary. On six DLPFC slices its mean L1 imputation distance was lower than Impeller's, for example 0.239 versus 0.254 on slice 151510; on Stereo-seq and Slide-seq V2, Impeller had lower mean L1 and RMSE.
Interpretation
Spatial transcriptomics modeling gains a layer that can be retained: each cell–gene entry keeps its own contextual state through the encoder, while a separate cell-level summary serves clustering. Earlier spatial methods mainly exchanged cell- or spot-level representations over a spatial graph, with gene-level information entering indirectly through the associated expression profile; gene-token models represent genes explicitly, but their cross-cell attention is computed at the cell-pair level with one attention map shared across genes. Across six DLPFC slices, SpaAxFormer's mean L1 distance for gene-expression imputation was lower than Impeller's on all six, 0.239 versus 0.254 on slice 151510; RMSE was lower on four slices and tied on two, and cosine similarity was higher on three and tied on three.
Information exchange is split into two directions: genes within a cell first reference each other, then the same gene's state is exchanged separately among spatial neighbors. This factorization follows the idea of alternating attention along cell and gene axes, but the cross-cell step computes attention weights separately for each retained input gene rather than sharing cell-pair weights across all genes. Spatial coordinates only decide which cells may exchange information, and the radius graph adds the nearest non-self neighbor when none lies within the radius; during training each target-cell batch is expanded with its spatial neighbors, and reconstruction supervision falls only on masked entries of target cells.
The cell-level representation is carried by a separate CLS state and receives extra supervision from CLS-conditioned reconstruction and contrastive learning. With gene reconstruction alone, CLS clustering reached a mean ARI of 0.103 on DLPFC slice 151675; adding CLS-conditioned reconstruction raised it to 0.400, and adding the expression-PCA-weighted multi-positive contrastive objective brought it to 0.511. This stepwise comparison ran on the single DLPFC slice 151675, with all three configurations sharing the same encoder architecture, experimental setup and five random seeds, scored by unrefined ARI.
Perspective
Evaluation covers imputation on six DLPFC slices, spatial-domain identification on eight DLPFC slices plus BC1 and STARMap, and two additional technologies, Stereo-seq and Slide-seq V2; spatial label refinement is applied only to Visium data, and STARMap is scored on unrefined assignments. The intended readers are researchers and engineers working on spatial-transcriptomics representation learning.
The imputation advantage concentrates on DLPFC, while Impeller has lower mean L1 and RMSE on Stereo-seq and Slide-seq V2; unrefined SpaAxFormer has lower mean ARI than ADEPT in all ten settings, and after refinement ADEPT remains higher in all nine Visium comparisons. Refinement only changes cluster labels without retraining the encoder, so those gains come from a downstream label operation rather than the CLS representations themselves. The authors also note that explicit computation grows with gene coverage and neighborhood context, and that the neighborhood-expanded batch scheme limits information propagation to the local context.