Paper. DenAdel A, Hughes M, Thoutam A, Gupta A, Navia AW, Fusi N, Raghavan S, Winter PS, Amini AP, Crawford L. Evaluating the role of pre-training dataset size and diversity on single-cell foundation model performance. Nature Methods (2026). doi:10.1038/s41592-026-03120-y
Presented by David Hirst · 18 August 2026
The study
The paper asks whether single-cell foundation models (scFMs) obey the scaling behaviour established for language models. The authors subsampled the 22.2 million-cell scTab corpus (164 cell types, 5,052 donors, 56 tissues, drawn from the CELLxGENE census) at 1, 10, 25, 50 and 75%, under three sampling schemes — uniform, cell-type re-weighted, and geometric sketching — and pre-trained five architectures of increasing complexity on each: PCA, scVI, a masked autoencoder (SSL), Geneformer, and SCimilarity. This yields 400 pre-trained models and 6,400 downstream evaluations spanning zero-shot and fine-tuned cell-type classification, batch integration, and perturbation response prediction.
Performance saturates at roughly 1% of the corpus, approximately 200,000 cells and under a gigabyte of counts. Diversity-enriched sampling confers no advantage over uniform sampling. In every task, the pre-trained models are matched or beaten by unsupervised PCA computed directly on the evaluation dataset and by logistic regression on highly variable genes.
Terminology clarified in discussion
Two terms were first discussed, both used loosely in the literature. Foundation model was initially equated with large language model, but the paper adopts the broader sense of any model whose learned representation is reused as a scaffold for downstream tasks — which is why a pre-trained PCA qualifies. Every architecture here is used identically: it supplies a low-dimensional embedding (loading vectors for PCA, encoder weights for scVI), and that embedding is transferred to the new dataset. Zero-shot denotes use of the embedding without further training, with classification performed by nearest neighbours; fine-tuned denotes retraining of some or all parameters, which for PCA means fixing the loadings and appending a neural network classifier.
Jiadong proposed a reframing of the plateau in terms of effective sample size. In single-cell data the effective sample size lies between the number of donors and the number of cells. Pseudo-bulking sits at the conservative extreme: it guarantees independence but discards within-donor variation. On this reading the saturation point is an empirical estimate of where the effective sample size sits for a given learning task — the authors did not explain this.
Criticisms
The dominant concern was that the study varies cells rather than donors. Sample size in this context means biological replication; 5,052 donors is adequate for biological inference and far short of what a large model requires. Downsampling cells within a fixed corpus therefore adds repetition and technical noise rather than information, and the observed plateau may say more about scTab than about scFMs. Rerunning the design on an independent pre-training corpus, or subsampling studies rather than cells, would separate the two.
Related to this, several attendees argued that the value of a foundation model should be assessed against deliberately heterogeneous, noisy training data. The study considered the training on a corpus assembled under (potentially) broadly uniform protocols.
The provenance of the ground-truth cell-type labels was raised and remains unresolved. Public atlas annotations are routinely generated by PCA followed by clustering and manual labelling. If the reference labels were produced this way, a PCA-derived classifier recovering them is close to circular, and the headline result that simple baselines win loses much of its force. It was also noted that PCA is the current practical default rather than an optimal reduction, and that cell-type-discriminating variance can fall outside the retained components.
On evaluation design, fine-tuning splits a single dataset 80/10/10 and reports test performance from the same study, which inflates accuracy relative to the realistic case of training and testing on independent studies. In fairness, the authors did guard against leakage at the corpus level by restricting evaluation datasets to studies published after the 2023-05-15 census release, so none appear in scTab; the objection applies to the fine-tuning split alone.
The choice of downstream tasks drew the sharpest criticism. Cell-type annotation and batch correction are largely solved problems, so demonstrating that foundation models fail to improve on them is a weak test of the paradigm. The question left standing is what a general-purpose scFM is actually for: if the only tasks that generalise across studies are ones already handled by conventional methods, the biologically interesting questions may be irreducibly study-specific.
Perturbation response prediction supports that reading. For three of the four compounds tested, every fine-tuned model was outperformed by a baseline predicting no change in expression. Discussion attributed this partly to uncharacterised technical effects in the underlying data, and noted that this is precisely the regime in which robust linear methods should be expected to win.
The absence of pre-training learning curves was noted as a gap. A study framed around dataset size and generalisation should report training and validation loss as a function of corpus size and show where the two diverge; the reported downstream F1 curves are only indirect evidence.
Transfer works for language but maybe not for expression?
The most developed technical argument concerned batch effects. In natural language, batch effects are enumerable — regional variants differ in orthography, idiom and lexical choice — so a sufficiently large corpus contains them all and the model learns to marginalise over them. Transcriptomic batch effects are not enumerable: each operator, day and reagent lot generates a novel effect, so no corpus contains the full space. A review by Shihua Zhang’s team was cited as making this argument. A complementary point was raised on target complexity: token combinatorics in language are effectively unbounded, whereas cell-type space is small, and separate ongoing work indicates that roughly 150 features suffice for cell-type prediction. Most genes contribute nothing to the task, so additional data and features dilute rather than sharpen the signal.
Three constructive directions followed: importing the statistical genetics toolkit (GWAS, fine-mapping, causal inference) rather than relying on scale alone; aligning the pre-training objective with the intended downstream task, for which the paper supplies direct evidence in SCimilarity’s contrastive objective and its correspondingly strong cell-type performance; and moving toward automated experimental design rather than passive accumulation of data.
Follow-up reading
Chengyi proposed a companion paper: Rosen Y, Roohani Y, Agrawal A, Samotorčan L, Tabula Sapiens Consortium, Quake SR, Leskovec J. Universal cell embedding provides a foundation model for cell biology. Nature 656:183–191 (2026). doi:10.1038/s41586-026-10689-z
UCE pursues the same premise — self-supervised pre-training on a large corpus to yield a universal per-cell embedding — but reports strong zero-shot downstream performance without fine-tuning, across 36 million cells and over 1,000 cell types, including generalisation to species absent from training. The apparent contradiction with DenAdel et al. is the reason to read the two together, and turns on evaluation task, corpus composition, and whether the reported gains survive a PCA baseline.
Compiled from the meeting recording. Speaker separation is not available in the room-microphone transcript, so contributions are attributed only where a name was used aloud.