An NIH multi-institution study in Lancet Oncology classified 52 CNS tumor types from tissue images at 80% accuracy across 5,516 test samples. A simultaneous Cancer Science paper documented that label noise in pathologist annotations causes AI pathology accuracy to be systematically overstated. These two findings define the current state of computational pathology.
The NIH CNS Tumor Study
Tio et al. 2024 trained a deep learning classifier on digitized H&E stained slides from 19 institutions to distinguish 52 CNS tumor subtypes. The 80% accuracy figure represents performance on held-out cases across three continents. For 12 of the 52 subtypes, accuracy exceeded 90%. For rare subtypes with fewer than 50 training cases, accuracy fell significantly, reflecting the training data bottleneck that affects all rare pathology classification tasks.
The Label Noise Problem
Computational pathology models learn from pathologist annotations. When those annotations contain errors, the models inherit them. The Cancer Science paper by Komura et al. 2024 documented systematic overstatement of AI pathology accuracy: because model performance is typically measured against the same annotations used for training, disagreements between the model and ground truth are counted as model errors even when the model is correct and the annotation is wrong. The true accuracy ceiling for any pathology AI system is bounded by inter-pathologist agreement on the training labels.
Limitations
Staining variability across institutions, scanner hardware differences, and tissue processing protocols all create distribution shift. Computational pathology models trained at academic centers with standardized protocols may underperform at community hospitals.
Related coverage: FDA Clearance for AI Medical Devices: What 510(k), De Novo, and PMA Mean | AI in Radiology: Three Phases and What the Clinical Evidence Shows | AI-Driven ADMET Prediction: What the Blind Challenge Results Show
Primary sources: Tio et al., Lancet Oncology 2024; Komura et al., Cancer Science 2024.