A transformer-based weakly supervised computational pathology method for clinical-grade diagnosis and molecular marker discovery of gliomas

Ostrom, Q. T. et al. The epidemiology of glioma in adults: a ‘state of the science’ review. Neuro-oncology 16, 896–913 (2014).Article 

Google Scholar 
Weller, M. et al. Glioma. Nat. Rev. Dis. Primers 1, 15017 (2015).Chen, R., Smith-Cohn, M., Cohen, A. L. & Colman, H. Glioma subclassifications and their clinical significance. Neurotherapeutics 14, 284–297 (2017).Article 

Google Scholar 
Louis, D. N. et al. The 2021 WHO classification of tumors of the central nervous system: a summary. Neuro-oncology 23, 1231–1251 (2021).Article 

Google Scholar 
Horbinski, C. et al. NCCN guidelines® insights: central nervous system cancers, version 2.2022: featured updates to the NCCN guidelines. J. Natl Compr. Cancer Netw. 21, 12–20 (2023).Article 

Google Scholar 
Van Den Bent, M. J. Interobserver variation of the histopathological diagnosis in clinical trials on glioma: a clinician’s perspective. Acta Neuropathol. 120, 297–304 (2010).Article 

Google Scholar 
Ker, J., Bai, Y. Q., Lee, H. Y., Rao, J. & Wang, L. P. Automated brain histology classification using machine learning. J. Clin. Neurosci. 66, 239–245 (2019).Article 

Google Scholar 
Bera, K., Schalper, K. A., Rimm, D. L., Velcheti, V. & Madabhushi, A. Artificial intelligence in digital pathology – new tools for diagnosis and precision oncology. Nat. Rev. Clin. Oncol. 16, 703–715 (2019).Article 

Google Scholar 
Niazi, M. K. K., Parwani, A. V. & Gurcan, M. N. Digital pathology and artificial intelligence. Lancet Oncol. 20, e253–e261 (2019).Article 

Google Scholar 
Echle, A. et al. Deep learning in cancer pathology: a new generation of clinical biomarkers. Br. J. Cancer 124, 686–696 (2021).Article 

Google Scholar 
Mobadersany, P. et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl Acad. Sci. USA 115, E2970–E2979 (2018).Article 

Google Scholar 
Ertosun, M. G. & Rubin, D. L. Automated grading of gliomas using deep learning in digital pathology images: a modular approach with ensemble of convolutional neural networks. AMIA Annu. Symp. Proc. Arch. 2015, 1899–1908 (2015).
Google Scholar 
Jin, L. et al. Artificial intelligence neuropathologist for glioma classification using deep learning on hematoxylin and eosin stained slide images and molecular markers. Neuro-oncology 23, 44–52 (2021).Article 

Google Scholar 
Im, S. et al. Classification of diffuse glioma subtype from clinical-grade pathological images using deep transfer learning. Sensors 21, 3500 (2021).Article 

Google Scholar 
Ocampo, P. et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. J. Thorac. Oncol. 13, S562 (2018).Article 

Google Scholar 
Hsu, W. W. et al. A weakly supervised deep learninÿg-based method for glioma subtype classification using WSI and mpMRIs. Sci. Rep. 12, 6111 (2022).Article 

Google Scholar 
Campanella, G. et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25, 1301–1309 (2019).Article 

Google Scholar 
Lu, M. Y. et al. Data-efficient and weakly supervised computational pathology on whole-slide images. Nat. Biomed. Eng. 5, 555–570 (2021).Article 

Google Scholar 
Shao, Z. C. et al. TransMIL: transformer based correlated multiple instance learning for whole slide image classification. In Proc. 35th Conference on Neural Information Processing Systems (eds Ranzato, M. et al.) 2136–2147 (Curran Associates, 2021).Li, B., Li, Y. & Eliceiri, K. W. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 14318–14323 (IEEE, 2021).Zhang, H. R. et al. DTFD-MIL: double-tier feature distillation multiple instance learning for histopathology whole slide image classification. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 18802–18812 (IEEE, 2022).Zhang, J. et al. Attention multiple instance learning with transformer aggregation for breast cancer whole slide image classification. In Proc. 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 1804–1809 (IEEE, 2022).Chen, R. J. et al. Scaling vision transformers to gigapixel images via hierarchical self-supervised learning. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 16144–16155 (IEEE, 2022).Hou, W. et al. H2-MIL: exploring hierarchical representation with heterogeneous multiple instance learning for whole slide image analysis. In Proc. AAAI Conference on Artificial Intelligence 933–941 (AAAI Press, 2022).Zheng, Y. et al. A graph-transformer for whole slide image classification. IEEE Trans. Med. Imaging 41, 3003–3015 (2022).Article 

Google Scholar 
Lee, Y. et al. Derivation of prognostic contextual histopathological features from whole-slide images of tumours via graph deep learning. Nat. Biomed. Eng. https://doi.org/10.1038/s41551-022-00923-0 (2022).Chefer, H., Gur, S. & Wolf, L. Transformer interpretability beyond attention visualization. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 782–791 (IEEE, 2021).Dosovitskiy, A. et al. An image is worth 16×16 words: transformers for image recognition at scale. Preprint at https://arxiv.org/abs/2010.11929 (2020).Vaswani, A. et al. Attention is all you need. In Proc. 30th Conference on Neural Information Processing Systems 5998–6008 (eds Guyon, I. et al.) (Curran Associates, 2017).Liu, Z. et al. Swin transformer: hierarchical vision transformer using shifted windows. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 10012–10022 (IEEE, 2021).Ilse, M., Tomczak, J. & Welling, M. Attention-based deep multiple instance learning. In Proc. 35th International Conference on Machine Learning (eds Dy, J. & Krause, A.) 2127–2136 (PMLR, 2018).He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 770–778 (IEEE, 2016).Russakovsky, O. et al. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015).Article 
MathSciNet 

Google Scholar 
Louis, D. N. et al. The 2016 World Health Organization classification of tumors of the central nervous system: a summary. Acta Neuropathol. 131, 803–820 (2016).Article 

Google Scholar 
Wang, X. et al. RetCCL: clustering-guided contrastive learning for whole-slide image retrieval. Med. Image Anal. 83, 102645 (2023).Article 

Google Scholar 
Yang, P. et al. CS-CO: a hybrid self-supervised visual representation learning method for H&E-stained histopathological images. Med. Image Anal. 81, 102539 (2022).Article 

Google Scholar 
Wang, X. et al. Transformer-based unsupervised contrastive learning for histopathological image classification. Med. Image Anal. 81, 102559 (2022).Article 

Google Scholar 
Ciga, O., Xu, T. & Martel, A. L. Self supervised contrastive learning for digital histopathology. Mach. Learn. Appl. 7, 100198 (2022).
Google Scholar 
Bach, S. et al. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10, e0130140 (2015).Article 

Google Scholar 
Selvaraju, R. R. et al. Grad-cam: visual explanations from deep networks via gradient-based localization. In Proc. IEEE Conference on Computer Vision 618–626 (IEEE, 2017).Vahadane, A. et al. Structure-preserving color normalization and sparse stain separation for histological images. IEEE Trans. Med. Imaging 35, 1962–1971 (2016).Article 

Google Scholar 
Carpenter, J. & Bithell, J. Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. Stat. Med. 19, 1141–1164 (2000).Article 

Google Scholar 
Yin, X. TCGA glioma WSI dataset used for evaluating ROAM. Zenodo https://doi.org/10.5281/zenodo.11469546 (2024).Yin, X. ROAM. GitHub https://github.com/whiteyunjie/ROAM (2024).Yin, X. A transformer-based weakly supervised computational pathology method for clinical-grade diagnosis and molecular state revelation of gliomas (v1.0.0). Zenodo https://doi.org/10.5281/zenodo.11469423 (2024).