Albahri, A. S. et al. A systematic review of trustworthy and explainable artificial intelligence in healthcare: Assessment of quality, bias risk, and data fusion. Inf. Fusion 96, 156–191 (2023).Article
Google Scholar
Tjoa, E. & Guan, C. A survey on explainable artificial intelligence (XAI): toward medical XAI. IEEE Trans. Neural Netw. Learn. Syst. 32, 4793–4813 (2021).Article
PubMed
Google Scholar
van Lent, M., Fisher, W. & Mancuso, M. An explainable artificial intelligence system for small-unit tactical behavior. IAAI Emerging Applications. 900-907 (2004)Graziani, M. et al. A global taxonomy of interpretable AI: unifying the terminology for the technical and social sciences. Artif. Intell. Rev. 56, 3473–3504 (2023).Article
PubMed
Google Scholar
Reyes, M. et al. On the interpretability of artificial intelligence in radiology: challenges and opportunities. Radiol. Artif. Intell. 2, e190043 (2020).Article
PubMed
PubMed Central
Google Scholar
Lipkova, J. et al. Artificial intelligence for multimodal data integration in oncology. Cancer Cell 40, 1095–1110 (2022).Article
CAS
PubMed
PubMed Central
Google Scholar
Boehm, K. M., Khosravi, P., Vanguri, R., Gao, J. & Shah, S. P. Harnessing multimodal data integration to advance precision oncology. Nat. Rev. Cancer 22, 114–126 (2022).Article
CAS
PubMed
Google Scholar
Acosta, J. N., Falcone, G. J., Rajpurkar, P. & Topol, E. J. Multimodal biomedical AI. Nat. Med. 28, 1773–1784 (2022).Article
CAS
PubMed
Google Scholar
Boonn, W. W. & Langlotz, C. P. Radiologist use of and perceived need for patient data access. J. Digit. Imaging 22, 357–362 (2009).Article
PubMed
Google Scholar
Huang, S.-C., Pareek, A., Seyyedi, S., Banerjee, I. & Lungren, M. P. Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines. Npj Digit. Med. 3, 1–9 (2020).Article
Google Scholar
Troyanskaya, O. et al. Artificial intelligence and cancer. Nat. Cancer 1, 149–152 (2020).Article
PubMed
Google Scholar
Bi, W. L. et al. Artificial intelligence in cancer imaging: clinical challenges and applications. CA Cancer J. Clin. 69, 127–157 (2019).Article
PubMed
PubMed Central
Google Scholar
Heiliger, L., Sekuboyina, A., Menze, B., Egger, J. & Kleesiek, J. Beyond medical imaging: a review of multimodal deep learning in radiology. https://www.zora.uzh.ch/id/eprint/219067/ (2022).Steyaert, S. et al. Multimodal data fusion for cancer biomarker discovery with deep learning. Nat. Mach. Intell. 5, 351–362 (2023).Article
PubMed
PubMed Central
Google Scholar
Taleb, A., Kirchler, M., Monti, R. & Lippert, C. ContIG: self-supervised multimodal contrastive learning for medical imaging with genetics. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 20876–20889. https://doi.org/10.1109/CVPR52688.2022.02024 (2022).Soenksen, L. R. et al. Integrated multimodal artificial intelligence framework for healthcare applications. Npj Digit. Med. 5, 1–10 (2022).Article
Google Scholar
Joshi, G., Walambe, R. & Kotecha, K. A review on explainability in multimodal deep neural nets. IEEE Access 9, 59800–59821 (2021).Article
Google Scholar
Venkadesh, K. V. et al. Prior CT improves deep learning for malignancy risk estimation of screening-detected pulmonary nodules. Radiology 308, e223308 (2023).Article
PubMed
Google Scholar
Rojat, T. et al. Explainable artificial intelligence (XAI) on TimeSeries data: a survey. Preprint at http://arxiv.org/abs/2104.00950 (2021).Baltrušaitis, T., Ahuja, C. & Morency, L.-P. Multimodal machine learning: a survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. 41, 423–443 (2019).Article
PubMed
Google Scholar
Yala, A., Lehman, C., Schuster, T., Portnoi, T. & Barzilay, R. A deep learning mammography-based model for improved breast cancer risk prediction. Radiology 292, 60–66 (2019).Article
PubMed
Google Scholar
Joo, S. et al. Multimodal deep learning models for the prediction of pathologic response to neoadjuvant chemotherapy in breast cancer. Sci. Rep. 11, 18800 (2021).Article
CAS
PubMed
PubMed Central
Google Scholar
Reda, I. et al. Deep learning role in early diagnosis of prostate cancer. Technol. Cancer Res. Treat. 17, 1533034618775530 (2018).Article
PubMed
PubMed Central
Google Scholar
Hyun, S. H., Ahn, M. S., Koh, Y. W. & Lee, S. J. A machine-learning approach using PET-based radiomics to predict the histological subtypes of lung cancer. Clin. Nucl. Med. 44, 956 (2019).Article
PubMed
Google Scholar
Liu, J. et al. Prediction of rupture risk in anterior communicating artery aneurysms with a feed-forward artificial neural network. Eur. Radiol. 28, 3268–3275 (2018).Article
PubMed
Google Scholar
Yoo, Y. et al. Deep learning of brain lesion patterns and user-defined clinical and MRI features for predicting conversion to multiple sclerosis from clinically isolated syndrome. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 7, 250–259 (2019).Article
Google Scholar
Mueller, S. G. et al. The Alzheimer’s disease neuroimaging initiative. Neuroimaging Clin. N. Am. 15, 869–877 (2005).Article
PubMed
PubMed Central
Google Scholar
Thung, K.-H., Yap, P.-T. & Shen, D. Multi-stage diagnosis of alzheimer’s disease with incomplete multimodal data via multi-task deep learning. Deep Learn. Med. Image Anal. Multimodal Learn. Clin. Decis. Support 10553, 160–168 (2017).Article
Google Scholar
Bhagwat, N., Viviano, J. D., Voineskos, A. N. & Chakravarty, M. M. Modeling and prediction of clinical symptom trajectories in Alzheimer’s disease using longitudinal data. PLOS Comput. Biol. 14, e1006376 (2018).Article
PubMed
PubMed Central
Google Scholar
Li, H. & Fan, Y. Early prediction of Alzheimer’s disease dementia based on baseline hippocampal MRI and 1-year follow-up cognitive measures using deep recurrent neural networks. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) 368–371. https://doi.org/10.1109/ISBI.2019.8759397 (2019).Spasov, S. E., Passamonti, L., Duggento, A., Liò, P. & Toschi, N. A multi-modal convolutional neural network framework for the prediction of Alzheimer’s disease. In: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 1271–1274. https://doi.org/10.1109/EMBC.2018.8512468 (2018).Qiu, S. et al. Fusion of deep learning models of MRI scans, Mini–Mental State Examination, and logical memory test enhances diagnosis of mild cognitive impairment. Alzheimers Dement. Diagn. Assess. Dis. Monit. 10, 737–749 (2018).
Google Scholar
Sheng, J. et al. Predictive classification of Alzheimer’s disease using brain imaging and genetic data. Sci. Rep. 12, 2405 (2022).Article
CAS
PubMed
PubMed Central
Google Scholar
Cao, R. et al. Development and interpretation of a pathomics-based model for the prediction of microsatellite instability in Colorectal Cancer. Theranostics 10, 11080–11091 (2020).Article
CAS
PubMed
PubMed Central
Google Scholar
Jurenaite, N., León-Periñán, D., Donath, V., Torge, S. & Jäkel, R. SetQuence & SetOmic: deep set transformer-based representations of cancer multi-omics. In: 2022 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) 1–9. https://doi.org/10.1109/CIBCB55180.2022.9863058 (2022).Prelaj, A. et al. Real-world data to build explainable trustworthy artificial intelligence models for prediction of immunotherapy efficacy in NSCLC patients. Front. Oncol. 12 (2023).Arya, V. et al. One explanation does not fit all: a toolkit and taxonomy of ai explainability techniques. Preprint at https://doi.org/10.48550/arXiv.1909.03012 (2019).Klaise, Janis, J., Van Looveren, A., Vacanti, G. & Coca, A. Alibi explain: algorithms for explaining machine learning models. JMLR. 22, 1–7 (2021).Kokhlikyan, N. et al. Captum: a unified and generic model interpretability library for PyTorch. Preprint at https://doi.org/10.48550/arXiv.2009.07896 (2020).The Institute for Ethical Machine Learning. XAI – An eXplainability toolbox for machine learning. https://github.com/EthicalML/xai (2023)Alber, M. et al. iNNvestigate neural networks! JMLR 20, 1–8 (2019).Hedström, A. et al. Quantus: an explainable AI toolkit for responsible evaluation of neural network explanations and beyond. JMLR 24, 1–11 (2023).Di Martino, F. & Delmastro, F. Explainable AI for clinical and remote health applications: a survey on tabular and time series data. Artif. Intell. Rev. 56, 5261–5315 (2023).Article
PubMed
Google Scholar
Reel, P. S., Reel, S., Pearson, E., Trucco, E. & Jefferson, E. Using machine learning approaches for multi-omics data analysis: a review. Biotechnol. Adv. 49, 107739 (2021).Article
CAS
PubMed
Google Scholar
Berisha, V. et al. Digital medicine and the curse of dimensionality. Npj Digit. Med. 4, 1–8 (2021).Article
Google Scholar
Ben Ahmed, K., Hall, L. O., Goldgof, D. B. & Fogarty, R. Achieving multisite generalization for CNN-based disease diagnosis models by mitigating shortcut learning. IEEE Access 10, 78726–78738 (2022).Article
Google Scholar
Gichoya, J. W. et al. AI recognition of patient race in medical imaging: a modelling study. Lancet Digit. Health 4, e406–e414 (2022).CAS
Google Scholar
Geirhos, R. et al. Shortcut learning in deep neural networks. Nat. Mach. Intell. 2, 665–673 (2020).Article
Google Scholar
Yu, Y., Lee, H. J., Kim, B. C., Kim, J. U. & Ro, Y. M. Investigating vulnerability to adversarial examples on multimodal data fusion in deep learning. Preprint at https://doi.org/10.48550/arXiv.2005.10987 (2020).Simon-Gabriel, C.-J., Ollivier, Y., Bottou, L., Schölkopf, B. & Lopez-Paz, D. First-order adversarial vulnerability of neural networks and input dimension. Proceedings of the 36th International Conference on Machine Learning. PMLR. 97, 5809–5817 (2019).Chen, J., Jia, C., Zheng, H., Chen, R. & Fu, C. Is multi-modal necessarily better? robustness evaluation of multi-modal fake news detection. IEEE Trans. Netw. Sci. Eng. 1–15 https://doi.org/10.1109/TNSE.2023.3249290 (2023).Shaik, T., Tao, X., Li, L., Xie, H. & Velásquez, J. D. Multimodality fusion for smart healthcare: a journey from data, information, knowledge to wisdom. Preprint at http://arxiv.org/abs/2306.11963 (2023).Rahim, N. et al. Prediction of Alzheimer’s progression based on multimodal deep-Learning-based fusion and visual Explainability of time-series data. Inf. Fusion 92, 363–388 (2023).Article
Google Scholar
Anguita-Ruiz, A., Segura-Delgado, A., Alcalá, R., Aguilera, C. M. & Alcalá-Fdez, J. eXplainable Artificial Intelligence (XAI) for the identification of biologically relevant gene expression patterns in longitudinal human studies, insights from obesity research. PLOS Comput. Biol. 16, e1007792 (2020).Article
CAS
PubMed
PubMed Central
Google Scholar
Shashikumar, S. P., Josef, C. S., Sharma, A. & Nemati, S. DeepAISE—an interpretable and recurrent neural survival model for early prediction of sepsis. Artif. Intell. Med. 113, 102036 (2021).Article
PubMed
PubMed Central
Google Scholar
Ibrahim, L., Mesinovic, M., Yang, K.-W. & Eid, M. A. Explainable prediction of acute myocardial infarction using machine learning and shapley values. IEEE Access 8, 210410–210417 (2020).Article
Google Scholar
Vielhaben, J., Lapuschkin, S., Montavon, G. & Samek, W. Explainable AI for time series via virtual inspection layers. Pattern Recognit. 150, 110309 (2024).Sandoval, Y. et al. High-sensitivity cardiac troponin and the 2021 AHA/ACC/ASE/CHEST/SAEM/SCCT/SCMR guidelines for the evaluation and diagnosis of acute chest pain. Circulation 146, 569–581 (2022).Article
CAS
PubMed
Google Scholar
Sallam, M. The utility of ChatGPT as an example of large language models in healthcare education, research and practice: systematic review on the future perspectives and potential limitations. https://doi.org/10.1101/2023.02.19.23286155 (2023).Lee, J. et al. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36, 1234–1240 (2020).Article
CAS
PubMed
Google Scholar
Rasmy, L., Xiang, Y., Xie, Z., Tao, C. & Zhi, D. Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. Npj Digit. Med. 4, 1–13 (2021).Article
Google Scholar
Wang, S., Zhao, Z., Ouyang, X., Wang, Q. & Shen, D. ChatCAD: interactive computer-aided diagnosis on medical image using large language models. Preprint at https://doi.org/10.48550/arXiv.2302.07257 (2023).Huang, S.-C., Shen, L., Lungren, M. P. & Yeung, S. GLoRIA: a multimodal global-local representation learning framework for label-efficient medical image recognition. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV) 3922–3931. https://doi.org/10.1109/ICCV48922.2021.00391 (2021).Wang, Z., Wu, Z., Agarwal, D. & Sun, J. MedCLIP: contrastive learning from unpaired medical images and text. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 3876–3887 (2022).OpenAI Platform. https://platform.openai.com (2023).Wu, C. et al. Can GPT-4V(ision) Serve medical applications? Case studies on GPT-4V for multimodal medical diagnosis. Preprint at http://arxiv.org/abs/2310.09909 (2023).Bienefeld, N. et al. Solving the explainable AI conundrum by bridging clinicians’ needs and developers’ goals. Npj Digit. Med. 6, 1–7 (2023).Article
Google Scholar
Berrevoets, J., Kacprzyk, K., Qian, Z. & van der Schaar, M. Causal deep learning. Preprint at https://doi.org/10.48550/arXiv.2303.02186 (2023).Ribeiro, F. D. S., Xia, T., Monteiro, M., Pawlowski, N. & Glocker, B. High fidelity image counterfactuals with probabilistic causal models. Proceedings of the 40th International Conference on Machine Learning. PMLR202. (2023).Castro, D. C., Walker, I. & Glocker, B. Causality matters in medical imaging. Nat. Commun. 11, 3673 (2020).Article
CAS
PubMed
PubMed Central
Google Scholar
Yue, K., Jin, R., Wong, C.-W., Baron, D. & Dai, H. Gradient obfuscation gives a false sense of security in federated learning. Preprint at https://doi.org/10.48550/arXiv.2206.04055 (2022).Mo, F. et al. Quantifying and localizing usable information leakage from neural network gradients. Preprint at https://doi.org/10.48550/arXiv.2105.13929 (2022).Mujawar, S., Deshpande, A., Gherkar, A., Simon, S. E. & Prajapati, B. in Human-Machine Interface 1–23 (John Wiley & Sons, Ltd, 2023). https://doi.org/10.1002/9781394200344.ch1.Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J. & Fernández-Leal, Á. Human-in-the-loop machine learning: a state of the art. Artif. Intell. Rev. 56, 3005–3054 (2023).Article
Google Scholar
Parcalabescu, L. & Frank, A. On measuring faithfulness of natural language explanations. Preprint at https://doi.org/10.48550/arXiv.2311.07466 (2023).Wu, C., Zhang, X., Zhang, Y., Wang, Y. & Xie, W. MedKLIP: medical knowledge enhanced language-image pre-training for X-ray Diagnosis. IEEE/CVF International Conference on Computer Vision (ICCV). 21315–21326 (2023).Filice, R. W. & Ratwani, R. M. The case for user-centered artificial intelligence in radiology. Radiol. Artif. Intell. 2, e190095 (2020).Article
PubMed
PubMed Central
Google Scholar
Ejaz, H. et al. Artificial intelligence and medical education: a global mixed-methods study of medical students’ perspectives. Digit. Health 8, 20552076221089099 (2022).PubMed
PubMed Central
Google Scholar
Agrawal, A. et al. A survey of ASER members on artificial intelligence in emergency radiology: trends, perceptions, and expectations. Emerg. Radiol. 30, 267–277 (2023).Huisman, M. et al. An international survey on AI in radiology in 1,041 radiologists and radiology residents part 1: fear of replacement, knowledge, and attitude. Eur. Radiol. 31, 7058–7066 (2021).Article
PubMed
PubMed Central
Google Scholar
Huisman, M. et al. An international survey on AI in radiology in 1041 radiologists and radiology residents part 2: expectations, hurdles to implementation, and education. Eur. Radiol. 31, 8797–8806 (2021).Article
PubMed
PubMed Central
Google Scholar
van Hoek, J. et al. A survey on the future of radiology among radiologists, medical students and surgeons: Students and surgeons tend to be more skeptical about artificial intelligence and radiologists may fear that other disciplines take over. Eur. J. Radiol. 121, 108742 (2019).Article
PubMed
Google Scholar
Codari, M. et al. Impact of artificial intelligence on radiology: a EuroAIM survey among members of the European Society of Radiology. Insights Imaging 10, 105 (2019).Article
Google Scholar
Keeney, S., Hasson, F. & McKenna, H. P. A critical review of the Delphi technique as a research methodology for nursing. Int. J. Nurs. Stud. 38, 195–200 (2001).Article
CAS
PubMed
Google Scholar
Schotman, E. & Iren, D. Algorithmic decision making and model explainability preferences in the insurance industry: a Delphi study. In: 2022 IEEE 24th Conference on Business Informatics (CBI) 01 235–242 (IEEE, 2022).Mittelstadt, B., Russell, C. & Wachter, S. Explaining explanations in AI. In: (ed) IEEE staff Proceedings of the Conference on Fairness, Accountability, and Transparency 279–288. https://doi.org/10.1145/3287560.3287574 (2019).Ates, E., Aksar, B., Leung, V. J. & Coskun, A. K. Counterfactual explanations for multivariate time series. In: 2021 International Conference on Applied Artificial Intelligence (ICAPAI) 1–8. https://doi.org/10.1109/ICAPAI49758.2021.9462056 (2021).Siddiqui, S. A., Mercier, D., Munir, M., Dengel, A. & Ahmed, S. TSViz: demystification of deep learning models for time-series analysis. IEEE Access 7, 67027–67040 (2019).Article
Google Scholar
Küsters, F., Schichtel, P., Ahmed, S. & Dengel, A. Conceptual explanations of neural network prediction for time series. In: 2020 International Joint Conference on Neural Networks (IJCNN) 1–6. https://doi.org/10.1109/IJCNN48605.2020.9207341 (2020).Guidotti, R., Monreale, A., Spinnato, F., Pedreschi, D. & Giannotti, F. Explaining any time series classifier. In: 2020 IEEE Second International Conference on Cognitive Machine Intelligence (CogMI) 167–176. https://doi.org/10.1109/CogMI50398.2020.00029 (2020).Binder, A. et al. Shortcomings of top-down randomization-based sanity checks for evaluations of deep neural network explanations. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 16143–16152 (2023).Baniecki, H., Kretowicz, W., Piatyszek, P., Wisniewski, J. & Biecek, P. dalex: responsible machine learning with interactive explainability and fairness in Python. JMLR 22, 1–7 (2021).H2O.ai. https://github.com/h2oai (2023).Li, X. et al. InterpretDL: explaining deep models in PaddlePaddle. JMLR 23, 1–6 (2022).
Google Scholar
People+AI Research (PAIR) Initiative. Saliency Library. PAIR code. https://github.com/PAIR-code/saliency (2023).Ancelin, M., Anne, E., Cavy, B. & Desmier, F. shapash. https://github.com/MAIF/shapash, (2023).Meudec, R. tf-explain. https://doi.org/10.5281/zenodo.5711704 (2021).Fernandez, F.-G. TorchCAM: class activation explorer. https://github.com/frgfm/torch-cam (2023).Fong, R., Patrick, M. & Vedaldi, A. Understanding deep networks via extremal perturbations and smooth masks. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2950–2958 (2019).Krakowczyk, D. et al. Zennit. https://github.com/chr5tphr/zennit (2023).