Self-supervised learning for accurately modelling hierarchical evolutionary patterns of cerebrovasculature

FrameworkThe proposed pipeline for hierarchical quantitative analysis of cerebrovasculature is illustrated in Fig. 1. Paired T1 and TOF-MRA images were acquired from each subject. TOF-MRA volumes were utilized for cerebrovascular reconstruction using CereVessSeg, a segmentation model refined from nnUNet28 by incorporating attention module and loss function emphasizing learning from hard examples. To further improve the model performance, a contrastive learning strategy, CereVessPro, was adopted during model pretraining. TOF-MRA volumes were inputted into this model for individualized cerebrovascular reconstruction. T1 volumes were used to generate an average group-level structural template in MNI space and individualized deformation field in subject space. The deformation fields characterize the biprojection between MNI and individual spaces. CVs and AVs were quantified and contrasted across three spatial hierarchies, including a large spatial hierarchy defined by the whole brain, a medium spatial hierarchy defined by an arterial atlas with four typical areas, and a small spatial hierarchy defined by the conventional Brodmann atlas consisting of 82 Brodmann area. Normative models were further constructed, using locally estimated scatterplot smoothing15,42, for investigating arterial and cortical evolutionary trajectories across these spatial hierarchies.Fig. 1: The proposed pipeline for hierarchical quantitative analysis of cerebrovasculature.To start with, subjects were admitted into different hospitals to create records. Basic demographics were measured and recorded, followed by guiding the subjects to different MR scanners within each hospital for image acquisition. Typically, paired T1 and MRA volumes were acquired. Each T1 image was used to generate a deformation field for individualized biprojection between the subject and MNI space. Each subject’s cerebrovascular segmentation was obtained by applying the deep-learning segmentation model to the corresponding TOF-MRA volume. For feature quantification, the atlas defined in MNI space was transformed to subject space using the previously created deformation field. Normative models were then built using group-wise statistical vascular and cortical features.DatasetWe constructed a large-scale, multi-center dataset comprising 2841 pairs of T1 and TOF-MRA volumes, totaling 475,754 MRA slices and 322,944 T1-weigted slices. The dataset includes 1126 pairs acquired from healthy UK subjects (termed as UK-Hset), 950 pairs from healthy Chinese subjects (termed as CN-Hset), 642 pairs from Chinese subjects with brain pathology (termed as CN-Pset), and 123 additional pairs from Chinese subjects with Alzheimer’s Disease (termed as CN-ADset). Demographic information for each dataset is illustrated in Table 1. For UK-Hset, data of 570 subjects were collected from IXI dataset (http://brain-development.org/ixi-dataset/), wherein MRI scans were acquired at Hammersmith Hospital (IXI-HH), Guy’s Hospital (IXI-Guys), and the Institute of Psychiatry (IXI-IOP) in London, UK. The remaining data of 556 UK healthy subjects were sourced from OASIS-3 dataset43. For CN-Hset, data were obtained from the Affiliated Hospital of Guizhou Medical University (AHGMU) in Guizhou Province, China. For CN-Pset, data were collected from Datian General Hospital (DGH) in Fujian Province, China. For CN-ADset, data were collected from Zhongxiang Hospital of Traditional Chinese Medicine (ZHTCM) in Hubei Province, China. Based on the diagnosis information (see Supplementary Table 1), the subjects in CN-Pset can be subdivided into 165 subjects with large territorial stroke (termed as CN-LTSset) and 334 with lacunar stroke (termed as CN-LSset).Table 1 Demographic information for UK-Hset, CN-Hset, CN-Pset, CN-ADset, respectivelyMoreover, we manually annotated 271 (healthy:pathological = 150:121) TOF-MRA volumes, including 28,128 slices. This annotated dataset, named CereVessMRA, supports the development of deep-learning methods for cerebrovascular segmentation. The 150 annotated healthy TOF-MRA volumes from UK subjects were sourced from three different institutions, as part of the UK-Hset. The 121 annotated pathological TOF-MRA volumes from Chinese subjects were provided by DGH in Fujian Province, China. This large-scale and multi-center cerebral arterial annotation dataset facilitated the development of generalizable deep-learning models for automatic and precise cerebral arterial segmentation.Evolutionary patterns of AVs and CVs were consistent between UK and CN healthy subjects across the whole brain and the four typical brain regionsBoth groups showed similar general trends in CVs and AVs evolution, as observed in the left panel of Fig. 2A, B (please refer to Supplementary Figs. S1, S2 for age and gender distributions of UK-Hset and CN-Hset. No statistical differences in gender or age were identified between these two groups, as shown in Supplementary Tables S1, S2). To be specific, at the whole-brain level, decreasing trends of CVs and AVs were observed. This declining evolutionary trend was preserved in other three typical brain regions, including ACA, MCA, and PCA. In contrast, while both groups showed a similar rising trend of AVs in CoW region, CVs in CoW region exhibited a decreasing trend. Comparatively, CN subjects displayed a slower trend than their UK counterparts in the evolution of both AVs and CVs. Additionally, larger variations in the evolutionary curves were observed in CN subjects compared to UK subjects for both AVs and CVs.Fig. 2: Consistent evolutionary patterns of AVs and CVs for UK and Chinese healthy subjects.Data are presented as smooth curves obtained by locally estimated scatterplot smoothing (LOESS) together with 95% confidence interval (CI) curves. A Evolutionary patterns of AVs across the whole brain and four typical brain regions for CN (top panel) and UK (bottom panel) subjects. B Evolutionary patterns of CVs across the whole brain and four typical brain regions for CN (top panel) and UK (bottom panel) subjects. AV arterial volume, CV cortical volume, w/i within, ACA anterior cerebral area, MCA middle cerebral area, PCA posterior cerebral area, CoW circle of Willis. Source data are provided as a Source Data file.Within each group, statistical significance existed in the distributions of AVs and CVs across genders, as shown in Supplementary Tables S3–S6. To be specific, for UK healthy subjects, AVs distributions showed significant statistical differences across genders in spatial hierarchies including the whole brain (pcorrected < 0.01), CoW (pcorrected < 0.001), ACA (pcorrected < 0.01), and MCA (pcorrected < 0.05) across genders, with PCA (puncorrected = 0.0796) as an exception. For CN healthy subjects, significant statistical differences in AVs distributions were found in all these spatial hierarchies (pcorrected < 0.001, 0.05, 0.05, 0.001 for the whole brain, CoW, ACA, MCA, and PCA, respectively) across genders. For CVs distributions, gender differences were clearly recognized. Both groups exhibited the same level of statistical differences across genders (pcorrected < 0.001 for each spatial hierarchy).The degree of variation in CVs and AVs differs among large territorial stroke, lacunar stroke, and ADAs displayed in Fig. 3A and Supplementary Fig. S3A, subjects with AD were accompanied by significant reduction of AVs and CVs (refer to Supplementary Fig. S4 for the demographic information of CN-ADset, Supplementary Figs. S5, S6 for histograms of age-matched subjects from CN-Hset and CN-ADset for further statistical analysis using two-sample t-test). In average, AD males were found to have AVs that were 23.5%, 36.4%, 12.0%, 19.9%, 30.2% smaller than those of their healthy control (HC) counterparts at the whole-brain level, CoW, ACA, MCA, and PCA, respectively, as detailed in Supplementary Table S7. For CVs, the reductions were 24.7%, 48.1%, 24.4%, 22.7%, 13.9%, as shown in Supplementary Table S8. For females, AVs were reduced by 19.8%, 40.0%, 17.2%, 16.4%, and 26.3% as shown in Supplementary Table S9, while CVs were reduce by 20.4%, 40.7%, 20.0%, 18.0%, 10.3% as shown in Supplementary Table S10.Fig. 3: Comparisons of CN healthy subjects and subjects with brain pathology in terms of AVs.Data are presented as smooth curves obtained by locally estimated scatterplot smoothing (LOESS) together with 95% confidence interval (CI) curves. A Comparison of CN healthy subjects and AD subjects in terms of AVs. B Comparison of CN healthy subjects and subjects with large territorial stroke in terms of AVs. C Comparison of CN healthy subjects and subjects with lacunar stroke in terms of AVs. Corresponding comparisons in CVs are provided in Supplementary Fig. S3. AV arterial volume, w/i within, ACA anterior cerebral area, MCA middle cerebral area, PCA posterior cerebral area, CoW circle of Willis. Source data are provided as a Source Data file.Subjects with LTS also exhibited reduction in CVs and AVs as shown in Fig. 3B and Supplementary Fig. S3B, but at a relatively milder extent compared to those observed in AD subjects (refer to Supplementary Fig. S7 for the demographic information of CN-LTSset, Supplementary Figs. S8, S9 for histograms of age-matched subjects from CN-Hset and CN-LTSset for further statistical analysis using two-sample t-test). Significant differences in CVs and AVs were not consistently observed across hierarchies, as demonstrated in Supplementary Tables S11–S14. For both genders, AVs and CVs provided complementary information. To be specific, for males, significant differences were found in AVs between HC and LTS subjects within ACA, but not in CVs. For females with LTS, while no significant differences existed in CVs within ACA and PCA, AVs showed significant reductions, being 25.5% and 16.1% smaller in ACA and PCA, respectively, compared to HC.Reduction of CVs and AVs were also observed in subjects with LS as displayed in Fig. 3C and Supplementary Fig. S3C (see Supplementary Fig. S10 for the demographic information of CN-LSset, Supplementary Figs. S11, S12 for histograms of age-matched subjects from CN-Hset and CN-LSset for further statistical analysis using two-sample t-test). As demonstrated in Supplementary Tables S15–S18, LS deteriorated AVs in both genders, but males experienced greater reductions in CVs compared to females, particularly within ACA and PCA.Similar region-specific evolutionary trends in AVs were observed in healthy subjects from both CN and the UK, while some regions exhibited distinct patternsWhile decreasing trends of AVs and CVs at large spatial scales, such as the whole brain and four typical brain regions were evident, these trends did not consistently apply to more localized regions, especially when examined at a higher resolution spatial hierarchy. As displayed in Fig. 4A, B, at the small spatial scale wherein Brodmann areas are defined, generally decreasing trends of AVs similar to those observed at the whole-brain level were identified in regions including PMC, SMC, APC, OA, PVC, SVC, AVC, MTG, STG, PiriC, PCC, DEC, FG, TA, POper, PT, DPC2, POrb, and RA (see Supplementary Table S19 for the full names of these abbreviations) for both CN and UK healthy subjects (refer to Supplementary Fig. S13 for normative models of AVs in each Brodmann area). However, this decreasing trend was not consistent across all Brodmann areas. Different evolutionary patterns of AVs, including parabolic or even increasing trends, were observed in other Brodmann areas including PSC, ITG, VACC, SA, VEC, RCC, DACC, EA, TA, AG, SG, AC1, and AC2. In contrast to CVs, where evolutionary trends generally showed consistent decreasing patterns across different Brodmann areas (refer to Supplementary Fig. S14 for normative models of CVs in each Brodmann area), AVs exhibited region-specific evolutionary trends. Evolutionary trends of AVs were not uniformly preserved across spatial hierarchies. Supplementary Videos 1–16 provide dynamic visualizations of the evolutionary trends of CVs and AVs. While CVs exhibited mostly consistent decreasing trends across Brodmann areas, AVs in some parietal lobe regions were found to peak during middle or even old ages.Fig. 4: Mappings of regional arterial and cortical evolutionary trends for both genders at the small spatial scale defined by Brodmann areas.The horizontal axis represents increasing age. Value from each Brodmann area is color-mapped by the deviation from the regional average across the age range. A Lateral and medial views of arterial and cortical evolutionary trends for CN and UK healthy females. B Lateral and medial views of arterial and cortical evolutionary trends for CN and UK healthy males. Source data are provided as a Source Data file.Segmentation model equipped with channel-wise attention, hard example mining strategy and contrastive learning of voxel propagationA supervised-learning model CereVessSeg with a contrastive learning method CereVessPro was developed for cerebral vascular segmentation. With nnUNet28 as the backbone, the model CereVessSeg additionally introduced attention mechanism-based module (AM) to enhance discriminative capacity of feature presentation for cerebral vessels. Further, a hard example mining strategy was incorporated in the conventional cross-entropy loss, named LmCE, to mitigate segmentation errors caused by complex background tissues as well as low-contrast vascular branches. The contrastive learning CereVessPro was proposed to pretrain CereVessSeg model, which exploits position consistency of cerebral vessels among different subjects, thus providing a robust initialization for CereVessSeg in the cerebrovascular segmentation task. Subsequently, CereVessSeg was finetuned using our annotated CereVessMRA dataset. We compared our method with several convolution-based medical segmentation methods, including 3D U-Net, V-Net, DenseVoxNet, VoxResNet, U-ception, DeepvesselNet, and nnU-Net28,32,34,35,36,37,38, and also transformer-based methods, including UNETR and SwinUNETR39,40. All these compared methods were trained from scratch using our CereVessMRA dataset.Four metrics, including intersection over union (IoU), DSC, average symmetric surface distance (ASD) and 95% Hausdorff distance (HD95)41, were used to evaluate the segmentation performances of different methods as well as for quantitative comparisons among different methods. The IoU and DSC, defined as follows, quantify the degree of similarities between ground truth and model predictions:$$\,{\mbox{IoU}}\,({{\bf{y}}},\hat{{{\bf{y}}}})=\frac{| {{\bf{y}}}\cap \hat{{{\bf{y}}}}| }{| {{\bf{y}}}\cup \hat{{{\bf{y}}}}| },$$
(1)
$$\,{\mbox{DSC}}\,({{\bf{y}}},\hat{{{\bf{y}}}})=\frac{2| {{\bf{y}}}\cap \hat{{{\bf{y}}}}| }{| {{\bf{y}}}|+| \hat{{{\bf{y}}}}| },$$
(2)
where y and \(\hat{{{\bf{y}}}}\) represent ground truth and the predicted binary segmentation results, respectively. ∣.∣ represents the number of voxels. ASD measures the average distance between the closest points on the surface of ground truth ys and the surface of predicted binary segmentation \({\hat{{{\bf{y}}}}}_{s}\), formulated as following:$$\,{{\mbox{ASD}}}\,({{{\bf{y}}}}_{s},{\hat{{{\bf{y}}}}}_{s})=\frac{1}{| {{{\bf{y}}}}_{s}|+| {\hat{{{\bf{y}}}}}_{s}| }\left(\sum_{p\in {{{\bf{y}}}}_{s}}\mathop{\min }_{q\in {\hat{{{\bf{y}}}}}_{s}}d(p,q)+\sum_{q\in {\hat{{{\bf{y}}}}}_{s}}\mathop{\min }_{p\in {{{\bf{y}}}}_{s}}d(q,p)\right),$$
(3)
HD95 is similar to conventional Hausdorff distance, which describes the maximum minimal distance between points on ys and \({\hat{\bf{y}}}_{s}\) as follows:$${\rm{HD}} \, ({\bf{y}}_{s},{\hat{\bf{y}}}_{s})= \max \left(\mathop{\max}\limits_{p \in {\bf{y}}_{s}} \mathop{\min}\limits_{q \in {\hat{\bf{y}}}_{s}}d(p , q) , \mathop{\max}\limits_q \in {\hat{{\bf{y}}}_{s}} \min_{p \in {\bf{y}}_{s}}d(q,p)\right),$$
(4)
Notably, HD95 eliminates the impact of a small subset of outliers by using the 95th percentile of distances.The performances of different methods on TOF-MRA volumes from healthy subjects and brain pathological subjects are displayed separately in Table 2. Mean squared distance (MSD) accesses the boundary consistency on average, with smaller MSD values suggesting fewer boundary gaps between ground truth and predictions. HD95 measures the network’s capability to suppress the influence of non-vascular regions during segmentation, while allowing for some outliers. Thus, a combination of a low DSC and a high MSD suggests compromised segmentation performance due to low capability in accurately recognizing vascular boundaries. And a combination of a low DSC and a high HD95 implies that deteriorated segmentation performance due to limited capability in filtering out some non-vascular regions. For MRA volumes from healthy subjects, while nnUNet28 was able to outperform other existing popular methods including DenseVoxNet, VoxResNet, 3D U-Net, DeepVesselNet, U-ception, V-Net, UNETR, and SwinUNETR, the proposed method can further advance its capability in boundary detection and outlier suppression. Interestingly, high accuracy in MRA volumes from healthy subjects did not guarantee equally good performance on MRA volumes from subjects with brain pathology. Besides our methods, methods including DenseVoxNet, 3D U-Net, DeepVesselNet, V-Net, and nnUNet showed higher accuracies for MRA volumes from pathological subjects compared to MRA volumes from healthy subjects, whereas VoxResNet, U-Ception, UNERT, and SwinUNETR exhibited decreased performance. Brain pathology always includes a decrement in the size of brain cells (please refer to Supplementary Fig. S15 for an illustration of brain pathology). Pathology can be either generalized, which means that all of the brain has shrunk or it can be focal, affecting only a limited area of the brain. Thus, changed or collapsed cerebral vasculature, especially in distal part of vessels, may be observed in pathology brains. In return, this morphological change of vasculature may lower the difficulties of segmentation due to losses of distal small vessels, which may explain the reason why improved performance were observed as mentioned above. Overall, among all other methods, our proposed method also achieves best segmentation performance for MRA volumes from pathological subjects.Table 2 Quantitative evaluation of different segmentation methodsSupplementary Tables 20, 21 display the statistical significance results from t-tests comparing our proposed CereVessSeg model, with and without CereVessPro pretraining, against other methods on MRA volumes from healthy and pathological subjects. CereVessSeg, both with and without pretraining, performed significantly better than all compared methods for MRA volumes from subjects with brain pathology. But for MRA volumes from healthy subjects, CereVessSeg without pretraining was not significantly better than SwinUNETR and nnUNet in terms of IoU and MSD. With pretraining, CereVessSeg became significantly better than all other compared methods, indicating the boost by CereVessPro in generally improving model performance for multi-center data.Visualization comparisons are shown in Fig. 5 (refer to Supplementary Fig. S16 for more visualized comparisons). Figure 5 reveals that DenseVoxNet, VoxResNet, 3D U-Net, DeepVesselNet, U-ception, UNETR, SwinUNETR, and V-Net exhibited many false positives (blue) and false negatives (green) in their segmentation results, compared to ground truth, particularly in the areas highlighted by arrows. In comparison, nnUNet produced fewer segmentation errors. Moreover, CereVessSeg, enhanced with the proposed AM and LmCE, further reduced both false positives and false negatives. Finally, with CereVessPro pretraining, CereVessSeg achieved the most complete and accurate cerebrovascular segmentation, closest to the ground truth compared to all other methods.Fig. 5: Visualized segmentation results from different methods.For every method, the segmentation result for a whole TOF-MRA volume is presented on the middle raw. Then two regions marked by the yellow and gray boxes are zoomed in and then displayed above and below to show details of segmentation. In every sub-figure, true positives are marked in red; false positives are marked in blue; false negatives are marked in green. White arrows point to areas with significant segmentation differences.We conducted ablation studies to measure the effectiveness of the proposed AM and LmCE in boosting the performance of CereVessSeg. The results are exhibited in Table 2. As seen from Table 2, each of the two proposed modifications independently improved segmentation performance, surpassing other state-of-the-art models.Additionally, we ablated different SSL methods for CereVessSeg, including masked volume inpainting44, BYOL45, and SimSiam46. The segmentation performances of CereVessSeg using these methods for pretraining are listed in Table 2. As shown, inpainting, BYOL or SimSiam did not enhance CereVessSeg’s performance for MRA volumes from healthy or pathological subjects. In comparison, the proposed CereVessPro significantly improved performance on both types of MRA volumes, indicating its superiority for our segmentation task.Supplementary Fig. S17 shows some failed cases by our proposed CereVessSeg, pretrained with CereVessPro. One kind of failed cases are vascular branches with very low intensity contrast against the background. Despite incorporating the proposed AM and LmCE, our methods still struggle to detect a few dim peripheral vessels and vascular ends, leading to under-segmentation. Another kind of failed cases are vessels in the head scalp and skull, which resemble cerebral vessels in appearance but are not part of the cerebrovascular system. Due to the patch-wise segmentation strategy used for whole-brain MRA inference, the segmentation model has difficulty perceiving the spatial locations of each patch in relation to the entire brain, making it challenging to tell apart cerebral vessels from vessels in the head scalp or skull. However, it is important to note that the first kind of failed cases are rare and involve only small regions, exerting minimal effects on cerebrovasculature quantification. In the second kind of cases, the over-segmented vessels in the head scalp and skull are excluded from subsequent analysis of cerebral vascular structures. Thus, these failed cases would not affect the overall quantification of cerebrovasculature.External validationWe conducted an experiment to validate our proposed CereVessSeg with CereVessPro on an external dataset, TubeTK (https://public.kitware.com/Wiki/TubeTK/Data). The TubeTK dataset contains MRA volumes from healthy subjects, acquired using a Siemens Allegra head-only 3T MR system. Voxel spacing of these MRA volumes is 0.5 × 0.5 × 0.8 mm3. Supplementary Fig. S18 presents segmentation examples of our segmentation method on TubeTK dataset. Compared to the provided annotations, which miss many vascular branches, our proposed CereVessSeg produced more complete cerebrovasculature segmentation. Besides, upon the training process not using MRA volumes from AD subjects, CN-ADset was utilized for external validation as well. Supplementary Fig. S19 presents segmentation examples on CN-ADset, showing our segmentation method accurately segmented almost every vascular region and thereby achieved highly complete and precise cerebrovascular segmentation results. The segmentation performances on both the TubeTK and CN-ADset demonstrated the generalizability of our CereVessSeg with CereVessPro, thanks to multi-center training data and the efficient contrastive learning method.