We categorize the generation of Soul datasets into three primary stages: initial clinical data acquisition, subsequent data filtering and pre-processing, and ultimately the implementation of a Human machine collaborative annotation framework (HMCAF). The overall workflow is illustrated in Fig. 2.Fig. 2Overall framework of Human machine collaborative annotation framework (HMCAF).Clinical data acquisitionSoul is a multi-diagnosis dataset collected from 53 subjects. The data collected at baseline include OCTA retinal images among BRVO patients in ages between 31 and 82 years old retrospectively collected from the Affiliated hospital of Shandong Second Medical University between 2020 to 2021. This study was received approval from the Institutional Review Board at the Affiliated hospital of Shandong Second Medical University (wyfy-2020-ky-11), following the principles outlined in the Declaration of Helsinki and with the include informed consent of all participants. OCTA images were included if (1) They do not have noticeable quality issues, such as severe smudges, artifacts, out-of-focus, blurriness, incorrect exposure, etc., that would affect the clarity of the observed target area. (2) To ensure spatial alignment of longitudinal tracking data, the macular fovea area is first labeled by a professional ophthalmologist during each patient visit. Subsequently, the 6*6mm area of the fovea of the macula is identified by automatic spatial alignment using the Optovue Angio OCT RTVueXR, a specialized ophthalmic device. This process uses high-resolution optical coherence tomography angiography to ensure accurate alignment of data from each image, helping doctors better monitor the progression of lesions and develop more effective treatment plans. An example is shown in Fig. 3, The results of multiple imaging data comparison showed that the blood vessels were reconstructed to a certain extent after multiple treatments. (3) Since the anti-VEGF Ranibizumab Injections drug is only valid for one month, in order to ensure the validity of follow-up data, only the information of visits on the day of Injections and visits within one week of Injections are kept in the follow-up data, and the follow-up should be ensured within 2-3 days after surgery as far as possible. Images were excluded if they showed any evidence of treatment, severe exposure abnormalities, severe refractive interstitial opacities, large-scale contaminations or if information about its origin was missing.Fig. 3The spatially aligned OCTA image data of BRVO patients corresponded to surgery 1, surgery 2 and surgery 3, respectively.Text RecordThe data primarily encompasses diverse medical information of patients, encompassing collection numbers, gender, age, diseased eye (left and right), disease progression, surgical dates, follow-up visit count, macular center thickness measurements, visual acuity assessments etc., with the corresponding records stored in an Excel file. To ensure participant anonymity and confidentiality, personal identifiers such as names are removed which can identify the subjects identity.ImageThe OCTA images typically acquired and stored grayscale in nature, saved in JPG format. The scanning process employed the Optovue Angio Oct RTVueXR system. It took approximately three years to collect and annotate these images. All subjects have complete registration information, with diagnosed diseases provided by ophthalmologists. Figure 4 (left) illustrates an sample of the scanned images alongside Projection maps at various levels. Although OCTA can produce projection Superificial layer (SVC), Deep layer (DVC), Outer Retina layer, and Choriocapillaris layer according to different retinal projection map, since BRVO disease image features are mostly based on the Superificial layer information, we have only take the SVC layer as the research object and build labels for its images. The retinal hierarchy corresponding to different projections is shown in Table 1 and Fig. 4 (right).Table 1 Superificial layer (SVC), Deep layer (DVC), Outer Retina layer, and Choriocapillaris layer corresponding retinal layers.Fig. 4Original scanned images(left) and Projection maps at different levels(right).Model Pre-training and Data pre-processingSoul encompasses three subsets, which have been categorized based on the number of injections and follow-up periods. These subsets include Soul-1(s1t1 & s1t2), Soul-2(s2t1 & s2t2), and Soul-3(s3t1 & s3t2), corresponding to patients who underwent a minimum of one, two, and three surgical treatments respectively. The characteristic of HMCAF is to reduce the time investment of expert manual annotation as much as possible while ensuring the authenticity and accuracy of labels. To ensure image quality and model performance, before original images imported into the baseline models of the HMCAF framework, A series of data processing operations such as data normalization, scale transformation, brightness change and contrast change were carried out. All images were uniformly cropped to eliminate any unused or unimportant boundaries and resized images with 304 * 304 pixels samed as OCTA-500 and ROSE. Before generating initial labels, the baseline model in the HMCAF framework needs to be pre-trained on the OCTA dataset to produce more accurate results. We pre-trained each of the four baseline models on the open source ROSE dataset and achieve optimal performance on the ROSE dataset. The baseline model performance is shown in Table 2.Table 2 Performance of the baseline model of pre-trained.Human machine collaborative annotation frameworkIn order to balance labeling accuracy and expert annotation cost, we propose a framework for automatically generating Pseudo-expert label, which consists of two modules: deep learning module and manual correction medules. Its framework is shown in the following Fig. 5.Fig. 5Detailed model of the HMCAF: OCTA-NET26, U-Net27, AttResU-Net28 and AttU-Net29.Machine learningFor the beginner learner, we select the basic framework model OCTA-NET26, U-Net27, AttResU-Net28 and AttU-Net29, which are pre-train on the ROSE-1 dataset using two-level labels of the SVC, to generate primary labels. However, the results are controversial due to the nature of the data for tiny blood vessel. Subsequently, we used the weighted fusion method to integrate the results of different models to obtain an improved fine vessel fusion label, as shown in Fig. 6.Fig. 6Results of different models and the fusion labels.Human correctionThrough deep learning, we have achieved labeling results with a certain level of accuracy. However, the inherent characteristics of diseases, such as patients’ poor fixation and other factors. Hence, to further enhance the accuracy and clinical applicability of our results, our ophthalmologists employ Labelme software (https://github.com/labelmeai/labelme)30 to correct the fusion label results (including additions, deletions, modifications, etc.). An example of artifacts and a comparison chart of expert corrections are shown in the Fig. 7.Fig. 7Diagram of the Expert Correction Module.
