Evaluation indexIn the experiment, a variety of criteria are used to evaluate the classification performance of the model, such as confusion matrix, ROC curve and AUC region, accuracy, precision, recall rate and F1 value. Confusion matrix can intuitively show the specific situation of classification of the model, and the confusion matrix is shown in Table 1. TP, FP, TN and FN represent true positive, false positive, true negative and false negative respectively.Table 1 Confusion matrix.Accuracy refers to the ratio of the number of samples correctly predicted by the classification model to the total number of samples, which is used to measure the accuracy of the overall prediction of the model. Precision refers to the ratio of the number of samples that are actually positive to the total number of samples that are predicted to be positive. Recall refers to the ratio of the number of samples correctly predicted by the model to the total number of samples that are actually positive among all samples. The F1 value is a harmonic average of accuracy and recall, which is used to comprehensively evaluate the classification performance of the model. The relevant expressions of each evaluation index are shown in Eqs. (7)–(10).$$\begin{aligned} Accuracy= & {} \frac{TP+TN}{TP+FP+TN+FN} \end{aligned}$$
(7)
$$\begin{aligned} Precision= & {} \frac{TP}{TP+FP} \end{aligned}$$
(8)
$$\begin{aligned} Recall= & {} \frac{TP}{TP+FN} \end{aligned}$$
(9)
$$\begin{aligned} F1-Score= & {} \frac{2\times Precision \times Recall }{Precision+Recall} \end{aligned}$$
(10)
Neuron data setNeuroMorpho.Org42is an open neuromorphologic database that collects and shares neuronal morphological data from different species, different brain regions, and different cell types. It contains morphological data on thousands of neurons and is currently the largest publicly accessible 3D neuronal reconstruction and related neuronal dataset.The two-dimensional image dataset of neurons in the Neuromorph-RAT dataset made by Zhang Tielin’s team22 is used for the experiment, including original neuron images (Img_raw) obtained by web crawler, the neuronal images with Z-jumping issues repaired using the tree toolbox (Img_resample), and the neuronal image data with XY alignment (Img_XYalign). In the experiment, 4-category classification and 12-category classification were carried out respectively. The 4-category classification includes principal cells, interneurons, glial cells, and sensory receptor cells, and 12-category classification consists of 12 subclasses, including six principal cells, three interneurons, two glial cells, and one sensory receptor cell.Experimental results and analysisImproved tests of AlexNet, VGG11_bn and ResNet-50In order to verify the effectiveness of each improved network, the network model before and after AlexNet improvement, VGG11_bn improvement and ResNet-50 improvement were tested on three datasets under the same experimental conditions. The test results of 4-category classification are shown in Fig. 8. The test results of 12-category classification are shown in Fig. 9.As can be seen from Fig. 8, the accuracy of the improved AlexNet network, the improved VGG11_bn network and the improved ResNet-50 network were all better than those before the improvement in the neuron morphology classification experiment. In addition, the accuracy of the improved AlexNet has increased by 3.45%, 3.41% and 3.06% compared to the original in the Img_raw dataset, Img_resample dataset, and Img_XYalian dataset, respectively. The accuracy of the improved VGG11_bn has increased by 1.78%, 1.89% and 0.83% on the three datasets, respectively, while the accuracy of the improved ResNet-50 has increased by 0.91%, 0.09% and 0.20% on the three datasets, respectively. This proves that the improved model has better performance in neuronal morphology-4 classification.Figure 8Comparison results of accuracy before and after model improvement (4-category classification).As shown in Fig. 9, in the 12-category classification experiment on the three datasets, the improved AlexNet’s accuracy has increased by 6.91%, 3.77%, and 10.09% respectively compared to the original in the Img_raw dataset, Img_resample dataset, and Img_XYalian dataset. The accuracy of the improved VGG11_bn has been improved by 2.63%, 5.13% and 1.60% on the three datasets, respectively, and the accuracy of the improved ResNet-50 has been improved by 2.29%, 0.50% and 0.08% on the three datasets, respectively.Figure 9Comparison results of accuracy before and after model improvement (12-category classification).Analysis of results of ablation experiment and MCF-Net experimentUnder the same experimental conditions, four different fusion methods of AlexNet network, VGG11_bn network and ResNet-50 network were used for ablation experiments. The experiment also used Img_raw dataset, Img_resample dataset and Img_XYalian dataset. The four fusion schemes include: (1) AlexNet, VGG11_bn and ResNet-50 models before the improvement (2) AlexNet, VGG11_bn before improvement and ResNet-50 after improvement (3) The improved AlexNet, VGG11_bn and ResNet-50 models before the improvement (4) Improved AlexNet, VGG11_bn, and ResNet-50 models. For each fusion method, the following steps are followed to conduct experiments: a. Train three network models to extract features from the images in the datasets. b. Save the probability predicted by the three classifiers for each category in csv file format. c. Use the Sugeno fuzzy integral to fuse the output of each classifier. d. Use the fused features for testing and calculate various experimental indicators. The experimental results are shown in Tables 2, 3 and 4.By observing the results in Tables 2, 3 and 4, it can be found that the fusion improved model AlexNet, VGG11_bn and ResNet-50, namely MCF-Net model, has the highest classification accuracy in the three datasets, reaching 97.82%, 91.75% and 93.13% respectively. These prove that the proposed fusion scheme of MCF-Net can obtain relatively optimal results, and also demonstrates that the proposed method is effective, and the improvement of a single network can still play its role after the network fusion.Table 2 Test results of different fusion methods in Img_raw dataset.Table 3 Test results of different fusion methods in Img_resample dataset.Table 4 Test results of different fusion methods in Img_XYalian dataset.In order to analyze the classification performance of MCF-Net network, confusion matrix and ROC curve were plotted on Img_raw dataset to compare the classification accuracy of different types of neurons. The confusion matrix is shown in Fig. 10, and the ROC curve obtained under this data set is shown in Fig. 11. To simplify the graph, 0 represents Glia, 1 represents Interneuron, 2 represents Principal cell, and 3 represents Sensory Receptor when drawing the confusion matrix and ROC curve of 4-category classification. When drawing the confusion matrix and ROC curve of 12-category classification, the labels are as follows: 0 represents GABAergic, 1 represents Nitrergic, 2 represents Pakachromaffin, 3 represents Purkinje, 4 represents Astrocyte, 5 represents Basket, 6 represents Ganglion, and 7 represents Granule, 8 represents Medium Spiny, 9 represents Microglia, 10 represents Pyramidal, and 11 represents Sensory Receptor.In Fig. 10a, it is indicated that when using the Img_raw dataset for 4-category classification testing, except for the slightly lower accuracy of the sensory receptor cell classification, the other categories can be correctly classified. It is evident from Fig. 10b that the MCF-Net model can correctly predict most types of neurons, especially Nitrergic, Pakachromaffin, and Pyramidal cells with an accuracy of 1. The area under the ROC curve (AUC) is a common indicator to measure the performance of a classifier, ranging from 0 to 1, and the closer the curve area is to 1, the better the performance of the classifier. As can be seen from the results in Fig. 11, the area under the ROC curve of the method based on multi-classifier fusion is relatively high, which indicates that the method can classify neurons more accurately.Figure 10Confusion matrix using MCF-Net on Img_raw dataset (a) 4-category classification, (b) 12-category classification.Figure 11ROC curve using MCF-Net on Img_raw dataset (a) 4-category classification, (b) 12-category classification.Comparison with existing methodsIn order to further verify the validity of the model, we compared the existing classical classification models on three neuronal morphological datasets with 4-category classification and 12-category classification, including the model used for integration in this paper. Zhang Tielin’s team22 used CNN(ResNet18) method to classify neuronal image dataset into 12 categories. The comparison results of 4-category classification are shown in Table 5, and the comparison results of 12-category classification are shown in Table 6. According to the experimental results, the MCF-Net network model has a higher accuracy in the task of neuron morphology classification than a single network, which indicates that the network can better capture the characteristics of neuron morphology by combining the classification results of multiple models. In addition, the experimental results of MCF-Net network are verified on three datasets, which further proves the validity and reliability of the network model. In general, the Sugeno fuzzy integral algorithm can be used to fuse the output results of multiple classifiers, which can achieve the morphological classification of neurons well.Table 5 Performance comparison results of different classification methods (4-category classification).Table 6 Performance comparison results of different classification methods (12-category classification).As shown in Table 7, different integration techniques were used to classify neurons on the Img_raw dataset, including the majority voting rule, the simple average method, and the multiplication rule. The experimental results show that the Sugeno fuzzy integral algorithm has high classification accuracy and good classification performance in neuron classification.Table 7 Comparison with existing integration methods.The computational complexity analysisThe number of parameters refers to the sum of the number of ownership weight and bias terms in the neural network, which are optimized during the training process to improve the predictive power of the model. The number of parameters depends mainly on the number of filters in the CNN model, the size of each filter, and the number of layers in the network, which together determine the complexity of the model, which in turn has a significant impact on memory requirements and training time. On the other hand, the testing time of a CNN model refers to the time required to predict or classify new and unseen data after the completion of model training, which reflects the response speed and performance of the model in practical applications.Table 8 presents the detailed parameter information of the three optimized basic networks, including their number of parameters, test time and test accuracy, which are utilized in the Sugeno fuzzy integral integration process. Compared to these three basic networks, the MCF-Net model has a significantly higher number of parameters, almost equal to the sum of the parameters in the three basic networks. The model contains 37.5M parameters and the test time is 252 seconds. As shown in the table, the testing duration of the MCF-Net model includes the individual testing time of the three fundamental networks and the time needed for model integration. Since the model integration time is relatively short, we can roughly consider that the testing time of the MCF-Net model mainly comes from the cumulative testing time of three basic networks. Although the MCF-Net model has a large number of parameters and requires the longest training time, it demonstrates a high accuracy and successfully realizes the goal of neuron classification without considering the limitations of computational cost and experimental conditions.Table 8 Total parameters, test time and test accuracy of the model.GradCAM analysisGradCAM43is a method for visualizing image regions of concern to convolutional neural network (CNN) models. By combining the convolution layer features and gradient information of the CNN model, it generates gradient-weighted activation maps to highlight the image regions that have an important impact on classification tasks.Figure 12Some neuron images (taken from the Img_raw dataset) along with their GradCAM activations by the three models used for forming the model fusion in this study.Figure 12 shows GradCAM activation graphs generated by three different models that serve as the basic learning network for ensemble learning, using sample images from the Img_raw dataset. As can be seen from the Fig. 12, different models focus on different parts of the same neuron image. For example, Fig. 12a shows Glia images, and Fig. 12b–d show class activation graphs of AlexNet, VGG11_bn, and ResNeXt-50, respectively. It’s clear from Fig. 12 that AlexNet focuses on both sides of the neuron’s left and right boundaries, and VGG11_bn focused more on the upper right and lower left regions of neurons, while ResNet50 focused on the middle of neurons.Different models focus on different regions of the same neuron image, which shows that there are differences and complementarities between models, which makes model fusion meaningful. From the examples, it is obvious that the three models focus on different image feature areas. Therefore, the method based on multi-classifiers fusion can synthesize the advantages of multiple models, improve the accuracy and stability of models and achieve better results in practical applications.
Morphological classification of neurons based on Sugeno fuzzy integration and multi-classifier fusion
