Retinal Image Segmentation and Disease Classification using Deep Learning

Abstract

Segmenting and classifying retinal blood vessels are essential ophthalmology tasks because they reveal important details regarding the health and condition of the retina and its blood vessels. However, there is still room for improvement in terms of accuracy and robustness. Deep learning models have demonstrated significant promise in precisely and quickly segmenting and categorizing retinal pictures. In this study, we suggest the Residual U-Net (ResUNet) model for segmenting retinal arteries and the Resnet50 model for classifying retinal arteries. For enhanced feature extraction and picture segmentation, the ResUNet model, a deep learning architecture, combines the benefits of the U-Net and ResNet50 models. Accuracy was required for segmentation and classification, which gave our study\'s focus on the project and potential survey results in its distinctiveness. The DRIVE, CHASE-DB1, STARE, and HRF datasets, which include retinal pictures with ground truth annotations, were used to train and test our model. These data were the resource for obtaining the segmented output with analysis in accordance with the initial input supplied to the model. Another sign of the robustness of our model is its strong generalization capacity. Overall, our study shows the usefulness of the ResUNet model for retinal vascular segmentation and classification and highlights its potential for enhancing the diagnosis and treatment of retinal illnesses, which has the potential to improve further with the potential for further development.

Introduction

I. INTRODUCTION

In the discipline of ophthalmology, segmenting and classifying retinal vascular structures is crucial because it helps with the detection and monitoring of retinal illnesses such as diabetic retinopathy, age-related macular degeneration, and glaucoma. In order to aid in early detection and intervention, accurate segmentation and classification of retinal vessels can offer useful information regarding the development and severity of various illnesses. Deep learning models have recently demonstrated considerable potential in obtaining high accuracy and efficiency in tasks involving retinal image analysis, such as segmenting and categorizing retinal vessels. However, reliable segmentation and classification of retinal pictures remain a problem for current deep-learning models, particularly when noise and fluctuations in image quality are present. In this study, we suggest a Residual U-Net (ResUNet) model for segmenting retinal arteries. We trained and evaluated our proposed ResUNet model on publicly available datasets: the DRIVE and CHASE-DB1 datasets, which contain retinal images with ground truth annotations. In addition to causing 6.7 million incidences of blindness, glaucoma affects 66.8 million people globally. In addition, if other eye conditions like diabetic retinopathy and cataract illnesses are not treated early enough, they might cause total blindness. As a result, we jointly created a deep learning-based model to classify the three types of retinal diseases—glaucoma, diabetic retinopathy, and cataract—into different groups. When dividing up retinal vessels and identifying disorders, the CNN network provides many benefits. The information flow from the initial, recall, precision, and F1-Score measurements provided by it regularly outperforms the state-of-the-art in vessel segmentation and classification. A variety of models were trained and evaluated to segment and forecast diseases with improved accuracy.

II. LITERATURE SURVEY

The chained outcomes technique is presented in the paper [1] and is accurate for early treatment planning, diagnosis, and visualization of ocular disease. Manual retinal segmentation is laborious and difficult where ophthalmology expertise is required. Two difficulties that typically occur and affect accuracy are inconsistent lighting of small vessels and a low-contrast optic disc area.

In the study [2], the technique of employing two Convolutional Neural Networks (CNNs) chained to one another was suggested. The second CNN, constructed from unused network blocks, joins the first CNN. As a result, we get information like recall and F1-Score. The creation of this model required the use of databases like DRIVE and CHASE.

To improve the image quality and allow CNN to recognize certain characters, pre-processing is done during this procedure.

In Paper [3], a deep learning supervised approach is proposed to address the identification problem for very thin arteries and sick conditions. To be more precise, it matched the average input of retinal vascular width with layer-wise effective receptive fields. The main focus of this research is on improving thin vessels. To achieve this purpose, extensive supervision and supplemental categorization have been used. Much more effort is required for deep supervision with data awareness. The layer that is most useful for obtaining vessel attributes is identified by the data-aware technique. The dominant layer is another name for the dominating layer. In terms of accuracy and sensitivity, this model outperforms the most advanced techniques.

In Paper [4], the segmentation of 2D images using the research's U-net model is discussed. They employed a deep guiding network to segment biological images, like the segmentation of retinal veins. The key components of the fundus image are the optic disc, optic cup, and vessels. Fundus imaging plays a significant role in the detection of retinal illnesses like hypertensive retinopathy, diabetic retinopathy, and retinal vein or artery blockage. Many different kinds of sensors are employed to gather biological signals since they are contaminated by a wide range of sounds. These sounds are removed from medical imaging by using picture filters like the guided image filter and the bilateral filter. Encoders and decoders are components of the U-net model.

The work [5] emphasizes the importance of deep learning for retinal vascular segmentation, which has surged in recent trends. The peculiarities of the retinal vessels work as an extraordinary benefit to recognizing a variety of eye diseases for later diagnosis. The doctor should be capable of monitoring, treating, diagnosing, and evaluating retinal sickness for diagnosis. This process takes a long time and is highly tiresome. However, it calls for specialized education and human knowledge, both of which are critical in a given circumstance. Therefore, the novel suggestion of developing a rapid method for retinal vascular segmentation would be useful for accuracy and require less maintenance. This describes how a modified residual U-NET can be used to segment retinal vascular architecture.

The research [6] uses an enhanced U-net convolutional neural network (CNN) architecture to segregate the optic disc and the optic cup from the retinal picture. The process involves procedures like cropping an RGB image after OD spots. Only OC segmentation, not OD segmentation, calls for this strategy. RGB images are subjected to spline interpolation using the nearest filling mode and the binomial order. Image scaling to a format of 128x128.Training images go through histogram equalization. All picture values are changed to be between 1 and 0 by rescaling images with the formula x = x/255. training the proposed CNN model with the scaled images the RIM-ONE and DRISHTI-GS databases provided the datasets that were utilized.

The Foundation for the paper [7] is that a classifier's ability to detect glaucoma can be enhanced by combining actual and synthetic images. The authors are able to create a sizable and varied training set using synthetic images, which helps to prevent overfitting and boosts the classifier's generalization. The scientists' use of a small collection of labeled images from the semi-supervised learning strategy also eliminates the necessity for costly manual labeling. Overall, this study offers a promising method for glaucoma identification utilizing artificial images and semi-supervised learning, which may enhance the effectiveness and precision of glaucoma screening in clinical settings. To prove the efficacy and viability of this strategy, however, additional research and validations

In Paper [8], The segmented blood vessels are then used to extract features from the images, which are fed into a support vector machine (SVM) classifier to classify the images as healthy or diseased. The Hessian-based and multiscale line detectors are two methods that are combined in the proposed method to find blood vessels in retinal pictures. The program then separates the vessels that were found from the surrounding area. The features from the images that were extracted using the segmented blood vessels were then fed into a support vector machine (SVM) classifier to determine whether the images were healthy or unhealthy. Overall, the strategy is promising for identifying and categorizing retinal pictures as healthy or sick. This could be helpful for diagnosing and tracking retinal illnesses in clinical settings, including diabetic retinopathy and age-related macular degeneration.

The segmented blood vessels are then used to extract features from the images, which are fed into a support vector machine (SVM) classifier to classify the images as healthy or diseased. For a fully automated system, the proposed method's paper [9] overall accuracy for diagnosing diabetic retinopathy was 87.25%, which is highly encouraging. Also quite high, at 90.25% and 85.75%, respectively, were the system's sensitivity and specificity. From the publicly accessible Messidor database, the authors used a dataset of retinal pictures. 800 photos were used for training and 400 for testing out of the 1200 total images in the dataset. The scientists also assessed the accuracy of their method in comparison to other cutting-edge methods for detecting diabetic retinopathy and discovered that it performed better than the majority of them. The article stresses the significance of early diabetic retinopathy detection because it can result in prompt therapies and shield diabetic individuals from going blind.

In the paper [10], a weighted attention mechanism and a U-net model are used to improve the recognition of small, thin vessels and to lower background noise. The appropriate illumination is used in conjunction with this.

The contrast of a fundus picture is enhanced using the Contrast Limited Adaptive Histogram Equalization (CLAHE) procedure at the pre-processing stage. To better train picture augmentation, such as horizontal flips, width shift range has been performed using overlapping 64*64 patches for roughly 500 images. They have put in place an attention mechanism that takes the ROI (region of interest) into account and ignores the background noise. Benchmark datasets like DRIVE and STARE were utilized. A Gaussian filter was used on the STARE dataset. Additionally, this model is cascaded using a skip connection, which combines identical mapping with convolutional processes like max pooling, and upsampling.

III. METHODOLOGY

A. Datasets

For the present research, we worked with public databases for the segmentation of retina images, DRIVE, and CHASEDB which are described below:

1. Drive: Consists of 40 color images of the retina, with dimensions of 565×584. This set is already divided into 20 images for training, which were separated into 15 to train the proposed neural network and 5 to validate them, as well as 20 other images for tests.
2. Chase-DB: Consists of 28 images of the retina of 14 children, cantered in the optic nerve, each with a dimension of 1280 × 960 pixels. The dataset provides a training set of 20 images and a test set of 8 images.
3. Stare: Consists of 20 retinal images with annotations for vessel segmentation and detection of lesions. It consists of images and provides training and testing sets.  It contains 20 equal-sized (700×605) color fundus images. For each image, two groups of annotations are provided.
4. HRF: Consists of 45 images and is organized as subsets of 15 with the image size are 3,504 x 2,336, with 22 training and 23 test images.

All the datasets provide ground truth annotations for vessel segmentation and classification, Augmentation was performed in the creation of the dataset where 1 image was augmented to 5 images which were further used. For classification, the dataset was taken from the Kaggle dataset with almost 4000 images for different diseases such as cataracts, diabetic retinopathy, glaucoma, and Normal which is 1000 images for each.

B. Segmentation              

The proposed methodology is divided into 4 steps:

1)  Data Augmentation: Data augmentation is a technique used to artificially increase the size of a dataset by creating modified versions of the original data. In this study, data augmentation was used to improve the performance and robustness of the deep learning model for retinal image analysis. The following data augmentation techniques were applied to the original retinal images:

a)  Image rotation: The original image was rotated by angles of 30, 45, 60, 90, 120, 180, and 270 degrees.

b)  Image flipping: The original image was flipped horizontally and vertically.

A total of 500 augmented images were generated for each original image, resulting in a dataset that was five times larger than the original dataset. During the augmentation process, the images were randomly modified according to the specified parameters, except the flipping operation which was applied deterministically. And was also split with the test data, trained data, and validation set.

2)  Data Preprocessing: The segmented blood vessels are then used to extract features from the images, which are fed into a support vector machine (SVM) classifier to classify the images as healthy or diseased. For our research, we used three publicly accessible retinal image datasets: the Child Heart and Health Study in England Database 1 (CHASE-DB1) dataset, the Structured Analysis of the Retina (STARE) dataset, and the HRF database. Both databases offer retinal pictures with real-world annotations for classifying and segmenting vessels. prior to the data being fed into the ResUNet model. To achieve a high-contrast background for the original image, the RGB photographs were first converted to a grey scale. Second, we performed preprocessing on the retinal images by scaling them to 512 512 pixels in resolution and normalizing the pixel values to the [0, 1] range. To improve the contrast, we used contrast-limited adaptive histogram equalization (CLAHE).

3)  Data Normalization: Data normalization is a method for putting a dataset's features into a similar scale and range so that machine learning algorithms can use them to produce predictions that are more relevant and comparable. In order to improve the performance of machine learning models, various feature sizes must be eliminated by normalization. Data normalization can be done in a number of ways, but two are frequently used: With this technique, the dataset's values are scaled to a specified range, often between 0 and 1. This scaling technique has the following formula:

a)  A is normalized as (a - min(a)) / (max(a) - min(b)). 

where a stands for the original value, a_normalized for the normalized value, and min(a) and max(a) stand for the least and maximum values of x in the dataset.

b)  Normalization should be performed on the training data and applied to the validation and test data as well. However, it is important to avoid using information from the validation and test data during normalization to prevent information leakage and overfitting.

Model for Segmentation

The model which was used for the segmentation is a pre-trained model which was trained to perform the segmentation of the input data and was evaluated accordingly which outputs the segmented image was RESUNET. This model has combined advantages of two models i.e., ResNet and U-Net which will increase the accuracy and robustness of the output image.

Conclusion

DRIVE, CHASE, HRF, and STARE datasets were used in our study to create a deep learning-based model for retinal vascular segmentation and classification and assess its performance. High accuracy of, sensitivity, specificity, precision, Recall, and F1 score was attained by the proposed ResUNet model across all datasets, demonstrating its efficiency and robustness in precisely segmenting and classifying retinal vascular segmentation. The findings of our work show the potential of deep learning-based retinal vascular segmentation and classification in assisting with the diagnosis and monitoring of various eye illnesses. Additionally, the retinal images were also accurately classified as ‘Cataract’, ‘Glaucoma’, ‘Diabetic retinopathy’, or a ‘Normal’ category and provided satisfactory results. Future studies in this area should concentrate on overcoming difficulties including generalization to new data, handling unbalanced datasets, and investigating more complicated models. Overall, the results of this work add to the expanding body of research on the application of deep learning for the segmentation and classification of retinal vessels and indicate that it has a substantial potential to enhance clinical outcomes for patients with eye illnesses. Acknowledgement: The authors would like to extend sincere Thanks to the researchers who have made their datasets publicly available, namely the Digital Retinal Images for Vessel Extraction (DRIVE), the CHASEDB1 dataset, the High-Resolution Fundus (HRF) dataset, and the Structured Analysis of the Retina (STARE) dataset.

References

[1] S Lian, L Li, G Lian, et al (2021). A global and local enhanced residual U-Net for accurate retinal vessel segmentation, IEEE/ACM Transactions on Computational Biology and Bioinformatics, 18(3), 852- 862, Available at: https://doi.org/10.1109/TCBB.2019.291718 8. [2] G Alfonso Francia, C Pedraza and S TovarArriaga (2020). Chaining a U-Net with a Residual U-Net for retinal blood vessels segmentation, IEEE Access, 8, 38493- 38500, Available at: https://ieeexplore.ieee.org/stamp/9006842. [3] S Mishra, D Z. Chen and X. Sharon Hu (2020). A data-aware deep supervised method for retinal vessel segmentation. 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE, Available at: https://doi.org/10.1109/ISBI45749.2020.90 98403. [4] P Yin, R Yuan, Y Cheng and Q Wu (2020). Deep guidance network for biomedical image segmentation, IEEE Access, 8, 116106-116116, Available at: https://doi.org/10.1109/ACCESS.2020.3002 835. [5] D Li, D Arief Dharmawan, B Poh Ng and S Rahardja (2019). Residual U-Net for retinal vessel segmentation. 2019 IEEE International Conference on Image Processing (ICIP). IEEE, Available at: https://doi.org/10.1109/ICIP.2019.8803101. [6] Ai O. Joshua, F V. Nelwamondo and G Mabuza-Hocquet (2019). Segmentation of optic cup and disc for diagnosis of glaucoma on retinal fundus images. 2019 Southern African Universities Power Engineering Conference/Robotics and Mechatronics/Pattern Recognition Association of South Africa (SAUPEC/RobMech/PRASA). IEEE, Available at: https://doi.org/10.1109/RoboMech.2019.87 04727. [7] Diaz-Pinto, Andres, et al. \"Retinal image synthesis and semi-supervised learning for glaucoma assessment.\" IEEE Transactions on medical imaging 38.9 (2019): 2211-2218. Available at: https://ieeexplore.ieee.org/abstract/document/8662628. [8] Barua, Bithi, and Md Mehedi Hasan. \"A new approach of detection and segmentation of blood vessels for the classification of healthy and diseased retinal images.\" 2016 3rd International Conference on Electrical Engineering and Information Communication Technology (ICEEICT). IEEE, 2016., Available at: https://ieeexplore.ieee.org/abstract/document/7873141. [9] T Mostafiz, I Jarin, S A. Fattah and C Shahnaz (2018). Retinal blood vessel segmentation using residual block incorporated U-Net architecture and fuzzy inference system. 2018 IEEE International WIE Conference on Electrical and Computer Engineering (WIECON-ECE). IEEE, Available at: https://doi.org/10.1109/WIECONECE.2018.8783182. [10] X Xiao, S Lian, Z Luo and S Li (2018). Weighted Res-UNet for high-quality retina vessel segmentation. 2018 9th International Conference on Information Technology in Medicine and Education (ITME). IEEE, Available at: https://doi.org/10.1109/ITME.2018.00080.

Copyright

Copyright © 2023 Sreelatha. H. S, Sreeranjani. K, Sushumna. R, Vagesh Panditharadhya, Manu. H. M. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.