Multi-Lesion Segmentation of Diabetic Retinopathy Using Deep Learning

Abstract

Diabetic Retinopathy (DR) and Diabetic Macular Edema (DME) are the two major complications of diabetes and have a significant impact on working individuals of the world population. DR doesn’t give any early symptoms. Therefore, it is important to diagnose DR at an early stage. The two above mentioned diseases usually depend on the presence and areas of lesions in fundus images. The four main related lesions include soft exudates, hard exudates,microaneurysms, and haemorrhages. Since lesions in retinal fundus images are a pivotal indicator of DR, analyzing retinal fundus images is the most popular method for DR screening. The examination of fundus images is time-consuming and small lesions are hard to observe. Therefore, adopting deep learning techniques for lesion segmentation is of great importance. In this project, we use one of the deep learning techniques called U-Net, which is a variant of Convolutional Neural Networks (CNN) for multiple lesion segmentation.

Introduction

I. INTRODUCTION

In recent times, India and other parts of the world have been faced with an increase in age and society related diseases like diabetes. According to recent survey, 24% of the country population has been diagnosed of diabetes disease alone and it have been recognize and accepted as one of the main cause of blindness in the country if not properly treated and managed. Early detection and diagnosis have been identified as one of the way to achieve a reduction in the percentage of visual impairment caused by diabetes with more emphasis on routine medical check which the use of special facilities for detection and monitoring of the diabetes. Diabetic related eye diseases are the most common cause of blindness in the world. Diabetic Retinopathy is a severe and widely spread eye disease, which can be regarded as manifestation of diabetes on retina. Diabetic Retinopathy is a specific micro vascular complication of both insulin dependent(type 1) and non insulin dependent (type 2) diabetes. The prevalence of retinopathy s strongly linked to the duration of diabetes. After 20 years of diabetes nearly all patients with type one diabetes and over 60% of patients with type 2 diabetes have some degree of retinopathy. Vision losses often, late symptoms of advanced diabetic retinopathy, many patients remain undiagnosed even as their disease is causing severe retinal damage. Hence there is an urgent need for mass screening retinal examination for the early detection and treatment of diabetic retinopathy

II. PROBLEM STATEMENT

Diabetes effects the circulatory system of a person, including that of the retina, which leads to DR. The oxygen supply to the visual system is reduced to a huge extent and it causes swellings on the retinal vessels. Also retinal lesions are formed which includes haemorrhages, microaneurysms and exudates. These are the symptoms for the disease, which will not be visible in the initial stages of the disease. Therefore, unless the patient takes regular examination of the disease, it cannot be identified and thus not cured. For a given collection of retinal fundus images (1…N), where N is greater than 100, the purpose is: (i) Precise classification of input images into normal, mild, moderate, severe. (ii) To increase classification accuracy and analyze the efficiency of the proposed work with the existing algorithms.

A. Risk Factors

All people with diabetes mellitus (Type I diabetes and Type II diabetes) are at risk. The longer a person has diabetes, the higher the risk of developing some ocular problem. After 20 years of diabetes, nearly all patients with Type I diabetes and greater than 60% of patients with Type II diabetes have some degree of retinopathy. It has been shown that the widely accepted WHO and American Diabetes Association diagnostic cutoff for diabetes of a fasting plasma glucose ≥ 7.0 mmol/l (126 mg/dl) does not accurately identify diabetic retinopathy among patients.

III. RELATED WORK

A. Image Segmentation

It is the process of dividing an image into multiple segments. In this process, every pixel in the image is associated with an object type. By dividing the image into segments, we can make use of the important segments for processing the image. That, in a nutshell, is how image segmentation works. An image is a collection or set of different pixels. We group together the pixels that have similar attributes using image segmentation. There are two major types of image segmentation?—?semantic segmentation

and instance segmentation.In semantic segmentation, all objects of the same type are marked using one class label while in instance segmentation similar objects get their own separate labels

The different types of Image segmentation architectures are:

1. Fast-FCN: In this architecture, a Joint Pyramid Up sampling (JPU) module is used to replace dilated convolutions since they consume a lot of memory and time. It uses a fully-connected network at its core while applying JPU for up sampling. JPU up samples the low-resolution feature maps to high-resolution feature maps.
2. Gated-CNN: This architecture consists of a two-stream CNN architecture. In this model, a separate branch is used to process image shape information. The shape stream is used to process boundary information.
3. Mask-R CNN: In this architecture, objects are classified, localized using a bounding box and semantic segmentation that classifies each pixel into a set of categories. Every region of interest gets a segmentation mask. A class label and a bounding box are produced as the final output.

IV. EXISTING SYSTEM

In traditional methods various feature extraction techniques are used to first extract the features from the fundus images. CNN demonstrates better performance on a variety of tasks in image processing and computer vision. Large training data is needed for Convolutional Neural Networks and requires lots of training data for image classification and segmentation tasks. If the images contain some degree of tilt or rotation then Convolutional Neural Networks usually have difficulty in classifying or segmenting the image. Lack of ability to be spatially invariant to the input data makes CNNs less significant for image classification and segmentation tasks.The proposed method is a Convolutional Neural Network variant that has been designed for the detection and segmentation of Microaneurysms, Haemorrhages and Exudates from retinal fundus images which is the Unet. Unet is a U-shaped encoder-decoder network architecture, which consists of encoder blocks and decoder blocks that are connected via a bridge.

One important modification in our architecture is that in the upsampling part we also have a large number of feature channels, which allow the network to propagate context information to higher resolution layers. As a consequence ,the expansive path is more or less symmetric to the contracting path, and yields a U-shaped architecture.

V. PROPOSED SYSTEM

U-Net is a convolutional network architecture for fast and precise segmentation of images. Many Neural Networks have performed ‘image segmentation’ before, but U-Net beats its predecessors by being less computationally expensive and minimizing information loss.

U-Net is a U-shaped encoder-decoder network architecture, which consists of four encoder blocks and four decoder blocks that are connected via a bridge. The proposed method is a Convolutional Neural Network variant that has been designed for the detection and segmentation of Microaneurysms, Haemorrhages and Exudates from retinal fundus images which is the Unet. Unet is a U-shaped encoder-decoder network architecture.

.

VI. METHODOLOGY

Indian Diabetic Retinopathy Dataset: It is the first database representative of an Indian population. Moreover, it is the only dataset constituting typical diabetic retinopathy lesions and also normal retinal structures annotated at a pixel level. This dataset provides information on the disease severity of diabetic retinopathy, and diabetic macular edema for each image.It consists of 3 folders each of which is divided again into 2 other folders i.e, Original Images, Groundtruths. Original images are the images captured using the Fundus camera and hence called the Fundus images of Retina. Groundtruths are classified into 5 categories.They are: 1.Optic Disc 2.Hemorrhages 3.Soft Exudates 4.Hard Exudates 5.Microaneurysms.

The U-NET model is an variant of Convolutional neural networks for biomedical image segmentation facilitated data management, model prototyping and real-time performance monitoring. The U-NET architecture imports the fundus image folder into training and validation subsets of 320 and 160 images respectively. The images were cropped to area size 256x256 and used as input data by Imagenet models previously trained for generic classification tasks. The test subset folder contained 160 images from the Indian Diabetic Retinopathy Image Dataset(IDRiD) and was disjoint from training data. This training system, which offered extensive hyperparameter selections, was then used to build model prototypes over 10 epochs requiring approximately 20 minutes each to complete and producing an accuracy of 97.6%. The lesions are labelled using image segmentation techniques.The predictions are performed on the random fundus images of the test folder.

VII. RESULTS

The testing images are selected at random and tested upon the model which produced an accuracy of 97% and a loss of 8.67%. The model was run for 100 epochs with 50 steps per epoch. The below figures show the Input image, its corresponding mask/label and the mask as detected by the model.

The graphs of training & validation losses vs the number of epochs and training & validation accuracy vs the number of epochs are quite opposite to each other.The increase in the value of training and validation loss after a certain number of epochs is due to numerical instability of some weights or gradients. The below figure shows the graphs of training and validation loss with respect to the number of epochs.

Conclusion

Medical image segmentation has an essential role in computer-aided diagnosis systems in different applications. It divides an image into areas based on a specified description, such as segmenting body organs/tissues in the medical applications for border detection, tumor detection/segmentation, and mass detection. The unet architecture was introduced for biomedical Image segmentation to extract the factors in the image. The second part decoder uses transposed convolution to permit localization. It is again a Fully Convolutional connected layers network. The idrid dataset contains only a limited number of samples and U-Net architecture requires large data in order to perform image segmentation tasks efficiently. Therefore, data augmentation is performed on the idrid dataset, and 960 samples were generated. These samples are further processed and trained using the U-Net architecture. The data took 50 minutes to train and yield the result of the segmented mask using U-Net. U-Net architecture achieved very good performance on fundus image segmentation tasks. Though we studied and applied U-Net for multi lesion segmentation on fundus images, we believe that U-Net can be applied to other image segmentation tasks.

References

[1] V. Gulshan, L. Peng, M. Coram, M. Stumpe, D. Wu, A. Narayanaswamy, S. Venugopalan, K. Widner, T.Madams, J. Cuadros et al., “Development and validation of a deep learning algorithm for detection of diabeticretinopathy in retinal fundus photographs,” JAMA, vol. 316, no. 22, pp. 2402– 2410, 2016. [2] X. Wang, L. Ju, X. Zhao, and Z. Ge, “Retinal abnormalities recognition using regional multitask learning,”in MICCAI, 2019. [3] W. Wang, Z. Xu, W. Yu, J. Zhao, J. Yang, F. He, Z. Yang, D. Chen, D. Ding, Y. Chen, and X. Li, “Two-stream CNN with loose pair training for multi-modal AMD categorization,” in MICCAI, 2019. [4] H. Fu, J. Cheng, Y. Xu, D. Wong, J. Liu, and X. Cao, “Joint optic disc and cup segmentation based on multi-label deep network and polar transformation,” IEEE transactions on medical imaging, vol. 37, no. 7, pp.1597–1605, 2018 [5] Z. Gu,J. Cheng, H. Fu, K. Zhou, H. Hao, Y. Zhao, T. Zhang, S. Gao, and J. Liu, “CE-Net: Context encodernetwork for 2d medical image segmentation,” IEEE transactions on medical imaging, 2019. [6] T. Laibacher, T. Weyde, and S. Jalali, “M2U-Net: Effective and efficient retinal vessel segmentation for real-world applications,” in CVPRW, 2019. [7] T. Durand, T. Mordan, N. Thome, and M. Cord, “WILDCAT: Weakly supervised learning of deep convnetsfor image classification, pointwise localization and segmentation,” in CVPR, 2017. [8] W. Ge, S. Yang, and Y. Yu, “Multi-evidence filtering and fusion for multi-label classification, object detection and semantic segmentation based on weakly supervised learning,” in CVPR, 2018. [9] Y. Zhou, Y. Zhu, Q. Ye, Q. Qiu, and J. Jiao, “Weakly supervised instance segmentation using class peak response,” in CVPR, 2018. [10] S. D. Solomon, E. Chew, E. J. Duh, L. Sobrin, J. K. Sun, B. L. VanderBeek, C. C. Wykoff, and T. W. Gardner, “Diabetic retinopathy: A position statement by the American Diabetes Association,” Diabetes Care,vol. 40, no. 3, pp. 412–418, 2017.

Copyright

Copyright © 2022 R. Swetha, N. Sailaja, K. Sai Divya, N. Vishal, P. D. Mahesh Babu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.