Literature Survey on Skin Lesion Classification

Abstract

Skin cancer is a growing public health concern, and early detection is crucial for successful treatment. In recent years, advances in deep learning algorithms have led to the development of skin lesion classification systems, which have the potential to improve the accuracy and speed of skin cancer diagnosis. Skin lesion classification is the task of accurately identifying and categorizing different types of skin abnormalities, including moles, freckles, and skin cancers. Deep learning algorithms have demonstrated encouraging outcomes and tremendous capabilities in handling image analysis. It can effectively tackle intricate issues in a prompt and effective way. However, Deep learning models may not always resemble or reflect the decision-making methods of dermatologists. Therefore, an Explainable Artificial Intelligence (XAI) is employed to comprehend the machine learning verdicts via the visual depictions of the categories of skin lesions. This paper provides an overview of the skin lesion classification using a deep learning algorithm along with an explainable artificial intelligence.

Introduction

I. INTRODUCTION

Skin cancer is a type of cancer that develops in the skin's cells. It is the most common type of cancer in the world, and its prevalence rises year after year. Skin cancer can be caused by UV radiation from the sun or tanning beds, genetics, or a weakened immune system. It can affect people of all ages and skin types, but those with fair skin, light-colored hair, and blue or green eyes are at a higher risk. Skin cancer can manifest as various types of skin lesions, which are abnormal growths or changes in the appearance of the skin. Skin lesions come in a variety of sizes and shapes, ranging from small, harmless moles to larger, potentially cancerous growths. They can be caused by a number of factors, including UV radiation exposure, infections, and skin injuries. It is critical to be aware of the signs and symptoms of skin lesions and to seek medical attention if they persist or change over time. The appearance and characteristics of skin lesions, such as size, shape, color, and texture, can be used to classify them. With the advancements in machine learning and artificial intelligence, deep learning models have shown great potential in improving the accuracy of skin lesion classification. These models can help dermatologists to identify various types of skin lesions. However, despite the high accuracy achieved by deep learning models, they often lack interpretability, which can make it challenging to understand the reasoning behind the model's predictions. This is where Explainable Artificial Intelligence (XAI) comes in, which provides a way to interpret and explain the inner workings of deep learning models. The aim of the survey was to provide an overview of the recent advancements in the field of skin lesion classification using deep learning and XAI. This survey will be of great interest to researchers and practitioners in the field of dermatology and medical imaging who are interested in the use of deep learning and XAI to improve the accuracy and interpretability of skin lesion classification.

II. LITERATURE SURVEY

Nigar et al. [1] proposed an machine learning model to classify the skin lesions using an pre-trained ResNet-18 deep learning algorithm by utilizing ISIC 2019 dataset [2,3,4] .The visual explanations for identifying the skin lesions classes is given by explainable artificial intelligence  which  helps the dermatologist for improved performance in earliest skin lesions detection. The skin cancer detection accuracy is improved by it. The eight classes of skin lesions such as actinic keratosis, basal cell carcinoma, benign keratosis, dermatofibroma, melanoma, melanocytic nevus, squamous cell carcinoma, vascular lesion are identified by the model. The model achieves 94.47% classification accuracy, 93.57% precision, 94.01% recall and 94.45% F1-score respectively. The Local Interpretable Model-agnostic Explanations (LIME) framework is used to generate visual explanations.

Rehman et al. [5] utilized pre-trained deep learning models, namely MobileNetV2 and DenseNet201, to categorize skin diseases into two classes: benign and malignant. This was achieved by utilizing the International Skin Imaging Collaboration (ISIC) archive dataset [6] .The models were modified by appending convolutional layers at the end of the both models to detect the skin cancer effectively. Furthermore, the final classification layer and the classification head were also modified. The modified models exhibited superior performance compared to the original pre-trained MobileNetV2 and DenseNet201 models.

Both the classes can be detected using the modified models. The Modified DenseNet201 model attains an accuracy of 95.50%, sensitivity of 93.96%, and specificity of 97.03%. With minor modifications, the model can be employed for diagnosing skin cancer across multiple categories.

Chowdhury et al. [7] employed a custom Convolutional Neural Network (CNN) to recognize 7 classes of skin diseases through the  HAM10000 dataset [2]. Class Activation Mapping (CAM) was used as an XAI method. This study aimed to explore whether the features extracted from deep models like convolutional networks, self-attention models, and attention as activation models have a correlation with clinically significant features. Three main categories of raw pixel-oriented machine learning algorithms, which include convolution, spatial self-attention, and attention as activation, were examined and contrasted with the Asymmetry, Border, Color, Diameter (ABCD) skin lesion clinical characteristics-based machine learning algorithms, employing both qualitative and quantitative interpretations. A visual analysis was done to see if the activation maps of deep models were similar to the segmentation maps used for clinical feature extraction. The maximum achieved accuracy is 82.7% and 78% of precision.

Nunnari et al. [8]  presented on how saliency maps can be used to identify areas of interest in the diagnosis of skin cancer. VGG16 and ResNet-50 are used to classify 8 skin classes and Gradient-weighted Class Activation Mapping (Grad-CAM) was used as an explainable method. The two neural  architectures distinguished by varying layers and distinct resolution of the final convolutional layer, measure how effectively thresholded Grad-CAM saliency maps can detect visual attributes of skin cancer. Research indicates that saliency maps subjected to a threshold can achieve a success rate of nearly 50%. The optimal threshold for obtaining the highest Jaccard index differs significantly depending on the features. Furthermore, the Jaccard index attained was as high as 0.143, which is approximately half the performance of cutting-edge architectures designed for predicting feature masks at the pixel level, like U-Net. The models attain an accuracy of  72.2% and 76.7%  respectively.

Wang et al. [9] introduced an Interpretability-based Multimodal Convolutional Neural Network (IM-CNN), a convolutional neural network that prioritizes interpretability. The model is designed for multiclass classification of skin lesion images and patient metadata. The IM-CNN is composed of three primary components that handle metadata, segmented skin lesion features utilizing domain expertise, and skin lesion images. The visual modules of the IM-CNN include explanations for both images and metadata, making it interpretable. Area Under the ROC Curve (AUC), specificity, and sensitivity, a fresh metric named AUC curve with sensitivity greater than 80% (AUC_SEN_80) has been introduced for assessing performance. For the HAM10000 dataset, Grad-CAM and SHapley Additive exPlanations (SHAP) served as visual explanation techniques. Compared to the single-model, IM-CNN achieves a 72%, 9%, and 21% increase in sensitivity, AUC, and AUC_SEN_80, respectively.

Kassem et al. [10] put forth a scheme for categorizing eight types of skin diseases by utilizing the ISIC 2019 dataset [2,3,4]. Timely and exact identification of skin abnormalities can be a matter of life and death. To accomplish this, the model uses GoogleNet pre-trained model and transfer learning. The model effectively categorized Melanoma, melanocytic nevus, basal cell carcinoma, actinic keratosis, benign keratosis, dermatofibroma, and Squamous cell carcinoma. The accuracy, sensitivity, specificity, and precision percentages of the classification were 94.92%, 79.8%, 97%, and 80.36%, respectively. The model can also detect images that do not belong to any of the eight classes.

Xie et al. [11] introduced the Mutual Bootstrapping Deep Convolutional Neural Networks (MB-DCNN) as a solution for both skin lesion segmentation and classification. The MB-DCNN model comprises of three networks: a rough segmentation network (coarse-SN), a mask-directed Classification Network (mask-CN), and an improved Segmentation Network (improved-SN). The coarse-SN generates preliminary lesion masks that act as a priori bootstrapping for mask-CN, enabling it to precisely locate and classify skin lesions. Subsequently, the lesion localization maps created by mask-CN are fed into improved-SN, with the objective of transferring the location information learned by mask-CN to improved-SN for precise lesion segmentation. The transfer of knowledge between the two networks is done in a bootstrapping manner. To address the issues caused by class imbalance, the innovative rank loss and dice loss were jointly employed. The approach was employed for identifying skin diseases, with ISIC 2017 [3] and PH2 datasets [12] serving as the basis for the study. The model attained an 80.4% Jaccard index and 89.4% in identifying skin lesions, while exhibiting an average AUC of 93.8% and 97.7% in categorizing skin lesions. The findings propose that by training a unified model to perform both tasks in a mutually reinforcing manner, it is feasible to enhance the effectiveness of skin lesion detection and classification simultaneously.

Zunair et al. [13] utilized the MelaNet approach to categorize two skin diseases utilizing the ISIC 2016 dataset [14] and CAM as an explanation technique. A framework was proposed for identifying melanoma. Initially, inter-class variation of the data was utilized to distribute the task of conditional image synthesis by learning inter-class mapping and synthesizing under-represented class samples from over-represented ones using unpaired image-to-image translation. Secondly, a deep convolutional neural network was trained for skin lesion classification using the original training set along with the newly synthesized under-represented class samples. The newly synthesized samples were used as supplementary data to train a deep convolutional neural network by minimizing the focal loss function, which helps the classification in learning from difficult examples. The model achieved an accuracy of 81.18% and sensitivity of 91.76%.

Brinker et al. [15] utilized the ISIC 2018 dataset [2,16] for the purpose of melanoma classification. They utilized advanced deep learning techniques to train a CNN using 12,378 dermoscopic images that were available as open-source. The CNN's performance was compared to that of 157 dermatologists who were provided with 100 images for evaluation. Outliers were detected using the Local Outlier Factor (LOF) method. The network demonstrated a specificity of 86.5%, while human experts achieved only 60%. The network's sensitivity was also found to be 74.1%, which was comparable to that of medical professionals.

Kasani et al. [17] conducted a study in which they compared different deep learning models for diagnosing melanoma. They utilized ResNet-50 to identify melanoma through the ISIC 2018 datasets [2,16] .The study also tested the latest deep learning architectures for detecting melanoma in dermoscopic images. To enhance the quality of the images, the researchers employed various pre-processing methods, including illumination correction, contrast enhancement, and artifact removal, to improve image quality and achieve better generalization ability. Due to the imbalanced class distributions of skin lesions, the researchers utilized different augmentation approaches, such as horizontal and vertical flipping, random contrast, and random brightness. Data augmentation enabled them to increase the size of the training set and reduce the overfitting problem. According to their research, image pre-processing, data augmentation, and picture preparation techniques significantly enhanced the classification rates. The researchers were able to achieve 93% precision, 92% accuracy, 92% recall, and 92% F1-score.

TABLE I.  Literature summary

Conclusion

Deep learning models and Explainable Artificial Intelligence (XAI) has shown promising results in the classification of skin lesions. Through the analysis of various studies and research papers, it is evident that deep learning models have demonstrated impressive accuracy in detecting and classifying skin lesions, with some models achieving results that are comparable to or even surpassing those of dermatologists. XAI techniques have been used to provide insights into the decision-making process of these models, improving their interpretability and transparency. Overall, the use of deep learning and XAI techniques in skin lesion classification has the potential to improve diagnosis and aid in the early detection of skin cancer and assist dermatologists in their clinical practice. With continued research and development, it is hoped that these techniques can be further optimized to achieve even better results and have a significant impact on public health.

References

[1] Nigar, Natasha, Muhammad Umar, Muhammad Kashif Shahzad, Shahid Islam, and Douhadji Abalo. \"A Deep Learning Approach Based on Explainable Artificial Intelligence for Skin Lesion Classification.\" IEEE Access 10 (2022): 113715-113725. [2] Tschandl P., Rosendahl C. & Kittler H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5, 180161 doi.10.1038/sdata.2018.161 (2018) [3] Noel C. F. Codella, David Gutman, M. Emre Celebi, Brian Helba, Michael A. Marchetti, Stephen W. Dusza, Aadi Kalloo, Konstantinos Liopyris, Nabin Mishra, Harald Kittler, Allan Halpern: “Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)”, 2017; arXiv:1710.05006. [4] Marc Combalia, Noel C. F. Codella, Veronica Rotemberg, Brian Helba, Veronica Vilaplana, Ofer Reiter, Allan C. Halpern, Susana Puig, Josep Malvehy: “BCN20000: Dermoscopic Lesions in the Wild”, 2019; arXiv:1908.02288.` [5] Zia Ur Rehman, Muhammad, Fawad Ahmed, Suliman A. Alsuhibany, Sajjad Shaukat Jamal, Muhammad Zulfiqar Ali, and Jawad Ahmad. \"Classification of skin cancer lesions using explainable deep learning.\" Sensors 22, no. 18 (2022): 6915. [6] ISIC Archive. Available online: https://www.isic-archive.com/ (accessed on 20 May 2022). [7] Chowdhury, Tamal, Angad RS Bajwa, Tapabrata Chakraborti, Jens Rittscher, and Umapada Pal. \"Exploring the correlation between deep learned and clinical features in melanoma detection.\" In Medical Image Understanding and Analysis: 25th Annual Conference, MIUA 2021, Oxford, United Kingdom, July 12–14, 2021, Proceedings 25, pp. 3-17. Springer International Publishing, 2021. [8] Nunnari, Fabrizio, Md Abdul Kadir, and Daniel Sonntag. \"On the overlap between grad-cam saliency maps and explainable visual features in skin cancer images.\" In Machine Learning and Knowledge Extraction: 5th IFIP TC 5, TC 12, WG 8.4, WG 8.9, WG 12.9 International Cross-Domain Conference, CD-MAKE 2021, Virtual Event, August 17–20, 2021, Proceedings 5, pp. 241-253. Springer International Publishing, 2021. [9] Wang, Sutong, Yunqiang Yin, Dujuan Wang, Yanzhang Wang, and Yaochu Jin. \"Interpretability-based multimodal convolutional neural networks for skin lesion diagnosis.\" IEEE Transactions on Cybernetics 52, no. 12 (2021): 12623-12637. [10] Kassem, Mohamed A., Khalid M. Hosny, and Mohamed M. Fouad. \"Skin lesions classification into eight classes for ISIC 2019 using deep convolutional neural network and transfer learning.\" IEEE Access 8 (2020): 114822-114832. [11] Xie, Yutong, Jianpeng Zhang, Yong Xia, and Chunhua Shen. \"A mutual bootstrapping model for automated skin lesion segmentation and classification.\" IEEE transactions on medical imaging 39, no. 7 (2020): 2482-2493. [12] Teresa Mendonça, Pedro M. Ferreira, Jorge Marques, Andre R. S. Marcal, Jorge Rozeira. PH² - A dermoscopic image database for research and benchmarking, 35th International Conference of the IEEE Engineering in Medicine and Biology Society, July 3-7, 2013, Osaka, Japan. [13] Zunair, Hasib, and A. Ben Hamza. \"Melanoma detection using adversarial training and deep transfer learning.\" Physics in Medicine & Biology 65, no. 13 (2020): 135005. [14] Gutman, David; Codella, Noel C. F.; Celebi, Emre; Helba, Brian; Marchetti, Michael; Mishra, Nabin; Halpern, Allan. \"Skin Lesion Analysis toward Melanoma Detection: A Challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC)\". eprint arXiv:1605.01397. 2016. [15] Brinker, Titus J., Achim Hekler, Alexander H. Enk, Joachim Klode, Axel Hauschild, Carola Berking, Bastian Schilling et al. \"Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task.\" European Journal of Cancer 113 (2019): 47-54. [16] Noel Codella, Veronica Rotemberg, Philipp Tschandl, M. Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, Harald Kittler, Allan Halpern: \"Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)\", 2018; https://arxiv.org/abs/1902.03368 [17] Kassani, Sara Hosseinzadeh, and Peyman Hosseinzadeh Kassani. \"A comparative study of deep learning architectures on melanoma detection.\" Tissue and Cell 58 (2019): 76-83. [18] Kaur, R.; Hosseini, H.G.; Sinha, R.; Lindén, M. Melanoma Classification Using a Novel Deep Convolutional Neural Network with Dermoscopic Images. Sensors 2022, 22, 1134. [19] El-Khatib, H.; Popescu, D.; Ichim, L. Deep learning–based methods for automatic diagnosis of skin lesions. Sensors 2020, 20, 1753. [20] Ichim, L.; Popescu, D. Melanoma detection using an objective system based on multiple connected neural networks. IEEE Access 2020, 8, 179189–179202.

Copyright

Copyright © 2023 Ellen Rajathy S, Anusudha K. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.