Pneumonia Based Covid-19 Classification using CNN and LSTM

Abstract

Pneumonia causes inflammation of the air sacs in either one or both lungs. There may be fluid or pus (purulent material) filling the air sacs, causing breathing problems, a fever, chills, and a cough with pus or phlegm. A variety of microorganisms, including bacteria, viruses, and fungi, can cause pneumonia, but a modern virus called COVID-19 has spread over the world and is currently infecting millions of people. Numerous countries are struggling with a shortage of testing supplies, vaccines, and other resources as a result of the large and sudden rise in cases. To expedite the testing process, scientists from all over the world have worked to create new methods for identifying the virus. We present in this project a hybrid deep learning model based on a convolutional neural network (CNN) and gated long short term memory (LSTM) to identify the viral illness from chest X-rays because it is extremely difficult to tell whether a patient who is admitted to the hospital is suffering from pneumonia or COVID-19 because both of them share the same symptom (CXRs). The suggested model employs a CNN to extract features and an LSTM as a classifier. Three batches of 7470 CXR images were used to train the model (COVID-19, Pneumonia, and Normal). The accuracy rate of the proposed model is 97%. These findings show how deep learning may considerably enhance X-ray image processing for patient COVID-19\'s early diagnosis. These indicators may make it possible to lessen the disease\'s impact. This strategy, in our opinion, can better assist doctors in making an early diagnosis.

Introduction

I. INTRODUCTION

The phrase "digital image" describes how a digital computer transforms a two-dimensional image. It denotes digital processing of any two-dimensional data in a larger context. A digital image is made up of an array of real or complex integers that are each represented by a finite number of bits. A transparencies, snap, photograph, or X-ray that has been provided is first digitally converted and stored in computer memory as a matrix of binary numbers. A high-definition television monitor can then be used to process and/or display this digitized image. To provide a visually continuous display, the picture is kept in a rapid-access buffer memory for display, which refreshes at a speed of 25 frames per second on the monitor.

A. The Image Processing System

The many components of digital image processing are combined into an image processing system. The manipulation of a picture using a digital computer is known as "digital image processing." Different computer algorithms are used in digital image processing to conduct image processing on the digital pictures.

It contains the following components:

1. Digitizer:  An image is transformed by a digitizer into a numerical representation that can be entered into a digital computer.
2. Image Processor: An image processor performs the actions of acquiring, storing, pre-processing, segmenting, representing, recognising, and interpreting images before displaying or recording the finished product. The basic steps of an image processing system are depicted in the following block diagram.

As indicated in the illustration, the procedure starts with the gathering of the first image, which is accomplished using an imaging sensor and a digitizer to digitise the image. The following phase is pre-processing, when the picture is enhanced and sent into the other processes as an input. Pre-processing frequently involves improving, eliminating noise, separating areas, etc. Segmentation divides a picture into its individual items or components the end output of fragmentation is frequently raw pixel data, which either contains the region's border or each pixel within it. The process of representation involves converting the raw pixel data into a format that can be used by the computer for further processing. The task of description is to identify key characteristics that distinguish one class of items from another. Based on the details provided by an object's descriptors, recognition gives it a label. An ensemble of identified things must be given meaning in order to be considered as interpreted. The knowledge base includes the information about a certain issue domain. Each processing module is guided in its operation by the knowledge base, which also regulates how the modules communicate with one another. Not all modules are required for a specified functionality. The image recognition system's architecture is determined by the application. The image converter usually operates at a frame rate of 25 frames per second.

3. Image Enhancement: The properties of a picture are improved via image enhancement processes, which may include enhancing the contrasting and brightness properties of the picture, lowering if the noise level or the details' sharpness. Simply said, this improves the picture and makes exactly similar content easier to interpret. It doesn't provide any new details.

4. Image Restoration: Similar to image enhancement, image restoration enhances the quality of the picture, however all procedures are mostly based on known, measurable, or original image degradations. Images with issues including geometric distortion, poor focus, repeated noise, and camera motion can be fixed using image restoration techniques. It is employed to fix defined as image degradations.

5. Analysing of Images: Based on the qualities of the original image, image analysis techniques generate numerical or graphical information. They separate them into items before classifying them. The image statistics affect them. Automated measurements, object categorization, and the extraction and description of scene and picture characteristics are common procedures. The majority of machine vision applications involve image analysis.

6. Image Compression: The amount of data required to describe a picture is reduced by compression and decompression of the image. Compression eliminates all the unnecessary information that is present in the majority of photos. Compression reduces the size, allowing for more effective storage or transportation. When a picture is shown, it is decompressed. While loose compression offers good compression, it does not reflect the original image in any way. In the original image, perfect reduction preserves the precise info.

7. Image Synthesis: Images are produced via image synthesis processes using non-image or other image data. In general, image synthesis techniques produce pictures that are either impossible to capture physically or are not practicable to get.

8. Applications of Digital Image Processing: the use of satellites and other spacecraft for remote sensing, the transmission and storage of images for commercial purposes, the processing of images for use in medicine, the processing of radar, sonar, and acoustic images, robots, and automatic industrial part  are just a few of the many uses for digital image processing.

9. Applications in Medicine: In radiology, nuclear magnetic resonance (NMR), and ultrasonic scanning for medical applications, one is concerned with processing chest X-rays, cineangiograms, transaxial tomography projection images, and other medical images. These photos may be used to screen and monitor patients, as well as to find tumours or other diseases in patients.

10. Communication: Applications for image transmission and storage may be found in military communications, teleconferencing, facsimile image transfer, broadcast television, and computer network communication, and closed-circuit video-based security monitoring systems.

11. Imaging Systems for Radar: Radar and sonar pictures are utilised for aircraft or missile guidance and manoeuvring as well as for the detection and identification of a variety of targets.

12. Document Management: It is used to scan and transmit paper documents in order to turn them into digital images, compress the images, and store them on magnetic tape. It is also used to automatically identify and recognise written properties when reading documents.

13. Defence: It is utilised real sophisticated bombs and missile systems technologies, as well as surveillance picture for automatic translation of geo satellite measurements to find out critical targets or military actions. in target acquisition and guidance for detecting and monitoring targets.

B Advantages of Image Processing

1. Images are processed more quickly and efficiently. Less film, additional photographic supplies, and processing time are required.
2. A digital image may be easily copied, and its quality is maintained until it is compressed. For instance, a picture is compressed when saved in jpg format. Resaving the image in jpg format causes the previously compressed image to be recompressed, degrading the image's quality with each save.
3. Image correction and retouching have gotten simpler. With the new Healing Brush Tool in Photoshop 7, it just takes a few seconds to smooth out facial creases. Why The pricey reproduction is quicker and less expensive than repairing the image using a repro camera.

C. LSTM

An LSTM recurrent unit aims to "forget" extraneous data while "remembering" all of the prior content that the system has currently witnessed. This is accomplished by adding various "gates," or activation function layers, for a variety of reasons. A vector called as the Internal Cell State, which is tracked by each LSTM recurrent unit, theoretically describes the data that the preceding LSTM recurrent unit elected to maintain. Four distinct gates are present in a long short term memory network each serving a particular function as follows: -

1. The Forget Gate (f) establishes how much the prior data should be forgotten.
2. The amount of data to be recorded about the internal status of the cell is determined by the input gate I
3. Most LSTM literature either totally ignores the input modulation gate (g) or believes it is a component of the input gate, despite the fact that the input modulation gate (g) is usually viewed as a component of the input gate. The knowledge that the Input gate will record onto the Internally State Cell is changed by providing it non-linearity and turning it zero-mean. Due to the faster convergence of Zero-mean input, this is done to shorten learning time.
4. The output gate (o) chooses the upcoming concealed state (Hidden state) to be generated an interior cell state that is currently in effect.

D. CNN

In deep learning, convolutional neural networks (CNN/ConvNet) are a family of deep neural networks that are often utilized to analyse image information. Normally, matrix multiplications come to mind when we think of a neural network, but that is not the case with ConvNet. It makes use of a unique method called convolution. In mathematics, the convolution technique combines two features to produce a third functional which it describes how the structure of one is altered by the others.  Artificial neurons are arranged in numerous layers to form convolutional neural networks. Cnns are mathematical operations that, approximately mimicking their biological equivalent, estimate the weighted sum of numerous impulses and emit an activated value. When a picture is received, each layer of a ConvNet generates a series of activation factors that are forwarded on to the subsequent layer. Typically, the first layer extracts fundamental features like edges that run horizontally or diagonally. The following layer receives this output and detects more intricate features like corners or multiple edges. As we delve further into the network, it is able to recognise even more intricate features like objects, faces, etc.

Fig 4: CNN Network

E. Paper Description

The lung gets irritated and inflexible when a lung illness develops gradually, preventing the alveoli from fully expanding. The body's ability to remove carbon dioxide from the body and exchange oxygen into the blood is constrained by this situation.

The interstitial and alveolar walls thicken as the illness worsens, impairing pulmonary functioning. In order to first screen for lung problems, the clinical diagnosis is based on symptoms including wheezing, coughing, irregular temperature, and shortness of breath. Blood tests, pulmonary function tests, pulse oximetry, chest X-rays, chest CTs, and biopsy/surgical biopsy examinations can all be used to diagnose pulmonary diseases correctly and determine the underlying cause of a disease so that it can be treated surgically or with medicine.

F. Objective

Air gaps and potential diseases may show as white and black areas, respectively, in the chest cavity. Through front, posterior, or lateral perspectives, digital chest X-ray pictures may be utilised to immediately detect and determine the class of abnormality. However, there are certain practical limits to pattern identification with manual inspection:

The contents of the chest cavity should be improved in low-quality X-ray pictures. Diagnostic outcomes depend on the judgments and experiences of the clinicians/radiologists.

1. Multi-marker picture classification issues must be solved with a lengthy examination period.
2. In order to screen the ROI for identifying common lung illnesses, an automated computer-aided technique based on a machine vision classifier is presented in this work.

G. Scope of the Work

This work's primary objective is to raise medical proficiency in regions with sparse radiotherapist availability. In such distant locations, our study helps with the early detection of pneumonia to avoid negative effects, including mortality. Pneumonia detection from the aforementioned dataset has not received much attention so far. The creation of algorithms in this area can greatly improve the delivery of healthcare services. Additionally, we demonstrated how hyper-parameter modification during the classification step improved model performance. Through a series of tests, we hope to provide the most effective pre-trained CNN model and classifier for use in related future research. Our research will probably help to improve pneumonia detection methods.

II. LITERATURE SURVEY

P. M. Shah et al [1] The proposed model employs a GRU as a classifier and a CNN to extract features. 3 classes and 424 CXR pictures were used to train the model (COVID-19, Pneumonia, and Normal). In its 200th epoch of use, the suggested model achieves encouraging results of 93%. These results demonstrate how deep learning can significantly aid in the analysis of X-rays in the early detection of COVID-19 in patients. Such signs can open the door to reducing the disease's effects. We think that this model can help doctors make an early diagnosis more successfully.

S. Rajaraman et al [2] The most common radiographic test for illness diagnosis is a chest X-ray (CXR) study. In contrast to adult pneumonia, paediatric pneumonia has received less research. In order to enhance illness diagnosis and support decision-making while simultaneously closing the gap in efficient radiological interpretations during mobile field screening, computer-aided diagnostic (CADx) tools were developed. For visual recognition, these programmes employ manually created or convolutional neural networks (CNN) extracted picture characteristics.

However, because they function inexplicably and are not well understood, CNNs are thought of as "black boxes. Tools for visual interpretation and explanation of model predictions are suggested. In this work, they emphasise the benefits of illustrating and describing how CNN activations and predictions are applied to the problem of detecting pneumonia in paediatric chest radiographs.

L. Qu el.al [3] Due to their outstanding 3D context mining capabilities,  The use of  3D convolutional neural networks has increased popularity in volumetric image analysis. The number of trainable parameters will significantly rise because of the 3D convolutional kernels, though. Given that training data are frequently scarce in biomedical problems, a trade-off between model size and representational capability must be made. In order to allay this worry, we suggest a brand-new 3D Dense Separated Convolution (3D-DSC) module to take the place of the original 3D convolutional kernels in this study. The representational capability of the network may be greatly increased while preserving a compact topology by adding more nonlinear layers and dense connections between 1D filters. They show that 3D-DSC is superior at classifying and segmenting volumetric medical images, two difficult tasks that frequently arise in the field of biomedical image computing.

M. F. Hashmi et.al[4] The accuracy of diagnosis has to be increased. The radiologists' decision-making process may be aided by the effective model for the identification of pneumonia that is developed in this study and trained on digital chest X-ray pictures. This article introduces a unique method based on a weighted classifier that optimally integrates the weighted predictions from cutting-edge deep learning models including ResNet18, Exception, InceptionV3, DenseNet121, and MobileNetV3. This method is a form of supervised learning in which the network forecasts the outcome based on the calibre of the training dataset. To improve the deep learning models' training and validation accuracy, transfer learning is applied. Techniques for partial data augmentation are used to evenly expand the training dataset. All of the individual models can be outperformed by the suggested weighted classifier.

J.Son et.al[5] This work suggests a deep learning framework that generates both a categorization outcome identifying the occurrence of a targeting finding and a heatmap that displays abnormalities. 24 The categorization result's output, which varies from 0 to 1, correlates to the finding's projected likelihood of existence. A low-resolution, single-channel image of a lesions has normalization colour levels around 0 and 1. When this low-resolution image is superimposed onto the original fundus image after being enlarged, lesions that the algorithms identify as favourable for the target finding are highlighted with high-intensity values.

Lai CC et.al[6] The development a massive international recurrence of the 2019 novel coronavirus (2019-nCoV) infection, also termed as COVID-19, or severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), in China during its year of 2019. More than 43,000 confirmed cases have been found as of February 11, 2020, according to statistics from the WHO, with more than 99% of the cases being found in China. They have been found in 28 different nations and regions. The illness has a 6.4 days on average for the incubation time, and 2.24 to 3.58 for the basic reproduction number. It is transmitted from person to person via droplets or direct touch. The most common symptom in patients with new coronavirus pneumonia was temperature, followed by congestion. (also known as Wuhan pneumonia) brought on by the SARS-CoV-2 virus. The most frequent observation from Computable tomography pictures of the chest was participation of both lungs in addition to ground glass opacity. The only instance of SARS-CoV-2 pneumonia in the United States was successfully treated with high - risk category, which is presently undertaking a diagnostic test in China.

Zheng Wang et.al[7] An immediate computer-aided diagnostic (CAD) technique using deep learning (DL) is required to automatically discover and discriminate COVID-19 versus community-acquired pneumonia (CAP) using chest X-rays.. In order to help radiologists automatically identify and localise COVID-19, this work intends to create and evaluate a reliable deep learning strategy. A retroactive chest X-ray image dataset was produced using open image data and the Xiangya Healthcare, then it was divided into a training environment and a studies support the hypothesis. The Discrimination-DL and the Localization-DL are the two phases with DLs that make up the proposed CAD system. Using 3548 chest X-ray radiographs as training datathe very first DL was developed to separate COVID-19 lung features using chest X-ray images. The subsequent DL, generated with 406-pixel patches, was used to localization and allocate the left lung, right lung, or bipulmonary to the recognised X-ray images. For the purpose of assessing the model's sturdiness, CAP and normal counterparts' X-ray radiography have been included. When comparable to the radiologists' results, the efficiency of COVID-19 differentiation using the Discriminatory practices was 98.71%, while the efficiency of specificity using the Localization-DL was 93.03%.

L.Goyal et.al[8] The arrangement that can aid in the assessment of COVID-19 with routine chest X-rays using deep learning techniques is the main topic of this study. The main strategy is to compile every COVID-19 picture that might possibly exist and then use a convolutional neural network (CNN) to create more photos. This will aid in the most accurate virus detection from acceptable X-ray images. The dataset was generated using 748 photos for three distinct sorts of classes. The COVID-19, typical, and bacterial pneumonia are the classifications. In this study, three deep transfer models are chosen for analysis. The VGG19, VGG16, and Restnet50 are the models.

M. Awais et.al[9] This research suggests an integrated IoT architecture that permits wireless transmission Transferring physiological signals to a data processing facility for Long Short-Term Memory (LSTM)-based emotion detection. Real-time collaboration is made possible by the suggested architecture, and emotion identification to enhance remote learning and health monitoring during pandemics. The findings of this investigation are highly encouraging. proposed IoT protocols (R-MAC and TS-MAC) are able to achieve a very brief lag of one millisecond. In addition, R-MAC delivers more dependability when compared to cutting-edge. The proposed deep learning framework also provides great performance, with an f-score of 95%.

Serener et.al[10] This study uses numerous deep learning techniques on computed tomography (CT) images to identify patients with mycoplasma pneumonia, normal viral pneumonia, or COVID-19, therefore reducing the diagnostic challenges. ResNet-18 and MobileNet-v2 designs appear to work well for differentiating various disorders, according to this paper's findings. They also conducted experiments and performed analyses to differentiate between mycoplasma pneumonia and common viral pneumonia. They investigated the performance of these discernments' detection using seven various deep learning architectures.The findings indicate that ResNet-18 and MobileNet-v2 designs are suitable for use in separating mycoplasma pneumonia from common viral pneumonia and COVID-19.

N. Hilmizen et.al[11] In this study, we propose to classify X-ray and CT scan images are divided into two categories: normal and COVID-19 Pneumonia. We will do this by concatenating two distinct utilising open-source models for transfer learning dataset containing 2500 photos from CT scans and 2500 images from X-rays. In this research, we have employed the models for image recognition MobileNet, Xception, InceptionV3, ResNet50, and VGG16 along with DenseNet121 . As a consequence, we combine the networks ResNet50 and VGG16 to improve prediction performance that is 99.87% the best. A single modality of CT-Scan ResNet50 networks helped us obtain the best classification accuracy of 98.00%, and X-Ray VGG16 networks helped us get the greatest classification accuracy of 98.93%.

M. Mishra et.al[12] This paper proposes In order to create a tool for categorisation to identify Infection with COVID-19 and other lung diseases, this work integrates artificial intelligence (AI) with medical science. pneumonia, pneumonia without the presence of the Covid-19 virus, and Pneumonia with Covid 19 virus presence and healthy lungs were the four conditions that were examined. There are two steps to the suggested AI system. X-ray chest maxima are divided into pneumonia and non-pneumonia in level 1. Stage 2 receives information from Stage 1 on whether an X-ray falls into the pneumonic category and moving forward categorizes it as Covid-19 negative or Covid-19 positive.

Y. Wang et.al[13] In this work, patterns connected to B lines and PL lines (pleural line) are characterised in order to objectively evaluate the LUS pictures and determine how serious COVID-19 pneumonia is (BLs). There are 27 COVID-19 pneumonia patients enrolled, of which 13 have moderate cases, 7 have severe cases, and 7 have critical cases.. They employ using all of the attributes as input into the SVM classifier, the classification performance is at its best (Receiver operating characteristic (ROC) curve area under the curve is 0.96, sensitivity is 0.93, and specificity is 1. As a result of these findings, the recommended method may prove to be a valuable tool for automatically grading the diagnosis and follow-up of COVID-19 pneumonia patients.

X. Wang et al[14] In this study, as the first step in that approach, we offer DeepSC-COVID, a combined 3D lesion segmentation and categorization using a deep learning model. To be more specific, we create We provide 4 findings about the lesion differences between COVID-19 and community-acquired pneumonia using a large-scale CT database with 1,805 3D CT images and fine-grained lesion annotations (CAP). Our research served as the foundation for the construction of DeepSC-COVID, which consists consisting of three subnets: a classification subnet for illness diagnosis, a 3D lesion subnet for segmentation and classification, and a merge feature subnet for edge detection. They may analyse their COVID-19 diagnostic model using a fine-grained 3D lesion arrangement, which sets it apart from all other models currently in use.

M. Sevi et.al[15]The early diagnosis of diseases is crucial to halting the pandemic. The transmission of the virus is more tightly controlled the greater the success. The PCR test is often used to determine a person's viral status. Deep learning (DL)  techniques can be applied to categorise chest x-ray pictures as well as the PCR method. By analysing multiple-levelled pictures all at once and setting Deep learning approaches have become more common in scholarly research in machine learning, replacing manually supplied parameters. The few health databases benefited from this popularity. The purpose of this study was to identify patients with illness whose x-rays were done for suspected COVID-19. Such COVID-19 research has often used a binary categorization. Chest x-rays taken from COVID-19 patients, those with viral pneumonia, and healthy individuals are used to collect data. Before classification, the data set was treated to the data augmentation technique. Deep learning methods for multi-class classification have been used to classify these three groupings.

M. Yamaç et.al[16] The following is a summary of the study's primary hypotheses: 1) The creation of a benchmark X-ray data set called QaTa-Cov19 with more than 6200 radiographs.

The data collection contains 462 COVID-19 X-ray images. patients  images from 3 additional classes: pneumonia caused by bacteria, pneumonia caused by viruses, and normal. 2Whenever the averaged five-fold cross validation accuracy for the QaTa-Cov19 collected data is computed, CheXNet, a state-of-the-art deep NN technology for X-ray pictures, is used to extract features for the recommended CSEN-based classification system, which achieves under 98% responsiveness but over 95% sensitivity for COVID-19 detection straight from raw X-ray images. 3) The results of this investigation show that COVID-19 produces a distinctive pattern in X-rays that can be distinguished with great precision. due to its elegant COVID-19 helpful diagnosis performance.

K.S et.al[17] In the planned study, individuals with COVID-19 and those with pneumonia will be identified using X-rays, one of the medical imaging techniques used to evaluate the patient's inflammatory lung disease. For the discovered dataset, the appropriate convolutional neural network model is chosen. On the actual dataset of X-ray pictures of the lungs, the model can identify COVID-19 patients and patients with pneumonia. The outcome shows that COVID and COVID vs. Pneumonia were accurately detected. The accuracy of COVID versus Normal is higher than COVID vs Pneumonia among the two. This approach is capable of detecting COVID, Pneumonia, and its subtypes, such as viral or bacterial pneumonia, respectively, with 80% and 91.46% accuracy, respectively. The suggested model's ability to detect COVID, bacterial pneumonia, and viral pneumonia aids in quick diagnosis and helps to differentiate COVID from different forms of pneumonia, making it easier to apply the right treatments.

H. N. Monday etc. Al[18] As a result of the rapid development of COVID-19, their suggested method is a four-level deconstruction discrete fourier transforms embedded convolutional neural networks robust to accept minimal dataset.  Based on 3 types of publicly available datasets images from chest x-ray and chest tomography, we evaluated our model. With fewer training parameters, our suggested model achieves 98.5% accuracy, 99.8% sensitivity, 98.2% specificity, and 99.6% AUC for multiple class categories. The findings of this investigation demonstrate that our technique produces outcomes that are cutting edge.

X. Ouyang et al[19] According to their best knowledge, their technique is assessed using the most comprehensive COVID-19 multi-center CT data from 8 universities. During the training-validation step, 2186 CT scans from 1588 patients were used for a 5-fold cross-validation.

We use a different independent, sizable testing dataset that consists of 2796 CT images from 2057 individuals during the testing phase. With an area under the receiver operating characteristic curve (AUC) value of 0.944, accuracy of 87.5%, sensitivity of 86.9%, specificity of 90.1%, and F1-score of 82.0%, the results demonstrate that our system can recognise COVID-19 pictuThe findings show that our system can recognise objects with an F1-score of 82.0%, efficiency of 87.5%, sensibility of 86.9%, specific of 90.1%, and area under the receiver operating characteristic curve (AUC) value of 0.944. the COVID-19 images. With this performance, the suggested algorithm may be able to assist radiologists in the early stages of the COVID-19 epidemic with CAP-based COVID-19 diagnosis.

V. K. Gupta et.al[20] An uncommon viral pneumonia in patients SARS-CoV-2, a novel coronavirus, was initially discovered in late December 2019. World Health Organization later proclaimed it to be a pandemic due to its catastrophic impacts on public health. The COVID-19 epidemic is now affecting an exponentially growing number of people worldwide.

Here, they are solely looking for verified, fatal, and cured COVID-19 cases in India. They are doing this study based on the cases that occurred in the various Indian states in chronological order. We are doing multi-class classification on our dataset since it has several classes.

A. Issues in  Already Existing System

The datasets we currently utilise for pneumonia research contain recurrence events for two classifications as well as no recurrence events.

Data preprocessing followed by decision stump classification, which will serve as the GRU algorithm's basic classifier. The number of iterations should be set to 2, and the weight threshold for weight pruning should be set to 10. The GRU method, which employs the base classifier reweighting, is utilised. The weight threshold for weight pruning is set at 100, and the number of iterations is set at 10. When comparing instances that were correctly categorised and classification accuracy, decision stump implementation by GRU and CNN increases accuracy.

Issues in Existing System

1. Number of images getting uploaded is only 424.
2. The overall accuracy at the 200th epoch is only 93 %
3. Since the GRU is used the accuracy will be less.

B. Proposed System

The method for the suggested pneumonia detection system can be carried out in two parts. The first stage will employ image pre-processing methods to improve design performance, such as resizing and histogram equalisation. Use this to increase accuracy by excluding outliers that could have an impact on the result. In the following step, lung segmentation will be used to identify the area of interest. The classification algorithm will ultimately be used to determine whether pneumonia is present or absent. VGG16 will be used as the baseline algorithm; it is based on CNN and can be further modified to improve accuracy; it will be used as the algorithm for the detection and classification of pneumonia.

III. METHODOLOGY

A.  Pre-Processing

Preprocessing is the First step in our project which plays a major role in initializing the raw data. In our Project the Pre-Processing step takes place as the following. The pipeline was first used to pre-process the X-ray pictures. Data resizing, shuffling, and normalisation were done in the pre-processing pipeline. The system was then given the output images to begin feature extraction [1]. Information from the patients' chest X-rays used to distinguish COVID-19 status. Chest x-rays of the patients in this paper were categorised as Normal pneumonia, viral pneumonia, and COVID-19 [15].The CXRs contain areas outside of the lungs that are unimportant for the detection of pneumonia. Our lung boundary detection algorithm, which is atlas-based, finds the borders of the lungs. The segmented images are then cropped to fill the lung pixel-containing bounding box [2].The Pre-processing in our project is getting the dataset(Indiana dataset) from Tensor flow keras. In this process we will divide the data into three part namely Train data generator, Test data Generator and Valid data Generator using Image data Generator.

B. Features  Extraction

Each deep learning extraction of features model may easily be incorporated into our model, however our customised model with defined layers learned the best-fit features linked to COVID19 and pneumonia. [1].

Additionally, we tested adding and removing convolutional layers. However, doing so impaired the validity of our findings. The feature extraction method is depicted in the figure. Each picture (In this case, CXR) is sent through the convolutional layers. The convolutional layer divides the picture into n different dimensions in order to produce feature maps.. In Feature Extraction we give a set of Batches that is nothing but we will divide the whole set of images into a standard batch size in order to extract the features. In our project the batch size is 8 with this we need to assign the number of classes. The Classes will tell how many types of feature extraction we want in the end. As in our case we will have 3 classes namely Covid-19, Normal and Viral Pneumonia.

C. Testing

In our Project the data set is further separated into 3 sets, with a ratio of 70%, 10%, and 20%, respectively, for training, validation and  testing[1]. In the testing part we will have the target size of 256 (as we divided the data into batches so we need to give the target for testing also in batches). In testing I will give my batch size as 10. At the end we will check whether many images are present in these batches. At the end we will check which image falls into which class.

D. Training

The Training Blocks play an major role in all Machine learning(ML) or deep learning(DL) Projects. It is similar in our Project also. 70% of data from the dataset is given for training purposes. In the training process the algorithm is developed to practice or get trained with the respective features of the project as per our requirement. It is very important to train the algorithm with a large amount of information to grab the logic and working method. By doing this we can get a good accuracy level in our project. In our project at the training part we divide the data into 256 batches with the epoch level of 10.

E. Classification of LSTM and CNN

1. LSTM: A unique class of deep neural networks known as the LSTM network has the capacity to memorise long-term relationships found in time-series datasets. The addition of various memory cells and gating techniques inside an LSTM network enhances these capabilities. After executing a number of gating operations, the memory cells are updated, which in turn updates which values to remember and which to discard in the temporal sequence. As a result, it is well suited to accurately and robustly simulate temporal dynamics. Input gate, output gate, and forget gate are the three types of gating operations in LSTM[9]. The Project LSTM networks are excellent for categorising, analysing, and mining data since there may be delays of varying lengths between critical events in a time series, and making predictions based on time series data. Our approach, the CNN LSTM model, was created primarily for challenges involving sequence prediction using spatial inputs, such as images or videos. In this design, feature extraction from input data is done using Convolutional Neural Network (CNN) layers, and sequence prediction is done using LSTMs on the feature vectors. In essence, CNN LSTMs are a subset of deep models that straddle the fields of computer vision and natural language processing. They are both geographically and temporally deep. These models have a tonne of potential and are being utilised more frequently for a variety of challenging tasks, including text categorization, video conversion, and other similar ones. Here is a general CNN LSTM model architecture.
2. CNN: In our project the Convolutional neural networks (CNN or ConvNet) are a subclass of neural networks that are mostly employed in voice and image recognition applications. With no loss of information, its integrated convolutional layer lowers the high dimensionality of images. CNNs are therefore very well suited for this use case. The convolutional layer converts the picture into multiple dimensions (n is the number of designated channels). to create feature maps. Three channels of 256x256 input pictures (X-rays) are provided to the first convolutional block (256x256x3). The 256x256 symbol stands for the image's height and width, while the 3 symbol stands for the colours red, green, and blue (RGB). This block creates 64 feature maps in 112x112 dimensions, which are then further shrunk to 112x112x64 by the layer max-pooling. Similar to Using an input of 112x112 of size 64, the second convolutional block creates features maps of 11x112x128 that are then further condensed by a second max-pooling layer of 56x56x128.These feature maps also go through the third, fifth, and sixth convolutional blocks.

F. Evaluation Metrics

The attained validation and training accuracy are 98% and 97%, respectively, on the 200th epoch. The training and validation losses for the 200th epoch are 0.8 and 0.9, respectively.

TABLE 1: Precision, Recall, and F1-score outcomes of the proposed CNN-LSTM for individual Normal, Covid-19, and Pneumonia

The overall accuracy at the 200th epoch is 98%.

Conclusion

The most important requirement for correctly diagnosing any type of thoracic disease is the presence of experienced radiologists. This project\'s main goal is to raise medical proficiency in regions with sparse radiotherapist availability. In such remote locations, our study helps with the early detection of pneumonia to avoid negative effects, including mortality. Pneumonia detection from the aforementioned dataset has not received much attention to date. The creation of algorithms in this area can greatly improve the delivery of healthcare services. Additionally, we demonstrated how hyper-parameter modification during the classification step improved model performance. Through a series of tests, we hope to provide the most effective pre-trained CNN model and classifier for use in related future research. In the near future, improved algorithms for detecting pneumonia are likely to be developed as a result of our study.

References

[1] P. M. Shah et al., \"Deep GRU-CNN Model for COVID-19 Detection from Chest X-Rays Data,\" in IEEE Access, vol. 10, pp. 35094-35105, 2022. [2] S. Rajaraman, S. Candemir, G. Thoma, and S. Antani, ‘‘Visualizing and explaining deep learning predictions for pneumonia detection in paediatric chest radiographs,’’ in Medical Imaging 2019: Computer-Aided Diagnosis, vol. 10950. International Society for Optics and Photonics, 2019, p. 109500S [3] L. Qu, C. Wu, and L. Zou, ‘‘3D dense separated convolution module for volumetric medical image analysis,’’ Appl. Sci., vol. 10, no. 2, p. 485, Jan. 2020. [4] M. F. Hashmi, S. Katiyar, A. G. Keskar, N. D. Bokde, and Z. W. Geem, ‘‘Ef?cient pneumonia detection in chest x-ray images using deep transfer learning,’’ Diagnostics, vol. 10, no. 6, p. 417, Jun. 2020. [5] J. Son, J. Y. Shin, H. D. Kim, K.-H. Jung, K. H. Park, and S. J. Park, ‘‘Development and validation of deep learning models for screening multiple normal ?ndings in retinal fundus images,’’Ophthalmology,vol.127, no. 1, pp. 85–94, Jan. 2020. [6] Lai CC, Shi TP, Ko WC, Tang HJ, Hsueh PR. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and coronavirus disease-2019 (COVID-19): The epidemic and the challenges. Int J Antimicrob Agents. 2020:105924. [7] Z. Wang, Y. Xiao, Y. Li, J. Zhang, F. Lu, M. Hou, and X. Liu, ‘‘Automatically discriminating and localizing COVID-19 from community-acquired pneumonia on chest X-rays,’’ Pattern Recognit., vol. 110, Feb. 2021, Art. no. 107613. [8] L. Goyal and N. Arora, ‘‘Deep transfer learning approach for detection of COVID-19 from chest X-ray images,’’ Int. J. Comput. Appl., vol. 975, p. 8887. [9] M. Awais, M. Raza, N. Singh, K. Bashir, U. Manzoor, S. U. Islam, and J. J. P. C. Rodrigues, ‘‘LSTM based emotion detection using physiological signals: IoT framework for healthcare and distance learning in COVID-19,’’ IEEE Internet Things J., early access, Dec. 10, 2020. [10] A. Serener and S. Serte, \"Deep learning for mycoplasma pneumonia discrimination from pneumonias like COVID-19,\" 2020 4th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), 2020, pp. 1-5 [11] N. Hilmizen, A. Bustamam and D. Sarwinda, \"The Multimodal Deep Learning for Diagnosing COVID-19 Pneumonia from Chest CT-Scan and X-Ray Images,\" 2020 3rd International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), 2020, pp. 26-31. [12] M. Mishra, V. Parashar and R. Shimpi, \"Development and evaluation of an AI System for early detection of Covid-19 pneumonia using X-ray (Student Consortium),\" 2020 IEEE Sixth International Conference on Multimedia Big Data (BigMM), 2020, pp. 292-296. [13] Y. Wang, Y. Zhang, Q. He, H. Liao and J. Luo, \"Quantitative Analysis of Pleural Line and B-Lines in Lung Ultrasound Images for Severity Assessment of COVID-19 Pneumonia,\" in IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, vol. 69, no. 1, pp. 73-83, Jan. 2022. [14] X. Wang et al., \"Joint Learning of 3D Lesion Segmentation and Classification for Explainable COVID-19 Diagnosis,\" in IEEE Transactions on Medical Imaging, vol. 40, no. 9, pp. 2463-2476, Sept. 2021. [15] M. Sevi and ?. AYDIN, \"COVID-19 Detection Using Deep Learning Methods,\" 2020 International Conference on Data Analytics for Business and Industry: Way Towards a Sustainable Economy (ICDABI), 2020, pp. 1-6. [16] M. Yamaç, M. Ahishali, A. Degerli, S. Kiranyaz, M. E. H. Chowdhury and M. Gabbouj, \"Convolutional Sparse Support Estimator-Based COVID-19 Recognition From X-Ray Images,\" in IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 5, pp. 1810-1820, May 2021. [17] K. S. and D. Radha, \"Analysis of COVID-19 and Pneumonia Detection in Chest X-Ray Images using Deep Learning,\" 2021 International Conference on Communication, Control and Information Sciences (ICCISc), 2021, pp. 1-6. [18] H. N. Monday et al., \"Improved Convolutional Neural Multi-Resolution Wavelet Network for COVID-19 Pneumonia Classification,\" 2021 4th International Conference on Pattern Recognition and Artificial Intelligence (PRAI), 2021, pp. 267-273. [19] X. Ouyang et al., \"Dual-Sampling Attention Network for Diagnosis of COVID-19 From Community Acquired Pneumonia,\" in IEEE Transactions on Medical Imaging, vol. 39, no. 8, pp. 2595-2605, Aug. 2020. [20] V. K. Gupta, A. Gupta, D. Kumar and A. Sardana, \"Prediction of COVID-19 confirmed, death, and cured cases in India using random forest model,\" in Big Data Mining and Analytics, vol. 4, no. 2, pp. 116-123, June 2021.

Copyright

Copyright © 2023 Varshni S. V, D Kavitha, B Padmavathi. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.