Generating Report of Bone Fracture and Bleeding using X-ray Images

Abstract

The bone is a major component of the human body. Bone provides the ability to move the body. The bone fractures are common in the human body. The doctors use the X-ray image to diagnose the fractured bone. Therefore, an automated system needs to develop to diagnose the fractured bone The Deep Neural Network (DNN) is widely used for the modelling of the power electronic devices. This study showed that a deep learning model can be trained to detect fractures in radiographs with diagnostic accuracy similar to that of senior subspecialized orthopaedic surgeons. The aim of this study is to perform fracture detection by use of deep-learning on X-ray images to support physicians in the diagnosis of these fractures, particularly in the emergency Services. Hospitals, especially their emergency services, receive a high number of fracture cases. For correct diagnosis and proper treatment of these, images obtained from various medical equipment must be viewed by physicians, along with the patient’s medical records and physical examination. Recent advancement in image processing and deep learning create some hopes in devising more enhanced applications. Therefore, data augmentation techniques have been used to increase the size of the data set.

Introduction

I. INTRODUCTION

The human body consists of many types of bone. Bone fractures are mostly caused by the automobile accident or bad fall. The bone fractured risk is high in aged people due to the weaker bone .The fracture bone heals by giving proper treatment to the patient. The doctor uses x-ray or MRI (Magnetic Resonance Imaging) image to diagnose the fractured bone .The small fracture in the bone becomes difficult to analyse by the doctor. The manual process for the diagnosis of the fractured bone is time consuming and the error probability is also high. Therefore, it is a necessity to develop a computer based system to reduce the time and the error probability for the fracture bone diagnosis .The recent emerging machine learning technologies are widely used in medical imaging as well as in the power electronics fields. In this work, we developed a deep neural network to detect and localize fractures in radiographs .We trained it to accurately emulate the expertise of 18 senior sub-specialized orthopaedic surgeons by having them annotate 135,409 radiographs. Within the scope of the steps used in the detection of bone fractures, along with a complete medical history (including inquiry about the manner of occurrence of the fracture) and physical exam, physicians may require tests used for fractures. Mainly, three different devices are used for these tests, which are X-ray, MRI (Magnetic Resonance Imaging) and CT (Computed Tomography).The most preferred among these devices is the X-ray device, which is also more cost-efficient compared to the other options. X-ray images obtained from STANFORD UNIVERSITY were used for deep-learning based fracture detection in wrist images in this study. To generate a report of bone fracture detection and bleeding using the images of X-Ray .Which can be used easily by the user.

The rest of the paper is organized as follows: Section II provides data description; Section III Algorithm used; Section IV provided performance comparision. Section V concludes the paper.

II. REVIEW OF THE LITERATURE

A. Computer Vs Human: Deep Learning Versus Perceptual Training For The Detection Of Neck Of Femur Fractures

This study extends a previous study that conducted perceptual training in medically-naïve individuals for the detection of NoF fractures on a variety of dataset sizes. The same anteroposterior hip radiograph dataset was used to train two DCNNs (AlexNet and GoogleNet) to detect NoF fractures.deep learning was completed across a variety of dataset sizes (200, 320 and 640 images) with images split into training (80%) and validation (20%). An additional 160 images were used as the final test set. Multiple pre-processing and augmentation techniques were utilized. This study suggests that as impressive as recognizing fractures is for a DCNN, similar learning can be achieved by top-performing medically-naïve humans with less than 1 hour of perceptual training. AlexNet and GoogLeNet DCNNs NoF fracture detection accuracy increased with larger training dataset sizes and mildly with augmentation.  Accuracy increased from 81.9% and 88.1% to 89.4% and 94.4% for AlexNet and GoogLeNet respectively. Similarly, the test accuracy for the perceptual training in top-performing medically-naïve individuals increased from 87.6% to 90.5% when trained on 640 images compared with 200 images. Single detection tasks in radiology are commonly used in DCNN research with their results often used to make broader claims about machine learning being able to perform as well as subspecialty radiologists.

B. Deep Learning in Medical Imaging: General Overview

The artificial neural network (ANN)-a machine learning technique inspired by the human neuronal synapse system-was introduced in the 1950s. However, the ANN was previously limited in its ability to solve actual problems, due to the vanishing gradient and overfitting problems with training of deep architecture, lack of computing power, and primarily the absence of sufficient data to train the computer system. Interest in this concept has lately resurfaced, due to the availability of big data, enhanced computing power with the current graphics processing units, and novel algorithms to train the deep neural network. Recent studies on this technology suggest it’s potentially to perform better than humans in some visual and auditory recognition tasks, which may portend its applications in medicine and healthcare, especially in medical imaging, in the foreseeable future. This review article offers perspectives on the history, development, and applications of deep learning technology, particularly regarding its applications in medical imaging.

C.  Automated Detection and Classification of the Proximal Humerus Fracture by using Deep Learning Algorithm

We aimed to evaluate the ability of artificial intelligence (a deep learning algorithm) to detect and classify proximal humerus fractures using plain anteroposterior shoulder radiographs. Patients and methods - 1,891 images (1 image per person) of normal shoulders (n = 515) and 4 proximal humerus fracture types (greater tuberosity, 346; surgical neck, 514; 3-part, 269; 4-part, 247) classified by 3 specialists were evaluated. The ability of the CNN, as measured by top-1 accuracy, area under receiver operating characteristics curve (AUC), sensitivity/specificity, and Youden index, in comparison with humans (28 general physicians, 11 general orthopaedists, and 19 orthopaedists specialized in the shoulder) to detect and classify proximal humerus fractures was evaluated. The CNN showed a high performance of 96% top-1 accuracy, 1.00 AUC, 0.99/0.97 sensitivity/specificity, and 0.97 Youden index for distinguishing normal shoulders from proximal humerus fractures. In addition, the CNN showed promising results with 65-86% top-1 accuracy, 0.90-0.98 AUC, 0.88/0.83-0.97/0.94 sensitivity/specificity, and 0.71-0.90 Youden index for classifying fracture type. When compared with the human groups, the CNN showed superior performance to that of general physicians and orthopedists, similar performance to orthopedists specialized in the shoulder, and the superior performance of the CNN was more marked in complex 3- and 4-part fractures. Further studies are necessary to determine the feasibility of applying artificial intelligence in the clinic and whether its use could improve care and outcomes compared with current orthopedic assessments.

D. Intervertebral Disc Detection in X-ray Images using faster R-CNN

Automatic identification of specific osseous landmarks on the spinal radiograph can be used to automate calculations for correcting ligament instability and injury, which affect 75% of patients injured in motor vehicle accidents. In this work, we propose to use deep learning based object detection method as the first step towards identifying landmark points in lateral lumbar X-ray images. The significant breakthrough of deep learning technology has made it a prevailing choice for perception based applications, however, the lack of large annotated training dataset has brought challenges to utilizing the technology in medical image processing field. In this work, we propose to fine tune a deep network, Faster-RCNN, a state-of-the-art deep detection network in natural image domain, using small annotated clinical datasets. one can achieve much better performance compared to traditional sliding window detection method on hand crafted features. Furthermore, we fine-tuned the network using 974 training images and tested on 108 images, which achieved average precision of 0.905 with average computation time of 3 second per image, which greatly outperformed traditional methods in terms of accuracy and efficiency.

III. METHODOLOGY USED

The methods which are used in this are:

• PRE-PROCESSING
• DCNN MODEL
• DENSENET

A. X-RAY Image Detection

To detect the image of the X-ray uploaded in the web page, we need to train as many images as possible. The image is being pre-processed using the rescaling and colour gradient methods. Scaling of the data makes it easy for a model to learn and understand the problem and give the accuracy much better. A colour gradient specifies a range of position-dependent colours, usually used to fill a region and used to detect the difference in colour of the X-ray image as bone in white colour. Convolutional neural network used here to detect the bone, image classification which processes the image and make the model to train. Then the model is trained using the images which are classified into Positive and negative to determine whether it is fracture or not at the end. The positive and negative images are classified into Classes of bone that has the sample for bone which is actually fractured or not fractured. By using the same process, the detection of Brain haemorrhage also detected. The training features which are adapted here helps to get accurate result with more and more X-ray images are added and trained, to get a well-trained model.

1. Pre-Processing: Data pre-processing in Machine Learning refers to the technique of preparing (cleaning and organizing) the raw data to make it suitable for a building and training Machine Learning models. The image detection is usually based on the image which is been uploaded and the type of bone which is further divided into classes which needs accurate type of bone and class which is belonged and mainly based on rescaling and colour gradient which separates the colour differentiation on the image need to be detected the colour difference present in the image in which white colour is bone and black colour is empty. The colour detection here only refers to the separation of the black and white colour. Firstly, sequential model is used to run the model as epoch is being increased to get a fine result. And the epoch runs to get a value which is being refereed from the previous epoch.Then, we enhance the data according to the type of bones and its classes and also requires much data for final process to be fine. The amount of labelled data and data enhancement can make up for small data samples. However , the most minute  fracture is so difficult to find ,the data needs to be so large to train a model to even detect a most minute fracture performed a manual secondary check on the data set after data enhancement, which is equivalent to manually identifying the bone fracture report , proving that this will not change the sample. Finally, we use data balancing method to make the training data be balanced relatively. Because the number of times the epoch runs in each class of database is more similar to the last achieved result, there exist the phenomenon of data imbalance influencing the DCNN's training effect.
2. DCNN Model: The DCNN model includes convolution layers, max-pooling layers, and fully connected layers. A Convolutional Neural Network consists of an input and an output layer, as well as multiple hidden layers. The hidden layers of CNN typically consist of a series of convolutional layers that convolve with a multiplication or other dot product. The Activation function is subsequently followed by additional convolutions such as pooling layers, fully connected layers and normalization layers, referred to as hidden layers because their inputs and outputs are masked by the activation function and final convolution. And the activation function we use ReLu. After the last fully connected layer, a dropout layer is followed to minimize the influence of overfitting.

Figure.1 Architecture for DCNN

3. DENSENET: Dense-Net is one of the new discoveries in neural networks for visual object recognition. To improve the declined accuracy caused by the vanishing gradient in high-level neural networks. Due to the longer path between the input layer and the output layer, the information vanishes before reaching its destination.

The layers in the densenet used:-

Rectified linear unit (ReLu) has hidden layers of neural network.

Softmax function is sigmoid function but is handy when we are trying to handle classification problems.

The layers will result in the visual detection of bone fracture and ReLu Softmax function help to get a determined result of the report.

A DenseNet is a type of convolutional neural network that utilises dense connections between layers, through Dense Blocks, where we connect all layers (with matching feature-map sizes) directly with each other. The input is given as pre-processed image which is given to the ReLu function and the network runs epoch after epoch to get an accurate result. H1 is the first network of cycle and H2,H3,H4 and goes on according to the user’s specification.X0 is the input given and X1,X2,X3,X4 are the processed image result to train a model and the transition layer whether the result is achieved.

IV. PERFORMANCE

Model accuracy is the measurement used to determine which model is best at identifying relationships and patterns between variables in a dataset based on the input, or training, data. The graph is plotted using Matplotlib.There are a variety of metrics for scoring whether a model is “good” or “bad” such as R2, percentage accuracy, mean absolute percentage error (MAPE), and many more. Each of these has advantages and disadvantages, but share one common trait – they are designed to compare, not evaluate performance in a vacuum.

The goal of training a model is to find a set of weights and biases that have low loss, on average, across all examples.

Conclusion

In this paper, a deep learning approach is proposed to detect the bone fracture using the X-Ray images. It is simple and accurate in detecting the bone fracture and generating a report. Based on the experimental results, it is shown that the system is working fine and produces desired results such as: • Working GUI for the complete detection. • Accurate detection of the fracture. All these working advantages ensure that the application is widely usable and makes the detection of bone fracture simple and easy to use. From the observations, the model ensures the detection of bones truthful as per the type of fracture with respect to its classes.

References

[1] Olczak et al. 2017 - Wrist/Hand/Ankle - Radiographs - This study supports the use of Orthopaedic radiographs of artificial intelligence, which can perform at a human level. [2] Kim et al. 2018 - Wrist Radiographs - The AUC scores for this test were comparable to state-of- the-art providing proof of concept for transfer learning from CNNs in fracture detection on plain radiographs. [3] Chung et al. 2018 - Proximal Radiographs - The use of artificial intelligence can accurately detect and Detection: humerus classify proximal humerus fractures on plain shoulder AP radiographs. [4] Heimer et al. 2018 - Skull - CT - Classification based on the existence of skull fractures 0.97 (AUC) on CMIPs with deep learning is feasible. [5] Lindsey et al. 2018 - Wrist - Radiographs - Deep learning methods are a mechanism by which senior on Test set1 medical specialists can deliver their expertise to generalists on Test set2 on the front lines of medicine, thereby providing substantial improvements to patient care. [6] Pranata et al. 2019 - Calcaneus - CT - The feasibility using deep CNN and SURF for computer aided classification and detection of the location of calcaneus fractures in CT images.

Copyright

Copyright © 2022 Anand P, Gokul B, Sakthivel B, Vignesh R, Vigneswar S. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.