Fusion of CT and MRI Medical Images Using Dual Tree Complex Wavelet Transform

Abstract

Combining various medical pictures will improve illness diagnosis accuracy and illustrate the complex link between them for medical study. Existing approaches take a long time and require a large number of data to train the models. In this model, we will use multi-stage fusion networks to extract sophisticated information from medical pictures such as Magnetic Resonance Imaging (MRI) and Computed Tomography (CT). In the suggested approach, we will extract the difficult and correlated information from each picture using the Dual Tree Complex Wavelet Transform (DTCWT), and then segment the fused image to obtain the segmented image. The proposed approach entails, the fusion of multimodal medical pictures may be accomplished using the Dual Tree Complex Wavelet Transform, which converts the original medical image to grayscale and decomposes it before extracting the wavelet coefficients using DTCWT. Following that, wavelet approximation is utilized to produce the fused coefficients. To obtain a final fused picture, the Inverse Dual Tree Complicated Wavelet Transform is performed. Additionally, segmentation is carried out in order to provide a segmented image for better visual representation. The quality of the final fused picture can be increased using the proposed strategy.

Introduction

I. INTRODUCTION

Medical images like X-rays, Positron Emission Tomography (PET), Single Photon Emission Computed Tomography (SPECT), Magnetic Resonance Imaging (MRI), and Computed Tomography (CT) will not provide as much detail for medical research and diagnosis as detailed information. They might provide a significant amount of detailed clinical data. Medical imaging systems will provide different medical information regarding tissues that are complicated in most cases. For example, X-rays are used for identify bone injuries and bone fractures, CT images will give the elaborated information of internal organs, tumors and blood vessels.

MRI is used to provide information about tissues. Whereas the SPECT will show how blood flows to tissues and organs, and PET helps to reveal how the tissues and organs are functioning. As the expansion of clinical usage of diverse medical imaging systems, the merging of multi- modality pictures plays a significant role in the medical imaging sector. Various medical imaging modalities can produce scans with both complementary and redundant information. The integration of medical pictures can provide clinical information that was not visible in the individual scans.

The project’s main goal is to create an efficient fusion approach that uses the Dual Tree Complex Wavelet Transform to fuse single or multi-modal pictures. In a single image, all important information from several source photographs might be gathered. The Discrete Wavelet Transform (DWT) is the most often used approach for picture fusion. The resulting image will comprise spectral and directional information; furthermore, the orientation data will include more precise information on horizontal, vertical, and diagonal directions.

Some features of the discrete wavelet transform are perfect reconstruction, no redundancy, very low computation but it shows severe shift dependence and directionality. Because to tiny movements in the source picture, the shift variance will cause errors in fused images, and due to poor polarity, this original picture would be hard to evaluate geometric elements such as contours and edges.

To tackle this, the suggested technique would employ Dual Tree Complex Wavelet Transform to merge various medical pictures using DTCWT. DTCWT will address the drawbacks of DWT. The DTCWT will give greater directionality and shift variance, making it easier to analyze the source image’s edges and contours. Its higher shift variance and directionality attributes result in a fully functional picture fusion tool.

II. RELATED WORK

                                                                                         Table 1. Similar Works

III. EXISTING METHOD

    Figure 1: Fundamental phases for a basic pixel level Image  Fusion strategy [7].

Figure 1 shows the key phases of the general pixel-level image fusion approaches. They are as follows: 1) A mathematical transform is performed to the input images to obtain the transform coefficients 2) The fusion process is employed to create the fused coefficients, and 3) an inverse transform is performed to the fused coefficients to create the final fused image.

Imaging techniques such as Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) in the field of medical imaging have supplied doctors with information on the structural characteristics, soft tissue, and so on of the human body. Distinct imaging technologies retain different properties, and different sensors receive different imaging information from the same component. Traditional medical image fusion methods are classified into two categories: spatial domain and transform domain. Discrete Wavelet Transform was used to merge multimodal medical pictures such as computed tomography (CT) and magnetic resonance imaging (MRI). The advantages of transparent bone are combined by combining MRI and CT. To compensate for the absence of data in a single imaging, information in CT pictures and clear soft tissue in MRI images are used. A guided filtering-based MRI and CT fusion technique (GF) is suggested. The fused picture not just preserves the edge information of the source image but also extracts feature information, resolving the edge degree and clarity issues. The visual inspection of fusion findings shows that brightness and correlation have improved noticeably. The discrete wavelet transform may generate diverse input frequency signals while keeping stable output appropriate location in the temporal and frequency domains, which aids in the preservation of picture particular information. The wavelet transform overcomes the restrictions of principal component analysis and provides an effective visual and numerical fusion effect. The source picture is improved and preprocessed, and the intensity component of the CT image is extracted using the IHS transform, which maintains more biological information and decreases color distortion. To get high- and low-frequency subbands, the DWT transform is applied to the intensity components of MRI and CT. The high and low- frequency subbands are fused using distinct fusion criteria, and the fused picture is obtained using the inverse DWT transform. Figure shows a block schematic of a 1-step 2-D DWT. To fuse the decomposing high-frequency coefficients, the absolute high-value approach is utilized, the weighted average method is employed, the predator-optimizer is used to calculate and optimize the weights, and lastly, the inverted transform is used to generate the fused pictures. Fusion is accomplished by the application of two fusion rules. Because high and low frequency coefficients have various meanings, they were fused using different rules. The first set of principles is that bigger wavelet coefficients indicate important picture characteristics such as corners and edges, hence larger wavelet coefficients are preferred. The wavelet factor is the most often used method for fusing features since larger values indicate stronger edges and are considered as a significant component of relevant information. Low wavelet coefficient values represent source image approximation, hence averaging is employed to obtain information about both source pictures. An Inverse Discrete Wavelet Transform is used to get the fused picture after approximating the derived wavelet coefficients. The CT image of a patient brain diseased by Sarcoma is shown in figure 2 and the MRI image of the same patient is shown in figure 3. These two multimodal images were fused by applying the Discrete Wavelet Transform algorithm. After the extraction of the wavelet coefficients and further fusion rule is applied to fuse the coefficients.

After fusing the extracted wavelet coefficients using the derived fusion rules, the Inverse Discrete Wavelet Transform is applied to obtain the final fused image. The final fused image is shown in figure 4. The problems related with DWT are, it does not provide sufficient directional information and results in an image with shift variance and additive noise. It also does not preserve time and frequency information and hence it is not as efficient to represent the enhanced visual representation of the fused image.

IV. PROPOSED METHOD

Machine learning applied to radiological imaging is a  growing study area that is predicted to expand in coming years. Machine learning breakthroughs have the ability to  recognize and classify complicated patterns from medical  images. Machine learning-based systems have demonstrated  similar results to human decision-making in a variety of  situations. Machine learning applications are essential  components of future clinical decision-making and  monitoring systems. The basic concept beside machine  learning approaches and their implementations in several  radiological imaging areas, including medical image  segmentation, brain function studies, and neurological  disease diagnosis, as well as computer-aided devices, image  restoration, and content-based image retrieval systems, are  covered in this review. We hope that by providing insight into how to use machine learning-powered applications,  clinicians will be able to better prevent and detect diseases.

A great picture fusion technique should incorporate  complementary information from the source photos while  ignoring undesired and unexpected aspects. Image fusion  may be performed at three different levels: pixel,  characteristic, and decision. Following that is the features  level image fusion, also known as the intermediate level  image fusion. This technique may represent and evaluate multi-sensor data  for classification. This approach is an important theoretical tool for image classification tasks. The ultimate level of  picture fusion is the decision level, which is the act of  merging information at a higher level as well as combining  the findings of several algorithms to obtain a final fused  decision. Individual photos are processed for information  extraction in this step. MRI, or Magnetic Resonance  Imaging, offers information on the tissue anatomy of the  brain but does not provide functional information. Because  the density of protons is high in the neurological system, fat,  surrounding tissues, and articular cartilage lesions, the 17  picture is extremely clear and free of artifacts. This has high  precision and minimal radiation harm to a human body, and  the benefit of vast amounts of information makes it a  valuable tool in clinical diagnostics.

Because the density of  protons in bone is quite low, the MRI bone picture is blurry.  Individual photos are processed for information extraction in  this step. MRI, or Magnetic Resonance Imaging, provides  information on the tissue anatomy of the brain but does not  provide functional information. So because the density of  protons is high in the neurological system, fat, surrounding tissues, and articular cartilage lesions, the 17 image is  particularly clear and free of artifacts. This has high precision  and negligible radiation harm to a human body, and the benefit of  abundant information makes it a helpful resource in clinical  diagnostics. Because the density of protons in bone is quite low,  the MRI bone picture is blurry. The CT picture is referred to as  Computed Tomography imaging. Because bone tissue absorbed  more than soft tissue, the bone tissue in the CT picture is  extremely apparent. CT scans reveal less cartilage information,  which provides anatomical data. Single-Photon Emission  Computed Tomography (SPECT) is a functional picture that  shows the metabolism of tissues or organs as well as the blood  flow of arteries and veins. It gives both benign and malignant  tumor information and is widely used in the detection of different  tumor illnesses. However, SPECT has a limited resolution and  weak positioning ability. The Anaconda tool is used in  combination with Tensorflow, Keras, Numpy, Seaborn, Opencv,  Pandas, Matplotlib, Pytorch, and Python3.The Tensorflow and  Keras frameworks are critical for image fusion. The performance  of this image fusion framework utilizing DTCWT is measured by  entropy, standard error, peak signal - to - noise, root mean square  error, and fusion factor. 

The suggested framework implements Dual Tree Complex Wavelet Transform in picture fusion. The dual tree complex wavelet transform will address the drawbacks of the Discrete Wavelet Transform (DWT). The DTCWT will give greater directionality and shift variance, making it easier to analyze the source image’s edges and contours. The DTCWT’s higher shift variance and increased directionality characteristics result in a fully functional picture fusion tool. Medical imaging systems are beneficial in detecting numerous abnormalities and disorders in our bodies. Medical imaging systems include magnetic resonance imaging (MRI), computed tomography (CT), X-rays, 18 Positron Emission Tomography (PET), and single photon emission computed tomography (SPECT). Each will give distinct details, but none of these technologies will be able to deliver all of the necessary information in a single image. The solution to this challenge is medical image fusion. A single image might include all of the important information from several source photos.

Figure 4.1 depicts the steps taken to fuse two source medical pictures using the Dual Tree Complex Wavelet Transform and generate the fused image. First, register the CT and MRI pictures, then do wavelet decomposition, fuse the two images, then restore the decomposed images. The basic idea behind wavelet image fusion is to combine the wavelet decompositions of the two original pictures utilizing fusion algorithms applied to approximations and detailed coefficients. This gives resources for the Wavelet toolbox analysis program is a collection of several functions built on the technical computer environment PYTHON. It also aids in the synthesis of image deterministic and random signals using wavelets and wavelet packets in the PYTHON programming language.

1. Image Dataset

The CT and MRI images are in 512*512 in size, The submitted image is in RGB format. This RGB image is  transformed to a grayscale image with intensity  information. The source grayscale images CT and MRI  images are then deconstructed using DTCWT.

2. Extracting Features using DTCWT

Following decomposition, the high and low amplitude  features will be extracted as wavelet coefficients. Feature  extraction is being used to reduce data dimension and, if the data is large enough, to uncover quiet and complementary characteristics. Because the medical photos are bigger, PCA is employed to minimize the data here. The PCA will eliminate the unneeded and redundant information in DTCWT. There are various phases to the PCA analysis. The very first phase is standardization, which aims to standardize the range of the continuous initial variables. So, each will contribute equally to the analysis. Standardization is conducted mathematically by comparing the distance between each value and the mean and then dividing by the standard deviation. The next step is to calculate the covariance matrix; certain variables are highly linked with others, resulting in duplicate information. As a result, calculate the correlation matrix to analyze the connection. To identify the major components, compute the eigenvectors and eigenvalues of the correlation matrix. The featured vector is then extracted from all of the calculated principal vectors. Finally, just use principal to reconstruct the data axes.

3. Image Fusion

The extracted coefficients are approximate. Finally, the approximated wavelet equations are obtained. The estimated coefficients are now merged to obtain the fused coefficients. To reconstruct the fused picture, we now employ the Inverse dual tree complex wavelet transform. Figure depicts the combined picture.

4. Image Segmentation

Filtration is used to differentiate between the minutiae of lines, curves, and borders. As a result, the segmentation is done on the final fused picture. Figure 8 depicts the segmentation results.

VI. FUTURE ENHANCEMENTS

Our suggested approach of medical image processing is  based on the Dual Tree Complex Wavelet Transform  (DTCWT), which has demonstrated that it gives the fused  picture with more precise representation of the tumor’s  spectrum, geographical, and soft tissue features. Concerning  performance, our suggested technique has high entropy,  fusion factor, and maximum signal noise ratio values. As a  result, our suggested algorithm outperforms previous fusion  approaches. In the future, block level fusion may be  employed, and the outcomes, such as performance  assessment as well as the final fused image in terms of  increased visual representation, can be improved.

Conclusion

This paper highlights medical image fusion based on the dual tree wavelet packet transformation in this research. The fused multi model picture will be computed using the dual tree wavelet transform, which has good specificity of directionality and lower shift variance. In this study, we suggest using DTCWT to dissect source pictures such as computed tomography (CT) and magnetic resonance imaging (MRI). Furthermore, coefficient extraction and estimation of extracted coefficients are conducted. The inversion dual tree wavelet transform is then employed to create the fused image, followed by segmentation to obtain the divided binary images. As a result, the merged picture of our suggested approach shows more precise tumor borders, tissues, spectral, contour, and spatial information.

References

[1] X. Xu, Y. Wang, and S. Chen, “Medical image fusion using discrete fractional wavelet transform,” Biomedical Signal Processing and Control, vol. 27, pp. 103–111, 2016. [2] Diwakar, M., Tripathi, A., Joshi, K., Sharma, A., Singh, P., Memoria, M., & Kumar, N., “A comparative review: Medical image fusion using SWT and DWT.”, Materials Today: Proceedings, vol. 37, pp. 3411–3416, 2021. [3] Z. Chao, X. Duan, S. Jia, X. Guo, H. Liu, and F. Jia, “Medical image fusion via discrete stationary wavelet transform and an enhanced radial basis function neural network,” Applied Soft Computing, vol. 118, no. 108542, 2022. [4] Richa, K. Kaur, and P. Singh, “A novel MRI and CT image fusion based on Discrete Wavelet Transform and Principal Component analysis for enhanced clinical diagnosis”, International Journal of Online and Biomedical Engineering (IJOE), vol. 18, no. 10, pp. 64–82, 2022. [5] S. Karamzadeh and M. Moshantat, “CT and MRI medical image fusion using discrete wavelet transform,” Cumhuriyet Science Journal, vol. 40, no. 4, pp. 939–945, 2019. [6] Sruthy S, Dr. Latha Parameswaran, Ajeesh P Sasi, “Image Fusion Technique using DT CWT”, IEEE, pp. 160164, 2018. [7] S. Li, X. Kang, L. Fang, J. Hu and H. Yin, \"Pixel-level image fusion: A survey of the state of the art\", Information Fusion, vol. 33, pp. 100-112, 2017. [8] Karishma C Bhataria, Bhumika K Shah, “A Review of Image Fusion Techniques,” Second International Conference on Computing Methodologies and Communication (ICCMC), 2018. [9] Rajasekhar K., Krishna S., Srinivasu V., Neeraj B., Jyothi K., “Image Fusion with CT and MRI Images for improving the quality of Diagnostics”, International Journal of Engineering Applied Sciences and Technology, vol. 6, no. 12 ,pp.318-322, 2022. [10] Nayera Nahvi, Deep Mittal, “Medical Image Fusion Using Discrete Wavelet Transform”, International Journal of Engineering Research and Applications, vol. 4, pp.165-170, 2014. [11] Neha Sharma, “Hybrid Image Fusion implemented in DTCWT”, International Journal of Engineering Technology and Computer Research (IJETCR), vol. 2, pp. 16-21, 2014.

Copyright

Copyright © 2023 Gajjala Jahnavi , Alla Deepthi Chowdary, Ananthula Jagadeesh, Kachapuram Basava Raju. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.