Image Deblurring using Deep Learning

Abstract

Image deblurring is a most considered problem in low-level computer vision with an objective to recover a high resolution image from a blurred input image. It is a mandatory task in image processing and has been studied for several years which is used in reconstruction of images which are blurred due to various reasons. Particularly in these years, deep learning approaches have shown great promises in various image restoration tasks, which includes image deblurring. Among these, DBSRCNN is a powerful deep learning approach that has been used for super-resolution and image deblurring. In this model developed we are going to implement De Blurring Super Resolution Convolutional Neural Network (DBSRCNN) for blurred reconstruction. The proposed method achieves superior results in both quantitative and qualitative evaluations, depicting its effectiveness in image deblurring tasks.

Introduction

I. INTRODUCTION

Image deblurring is a primary task in image processing that involves the restoration of images that have been blurred due to various reasons. It is concerned with problems caused by overexposure, underexposure and poor photographic conditions.     

The blurring of an image is due to:

1. Camera movement or, when long exposure durations are utilized to capture the image, movement of the subject of the image being captured.
2. Decreased photon capture due to out-of-focus optics, wide-angle lens being used, atmospheric turbulence, or short exposure times.

Image deblurring is a predominantly a tough task due to the inherent wicked behavior of the problem. Most of the images are blurred because of the motion of the camera or the subject being captured. The blurred nature may vary in intensity and direction too; making it a bit more difficult to find one single solution that works for all these cases. As a result, image deblurring requires sophisticated algorithms and techniques that can handle the complex nature of the problem.

II. RELATED WORK

Image deblurring is a complex issue in computer vision, where the goal is to output a clear image from a blurry input. Blurry images can be caused by various factors such as camera shake, motion blur, out-of-focus, and atmospheric turbulence. Traditional methods for image deblurring involve solving an ill-posed inverse problem, which is often computationally expensive and requires manual parameter tuning. In the very near years, deep learning approaches are showing promising results for image deblurring. Deep learning methods learn a mapping between the blurry and sharp images using large amounts of training data, without explicitly modeling the degradation process.

This makes them more efficient and robust compared to traditional methods. Some of the popular deep learning models for image deblurring include convolutional neural networks (CNNs), recurrent neural networks (RNNs), and generative adversarial networks (GANs). CNN-based methods typically use a convolutional neural network to directly estimate the sharp image from the blurry input. RNN use a recurrent neural network to model the temporal dependencies between the blurry frames and estimate the sharp image. In this particular proposed model we have implemented the DBSRCNN for image deblurring. One of the challenges in deep learning-based image deblurring is to generate visually pleasing results while preserving fine details and avoiding artifacts. To address these issues, various loss functions have been proposed, including perceptual loss, which analyses the variation in feature representations between the actual images, and adversarial loss, which encourages the generated image to be realistic and indistinguishable from the ground truth.

Overall, deep learning approach(DBSRCNN) for image deblurring have shown impressive results and have the potential to revolutionize this field. However, there are still many challenges to overcome, such as generalizing to different degradation types and dealing with real-world noise and artifacts.

      III. METHODOLOGY

Deblurring Super-Resolution Convolutional Neural Network is a type of convolutional neural network that is actually designed to perform joint deblurring tasks on blurry images. The architecture of DBSRCNN typically consists of multiple convolutional layers, which extract features from the input image, and deconvolutional layers, which upsample the image to the desired output resolution.

Here is a more detailed overview of the architecture of DBSRCNN:

1. Input Layer: The input layer receives the low-resolution, blurry image as input. The first layers completely deals with the feature extraction. It consists of  [22]32 filters with filter size 9*9. At this level we extract certain features like pixel information.
2. Convolutional Layers: Basically,[22] the second layer is responsible for feature enhancement which contains 32 filters with filter size (9*9). Several convolutional layers are used to get features from the input image. Convolution layers typically use small filters to capture local details in the image. In a convolutional layer, n number of filters are applied as the input image to extract different features. Each filter performs a convolution operation on the input image, multiplying a small portion of the input by the filter weights and summing the results to produce a single output value. The output values from each filter are then combined to produce the output feature map for the layer.
3. Deconvolutional Layers: Deconvolution is a common operation used in deblurring, super-resolution, and other related tasks performed by convolutional neural networks (CNNs). In the context of CNNs, deconvolution is typically used to obtain  the original highly resolute image from a down sampled, blurred image. As a part of this process concatenation takes place, which includes a layer consisting of 32 or 64 filters with filter size of (5*5). In deblurring tasks, the input image is typically blurred by convolution with a blur kernel. To recover the original sharp image, a deconvolution layer is used in the network to invert the effect of the blur kernel. This deconvolution layer is also referred to as a "transpose convolution" or "fractionally-strided convolution" layer. The deconvolution layer essentially performs an up-sampling operation by filling the gaps between the input pixels, which were lost during the down sampling process. In super-resolution tasks, the input image has a low resolution, and the primary moto is to generate a clear image. The deconvolution layers essentially perform the inverse operation of the down-sampling process. In both deblurring and super-resolution tasks, the deconvolution layer can be used in combination with other layers, such as convolutional layers and activation functions, to form a convolutional neural network. [28] The network can be trained on a dataset of sharp images and corresponding low-resolution or blurred images, using techniques such as gradient descent, to learn the mapping between the two types of images. In this second order mapping is followed with 32 filters of size (5*5). Overall, deconvolution is a powerful operation used in convolutional neural networks for various tasks, such as deblurring and super-resolution, and is essential for achieving high-quality results.
4. Output Layer: The output layer produces the high-resolution, deblurred image. This layer consists of 1 filter with size (5*5). Overall, the architecture of DBSRCNN is similar to other types of super-resolution convolutional neural networks, but with the addition of deblurring capabilities.

IV. EXPERIMENTAL EVALUATION

A. Generating Data

Generating data includes blurring images by Gaussian blur filter at different levels. i.e., sigma=1,2,3 and 4. These images are resized by the help of bicubic function using upscaling factor as three. To compare the model with SRCNN algorithm training set yang 91 is used

B. Input

Through data generation a dataset of paired low-resolution and truly clear images is obtained. [27]The low-resolution images can be obtained by down sampling the high-resolution images using a predefined scaling factor. The clear images should be containing the same number of pixels as the low-resolution images, and they should be aligned. This dataset is taken as our input.

???????C. Training

The DBSRCNN model is trained using the prepared dataset and defined architecture. The model is prepared to lessen the chosen loss function between the predicted and ground-truth of the high-resolution images. During this training process, the images with less clarity are given to the model as input, and the model generates a high-resolution output. The model is updated using backpropagation and stochastic gradient descent functions to adjust the model parameters and reduce the loss.

???????D. Testing

The trained DBSRCNN model is tested on new and unseen low-resolution images to output a high-resoluted image as output. The performance of the model on test images can be measured using the same metrics which are used for evaluation.

VI. FUTURE SCOPE

Image deblurring is an truly an area of research in computer vision computer vision and image processing that must be focused on and aims for image reconstruction. There are many potential future directions for image deblurring. There is still room for improvement and development of effective deep learning models can significantly improve the accuracy and speed of image deblurring. Real time deblurring is an important research direction to go with. Since there are many practical applications for real time deblurring which includes video processing, surveillance, autonomous driving etc. Developing real time algorithms that can efficiently carry out better results even in case of large and complex images in real-time can have significant images in these areas.  Not only this incorporating the prior knowledge about the image formation process can improve the accuracy of image deblurring. It can look for it’s applications in astronomical imaging and medical imaging.

Conclusion

In conclusion the use of DBSRCNN as proven an effective technique for image deblurring. The model has depicted the ability to restore blurred images with high levels of accuracy and efficiency. By leveraging the deep learning and image processing techniques, DBSRCNN has improved the performance in obtaining images of good clarity, making it a valuable tool for applications such as medical imaging, remote sensing, and surveillance. However, as with any deep learning model, the accuracy of DBSRCNN is strictly dependent on the quantity and the quality of training data. In addition to that, selecting appropriate hyperparameters and tuning them can significantly impact the performance of the model. Thus, to achieve optimal results, careful selection as well as preprocessing of the training data and thoughtful experimentation with chosen hyperparameters are required. Finally, DBSRCNN is a promising approach for image deblurring and can be further improved and expanded to address related image restoration tasks.

References

[1] J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process., vol. 19, no. 11, pp. 2861–2873, 2010. [2] M. Bevilacqua, A. Roumy, C. Guillemot, and M. AlberiMorel, “Low-complexity single-image super-resolution based on nonnegative neighbor embedding,” in Proc. BMVC. [3] J. Yang, Z. Wang, Z. Lin, S. Cohen, and T. Huang, “Coupled dictionary training for image superresolution,” IEEE Trans. Image Process., vol. 21, no. 8, pp. 3467–3478, 2012. [4] R. Timofte, V. De Smet, and L. Van Gool, “Anchored neighborhood regression for fast example-based superresolution,” in Proc. ICCV, 2013, pp. 1920–1927. [5] R. Timofte, V. De Smet, and L. Van Gool, “A+: Adjusted anchored neighborhood regression for fast superresolution,” in Proc. ACCV. Springer, 2014, pp. 111– 126. [6] S. Schulter, C. Leistner, and H. Bischof, “Fast and accurate image upscaling with super-resolution forests,” in Proc. CVPR, 2015, pp. 3791–3799. [7] C. Dong, C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image super-resolution,” in Proc. ECCV. Springer, 2014, pp. 184–199. [8] Z. Wang, D. Liu, J. Yang, W. Han, and T. Huang, “Deep networks for image super-resolution with sparse prior,” in Proc. ICCV, 2015, pp. 370–378. [9] D. Liu, Z. Wang, B. Wen, J. Yang, W. Han, and T. Huang, “Robust single image super-resolution via deep networks with sparse prior,” IEEE Trans. Image Process., vol. 25, no. 7, pp. 3194–3207, 2016. [10] J. Kim, J. Kwon Lee, and K. Mu Lee, “Accurate image super-resolution using very deep convolutional networks,” in Proc. CVPR, 2016, pp. 1646–1654. [11] J. Kim, J. Kwon Lee, and K. Mu Lee, “Deeply-recursive convolutional network for image super-resolution,” in Proc. CVPR, 2016, pp. 1637–1645. [12] W. Shi, J. Caballero, F. Huszar, J. Totz, R. Aitken,´ A.and Bishop, D. Rueckert, and Z. Wang, “Realtime single image and video super-resolution using an efficient sub-pixel convolutional neural network,” in Proc.CVPR, 2016, pp. 1874–1883. [13] W. Lai, J. Huang, N. Ahuja, and M. Yang, “Deep laplacian pyramid networks for fast and accurate superresolution,” arXiv preprint arXiv:1704.03915, 2017. [14] B. Lim, S. Son, H. Kim, S. Nah, and K. Lee, “Enhanced deep residual networks for single image superresolution,” in Proc. CVPR workshops. IEEE, 2017, pp. 1132–1140. [15] F. Abluwi, R. Dahyot, and V. Krylov, “Artifacts reduction in JPEG-Compressed Images using CNNs,” in Proc. Irish Machine Vision and Image Processing Conf. IMVIP, 2018. [16] D. P. Kingma and L. Ba, “ADAM: a method for stochastic optimization,” in Proc. ICLR, 2015. [17] J. Yang, J. Wright, T. Huang, and Y. Ma, “Image super-resolution as sparse representation of raw image patches,” in Proc. CVPR. IEEE, 2008, pp. 1–8. [18] R. Zeyde, M. Elad, and M. Protter, “On single image scale-up using sparse-representations,” in International conference on curves and surfaces. Springer, 2010, pp. 711–730. [19] YapengTian, “Srcnn-keras,” https://github. com/YapengTian/SRCNN-Keras, 2017. [20] Fatma Albluwi, Vladimir A. Krylov, Rozenn Dahyot. \"Image Deblurring and SuperResolution Using Deep Convolutional Neural Networks\" , 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP), 2018 [21] Zhenyu Wang, Wankou Yang, Xianye Ben. \"Low-resolution degradation face recognition over long distance based on CCA\", Neural Computing and Applications, 2015. [22] Fatma Albluwi, Vladimir A. Krylov, Rozenn Dahyot. \"Image Deblurring and SuperResolution Using Deep Convolutional Neural Networks\" , 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP), 2018. [23] Junaid Ahmed, Bin Gao, Gui Yun Tian.\"Wavelet domain based directional dictionaries for single image super-resolution\", 2017 IEEE International Conference on Imaging Systems and Techniques (IST), 2017.

Copyright

Copyright © 2023 Ainala Vinay Kumar Reddy, Akula Manasa, Chilupuri Varsha, Mrs. Ruqqaiya Begum. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.