Image Dehazing from Repeated Averaging Filters with Artificial Neural Network

Abstract

The physical process of converting an image signal into a physical image is known as image processing. Either a digital or analogue image signal can be used. An actual physical image or the attributes of an image can be the actual output. The logical process of detecting, identifying, classifying, measuring, and evaluating the relevance of physical and cultural items, their patterns, and spatial relationships is referred to as image processing. This paper presents a method for measuring ambient light from a single foggy image using Repeated Averaging Filters, which adds to greater radiance recovery. The problem of halo artefacts in the final output image after dehazing plagues existing dehazing algorithms. An averaged channel is created from a single image using recurrent averaging filters using integral images with artificial neural network, which is a faster and more efficient method of reducing halo artefacts. In terms of quantitative and computational analysis, the suggested dehazing method achieves competitive results and outperforms many earlier state-of-the-art solutions.

Introduction

I. INTRODUCTION

A numerical representation of an object is a digital image. Pixels are the visual elements that make up the image. Each pixel has its own coordinates and value. A pixel is a representation of the brightness at a specific place in an image. All image processing actions are performed on these pixels. The use of computer algorithms to perform image processing on digital images in order to obtain an improved image or to extract relevant information from it is known as digital image processing. Digital image processing has the advantages of flexibility, adaptability, and data storage and transport. Digital image processing does not necessitate hardware upgrades, and data within the computer can be transferred from one location to another. Memory and processing speed are two of digital image processing's drawbacks. We need to keep digital photographs on a storage device so that we can utilise them again in the future. For storing image data, there are numerous storage devices available. Optical discs, magnetic discs, and floppy discs are examples of these storage devices.

Removing haze from a single image or several images is an important and necessary work in video processing, computer vision, and digital photography [1], since it improves the visibility of distinct objects and reduces colour shift caused by air light. Another benefit of haze removal is that it aids computer vision algorithms for picture analysis at all levels, from low to high, and provides depth information [2].

Picture dehazing can be split into two categories: image enhancement and image restoration [3]. To address dehazing, image restoration-based approaches create an atmospheric scattering model and then apply the inversing degradation process [4]. Further Image restoration approaches can be divided into two groups: those that consider several photos and those that focus on a single image [5]. Other approaches, such as Retinex [6], homomorphic [7], and wavelet transform [8,] were also introduced.

Previously proposed techniques were tested on several photos for haze reduction. However, multiple-based image approaches have run into some issues in online imaging dehazing applications, which necessitate a high-resolution sensor. As a result, numerous studies [2], [9], [10], [11] concentrated on single image dehazing.

Later on, the Dark Channel Prior [2] is proposed for single picture dehazing, and dark channel prior-based techniques receive a lot of attention. Using the simple principle of dark channel prior techniques, a lot of work has been done [2, 9].

Single image dehazing is classified using the dark channel previous approach in four steps: first, estimate the air light (atmospheric light), second, transmission, and third, refining of the estimated transmission map. The fourth and last step is to restore the scene's brilliance. Our proposed method is based on the dark channel prior (DCP) method and addresses some of the DCP method's issues, such as removing halo artefacts from the final recovered scene radiance map. Aside from that, our method gives a comprehensive solution for dehazing a single foggy image in the outdoors.

II. LITERATURE REVIEW

Linear filters were the principal instruments for image enhancement and restoration in the early days of image processing. They perform poorly in the presence of non-additive noise, as well as in cases involving system nonlinearities or Gaussian statistics [19].

Images generated from any source nowadays will degrade in some way during the transmission and processing process. We are unable to extract useful information from the photos due to their deterioration. As a result, we see a need for a technique that can restore the original image from a distorted one. As a result, Image Restoration is extremely significant in the field of image processing. Image Restoration is used to restore images that have blur kernal and additive noise that are unknown.

Image restoration and enhancement procedures are used to enhance the image's look or extract finer information from deteriorated photos. The goal of picture restoration and enhancement is to modify an image so that the final product is more suited to a certain application than the original. Computer vision, video surveillance, satellite and medical image processing and analysis are only some of the uses for these approaches. Filtering the observed image to reduce the effect of degradations is what image restoration is all about.

Sensor noise, random atmospheric turbulence, and other factors may degrade the photos. Random noise degrades images frequently. Noise can arise during the collection, transmission, or processing of images, and it can be reliant on or independent of the image content. The probabilistic features of noise are commonly used to describe it. The extent and accuracy of knowledge of the degradation process, as well as the filter design criterion, determine the effectiveness of picture restoration filters [Jain, 1989]. For image restoration, traditional filters such as mean, median, and so on are commonly utilised. However, these traditional filters have drawbacks, leading to the creation of improved filters such decision-based median filters, switching median filters, wavelet filters, and fuzzy filters [Gonzalez and Woods, 2008].

[Pratt, 2001] The goal of image enhancement is to increase the interpretability or perception of information in images for human viewers or to give better input for other automated image processing approaches. Image enhancement is a technique for improving the depiction of fine features in photographs. Image contrast enhancement is a type of image enhancement procedure that entails changing one image into another in order to improve the look and feel of an image for machine analysis or human perception [Acharya and Ray, 2005]. It's a must-have tool for researchers in a range of domains, including medical imaging, forensics, and atmospheric sciences, among others. Aghi and Ajami introduced a new artificial neural network-based colour picture denoising method. Their major goal is to use appropriate neural networks to develop an adaptive noise canceller. A. De Stefano and colleagues demonstrated an automatic method for removing grain in film images. This technique minimises noise by using a parameterized set of functions to threshold the image's wavelet components. The Vector Rank M-type K-Nearest Neighbor (VRMKNN) filter was presented by Volodymyr P and Francisco G. F to reduce impulsive noise from colour static and dynamic image sequences. This filter uses a vector approach and a rank M-type K-nearest neighbour method to handle multichannel images.

For many image processing and computer vision applications, image segmentation is a crucial step. Applications across a wide range of areas have sparked attention. Analyzing different parts of an aerial photo, for example, can help you better understand the vegetation cover. For content-based image retrieval, scene segmentation is useful for retrieving images from big image databases. The majority of segmentation methods necessitate picture attributes that define the segments to be segmented. Texture and colour, in particular, have been used extensively and independently. Because colour information is a multi-dimensional vector, grey image segmentation techniques cannot be used directly. Edge detection, region growth, clustering, neural networks, fuzzy, tree/graph based algorithms, probabilistic or Bayesian approaches, and histogram thresholding are all examples of existing colour image segmentation techniques. Non-linear filters, such as the Adaptive Median Filter (AMF) [Hwang and Haddad, 1995], can be used to distinguish between damaged and uncorrupted pixels before filtering. Uncorrupted pixels will be kept alone, while noisy pixels will be replaced by the median value. Because the erroneous pixels that are replaced by the median values are infrequent, AMF performs well at low noise densities. To accomplish better noise removal at greater noise densities, the window size must be raised, resulting in reduced correlation between corrupted pixel values and restored median pixel values. When using a switching median filter or a decision-based median filter, A predetermined threshold value is used to make the decision. The difficulty in constructing a robust decision measure is a major disadvantage of this strategy. Because these filters do not take into account local features, edge details may not be retrieved properly, especially when noise levels are high.

III. IMAGE RESTORATION AND ENHANCEMENT

Picture dehazing can be divided into two categories: image enhancement and image restoration. One of the most important areas of research in the world of digital image processing is picture restoration and enhancement. Using a priori understanding of the degradation phenomenon, image restoration aims to reconstruct or restore a degraded image. Visual enhancement, on the other hand, refers to the highlighting or sharpening of image elements such as edges, boundaries, or contrast in order to improve the usability of a graphic display for presentation and analysis. Image restoration and enhancement techniques are widely utilised in computer vision, video surveillance, medical image processing, and satellite image processing, among other fields.

A. Image Restoration

Images are often degraded by random noise which can occur during image acquisition, transmission or processing. The degradations may occur due to sensor noise, relative object-camera motion, random atmospheric turbulence, and so on. Noise may be either dependent or independent of image content, and is usually described by its probabilistic characteristics. During image transmission, noise which is usually independent of the image signal occurs. Gaussian noise is a very good approximation of noise that occurs in many practical cases. Image noise reduction has come to specifically mean a process of smoothing noise that has somehow corrupted the image. Image restoration is concerned with filtering the observed image to minimize the effect of degradations, where prior information of the degradation form is needed. The goal of image restoration is to recover an image that resembles the original image as closely as possible by reducing the noise. Image restoration techniques are basically divided in two categories namely: Deterministic process and stochastic process. Deterministic processes are those processes in which there is a prior knowledge of degradation function or point spread function and stochastic processes are those processes in which there is no prior knowledge of degradation function or point spread function like blind de-convolution method. Deterministic methods are subsequently divided into two parts: Parametric and Nom-parametric. Linear Filters do not necessarily maintain image non-negativity or signal-dependent noise. This has led to the development of non-linear and iterative restoration algorithms. Image restoration is different from image enhancement in the way that the latter is designed to emphasize features of the image to make the image more pleasing to the observer, but not necessarily produce realistic data from a scientific point of view. Image enhancement techniques (like contrast stretching or de-blurring by a nearest neighbor procedure) use no a priori model of the process that created the image.

B. Image Enhancement

Image enhancement includes sharpening, contrast manipulation, filtering, interpolation and magnification, pseudo coloring, and so on. The greatest difficulty in image enhancement is quantifying the criterion for enhancement. Therefore, a large number of image enhancement techniques are empirical and require interactive procedures to obtain satisfactory results. However, image enhancement remains very important because of its usefulness in virtually all image processing applications. Color image enhancement may require improvement of color balance or color contrast in a color image. Enhancement of color images becomes a more difficult task not only because of the added dimension of the data but also due to the added complexity of color perception [Gonzalez and Woods, 2008].

Image enhancement techniques are used to improve the appearance of the image or to extract the finer details in the degraded images. The principal objective of image enhancement is to process an image so that the result is more suitable than the original image for a specific application. A method that is quite useful for enhancing one category of images may not be necessarily be the best approach for enhancing other category of images. Color image enhancement using RGB color space is found to be inappropriate as it destroys the color composition in the original image. Due to this reason, most of the image enhancement techniques, especially contrast enhancement techniques, use HSV color space [Hanmandlu and Jha, 2006].

Image enhancement methods may be categorized into two broad classes: transform domain methods and spatial domain methods. The techniques in the first category are based on modifying the frequency transform of an image, whereas techniques in the second category directly operate on the pixels. However, computing a two dimensional (2-D) transform for a large array (image) is a very time consuming task even with fast transformation techniques and is not suitable for real time processing.

Image enhancement is basically improving the interpretability or perception of information in images for human viewers and providing `better' input for other automated image processing techniques. The principal objective of image enhancement is to modify attributes of an image to make it more suitable for a given task and a specific observer. During this process, one or more attributes of the image are modified. The choice of attributes and the way they are modified are specific to a given task.

IV. PERFORMANCE METRICS

It is a tedious task to assess the performance of haze image enhancement and restoration algorithms, as there are no ground truths available. To assess the enhanced visibility, we measure the performance of algorithms in two ways. First, we will assess the qualitative comparison of our method with other contemporary methods. This measure is subjective and hence proper quantification is not possible. Second approach is to follow quantitative comparison using the metrics which have been used by other researchers. Though, some researchers have used mean squared error (MSE) [19] and structural similarity index metric (SSIM) [20] they do not fit the test, particularly for the reason that these metrics need reference images for proper evaluation and specifically MSE is designed for applications such as image compression. For comparative evaluation with the existing methods, we have used it.

There are other techniques such as examining the number of visible edges before and after restoration, quantity of edges visible in output images to those not present in hazy images and mean ratio of the gradients at the visible edges. This metric was proposed Hautiere et al. in [60] and used for the purpose of visibility recovery assessment in [20]. To assess the quantitative comparison we have referred to blind contrast enhancement indicators to quantify the quality of the restoration.

The details of these metrics are explained in following sections, followed by a qualitative and quantitative comparison as applied to one of the haze image.

A. Peak Signal to Noise Ratio (PSNR) and Mean Squared Error (MSE)

The term peak signal-to-noise ratio (PSNR) is an expressed as the ratio of maximum possible value of a signal and the power of distorting noise that affects the quality of its representation. The dimensions of the two images must be the same. Mathematical representation of the PSNR is as follows:

Where, f is the matrix data of the original image, g is the matrix data of processed image, m and n represents the numbers of rows and the columns of pixels of the images and i and j represents the index of row and the column respectively. MAXf is the maximum signal in image f. The major shortcoming of PSNR metric is that it relies on numeric comparison and does not actually take into consideration the biological factors of the human vision system such as the structural similarity index (SSIM).

B. Structural Similarity Index (SSIM)

Wang et al. proposed SSIM metric for assessing image quality. The structural similarity (SSIM) index computes the similarity index between two images. It is more consistent with human perception as opposed to conventional methods such as mean square error (MSE). As it is correlated to human visual perception, SSIM has become a universal quality metric for image and video applications for quantitative analysis. For input image O and R, let μO, σO and σOR denote the mean of O, the variance of O, and the covariance of O and R respectively, SSIM is mathematically given as

Where C1 and C2 are constants, this metric has been suggested for the haze environment for quantitative analysis in Lu et al.

V. IMAGE DEHAZING METHODS AND MODELS

Our work is based on restoration-based picture dehazing approaches, which are classified into single-image and multiple-image methods. Polarization approaches [12], [13], [14] refer to image dehazing based on several images. By varying the intensities of image structure under different weather circumstances, the proposed technique [12] examined scene points and determined depth discontinuities. Similarly, the proposed approach [13] noticed that a polarised filter alone is insufficient to dehaze the haze image and applied several polarisation orientations to improve estimations. Another regularization-based approach [14,18] modelled inheriting body constraint and contextual regularisation to estimate scene transmission mutually.

[10] shows that for a single image, a typical image without haze has higher contrast, however when haze and fog are present, the contrast decreases.

As a result, a local contrast is increased for this purpose, and visibility improves, but the method still suffers from halo artefacts in the final output map. After transmission computation, the approach in [11,19] focused on scene albedo and assumed that surface shading and transmission have no local co-relations. The well-known dark channel prior (DCP) [2] was recently introduced. The dark pixels phenomenon was discovered after extensive investigation on outdoor photos. The findings were based on the presence of black pixels in natural outdoor photos, and it was discovered that at least one colour channel in an RGB image has the lowest pixel intensities, neglecting the sky region, which tends to be dark. Researchers were led in new directions by the DCP approach. The DCP approach had some drawbacks, such as the use of soft matting to refine the transmission, which is a computationally costly job. Furthermore, it is ineffective for photos with brighter objects because it selects the maximum pixel intensities, which can cause problems in the final out map. Guided filters, which keep much of the edges and function as smooth operators, are recommended for this purpose [9,20].

Similarly, [1] presented an approach to avoid halos in single picture dehazing by computing fixed points using nearest neighbours (N-N) for recovering smooth transmission using feed forward neural. The above discussion motivates us to suggest a new process of repeated averaging filters that addresses the issues raised.

VII. RESULT AND DISCUSSIONS

The system specification for our experiments is an ASUS machine with an Intel Core i7-6700HQ 2.60 GHz CPU, 8.00 GB of installed memory (RAM), and MATLAB 2017b running on Windows 7. We tested our hypothesis on a large data set of outdoor hazy photos, such as cityscapes, aerial views, and landscapes, which had previously been used in the literature. Our approach is applicable to any scene or input image that is polluted by fog, haze, or dust, as demonstrated by our testing results.

This method was evaluated in terms of both qualitative and quantitative analysis. Prior to this, there were some disadvantages to using the dark channel approach. This may not be useful in photographs containing brilliant objects with greater brightness, for example. Because it chooses the brightest pixels, such as pixels from an automobile in the input image as ambient light, it can result in a faulty transmission map. Another drawback is that it refines the transmission map using the soft-matting approach, which is a time-consuming process. However, because our proposed method uses some sigma with repeated averaging filters and a feed-forward Neural Network, it avoids the techniques previously used for transmission map refining. As a result, a smooth and filtered transmission map is obtained, free of the halo artefacts that the DCP approach produces.

A. Qualitative Evaluation

For qualitative evaluation we compared our results with [9], [14], [15] and [17] methods. Our proposed method dominates all the previous methods in terms of qualitative evaluation. The qualitative results of our proposed method are shown in Figure 3 with different data images.

Conclusion

The operational complexity, successful dehazing of the dense haze image, and applicability to real-time systems are all encountered in this work technique. The usage of integral image operations is used to solve the operational complexity. The use of multiple averaging filters predicted improved air light, which improved the restored scene radiance even more. With the help of the sigma amount, we proposed a method that refined the transmission map and removed the halo artefacts that earlier approaches had. Subjective visual analysis as well as quantitative methodologies were used to assess the outcomes. Mean Square Error (MSE) and structural similarity (SSIM) were utilised to analyse the haze-free photos objectively. The signal intensity, the extent of feature preservation, and the recovery of structural details gained in the haze-free image are all quantified using these measures.

References

[1] Zhang, Shengdong, and Jian Yao. \"Single image dehazing using fixed points and nearest-neighbor regularization.\" Asian Conference on Computer Vision. Springer, Cham, 2016. [2] He, Kaiming, Jian Sun, and Xiaoou Tang. \"Single image haze removal using dark channel prior.\" IEEE transactions on pattern analysis and machine intelligence 33.12 (2011): 2341-2353. [3] Xu, Yong, et al. \"Review of video and image defogging algorithms and related studies on image restoration and enhancement.\" Ieee Access 4 (2016): 165-188. [4] McCartney, Earl J. \"Optics of the atmosphere: scattering by molecules and particles.\" New York, John Wiley and Sons, Inc., 1976. 421 p. (1976). [5] Zhou, X., L. Bai, and C. Wang. \"Single Image Dehazing Algorithm Based on Dark Channel Prior and Inverse Image.\" International Journal of Engineering-Transactions A: Basics 30.10 (2017): 1471-1478. [6] Cooper, Ted J., and Farhan A. Baqai. \"Analysis and extensions of the Frankle-McCann Retinex algorithm.\" Journal of Electronic Imaging 13.1 (2004): 85-93. [7] Seow, Ming-Jung, and Vijayan K. Asari. \"Ratio rule and homomorphic filter for enhancement of digital colour image.\" Neurocomputing 69.7-9 (2006): 954-958. [8] Dippel, Sabine, et al. \"Multiscale contrast enhancement for radiographies: Laplacian pyramid versus fast wavelet transform.\" IEEE Transactions on medical imaging 21.4 (2002): 343-353. [9] He, Kaiming, Jian Sun, and Xiaoou Tang. \"Guided image filtering.\" IEEE transactions on pattern analysis & machine intelligence 6 (2013): 1397-1409. [10] Tan, Robby T. \"Visibility in bad weather from a single image.\" Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on. IEEE, 2008. [11] Fattal, Raanan. \"Single image dehazing.\" ACM transactions on graphics (TOG) 27.3 (2008): 72. [12] Narasimhan, Srinivasa G., and Shree K. Nayar. \"Contrast restoration of weather degraded images.\" IEEE transactions on pattern analysis and machine intelligence 25.6 (2003): 713-724. [13] Schechner, Yoav Y., Srinivasa G. Narasimhan, and Shree K. Nayar. \"Instant dehazing of images using polarization.\" null. IEEE, 2001. [14] Meng, Gaofeng, et al. \"Efficient image dehazing with boundary constraint and contextual regularization.\" Proceedings of the IEEE international conference on computer vision. 2013. [15] M. Boffety, F. Galland and A. G. Allais, \"Influence of polarization filtering on image registration precision in underwater conditions,\" Optics Letters, vol. 37, no. 15, pp. 3273-3275, 2012. [16] N. Hautiere, J. P. Tarel, D. Aubert and E. Dumont, \"Blind contrast enhancement assessment by gradient ratioing at visible edges,\" Image Analysis Stereology, vol. 27, no. 2, pp. 87-95, 2011. [17] S. Wang, K. Ma, H. Yeganeh, Z. Wang and W. Lin, \"A patch-structure representation method for quality assessment of contrast changed images,\" IEEE Signal Processing Letters, vol. 22, no. 12, pp. 2387-2390, 2015. [18] J. Wu, H. Huang, Y. Qiu, H. Wu, J. Tian and J. Liu, \"Remote sensing image fusion based on average gradient of wavelet transform,\" in Proceedings of the IEEE International Conference on Mechatronics & Automation, Niagara Falls, Canada, 2005. [19] N. Dhanachandra, K. Manglem and J. Y. Chanu, \"Image segmentation using k-means clustering algorithm and subtractive clustering algorithm,\" in Eleventh International Multi-Conference on Information Processing, Bangalore, India, 2015. [20] Q. Zhu, J. Mai and L. Shao, \"A fast single image haze removal algorithm using color attenuation prior,\" IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 3522-3533, 2015.

Copyright

Copyright © 2022 Vinaya Kumar, Madhu Lata Nirmal. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.