An Effective Framework for Enhancement of Hazed and Low-Illuminated Images

Abstract

Haze removal is important for computer photography and computer vision applications. However, most of the existing methods for removing the ha- ziness are designed for daytime images and may not always work well at hazy night images. Unlike image conditions during the sunny day, images captured in winter night conditions can suffer from irregular lighting due to artificial light sources with varying colors and non-uniform illumination, which show low brightness, contrast and color distortion. In this paper, we propose a new frame- work for presenting night-time hazy imaging, which works on haze removal and low-illumination correction algorithm taking into consideration both the non-uni- form illumination of artificial light sources and the effects of dispersion and at- tenuation of fog. Therefore, firstly, we will give a hazy low-illuminated image having low light as input and then apply a technique to clarify the visibility of the input image. Then, apply the contrast enhancement and after that apply the LIME technique and finally, apply the white balance technique and we will get our im- proved output image. The experimental results show that the proposed algorithm can achieve an illumination balance, results without haziness and good color cor- rection capacity.

Introduction

I. INTRODUCTION

The haze is able to change natural colors and decrease the contrasts of the captured images. The degradation is generally caused by the dispersion of light and the attenua- tion of light in the atmosphere. It is important to eliminate the haziness of degraded images for different applications. Based on the simplification of the reflection of light, the observed images can be modeled as a product of illumination and reflection. Many methods have been used that decompose an image in illumination and reflection in a series of applications, such as the improvement of the contrast, the improvement of the image of the non-uniform illumination, the mapping of the tones, the remote sensing, the correction of the image, image segmentation, target selection, and tracking, etc [2,4, 7] [8] [17].

This diagram (Fig 2) showing the problems that are faced in the hazy winter night low- illuminated image, the difficulties like distortion, noise, blur-ness, haziness, low clarity, visualization of the image, absence of natural colors occurs due to the presence of fog, mist, clouds, aerosols, obstacles causing night images look very poor in quality [8]. Due to all these elements present in the atmosphere, the images suffer multiple scattering phenomena. We will try to remove all the problems that can cause distortion of the images by using enhancement and illumination techniques and comparing our results with different quality estimation parameters [2].

In poorly lit situations, the qualities of pictures and videos captured by optical imaging devices frequently deteriorate. This could decrease the performance of specific systems, which could be used in intelligent traffic analysis systems, visual surveillances. Low- light conditions in night-time environments can produce low-contrast images and vid- eos, reducing visibility [8].

Backlighting is one example in which objects and details cannot be captured simulta- neously in bright regions (background) and dark regions (foreground) due to limitations in the exposure settings of many imaging systems [17]. In everyday life, we receive a lot of information from images and we try to identify and effectively process this visual information [2].

These conditions which sometimes lead to degradation in the quality of the images include bad weather, poor lighting, moving objects, etc.

The impact of these conditions on image quality could make it problematic for computer vision appli- cations to clearly identifying the image content [17]. The clearly visible images repre- sent more details and useful contents can be identified easily. Enhancement techniques can make the hidden contents of the image visible and facilitates the usability of im- portant information for computers [7].

The elimination of darkness and the extraction of meaningful information are important in applications such as enhancing degraded images, medical imaging, object tracking, route detection, face detection, object detection, road safety. In low-light conditions, the quality of image and video captured through optical imaging cameras frequently deteriorates. This could result in a decrease in performance of some systems [7,9], which are used in night vision systems, intelligent traffic analysis, visual surveillance and detecting systems [16].

II. LITERATURE SURVEY

[S. Kuanar. 2019][2] A deep learning based De-Glow-De-Haze architecture for varying color illumination and glows. Convolution neural network (CNN) based De-Glow model is able to remove the glow effect and De-Haze network is included to remove the haze effect.

[Q, Wang. 2016] [17] Enhancement method which emphases on backlit image based on a technique called multi scale fusion. The fusion technique depends on inputs and weight maps, the features for the backlight are designed to find suitable inputs and cor- responding weights.

First, three inputs are derived which are attained by enhancing the luminance in the original value of the HSV space. Next, a weight map is intended to calculate the exposure function from the inputs which are derived.

Finally, to efficiently combine all the different types of useful information into a single one, all these inputs and the weight maps are merged and the improved image is ob- tained.

[Guo et al. 2017] He offered a technique to improve the quality of colors in images and the contrast of dark areas are highlighted based on the features extracted by observing human vision and the transformation of logarithms. Furthermore, the gamma transform can improve the contrast in images. Resulting in, an algorithm that has achieved better results in color restoration.

[Hao et al.] [8] Method for estimating lighting and used a guided filter for dividing the texture patterns into a refined lighting map. This technique used for the estimation of the lighting map in low light conditions, but the lighting estimate was slightly different in both strategies and this technique has one of the great limitations that it was still necessary to maintain the constancy of the natural color in final output images and they are improved.

A wavelet-based algorithm to improve color. The Euler-Lagrange formula worked with wavelets together to calculate detailed coefficients.

This technique eliminated the color tone of overexposed and underexposed images [5]. This distortion of light is frequently occurring and observed in images where the luminous environment has very complex conditions. After estimating weak lighting, the improved output results were achieved by combining multiple derived results to get the desired output with the help of a mul- tiscale pyramid [5]. This algorithm worked to perform several tasks, such as night, backlight and uneven lighting.

III. PROPOSED METHODOLOGY

A. Proposed Work Flow

B. Proposed Algorithm

• Step1: Take a dataset of all hazy low illumination images as inputs.
• Step2: Apply visibility clarification on that image.
• Step2 (a): Apply Non-Local Dehazing. Step2(b): Apply Dark Channel Prior. Step2(c): Apply Screened Poisson.
• Step3: Apply Contrast enhancement technique on the resultant image from the pre- vious step.
• Step4: Apply LIME technique.
• Step5: Apply White-Balance and we will get our resultant enhanced output image.

1. Visibility Clarification Technique: The methods that use a model as a state of art technique used for image dehazing by using spatial-temporal information obtained from the input image. The Visibility clar- ification technique is used to remove fog or haze present in the images and makes the background objects clearly visible [9]. It is responsible for showing the details of the objects clearly which hide due to the presence of haze. And make these images useful for various computer vision applica- tions [9].

a. Dark Channel Prior Algorithm: The dark channel prior (DCP) method [9] is statistics of the haze free outdoor images. Which relies on the observation that almost all local patches in haze free outdoor images have some pixels with very low-intensities in at least one-color channel. This key ob- servation has been validated in [9]. Based on this observation, the haze degrading the image, air-light (A) can be estimated along with the haze-free image. Dark channel prior dictates that minimizing over local patches in outdoor images over three intensity channels (red, green, blue) gives very low intensity values. Mathematically, this mini- mization step is carried out.

where Jc is a color channel corresponding to true radiance J and ?(x)is a local patch centered at pixel x.

In fact, there are three essential causes for this low intensity in outdoor scenes when minimizing over local patches:[9]

• Colorful objects or surfaces.
• Shadows like, shadows of buildings, cars, etc.
• Dark objects or surfaces.

To eliminate haze, DCP method works in four main steps: a) Estimation of the atmos- pheric light. b) Estimating the transmission t(x). c) Soft matting. d) Recovering the scene radiance.

b. Non- Local Dehazing Algorithm: One of the state of the art image dehazing methods that have been proposed recently is Non-Local Image Dehazing [9]. Non-Local dehazing is based on the supposition that colors of haze-free images are well approximated by a few hundred distinct colors that form tight clusters in RGB space. Moreover, pixels in a given cluster are often non- local, i.e. those pixels are spread over the entire image plane at different distances from the camera [9]. Due to the impact of haze on image pixels, each cluster in the undegraded image turns into a line in the RGB space in the hazy image. Those haze lines are used to recover the haze free image and depth map [9].

AI(x)= A(x)-I                                                                                                                                                                           (2)

Convert AI to spherical coordinates for obtaining [r(a), θ(a), nφ(a)] Cluster the pixels as [θ(a), φ(a)].

Each cluster H is a haze-line Estimate maximum radius

ˆsmax (a) = max (a∈Hr(a))                                                                                                                                                            (3) Estimate transmission and Performing regularization by u(x)

u(a) = r(a)ˆrmax                                                                                                                                                           (4)

c. Screened Poisson Enhancement: The Screened Poisson Equation based method [11] is a gradient domain method. It is a form of image sharpening where the method works like a high pass ?lter [9]. The ob- jective is to preserve the image gradients and at the same time remove the nonuniform illumination effects in the image. This is done by minimizing, with respect to the func- tion u, the functional J(u)consisting of two such corresponding terms, where λ is the trade-off parameter between the two terms [9].

J(u) = (Z?||∇u− ∇f||2dx) + (λ Z?(u−¯u)2dx)                                                                                                                                                   (5)

The nonlinear Euler-Lagrange equation for this minimization problem can be linearized by ?xing the mean value of u to be the same as the mean value of f. Then the solution can be obtained from the screened Poisson equation, with homogeneous Neumann boundary condition [9].

λ(u−¯f) −?u+ ?f= 0, over ?                                    (6)

∂u ∂n = 0                                                    (7)

over ∂? where n denotes the normal vector to the boundary.

Using the two-dimensional discrete Fourier transform, the solution then is obtained in the frequency domain. The Neumann boundary condition is imposed by extending the input image symmetrically across its sides, so that the new J×L image which is four times in size becomes symmetric and periodic. The frequency domain solution is ob- tained. The discrete inverse Fourier transform gives the spatial domain solution u [9].

2. Contrast Enhancement Technique: It is used to increase the sensitivity and increase the specificity of the image. Contrast is the evaluation of the image quality. The contrast is the difference in the visual quality and the properties that make objects different from the others. It is used to increase the sensitivity and increases the specificity of the images. Contrast is the evaluation of the image’s qualities. The contrast is the differences in visual quality and the properties that make objects different from the others. Contrast is the determination of the differ- ences between the color & brightness of the objects with other objects related to it [1]. The hazy low illuminated images suffer from lower contrasts and resolutions because of poor visibility conditions, so identifying objects becomes typical tasks [1]. These degradations include diminished colors, lower brightness and in-distinguishable objects in the images. For improving the qualities of these degraded images, hazy low illuminated image enhancement techniques which focuses on refining the contrasts and colors of night images using the contrasts enhancement technique, greatly contributing to Increasing image visibility [1]. Contrast is the determination of the difference between the color and brightness of the object with other objects related to it [1,13].

3. LIME Technique: When images are captured in little or no light conditions, images often suffer from poor visibility. A method of improving the low-light image (LIME), the illumination of every pixel is initially estimated individually by calculating the maximum value in the R, G and B channels. Also, we perfect the initial illumination map by imposing a pre- vious structure on it, like the final illumination map. Once the lighting map is created, an improvement can be achieved accordingly [10]. Retinex model is used for building this method, that explains the formation of low-light images, the captured image and the desired recovery are L and R, respectively. And the illumination map is represented by T, and element-wise multiplication denoted by the operator.

L = R?T                                                                                                                                                                     (9)

Low illumination L, Gamma transform parameter γ. Constructing weight matrix by es- timating the initial illumination map on L.

Then Re?ne illumination map T. Gamma(γ) correction applied on T from T ← T^γ; Enhancement of L including T from L = R?T.

4. White Balance Enhancement: White Balance first measure the ambient light and then make changes in the picture, setting white balance incorrectly can ruin a picture, adding all kinds of unwanted color cast and causing picture looks very unnatural, because of most light sources (candle, bulb, flashlight, cloudy, sunlight while rising and sunset). White Balance is the process of eliminating the caste of unrealistic colors caste so that objects will look to have nat- ural colors. White Balance measure of light source color temperature, which refers to the relative warmth of the coolness of white light. [1] Color Balance is the global adjustment in the intensities of colors and the goal is to render natural colors or neutral colors that include gray balance, white balance, neutral balance [13].

IV. RESULT ANALYSIS

For comparative study purposes, we have selected a few low light test images since they are captured under different conditions and varied types of weather conditions representing different scene configurations. The low illumination enhancement methods performance is being evaluated by objective, subjective, comprehensive, and comparative study of various low light images and the results obtained will tell you about the removal of low contrast, increase brightness, and the natural color balancing capabilities of the methods. Finally, we will test and prove the use of defined ap- proaches for the different scenes and environmental conditions of the low light images.

This table showing various techniques developed so far and so that we could be easily able to visualize the differences in the outputs obtained by different techniques.

V. QUALITY ASSESSMENT

A. MSE: [Dec.] (Mean Square Error)

Mean Squared Error is the measurement of a fitted line that is how much close to data points if the MSE is low then the image quality is high. It is the distortion and error measure in the image. The calculation of MSE is error metrics used for comparing im- age enhancement qualities[1,5].

MSE represents the cumulative squared-error between the enhanced and the original image.

B. PSNR: [Inc.] (Peak Signal to Noise Ratio)

The term PSNR is an expression for the relationship between the maximum possible value of a signal and the power of distorting the noise that affects image quality.

Peak Signal Noise Ratio is the error metric which compares images enhancement qual- ities, if PSNR is high then it means good quality and if PSNR is low means bad quality PSNR uses the term MSE in the denominator value, so high will be the PSNR value when the error is low.

It is a mathematical measurement of image qualities based on the pixel differences be- tween the two images [4,13].

C. NIQE: [Inc.] (Naturalness Image Quality Evaluator)

The term NIQE value is calculates the naturalness of the image, higher the value of NIQE better are the results [10,19,20].

D. SSIM: [Inc.] (Structural Similarity)

The structural similarity index (SSIM) is the to measurement of the similarity between two images. The SSIM index measures of the quality of one of the images that are compared, providing that the other image is considered to be of perfect quality [5,1].

Conclusion

In this paper the problem related to Low Illumination Image Enhancement for finding the visibility of long-distance objects in hazy low light images is a great challenge for us, and the presence of no light due to scattering and low contrast, little brightness phe- nomenon in low illumination pictures that results in the poor visibility create difficul- ties, we have examined a few of the enhancement algorithms which have been specifi- cally developed for the hazy low illumination pictures, and we will find results from the output image. we have compared the previously developed methods and we have evaluated the performance of all the techniques in relations to various calculations, these methods work on all the low light images, the elimination of image low contrast develops a simpler and more effective technique, so we aim to use these results as the base for developing furthermore advanced low light image enhancement techniques in future.

References

[1] S. Bhadouria, “Effective pre-processing technique for deep underwater image enhance- ment”. Springer 2019. [2] A.S. Bhadouria, “Low Light and non-uniform illumination-based night image enhancement techniques: An exhaustive study,” IJCSNT,2019. [3] Yurui Ren, Zhenqiang Ying, Thomas H. Li, Ge Li. \"LECARM: Low-Light Image Enhance- ment Using the Camera Response Model\", IEEE Transactions on Circuits and Systems for Video Technology, 2019. [4] TIP, 2018, “Structure-Revealing Low-Light Image Enhancement via Robust Retinex Model,”. [5] Ziaur Rahman, Muhammad Aamir, Yi-Fei Pu, Farhan Ullah, Qiang Dai. \"A Smart System for Low-Light Image Enhancement with Color Constancy and Detail Manipulation in Com- plex Light Environments\", Symmetry, 2018. [6] Z. Ying, L. Ge, Y. Ren, R. Wang, and W. Wang, “A new low-light image enhancement algorithm using camera response model,” in IEEE International Conference on Computer Vision Workshop, 2018. [7] Xu. Ren, Ma. Li, \"Joint Enhancement and Denoising Method via Sequential Decomposi- tion\", 2018 IEEE International Symposium on Circuits and Systems (ISCAS), 2018. [8] Li, M., Liu, J., Yang, W., Sun, X., & Guo, Z. (2018). Structure-Revealing Low-Light Image Enhancement Via Robust Retinex Model. IEEE Transactions on Image Processing, 27(6), 2828–2841. [9] G. G. Roy, “Image Visibility Clarification and Enhancement,” January 2017. [10] Guo, X.; Li, Y.; Ling, H. LIME: Low-light image enhancement via illumination map esti- mation. IEEE Trans. Image Process. 2017, 26, 982–993. [11] Z. Ying, Ge Li, “A New Low-Light Image Enhancement Algorithm Using Camera Response Model” 2017. [12] Z. Ying, Ge Li, Y. Ren, \"Chapter 4 A New Image Contrast Enhancement Algorithm Using Exposure Fusion Framework\", Springer Nature, 2017. [13] X. Fu, D. Zeng “A Fusion-based Enhancing Method for Weakly Illuminated Images,” 2016. [14] Q. Wang, Xu. Fu, “A Fusion-based Method for Single Backlit Image Enhancement” IEEE International Conference on Image Processing, 2016. [15] Xu. Fu, D. Zeng, “A Weighted Variational Model for Simultaneous Reflectance and Illumi- nation Estimation” IEEE Conference on Computer Vision and Pattern Recognition,2016. [16] X. Fu, D. Zeng, H. Yue, Y. Liao, X. Ding, and J. Paisley, “A fusion-based enhancing method for weakly illuminated images,” Signal Processing, vol. 129, pp. 82–96, 2016. [17] Qiuhong Wang, Xueyang Fu, Xiao-Ping Zhang, Xinghao Ding. \"A fusion-based method for single backlit image enhancement\" 2016 IEEE International Conference on Image Pro- cessing (ICIP), 2016 [18] B. Giang, G. Woodell \"Multi-Scale Retinex with Color Restoration on Graphics Processing Unit,” 2015. [19] H. Ying, Ge Li, and Wen Gao, “A Bio-Inspired Multi-Exposure Fusion Framework for Low- light Image Enhancement”,2015. [20] Yue huang, “A probabilistic method for image enhancement with simultaneous illumination and reflectance estimation, ” august 2015. [21] Jing Zhang, Yang Cao, Zengfu Wang. \"Nighttime haze removal based on a new imaging model\", 2014 IEEE International Conference on Image Processing (ICIP), 2014. [22] Xu. Fu, Ye Sun, “A novel Retinex based approach for image Enhancement with Illumination Adjustment,” 2014. [23] Xu. Fu, Ye Sun, “A variational framework for single low light image enhancement using bright channel prior,” 2013. [24] Whang S, Zhang J, (2013) “Naturalness Preserved Enhancement Algorithm for Non- Uni- form Illumination Images”. [25] C. Lee, Ch. Lee. “Contrast Enhancement Based on Layered Difference Representation of 2D Histograms\", IEEE Transactions on Image Processing, 2013. [26] T. Celik, T. Tjahjadi, “Contextual and Variational Contrast Enhancement,” 2011. [27] J He, K Sun, X Tang, “Single image haze removal using dark channel prior to” IEEE Trans- actions on Pattern Analysis and Machine Intelligence. 2341–2353 (2011).

Copyright

Copyright © 2022 Aashi Singh Bhadouria, Khushboo Agarwal. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.