Satellite Image Dehazing

Abstract

The images captured during haze, murkiness and raw weather has serious degradation in them. Image dehazing of a single image is a problematic affair. While already-in-use systems depend on high-quality images, some Computer Vision applications, such self-driving cars and image restoration, typically use input from data that is of poor quality.. This paper proposes a deep CNN model based on dehazing algorithm using U-NET, dynamic U-NET and Generative Adversarial Networks (CycleGANs). CycleGAN is a method that comprehends automatic training of image-to-image transformation without associated examples. To train the model network, we use SIH dataset as the training set. The superior performance is accomplished using appreciably small dataset, the corresponding outcomes confirm the adaptability and strength of the model.

Introduction

I. INTRODUCTION

Images captured from satellite and outdoor settings can go through limpidness depletion mainly due to atmospheric scattering brought about by clouds, haze and to some extent by air pollution. Several Image Dehazing Algorithms have been put out to enhance visibility of photographs taken in foggy settings. As a result, image dehazing is increasingly preferred in the domains of computational photography and computer vision.

Depending on variances in image denoising concepts, current methods may be broadly separated into three groups: image improvement-based approaches, picture merging and blending-based methods, and image patching and rebuilding-based strategies. Approaches centred on image improvement disregard the basis of image deterioration. Methods based on image merging and blending improve data from various sources without the need for a physical model.According on how well each of the aforementioned strategies really works, they may all be further divided into a number of subcategories The removal of undesirable visual effects is the theory behind picture dehazing, which is typically considered an image-enhancing technique.

II. RELATED WORK

Image dehazing and image processing is an interesting field to base research on. Many projects have been implemented and papers have been published under this field. Review of some of those papers are as follows.

1. In this study, an unique method for removing haze degradations from RGB photographs under stacking conditions is proposed. The hazy images serve as the conditioned entrance for the suggested GAN architecture, which trains to remove the haze it will be employed to get crisp images. This conceptualization guarantees a quick and homogeneous generalisation of the model. Experiments revealed that the suggested procedure produces crisp, high-quality photographs. aims to use a layered conditional to reduce haze degradations in RGB photos. Network of Generative Adversaries (GAN). The hazy photos from which the clear images will be obtained serve as the conditioned entrance for described GAN architectural structure , which understands to eliminate the haze. Numerous methods for enhancing image quality have been developed, including those that focus specifically on removing fog.
2. Due to its long wavelength, near-infrared (NIR) light may penetrate more deeply than visible light and is thus less likely to be dispersed by airborne particles. Because of this, it is desirable for picture dehazing to bring out features of far-off objects in landscape photos. In this paper, the author suggested an improved picture dehazing system that efficiently predicts the air, light, colour, and transfers information from the NIR utilising a pair of coloured and NIR images. In this article, various haze reduction techniques were analysed. Numerous computer graphics and vision applications struggle because haze makes the scene less visible. Haze removal techniques are used in many applications, including object identification, surveillance, consumer electronics, etc. The key picture dehazing approaches that have been developed during the past ten years are reviewed in this study. The method first applies the Directed filter to enhance the image quality, which is a simple to execute per-pixel alteration. With the exception of a continuous region devoid of edge information, we may utilise the spread map of the picture to determine the intensity of the image. Studies on hazy photographs that had been collected reveal that our technique outperforms existing image dehazing algorithms in terms of recovering details and distributing colours.
3. The article presents a contrasting examination of various image dehazing methods. Many researchers were able to clear the haze from their images by taking a set of pictures. These techniques make an effort to capture multiple scene-images with the goal of learning the various aspects of the image. Separate images can help to effectively capture details that are relevant to haze because they have different characteristics.Multiple photos are sometimes necessary for feature extraction, though. Since a single input image contains so little information, it can be difficult to extract important hazy properties from it. Numerous studies tried to remove the image blur. There are numerous single picture dehazing approaches that produced dehazed outputs that are appealing to the eye and have less complexity overall. Handling this difficulty in computer vision is essential in order to extract photos for informational purposes from the obscured areas. The primary goal of picture denoising methods is to provide results with the right amount of noise and appropriate visible objects. This document provides a concise discussion of many widely used image dehazing techniques. There are plenty of applications for image dehazing. Numerous methods have been lodged by scholars to clear distortion from an image and create better photographs. The study employs solutions based on existing understanding as well as on learning
4. It is a common practise to assess DHAs using hazy images created from reference photographs with no haze. The evaluation criteria employed in the current literature include FR IQA measures. However, no research has been done to determine whether FR IQA This paper has done study on this strategy. The building of a SHRQ (synthetic haze removal quality) database is explained. It is divided into two subsets: a conventional subgroup of a picture and an overhead picture, respectively, each of which contains 360 and 240 dehazed solutions that were produced from 45 and 30 artificially hazy photographs employing eight standard DHAs. The database is subject to a user study. It has been highlighted that some distinctions exist between FR DHA testing and FR IQA, making modern FR IQA measures inappropriate for the purpose of FR DHA testing. In order to address these troubles, researchers have looked at usual deviations brought on by dehazing and suggested a FR DHA testing technique that considers the restoration of the image's original framework, colour rendition, and amplification of low-contrast areas excessively. To accommodate for the distinctions between FR DHA and FR IQA analyses, we alter fundamental structural elements of the dehazed picture. By considering its special characteristics, aerial picture dehazing is enhanced, and the distinctions between dehazing regular and aerial photos are also explored. The suggested method's efficacy on both segments of the SHRQ dataset is validated, and all critical features have a role in the technique as a whole.Evaluation of DHAs using artificial hazy photos is a potential approach because real hazy images cannot be used as accurate quantitative metrics for DHA assessment.
5. In this paper, we suggest HR-Dehazer, a brand-new and precise image dehazing technique. An encoder-decoder neural network is constructed in order to learn a direct mapping between a hazy image and its corresponding clear form. In order to encourage consistency among local structures and push the network to take into consideration the semantics of the input image, we created a particular loss. This loss also increases the system's scalability invariance. Results in numbers for the recently made available Dense-Haze dataset. First, an examination of the impact of haze on various colour spaces is made. The suggested encoder-decoder architecture is then discussed, followed by the presentation of the loss functions and the training process. Then, the HR-quantitative Dehazer's and qualitative findings are contrasted with those of cutting-edge techniques. When evaluating dehazing algorithms on paired synthetic photos, the recovered hazy images are compared to the ground truth, which is an unhazed version of the image.
6. This work provides a architecture build on deep learning for picture restoration coupled with single image dehazing. As opposed to learning a mapping between pair of photos with haze  and its matching haze-free image, as most existing techniques do, we propose recasting the The detail component can be improved further by distinguishing the foundation and detail elements of the blurry image. By creating a CNN (convolutional neural network) specifically for modelling between base components with and without haze, haze can be reduced. The base with less haze and the elements with more detail are combined to create the final dehazed image. Three processes make up the proposed deep learning-based single picture dehazing architecture: (1) image decomposition-based pre-processing; (2) CNN-based dehazing; and (3) creation of the final picture after image denoising using the clean base image element in combination with the higher detail image element .
7. Dehazing principles can vary depending on the three different types of existing methods— based on picture augmentation, based on the fusion of images and the reconstruction of images. Image enhancement methods typically utilise specialised picture processing techniques to enhance the contrast and details as well as the aspects of the image, without putting into consideration the reasons the image has degraded. The required data is improved using picture fusion-based techniques without the need for a physical model from numerous origin connections to ultimately produce a high-quality image. By examining the physical mechanics of optical imaging, methods based on image restoration create a model for the deterioration of foggy images, reverse the processes, and correct any distortion brought on by these processes to provide clear, haze-free images. Histogram equalisation methods The Retinex technique and frequency domain enhancement are addressed under the title "Image enhancement-based approaches." Under the heading of image fusion-based methodologies, techniques such as blending of Images from Multiple Spectrums and Fusion with a unary Image are investigated. The Degradation Model and based on Atmospheric Diffracted Mathematical Models are the final two models covered in picture restoration-based methods.
8. The Attention-Unet network is presented in this research for the evaluation of ambient light. To solve the parameter invariability issue brought on by the challenge of self-learning for U-Net, a system that the attention gate developed a technique that uses implicit training to remove the unnecessary ambient light from the input picture and It is necessary to draw attention to the area's noteworthy features that have a substantial influence on ambient light. While atmospheric light is forecasted using an attention U-net, the transmission diagram is estimated using a pyramid dense network estimation network.The structure of this paper is one of networks. A joint discriminator was included after ambient light and transmission diagram estimates to train the GAN. Because ATTENtion-UNET was used instead of Unet, the author has concluded that the dehazing effect is more similar to the actual haze-free outcome.
9. An overview of deep CNN and deep GAN, their fundamental ideas, GAN versions, and applications for computer-aided understanding of visuals are discussed in this work. A contrast among biological and computer vision is made in order to better learn the development of neural networks and the history of computer vision. It offers a comprehensive comparison of contemporary and earlier surveys. The study looks at dropout/augmentation issues, different deep learning methods, and deep generative adversarial networks and deep convolutional neural networks used in computer vision applications. Additionally, quick fixes are discussed.  The findings indicate that using dropout and data augmentation significantly improves accuracy. Applications of deep convolutional neural networks, such as speech recognition, document analysis, and location and detection, are covered in detail. Deep generative adversarial network applications, such as face ageing, image denoising, and facial attribute manipulation, are thoroughly examined.
10. In this study, the authors present a dehazing algorithm based on fusion that does not directly estimate the transmitter map. The transition latent pictures are obtained by bringing the scenario back to radiance depending on the atmospheric scattering concept, a range of globally constant transmission values. This technique only properly removes haze from each image at the area where the specific broadcast accurately depicts the scene. These images are combined into a clear result using the fusion technique, which only uses the data from the patches in the latent pictures. Without further information, it may be challenging to gauge the complexity of the outside scene and an inaccurate depth estimate could lead to a bad dehazed picture.

V. IMPLEMENTATION

A. Basic definitions

Clouds must be eliminated in order to recover the ground image B from a single input satellite image S. The proposed cloud reduction strategy is split into two parts:

1. U-Net(U): In order to segment clouds, a satellite image is split into a background image with missing portions caused by dense clouds and a cloud image.
2. GMAN: to fill in these blank spaces and produces a cloud-free image with a uniform appearance.In Stage I, we use the U-Net to segment thin clouds.  The U-Net is frequently used in image-to-image translation jobs and is effective at segmenting neural structures in electron microscopic stacks. The U-Net design, which can be trained with very few images, consists

B. Proposed system

The U-Net can successfully segment neural structures in electron microscopic stacks and is commonly utilised in image-to-image translation tasks. A contracting path is used to gather context, and an expanding path with symmetry is used to provide exact localization, in the U-Net architecture, which can be trained with relatively few pictures. Using the well-known atmospheric scattering model from, we mix ground and cloud images to produce hazy shots. With the help of these images, a U-Net is trained to distinguish between synthetic images of the ground and clouds. As a consequence, human-labelled input data are not needed for the training process. Images of the ground will be designated as blank in places with heavy clouds and given to Stage II. We apply a GAN to recover lost areas with irregular shapes in Stage II due to its improved generating capabilities. A discriminator D and a generator G, both of which are DNNs, make up a GAN. The discriminator D wants to increase the likelihood that it correctly labels training examples and samples from G, as opposed to the discriminator G, which wants to increase the likelihood that it correctly recognises training examples and samples from G. The generator G fills the voids by using the statistical distribution of the training data instead of duplicating the surrounding pixels into the empty pixels.Depending on the situation, several down- and up-sampling approaches may be utilised in pyramidal form networks to improve segmentation accuracy. In the U-Net design, we investigate several levels of down- and up-sampling procedures to identify a suitable lightweight network topology for segmenting hand bones. The image depicts the first U-Net (U-Net4) that we used. The provided image is 256*256 in size. After each of the two 3 * 3 convolutions, the signal is downsampled using a rectified linear unit (ReLU) and a 2*2 max pooling operation with stride 2. (unpadded convolutions).

The feature map must be upsampled for each step of the expanding path, followed by a 2*2 convolution (also known as a "upconvolution") that reduces the number of feature channels in half, two 3*3 convolutions that are concatenated and each followed by a ReLU, and the similarly cropped feature map from the contracting path.For the identical hand bone segmentation job, we tested several U-Net designs with varying numbers of down- and upsampling operations figure and U-NetCC, and we discovered that the U-Net2 can offer the best results..

A system in which two networks compete with one another to perform better is known as an adversarial model. An innovative form of adversarial process used to produce new data is called generative adversarial networks (GAN). Two networks make up the framework: a discriminator and a generator. A classifier known as the discriminator network, D, has been trained to discriminate between input pictures that are either produced by the generator G or taken straight from the data set. The goal of D's conventional CNN supervised training is to lower error rates while classifying "fakes" as genuine pictures from a data set. D returns the likelihood that a picture was made by G for each image input. Images from the generator G are regularly sent to the discriminator. When the discriminator is given a fake image to aid in the development of convincing pictures, the discriminator's gradient function is a function of the generator's gradient function. This enables the generator to alter its weight in response to the discriminator's output. To give the generated images more diversity, random noise is also added to the generator. The discriminator must be fooled as much as possible by the generator in order to enhance its mistake rate. Since the discriminator will always return a probability of 1/2 regardless of whether the picture originates from the data set or the generator, the adversarial network is no longer able to distinguish between real photos and counterfeit ones created by the generator.The generated visuals may then be artificially produced using the constructed generator. Figure displays the basic relationship diagram of a GAN. By changing the labels and input photos, this framework may be further expanded to force the GAN to produce just a small range of synthetic images. A conditional GAN is the name for this modification. Conditional GANs have a subclass called adversarial U-nets. While the discriminator is updated, the generator network is built using the U-net design. The U-net design allows the generator to accept a picture as input instead of random noise. The primary distinction between adversarial U-nets and earlier systems is that the latter's generator tries to modify existing images rather than producing new ones. This output of G is compared to D, which was trained on manually altered photos. The generator should be able to provide the same level of change once it receives the necessary command.

The goal of network D is to classify all inputs from networks x and G as genuine or counterfeit in accordance. G prefers that its results be regarded as accurate.

C. Comparison with other Systems

Despite the availability of a number of deep learning segmentation models, in this part we will give a brief introduction to some of the most well-liked U-net alternatives, including FCN, Segnet, FPN, and DeepLab. Fully convolutional networks were one of the first deep learning models for semantic segmentation (FCN). FCNs offer a fully segmented image using a single upsampling layer and predictable downsampling paths to get contextual information. Optional skip connections are also included in FCNs, however because of the way they are built, the skipped gradients frequently have different lengths and need extra processing to be upscaled. Each area still receives the proper style, which is applied as necessary.

FCNs' incapacity to gather knowledge about the state of the world is among their main flaws. In the end, FCNs outperform other cutting-edge segmentation models. Segnet is a different encoder-decoder paradigm that was developed after U-net. Skip connections are not used by Segnet to communicate low-level contextual information to deeper layers, though. The main benefit of Segnet is that it has less training data than other segmentation algorithms. In order to recognise objects, feature pyramid networks (FPN) were originally developed utilising an encoder-decoder structure. Similar to U-net, skip connections are used to concatenate gradient information for the decoder. In contrast to U-net, the decoder also passes gradient data from each layer to a following set of convolution layers.FPNs are extremely helpful for creating multi-class segmentation maps since they are made to recognise items from all decoder tiers. Another well-liked segmentation technique that makes use of excited spatial pyramid pooling is DeepLab. DeepLab models can use spatial pyramid pooling to accept input of various sizes. Thanks to atrous-CNN or dilated convolution, the layer may collect contextual input from a larger area without raising the filter size. Combining these two methods can significantly increase DeepLab models' resilience without significantly increasing processing complexity.

VI. FUTURE SCOPE

This project can be further developed and enhanced by implementing more such algorithms and comparing the results for better accuracy. The proposed model can be advanced by adding a Dark Channel Prior (DCP) algorithm which can be used to analyse a range of outdoor images and then find dark primary colour for approximation of haze concentration.

Conclusion

The proposed model was evaluated using the testing dataset. An image dehazing framework by encoder – decoder of U-net architecture employing customized convolution sub-pixel convolution and multiplier is implemented.

References

[1] Suarez, Patricia L., Angel D. Sappa, Boris X. Vintimilla, and Riad I. Hammoud. “Deep Learning Based Single Image Dehazing.” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2018. doi:10.1109/CVPRW.2018.00162. [2] Feng, Chen, Shaojie Zhuo, Xiaopeng Zhang, Liang Shen, and Sabine Susstrunk. “Near-Infrared Guided Colour Image Dehazing.” 2013 IEEE International Conference on Image Processing, 2013. doi:10.1109/ICIP.2013.6738487. [3] Anil Singh Parihar, Yash Kumar Gupta, Yash Singodia, Vibhu Singh, Kavinder Singh, \"A Comparative Study of Image Dehazing Algorithms\" 2020 Proceedings of the Fifth International Conference on Communication and Electronics Systems (ICCES 2020) IEEE Conference Record # 48766; IEEE Xplore ISBN: 978-1-7281-5371-1 [4] Xiongkuo Min, Guangtao Zhai, Ke Gu, Yucheng Zhu, Jiantao Zhou, Guodong Guo, Xiaokang Yang, Xinping Guan and Wenjun Zhang, \"Quality Evaluation of Image Dehazing Methods Using Synthetic Hazy Images\", IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 21, NO. 9, SEPTEMBER 2019, pp.2319-2333, DOI: 10.1109/TMM.2019.2902097 [5] S. Bianco, L. Celona, F. Piccoli and R. Schettini, \"High-Resolution Single Image Dehazing Using Encoder-Decoder Architecture,\" 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019, pp. 1927-1935, doi: 10.1109/CVPRW.2019.00244. [6] Yeh, C.-H., Huang, C.-H., Kang, L.-W., & Lin, M.-H. (2018). Single Image Dehazing via Deep Learning-based Image Restoration. 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). doi:10.23919/apsipa.2018.8659733 [7] W. Wang and X. Yuan, \"Recent advances in image dehazing,\" in IEEE/CAA Journal of Automatica Sinica, vol. 4, no. 3, pp. 410-436, 2017, doi: 10.1109/JAS.2017.7510532. [8] Y. Cui, J. Wu, Y. Chen and C. Cai, \"A Comparative Study of Image Dehazing Based on Attention-Net and U-Net Atmospheric Light Estimation,\" 2020 IEEE 6th International Conference on Computer and Communications (ICCC), 2020, pp. 378-382, doi: 10.1109/ICCC51575.2020.9344955. [9] Abirami, R. Nandhini, P. M. Durairaj Vincent, Kathiravan Srinivasan, Usman Tariq and Chuan-Yu Chang. “Deep CNN and Deep GAN in Computational Visual Perception-Driven Image Analysis.” Complex. 2021 (2021): 5541134:1-5541134:30. [10] Hong, Soonyoung, Minsub Kim, and Moon Gi Kang. \"Single image dehazing via atmospheric scattering model-based image fusion.\" Signal Processing 178 (2021): 107798. [11] K. Vayadande, T. Adsare, T. Dharmik, N. Agrawal, A. Patil and S. Zod, \"Cyclone Intensity Estimation on INSAT 3D IR Imagery Using Deep Learning,\" 2023 International Conference on Innovative Data Communication Technologies and Application (ICIDCA), Uttarakhand, India, 2023, pp. 592-599, doi: 10.1109/ICIDCA56705.2023.10099964. [12] K. Vayadande et al., \"Heart Disease Prediction using Machine Learning and Deep Learning Algorithms,\" 2022 International Conference on Computational Intelligence and Sustainable Engineering Solutions (CISES), Greater Noida, India, 2022, pp. 393-401, doi: 10.1109/CISES54857.2022.9844406. [13] Kuldeep Vayadande, Aditya Bodhankar, Ajinkya Mahajan,Diksha Prasad, Shivani Mahajan, Aishwarya Pujari and Riya Dhakalkar (2022). Classification of Depression on social media using Distant Supervision, ITM Web Conf. 50. [14] Samruddhi Mumbare, Kunal Shivam, Priyanka Lokhande, Samruddhi Zaware, Vara Deshpande and Kuldeep Vayadande,” Software Controller using Hand Gestures”, ITM Web Conf. Volume 50, 2022. [15] K. Vayadande, V. Ingale, V. Verma, A. Yeole, S. Zawar and Z. Jamadar, \"Ocular Disease Recognition using Deep Learning,\" 2022 International Conference on Signal and Information Processing (IConSIP), Pune, India, 2022, pp. 1-7, doi: 10.1109/ICoNSIP49665.2022.10007470. [16] K. Vayadande, T. Adsare, T. Dharmik, N. Agrawal, A. Patil and S. Zod, \"Cyclone Intensity Estimation on INSAT 3D IR Imagery Using Deep Learning,\" 2023 International Conference on Innovative Data Communication Technologies and Application (ICIDCA), Uttarakhand, India, 2023, pp. 592-599, doi: 10.1109/ICIDCA56705.2023.10099964. [17] Vayadande, K. B., & Karande, N. D. Automatic Detection and Correction of Software Faults: A Review Paper. International Journal for Research in Applied Science & Engineering Technology (IJRASET) ISSN, 2321-9653.

Copyright

Copyright © 2023 Aditya Yeole, Jay Wanjare, Saket Waware, Yash Wagh, Yuvraj Susatkar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.