Efficient Deep CNN-Based Fire Detection and Localization in Machine Learning Video Surveillance Applications

Abstract

Early warning is important to reduce loss of life and various industries due to fire. Accidents caused by undetected fires cost the world a lot of money. The demand for effective fire alarm systems is growing. Existing fire and smoke detectors fail due to system inefficiency. Analysis of live camera data enables real-time fire detection. The properties of the fire flame are examined and the fire is recognized using edge detection and thresholding methods, resulting in a fire detected model. Detects dangerous fires identified based on size, speed, volume and structure. In this paper, we propose an emerging fire detection system based on a convolutional neural network. Experimental results of the model on our dataset show that it has good fire detection ability and real-time multi-level fire detection ability.

Introduction

1. I. INTRODUCTION

Fire,  especially  fire  in  buildings,  can  spread  quickly  and cause  great  loss  of  life  and  property.  Therefore,  early  fire detection  and  warning  is  imperative.  Fire  detectors,  smoke detectors and temperature detectors have  been  widely  used  to protect  property  and  give  warning  of  fires.  However,  smoke and temperature detection is slower than light detection, which is  the  substantive  detection  method  proposed  in  this  paper. Furthermore, to cover the entire area potentially subject to fire, many smoke or temperature fire detectors are required. Fire,  especially  fire  in  buildings,  can  spread  quickly  and cause  great  loss  of  life  and  property.  Therefore,  early  fire detection  and  warning  is  imperative.  Fire  detectors,  smoke detectors and temperature detectors have  been  widely  used  to protect  property  and  give  warning  of  fires.  However,  smoke and temperature detection is slower than light detection, which is  the  substantive  detection  method  proposed  in  this  paper. Furthermore, to cover the entire area potentially subject to fire, many smoke or temperature fire detectors are required. Fire,  especially  fire  in  buildings,  can  spread  quickly  and cause  great  loss  of  life  and  property.  Therefore,  early  fire detection  and  warning  is  imperative.  Fire  detectors,  smoke detectors and temperature detectors have  been  widely  used  to protect  property  and  give  warning  of  fires.  However,  smoke and temperature detection is slower than light detection, which is  the  substantive  detection  method  proposed  in  this  paper. Furthermore, to cover the entire area potentially subject to fire, many smoke or temperature fire detectors are required. Fire,  especially  fire  in  buildings,  can  spread  quickly  and cause  great  loss  of  life  and  property.  Therefore,  early  fire detection  and  warning  is  imperative.  Fire  detectors,  smoke detectors and temperature detectors have  been  widely  used  to protect  property  and  give  warning  of  fires.  However,  smoke and temperature detection is slower than light detection, which is  the  substantive  detection  method  proposed  in  this  paper.  Furthermore, to cover the entire area potentially subject to fire, many smoke or temperature fire detectors are required. Fire,  especially  fire  in  buildings,  can  spread  quickly  and cause  great  loss  of  life  and  property.  Therefore,  early  fire detection  and  warning  is  imperative.  Fire  detectors,  smoke detectors and temperature detectors have  been  widely  used  to protect  property  and  give  warning  of  fires.  However,  smoke and temperature detection is slower than light detection, which  is  the  substantive  detection  method  proposed  in  this  paper.  Furthermore, to cover the entire area potentially subject to fire, many smoke or temperature fire detectors are required. Fire,  especially  fire  in  buildings,  can  spread  quickly  and cause  great  loss  of  life  and  property.  Therefore,  early  fire detection  and  warning  is  imperative.  Fire  detectors,  smoke detectors and temperature detectors have  been  widely  used  to protect  property  and  give  warning  of  fires.  However,  smoke  and temperature detection is slower than light detection, which is  the  substantive  detection  method  proposed  in  this  paper. Furthermore, to cover the entire area potentially subject to fire, many smoke or temperature fire detectors are required. Fire,  especially  fire  in  buildings,  can  spread  quickly  and cause  great  loss  of  life  and  property.  Therefore,  early  fire detection  and  warning  is  imperative. 

Fire  detectors,  smoke detectors and temperature detectors have  been  widely  used  to protect  property  and  give  warning  of  fires.  However,  smoke and temperature detection is slower than light detection, which is  the  substantive  detection  method  proposed  in  this  paper. Furthermore, to cover the entire area potentially subject to fire, many smoke or temperature fire detectors are required. Fire,  especially  fire  in  buildings,  can  spread  quickly  and cause  great  loss  of  life  and  property.  Therefore,  early  fire detection  and  warning  is  imperative.  Fire  detectors,  smoke detectors and temperature detectors have  been  widely  used  to protect  property  and  give  warning  of  fires.  However,  smoke and temperature detection is slower than light detection, which is  the  substantive  detection  method  proposed  in  this  paper. Furthermore, to cover the entire area potentially subject to fire, many smoke or temperature fire detectors are required.

Convolutional Neural Networks (CNNs) have brought state-of-the-art performance in image classification and other computer vision tasks. Their application in fire detection systems will substantially improve detection accuracy, ultimately minimizing fires and reducing environmental and social impacts. However, the main problem for CNN-based fire detection systems is their implementation in real surveillance networks due to their high memory and computation requirements for inference. In this paper, we propose an original, energy-efficient, and computationally efficient CNN architecture, inspired by the Squeeze Net architecture, for fire scene detection, localization, and semantic understanding. It uses smaller convolutional kernels and contains no dense, fully connected layers, which helps keep computational requirements to a minimum.

Despite the low computational requirements, experimental results demonstrate that our proposed solution achieves an accuracy comparable to other, more complex models, mainly due to its greater depth. In addition, this paper shows how a trade-off between fire detection accuracy and efficiency can be achieved by considering the specific characteristics of the problem of interest and different fire data.

Fires became more common as society developed and people became more reliant on fire. The most dangerous thing about the rapid growth of the fire is that it directly affects people's lives as well as human-made structures. When it comes to data on this, forests destroyed by fire are major contributors to the Earth's increasing temperature. In China alone, a large number of people died in 2019 as a result of indoor fires. These results highlight the seriousness of this problem and the importance of fire detection.

In the past few years, fire detection methods have relied heavily on sensors such as weather and smoke sensors, however this approach can miss a distant flame. Fire detection methods using video applications are in demand in recent years with the advent of deep learning. a high dome environment or a fire in its early stages.

Object detection is visually and widely used to alert the necessary management to the open environment as soon as possible. Different kinds of fire detection-related technologies, such as vision-dependent technology, are in demand, according to researchers, due to limitations in previously designed fire detection systems, as well as advances in visual effect-dependent technologies. processed objects. Fire detection using vision technology provides three key advantages over traditional fire detection methods: accessible, controllable and immediate. The cameras ensure that the spaces in which we monitor will be accessible by a visual detection system that allows to prevent the guards from checking the status in real time. As a result, the ability to control is evident in the movements that have been collected and made available after the fire hazard has occurred. Finally, the immediacy of the fire is ensured by the low computational cost and efficient method.

II. LITARATURE REVIEW

The authors of the article [1] established a benchmark using the YOLOv4 protocol, developed a fire detection model using the Darknet deep learning framework. They used a large number of self-created fire datasets for training and testing in the experiment, including multilevel ones. Our model was found to outperform others in fire detection. This research based on convolutional neural networks developed a better fire detection algorithm YOLOv4 (CNN). They use an automated fire detection dataset to increase model accuracy, a modified loss function to improve small-scale flame detection, and a combination of Soft-NMS to improve the suppressive effect of a box that is redundantly bounded and has a lower low recall rate.

The authors of the article [2] studied some problems related to fire detection in forest areas, fire detection at early moments and false detection. They used traditional detection using object-based approaches for the first time in forest fire detection. Real-time features are enhanced by SSD, greater fire detection accuracy and the potential to detect fires earlier. To reduce the number of false alarms, they created a system using smoke and fire and used newly introduced modifications. Meanwhile, they tweak the tiny yolo YOLO structure and design a new structure. Experiments show that tiny-yolo-voc1 improves the fire detection accuracy rate. The document was really useful for forest safety and monitoring.

The authors of the article [3] proposed a support vector machine image technique for fire detection. To begin with, an inter-frame difference approach is used to detect a motion region, which is then considered a fire suspect region. The likely fire area is then resampled using a scene categorization algorithm. The texture and color moment features of the suspected fire area are then extracted. Finally, a trained support vector machine for the machine learning model is used to input these eigenvalues ??as eigenvectors for fire and non-fire classification and recognition. Experimental findings revealed that the proposed approach in this work can eliminate the drawback of artificially setting the flame characteristic threshold and has higher accuracy than the traditional flame identification algorithm. However, there are some phenomena that have been overlooked or misinterpreted. This is what the algorithm does.

The authors of the paper [4] used a new fire detection approach that uses gas leak concentration to predict explosions and fires, which was previously known as fire predictor and fire appearance detector. Fire detection is one of the functions of a smart home. Early detection of fire in the house is a crucial step in preventing a large-scale fire and saving many things. The fire predictor simply displays the concentration of the gas leak and triggers the alarm. The classification of fire detectors is performed using a fuzzy approach. Data can be sent to the MFC from the output simulation system, but not understood by the MFC reader in real time.

The authors of the paper [5] studied on the Raspberry Pi hardware the circumstances and the Raspberry Pi software, they were able to compensate for the shortcomings of standard fire detectors to improve the reliability of fire deep use paper YOLO v3 is a modest local video fire identification system in transit that uses a light technique of direct regression detection. RPI Fire obtains a high level of excellence, the speed of recalling video testing and fire simulation, which is able to meet the requirements of shipboard fire detection.

The authors of [9] propose a fire recognition approach based on the centroid diversity of consecutive image frames, which is based on the flickering characteristic of the flame. This method has the advantages of simplicity, speed and versatility compared to the standard flame recognition method. It is able to eliminate a wide range of non-fire sources of interference. This type of fire detection technology, which uses a standard camera, has many practical uses.

The authors of the article [10] proposed to use detailed information films of forest fires in different situations to assess an aerial approach to forest fire identification. To improve detection speed, moving features are first identified, which are later scaled to highlight the burning area. Second, the smoke is obtained using the technique we designed. In real-world application of forest fire detection, our system demonstrates its robustness with a high degree of accuracy. using optical flow to calculate motion vectors, then DBSCAN clustering that represents existing motion pattern models. Similarity and criteria of entropy classes result in fitting models of movement patterns. This helps in detecting movements with certain irregularities in the traffic.

III.  ALGORITHM

A. Convolution Neural Networks (CNNs)

In deep learning, a convolutional neural network (CNN or ConvNet) is a class of artificial neural network most commonly used to analyze visual images. They are also known as sliding invariant or spatially invariant artificial neural networks (SIANNs), based on a weight-shared architecture of convolutional kernels or filters that slide along the input features and provide translational equivariant responses known as feature maps. Counterintuitively, most convolutional neural networks are only equivariant, as opposed to invariant, to translation. They have applications in image and video recognition, recommender systems, image classification, image segmentation, medical image analysis, brain-computer natural language processing interfaces, and financial time series.

IV.  RESEARCH WORK

This study investigated the performance of their fire detection model by examining a large database of different scene conditions. It shows that the delicacy of the model was 93.97 generations, while the false alarms and failures were 7.08 generations and 6.86 generations, respectively. This study proposes the next phase of bank discovery as a strategy to improve the performance of the fire detection model by removing a thick bank that covers almost the entire fire. The bank detection phase is created using both the color and blending characteristics of the bank pixels. This stage is shown to be effective in fire area detection as the fire pixels are segmented using both the color and mixing characteristics of the bank.

Originally, fire-like areas were detected using the RGB-HIS model. The movement of the center of gravity of these areas was calculated using videotape shadows and a new fire detection methodology was developed. Test results show that this method is effective. The proposed system has a certain practical significance for indoor fire detection, as a number of experiments demonstrate its ability to exclude the influence of common interferences and trigger fire warnings in time.

Basically, they trained and tested their system on our own dataset, which consists of images taken from the internet and manually labeled to represent fire and banks. Compared to the performance of state-of-the-art infrastructure-based styles, our systems performed better and were more complex. Experimental results based on real-world data show that the performance and complexity of our system are excellent.

A. Outline

The steps listed below comprise the working model of our system are:

1. Collecting footage and converting it into useful information/data
2. Data acquisition and analysis
3. Decision making to check fire activity
4. Trigger the alert system via registered mail

B. Capturing Video And Turning It Into Actionable Information

Video is captured using an IP camera that is connected to a remote organization. After filming, the video is processed and divided into visual outlines. Machine learning models are fed these image outlines. During data cleaning, a lexicon of unique terms that occur in all photo captions in the dataset is created and stored on disk. Informing. Corresponding data is then passed on to the next stage.

In the above figure, we get complete information of how image capture to the how image detected and how we get alert message                                                 

1. Preprocessing: In this method, we collect all the raw data from the image capture frame and perform an operation on it called preprocessing. For example: A training algorithm i.e. CNN on the photo it captured results in poor classification results.
2. Feature Extraction: A CNN is a neural network that extracts image attributes from an input image and classifies them using another neural network. A feature extraction network works with the input image. The neural network uses the extracted feature signals.
3. Segmentation: R-CNN (Regions with CNN feature) is a real-world application of regional approaches. Based on the results of object detection, it performs semantic segmentation. R-CNN uses selective search to extract a large number of object ideas before computing CNN characteristics for each one.

C. Data Flow Diagram

In this diagram we can see how data is used in each step, first it is taken as input, then some operation is done on it before processing and then we apply CNN algorithm on this data. then we can detect fire using this data in this process data can be converted to other format like video to image.

V. OUTPUT RESULTS AFTER PROCESSING EACH FRAME ONE BY ONE:

1. When Normal Event Detected (No Fire)

A. Preprocessing of Dataset

1. Dataset: There are a limited number of fire videos available on the Internet, as well as a limited selection of situations, flame sizes, and video quality. As a result, we developed a high-quality fire dataset of 20,000 images that we used to train and test the model to improve its generalization and robustness. Using the original public fire dataset, we found many high-quality multidimensional fire images on the Internet. We also used flame movies to capture video in a number of test scenarios to increase the model's detection rate of small fires and performance in complicated environments. We used the labelImg tool to label our hand-crafted fire dataset. Compared with the crowd sourced tagging fire dataset, we will try to unify the tagging criteria and achieve the highest quality of flame tagging to improve the accuracy of our model. Our hand-built fire dataset contains a total of 20,000 fire photos. The training set contains 16,800 fire photographs, while the test set contains 3,200 fire photographs. The fire dataset contains various scenes, from simple to complex backgrounds, from small to large, from single object to multiple objects, indoor to outdoor environment, and day and night lighting, which ensures that the improved YOLOv4 model proposed in this paper has good generalization ability.
2. Training Details: The data argument in this experiment is based on Mosaic and Cutmix technology. Our fire detection model was also trained on two Nvidia GTX 1080Ti GPUs. In our training, we used the SGD optimizer with the following settings: input 608*608, batch size 128, maximum epochs 60,000, learning rate 0.001, momentum 0.949, and weight decay rate 0.0005. For this frequency, it is reduced to 0.0001 after 40,000 iterations; after 55,000 iterations, this rate is further reset to 0.00001. The training took approximately four days to complete.
3. Results: To evaluate the fire detection effect of our method for identifying multidimensional (especially small) flames in various complicated settings, we tested our trained fire detection model on several real and experimental fire movies. We tested our trained fire detection model on several real and experimental fire videos to verify the fire detection effect of our method for detecting multidimensional (especially small) flames in various complex scenes. Our own fire detection dataset was based on the scale of the flame to match the aspect ratio of the image, and we tested our trained fire detection model on several real and experimental fire videos to verify the fire detection effect of our method for detecting multi-level (especially small) flames in various complex scenes. Flame samples are available in small, medium, large and extra large sizes. Some forecast results are shown below.

Conclusion

This paper proposed a new network architecture based on the YOLOv4 protocol and used the Darknet deep learning framework to develop a fire detection model. In the experiment, we used a large number of self-created fire datasets, including A multi-level system is used for training and evaluation. Our approach outperforms others in terms of fire detection, according to the data. Finally, an aerial wildfire identification system was evaluated using a large database of wildfire movies under different scene conditions. First, the chromatic and motion features of the forest fire are extracted to increase the detection rate, and then the rule is used to align them and mark the fire area. Second, to solve the problem, we use our proposed technique to obtain smoke.

References

[1] AAA lkhatib, “Smart and Low Cost Technique for Forest Fire Detection using Wireless Sensor Network,” Int. J. Comput. Appl., vol. 81, no. 11, pp. 12–18, 2013. [2] J. Zhang, W. Li, Z. Yin, S. Liu, and X. Guo, “Forest fire detection system based on wireless sensor network,” 2009 4th IEEE Conf. Ind. Electron. Appl. ICIEA 2009, pp. 520–523, 2009. [3] A. A. A. Alkhatib, “A review on forest fire detection techniques,” Int. J. Dis- trib. Sens. Netw., vol. 2014, no. March, 2014. [4] P. Skorput, S. Mandzuka, and H. Vojvodic, “The use of Unmanned Aerial Ve- hicles for forest fire monitoring,” in 2016 International Symposium ELMAR, 2016, pp. 93–96. [5] F. Afghah, A. Razi, J. Chakareski, and J. Ashdown, Wildfire Monitoring in Remote Areas using Autonomous Unmanned Aerial Vehicles. 2019. [6] Hanh Dang-Ngoc and Hieu Nguyen-Trung, “Evaluation of Forest Fire De- tection Model using Video captured by UAVs,” presented at the 2019 19th International Symposium on Communications and Information Technologies (ISCIT), 2019, pp. 513–518. [7] C. Kao and S. Chang, “An Intelligent Real-Time Fire-Detection Method Based on Video Processing,” IEEE 37th Annu. 2003 Int. Carnahan Conf. OnSecurity Technol. 2003 Proc., 2003. [8] N. I. Binti Zaidi, N. A. A. Binti Lokman, M. R. Bin Daud, H. Achmad, and K. A. Chia, “Fire recognition using RGB and YCbCr color space,” ARPN J. Eng. Appl. Sci., vol. 10, no. 21, pp. 9786–9790, 2015. [9] C. E. Premal and S. S. Vinsley, “Image Processing Based Forest Fire Detection using YCbCr Colour Model,” Int. Conf. Circuit Power Comput. Technol. ICCPCT, vol. 2, pp. 87–95, 2014. [10] C. Ha, U. Hwang, G. Jeon, J. Cho, and J. Jeong, “Vision-based fire detection algorithm using optical flow,” Proc. - 2012 6th Int. Conf. Complex Intell. Softw. Intensive Syst. CISIS 2012, pp. 526–530, 2012. [11] K. Poobalan and S. Liew, “Fire Detection Algorithm Using Image Processing Techniques,” Proceeding 3rd Int. Conf. Artif. Intell. Comput. Sci., no. December, pp. 12–13, 2015. [12] Innovative Mechanisms for Industry Applications (ICIMIA) (pp. 335-339). IEEE.

Copyright

Copyright © 2022 Kartik Jadhav, Ganesh Gaikwad, Shubham Wagh, Adesh Navthar, Prof. Suryawanshi A. S.. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.