Traffic Sign Detection and Recognition using Image Processing

Abstract

Now a days, automobiles became most convenient mode of transportation for everyone. As we know one of the most important functions, TSDR has become a popular research . It primarily involves the use of vehicle cameras to collect real- time road pictures and then recognize and identify traffic signs seen on the road, therefore delivering correct data to the driving system. With the advancement of science and technology, an increasing number of scholars are turning to deep learning technology to save time in traditional processes. From the training samples, this model can learn the deep features inside the autonomously. The accuracy and great efficiency of detection and identification are the subject of this essay. A deep convolution neural network algorithm is proposed to train traffic sign training sets using Caffe[3], an open-source framework, in order to obtain a model that can classify traffic signs and learn and identify the most critical of these traffic sign features, in order to achieve the goal of identifying traffic signs in the real world.

Introduction

I. INTRODUCTION

In road traffic networks, traffic signs play an essential role. The main purpose of traffic signs is to display the contents that need to be noticed in the current road sections, to warn drivers ahead of the road of the danger and difficulty in the environment, to remind drivers to drive at the prescribed speed, and to provide a favourable guarantee for safe driving. As a result, traffic sign recognition and identification is an essential research direction that is critical for preventing road traffic accidents  and protecting drivers' personal safety. Despite the fact that the number of people killed or seriously injured in street accidents has decreased over the last three decades, there has been an increase in street traffic. This shows that, even though our streets are busier than at any other time in recent memory, they are safer thanks to fundamental advancements in car design, such as enhanced fold zones and side effect bars. Away innovation, such as safety belts, airbags, and electronically monitored slowing devices, can also be assigned to this. The two primary kinds of road traffic signs are main signs and auxiliary signs. Warning signs, prohibition signs, required signs, guiding signs, tourist signs, and road construction and safety signs are among the most common types of signage. Prohibition signs were the most common, and they were used to prohibit 43 different types of conduct. Near the junction, mandatory signs highlight the function of vehicles that are put in the necessity to signal vehicles. The purpose of warnings is to inform drivers, vehicles, and pedestrians to be aware of harmful targets, which are divided into  45 categories. In traffic signs, they all play a vital function. The most prevalent speed limit signs, as well as the restriction of  left and right turning signals, are extremely important for safe driving and are hence the subject of current traffic sign recognition study. The originality of the design of traffic signs is one of the excellent aspects of TSDR; hues often contrast well against the earth, the signs are carefully placed in relation to the earth, and they are frequently placed in an unmistakable sight to the driver. TSDR, on the other hand, still has a number of flaws. We can discern the opposing points of view: The conditions of lightning are unpredictable and uncontrollable. The time and season, climate conditions, and neighboring light variations, such as heading of light, all influence the appearance of lightning.

The goal of this article is to create a more accurate and robust system for automatically detecting and recognizing traffic signals in a variety of challenging conditions and disruptions.

II. METHODOLOGY

A. DATASET

Although the convolution neural network has made significant progress, there are currently few applications in the field of traffic sign identification and recognition. The absence of traffic sign data sets is one of the main reasons behind this. A considerable quantity of traffic sign data is required to train and verify a deep convolution neural network traffic sign recognition model. However, in comparison to other emerging nations, India has a scarcity of accessible traffic sign datasets. GTSRB in Germany, GTSDB in Germany, and KUL in Belgium are currently some of the most well-known traffic sign databases.

The GTSRB and GTSDB traffic sign databases are utilised in this work to identify and recognise traffic signs. Many forms of complicated traffic signs, such as sign tilt, uneven illumination, traffic signs with distraction, occlusion, and similar backdrop colors, as well as genuine scene maps, are included in these two datasets. The algorithm's abilities was tested using a range of complicated and difficult-to-distinguish traffic signals.

B. Proposed Method

The goal of this project is to provide a way for segmenting traffic signs from background images using various approaches such as filtering and edge detection. The YCBCR conversion method is utilized to compare the detected picture to the datasets, and feature selection extraction is used to extract the correct symbol from the compared images. As illustrated in Figure 1, feature extraction is divided into three steps.

III. FILTERING BY CNN

The Gabor filter is a linear texture analysis filter that examines if the picture contains any certain frequency content in specific directions in a limited region surrounding the point or region of examination. Many modern vision experts believe that Gabor filter frequency and orientation representations are comparable to those of the human visual system. They've been discovered to be very useful for texture representation and discriminating.

1. These filters have been demonstrated to have excellent localization capabilities in both the spatial and frequency domains, making them ideal for texture segmentation. A Gabor filter is a sinusoidal signal modified by a Gaussian wave with a certain frequency and direction.
2. When a Gabor filter is applied to an image, it gives the highest response at edges and at points where texture changes. When we say that a filter responds to a particular feature, we mean that the filter has a distinguishing value at the spatial location of that feature.
3. A bank of Gabor filters with a variety of orientations is used to evaluate texture or extract features from a picture.

IV. DETECTING EDGES

When an edge detector is applied to an image, it can provide a collection of linked curves that show object borders, surface marking boundaries, and curves that correspond to surface orientation discontinuities. Applying an edge detection method to a picture can minimize the quantity of data that has to be processed and therefore filter out information that isn't as vital while retaining the image's crucial structural features. If the edge detection stage is successful, the job of understanding the information content in the original image may be significantly streamlined.

V. DWT FEATURE SEGMENTATION

Segmentation is the process of dividing a picture into areas that are consistent under certain conditions. The objective of segmentation is to make a picture more intelligible or simpler to interpret by simplifying or changing its representation. For a monochrome image, the essential feature for colour segmentation is image luminance amplitude, and for a colour image, colour components. Segmentation can also benefit from image form and texture. Discrete Wavelet Transformation is used in this segmentation. The technique of segmenting a digital picture into several parts is known as image segmentation (sets of pixels also known as image objects). The objective of segmentation is to simplify and transform an image's representation into something more relevant and simpler to understand. The technique of giving a label to each pixel in a picture such that pixels with the same label share certain visual properties is known as image segmentation. Detection and recognition are the two basic stages of traffic sign identification. Pre-handling, including extraction, and division are all parts that can be identified during the detection stage. To finally recognize the traffic signals, a full chain of image processing procedures is required. Pre-preparing, which may include a few actions, is the first step in the detecting process. These assignments correct a picture that has been distorted by noise, haze, out-of- center haze, bending caused by low goals, and so on.

VI. FEATURE EXTRACTION

The system's specific characteristic is that it uses the colour, size, and orientation of the segmented picture to recognise the traffic sign from the backdrop image. Then, using correlation, quality of the segmented picture, homogeneity, variance, standard deviation, and accuracy, this sign is compared to the datasets, and the mean is calculated. If the input to an algorithm is too vast to process or if there is a lot of data but not much relevant information, the input will be reduced to a smaller collection of features. Feature extraction is the name for this transformation. Its goal is to find the optimal number of characteristics to adequately characterise the data without sacrificing accuracy. A feature space is the collection of all conceivable features.

VII. RESULT

Every stage of the procedure, such as edge detection and segmentation in MATLAB, displays the outcome. The final result is then displayed in a notice pop-up on the screen to indicate the modifications that the car must make for safe driving. This warning will be similar to advice to choose a different route or something similar in order to drive safely without causing any accidents or delays.

All the other models or TSDR detect the images of traffic sign but it may give different results as we know there are different weather conditions in which it is hard to detect it , so we are using Data Augmentation in which it will automatically detect the images no matter what is the weather condition or even if the image is left shifted , right shifted, or rotated it will detect the image On the issue of traffic sign detection and recognition, there are a variety of approaches that may be used. In this sector, statistical approaches appear to be restricted, thus there has been a lot of study to discover more reliable ways. Finally, the difficulty of the problem-specific job determines the method and methodology to be used. Finding the appropriate settings for the various  approaches may be a time-consuming task. More research into the three highlighted approaches for traffic sign detection and identification is simply possible. As a consequence, we may use a combination of approaches to get significantly better outcomes.

VIII. OUR UI

As we can see that the UI is looking good and it is predicting the same output.

Conclusion

We presented a traffic sign identification and recognition system for recognizing the sign in a given picture captured by a car camera in this article. Detection, segmentation, and classification are the three phases of image analysis. The edge of the sign is identified, and segmentation is used to separate it from the backdrop. We present a simple feature selection technique using two data sets for traffic sign detection and recognition: GTSRB and GTSDB. Many forms of complicated traffic signs, including sign tilt, uneven illumination, traffic signs with distraction, occlusion, and similar backdrop colors, as well as actual scene maps, are included in these datasets. On the issue of traffic sign detection and recognition, there are a number of approaches that may be used. In this sector, statistical approaches appear to be restricted, thus much study has been done to develop more precise ways. Finally, the choice of a method and the application of a methodology is determined by the intricacy of the situation at hand. Finding the correct settings for the various approaches may be a time-consuming task. More research into the three focused techniques of traffic sign detection and identification may simply be done. As a consequence, we may combine approaches to get far better outcomes.

References

[1] Zhu Yingying, et al. “Traffic Sign Detection and Recognition using Fully Convolutional Network Guided Proposals”, Neurocomputing 214(2016):758-766. [2] Bedi, Rajni, et al. “Neural Network Based Smart Vision System for Driver Assistance in Extracting Traffic Signposts.” Cube International Information Technology Conference ACM, 2012:246-251. [3] Zhao Guofeng, “Traffic Sign Recognition Based on Machine Learning”, Hanghou University of Electronics Science and Technology, 2015.. [4] Peng Jinhang, “A Study on Convolutional Neural Networks for Traffic Sign Recognition,” Beiging Jiaotong University,2017. [5] Houben, Seastian, et al. “ Detection of Traffic Sign in Real world Images: The German Traffic Sign Detection Benchmark.” International Joint Conference on Neural Networks IEEE, 2013:1-8. [6] Shi, Guangming, et al. “ Visualization and Pruning of SSD with the Base Network VGG16.” International Conference on Deep Learning Technologies ACM, 2017:90-94. [7] Liu, Wei, et al. “SSD:Single Shhot MultiBox Detector.”(2015). [8] L. Li, W. Huang, I.Y. Gu, and Q. Tian, “Foreground object detection from videos containing complex background”, Proc. of the Eleventh ACM international Conference on multimedia, Berkeley, CA, USA, November 02.08.2003. [9] K.M.M. Rao, “Overview of Image Processing”, Processing of a workshop on image processing and pattern recognition, IETE, Hyderabad, 2526 September 2004. [10] “Digital Image Processing, 3rd International Conference on Neural Networks,” by Rafael C. Gonalez and Richard E. Woods. Ed. in 2008. [11] Z. Jinglei, L. Zhengguang, and T. Univ, “A Vision-based road surveillance system using improved background subtraction and region growing approach,” 8th ACIS Int. Conf. on software engineering, Artificial Intelligence, networking, and Parallel/Distributed Computing, Qingdao, August 2007, pp. 819-822. [12] M. Siyal and J. Ahmed, “A new morphological edge detection and window-based technique for real-time road data control and management,” Fifth IEEE International Conference on Information, Communications, and Signal Processing, Bangkok, July 2005, pp.324-328 [13] Y. Zhu, M. Liao, M. Yang and W. Liu, \"Cascaded Segmentation-Detection Networks for Text-Based Traffic Sign Detection,\" in IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 1, pp. 209-219, Jan. 2018. [14] Mermann K., Gool L. Van., Multi-view traffic sign detection, recognition, and 3D localisation, 2009 Workshop on Applications of Computer Vision WACV, (2009) 1–8 [15] . Belaroussi R., Foucher P., Tarel J.-P., Soheilian B., Charbonnier P., Paparoditis N., Road Sign Detection in Images: A Case Study 20th International Conference on Pattern Recognition (ICPR), (2010), 484–8.

Copyright

Copyright © 2022 Mr. Mohammad Shabbir Sheikh, Mr. Mohammad Mateen Manjothi, Mr. Om Patil, Mr. Sahil Upadhyay, Asst. Prof. Chriranjit Das. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.