Literature Review on Vision-based Traffic Sign Detection and Recognition using Artificial Intelligence

Abstract

REFERENCES [1] Rajib Baran Roy, Md. Rokonuzzaman , Nowshad Amin “A Comparative Performance Analysis of ANN Algorithms for MPPT Energy Harvesting in Solar PV System” IEE Vol. 9, Issue 3 June 2021. [2] Catalina Gonzolez, Castano Carlos, “MPPT Algorithm based on Artificial Bee Colony for PV system” ,IEEE, Vol. 9, Issue 4, Feb 2021. [3] Jerome Cros, Rajib Baran “Maximum Power Tracking by Neural Network” , in Proc. ICRITO IEEE, Vol. 8, p. 257, June 2020. [4] Sabir Messati, Abd Ghani Harg \" New Neural Networks MPPT controller for PV “ Proc. 2nd IREC, Vol 2 , September 2015. [5] Zarko Zecevic , Maja Rolevski “Neural Network Approach to MPPT Control and Irradiance Estimation” Applied Science MDPI, Vol 10, Issue 15, July 2020. [6] Kian Jazayeri, Moein Jazayeri and Sener Uysal, “Comparative Analysis of Levenberg-Marquardt and Bayesian Regularization in photovoltaic power estimation using Artificial Neural Network”, Springer, LNAI 9728, pp 80-95, 2016. [7] R. Algarín, , D. R. Leal and D. S. Hernández “A low-cost maximum power point tracking system based on neural network inverse model controller”, Electronics, vol. 7, p. 244, Jan. 2018. [8] Brigitte Hauke, “Basic Calculation of a Boost Converter’s Power Stage”, Texas Instruments, US, pp. 1-7, Jan 2014. [9] L. Chen and X. Wang, ``Enhanced MPPT method based on ANN-assisted sequential Monte-Carlo and quickest change detection\'\', IET Smart Grid, vol. 2, Issue. 4, September 2019. [10] Henery P. Gavin “ The Levenberg-Marquardt algorithm for Non linear square Curve Fitting Problems” 3rd Edition, Duke University, 2018. [11] Ahmed Gundogdu, Resat Celikel “ANN Based MPPT Algorithm for Photovoltaic Systems, Turkish Journal of Science and Technology, Vol. 15, Issue 2, pp. 101-110, Aug 2020. [12] Samer Gowid, Ahmed Massoud, “A Robust Experimental-based Artificial Neural Network Approach for Photovoltaic Maximum PowerPoint Identification considering Electrical, Thermal and Meteorological Impact. Vol 15, Issue 9, pp 3699-3707, June 2020. [13] M. A. Elgendy, D. J. Atkinson “Experimental investigation of the incremental conductance maximum power point tracking algorithm at high perturbation rates,\'\' IET Renewable. Power Gener., vol. 10, no. 2, pp. 133-139, 2016. [14] Prof. L. Umanand ,“Design of photovoltaic System”. NPTEL , IISC Bangalore, 2018. [15] Jyothi Laxmi P.N “ An Artificial Neural Network Based MPPT Algorithm For PV System”, 4th ICEES, IEEE , PP 237-245 , Feb 2018. [16] Sasmita Nayak, Dr. Neeraj Kumar, “Scaled Conjugate Gradient Back propagation Algorithm for Selection of Industrial Robots”, IJCA, Vol. 7, Issue, 6, Nov 2017. With the expansion in the number of vehicles on the road, the development of technology, and other factors, safety has elevated to the top of automotive industries. Following traffic laws and regulations are essential in Advanced Driver Assistance Systems (ADAS) to prevent collisions. Over the past few decades, traffic sign recognition has gained significant interest due to its quick processing speed and the reduction of human interactions, which lowers risk with the assistance of Artificial Intelligence. While trying to focus on the road, drivers sometimes miss signage on the side of the road, which could be vulnerable for both them and the pedestrians. If the driver could be alerted effectively without having to change their focus, this issue may be avoided. When a sign is detected and recognised, the Traffic Sign Detection and Recognition (TSDR) system alerts the driver to any approaching signs. In addition to ensuring road safety, this also gives drivers a little more comfort while negotiating challenging or unfamiliar roads. This paper provides a comprehensive survey on traffic sign detection and recognition with details of algorithms, architectures, and methods implemented. Based on the investigation of several experts, this literature study was completed and may be used as a future reference.

Introduction

I. INTRODUCTION

According to the Ministry of Road Transport and Highways (MORTH) Government of India, the report states that road accidents stood at 1,51,417 in 2018 accounting for 11% of total death in India [8]. This is because the number of automobiles on the road nowadays is increasing exponentially due to travel, transportation, services, etc., and they have become an indispensable part of our daily routines. Rush-hour traffic makes it possible for people to miss traffic signs, and occasionally people do this on purpose (by speeding, for instance), which can result in fatal collisions. This may be dealt with by using autonomous cars, which minimize or even eliminates traffic accidents. Visual signals, such as traffic lanes and traffic signs, provide drivers with the most important information about the restrictions and state of the roads across every country. Traffic signs are an essential component of the road infrastructure since they offer diverse information on the condition of the roads, warnings, restrictions, and other valuable information.

1. Generally, Traffic signs are categorized into three groups: Mandatory/Regulatory signs
2. Warning/Cautionary signs
3. Informative/Guide signs

These signs are designed in a way to stand distinct in the form of colours, and shapes and are easily recognized by the human eye. However, due to the complexity of driving situations, passing vehicles, structures, and plants along the roadside frequently interfere with the detection and recognition of traffic signs. To properly judge the next driving manoeuvre, neither the driver nor the vehicle vision system can accurately interpret the meaning of traffic signs. Because of this, researchers have developed various techniques for real-time traffic sign recognition.

II. ARCHITECTURE OF TSDR SYSTEM

The two modules for traffic sign detection and recognition are detection and recognition. The image is initially taken at a specific distance and pre-processed to remove extra noise and data. To locate the traffic signs, the picture is subjected to colour transformation and shape analysis in the detection module. The recognition step receives the output from the preceding module and extracts the Region of Interest (ROI) and categorises it using the system's database. The output of this module starts with a text message or audio alert telling the driver what the traffic sign means and instructing them to react appropriately.

The different techniques of approach in the TSDR (Traffic sign Detection and Recognition) system are shown in Fig.5.  In this paper Recognition based on SVM and CNN is illustrated as this is mainly focused on these two techniques because of their reliability and robustness and training of large datasets. These two algorithms are the primary emphasis of this work due to their dependability, robustness, and capacity to train big datasets [6].

III. DIFFERENT TECHNIQUES

A. Support Vector Machine (SVM)

The fundamental task of the SVM is to find a hyperplane in an n-dimensional space that divides the data points into their respective classes. The data points and the hyperplane should be as wide apart as is practical. Maximizing the margin distance adds some support, increasing the confidence with which future data points may be categorised. Support vectors refer to the data points that are closest to the hyperplane. Due to their proximity to one another, they have a bigger influence on the precise location of the hyperplane than other data points.

C. Convolutional Neural Networks (CNN)

A form of Deep Learning technology, convolutional neural networks can take in an input image, give significance (learnable weights and biases) to distinct components and objects in the image, and be able to distinguish between them. Convolutional neural networks are highly good at identifying the patterns in the input picture, such as lines, gradients, circles, or even eyes and faces. This characteristic makes convolutional neural networks very useful for computer vision. In contrast to more traditional computer vision techniques, convolutional neural networks may start working on a raw picture right away and do not need any prior processing. Convolutional neural networks' strength comes from the convolutional layer, a special kind of layer [10].

It typically has three layers: input, hidden, and output. The input layer is made up of the input image's pixel matrix. Convolutional layers are frequently followed by activation layers, some of which are then followed by pooling layers, as the hidden layers. The number of layers in the concealed layer depends on how complicated the problem is. The output layer, which is a fully linked layer, flattens and sends the input from the preceding levels to turn the output into the necessary number of classes for the network.

IV. DATA BASE

The creation of any TSR system requires the use of the traffic sign database. It is employed for developing, training, testing, assessing, and validating models. Large numbers of traffic signs representing all categories with intricate scenarios and imagery may be found in the database. Table 1 lists some of the widely used datasets that are freely accessible to the scientific community [6].

TABLE I

Widely Used Traffic Sign Databases

V. RESEARCH TRENDS ON TRAFFIC SIGN DETECTION AND RECOGNITION

Recently, there has been an increase in interest in creating effective and trustworthy TSDR systems. To find papers published in journals that are indexed in the Scopus database and to limit the search to the last ten years, a basic search for "traffic sign detection and identification" has been conducted to demonstrate the present level of scientific study on this issue (2009-2018). Figure 3's Citation Trends illustrate the comparatively quick growth rate, with citations increasing by 60.34% between 2016 and 2018 [6].

VI.  LITERATURE SURVEY

A. T. Guofeng, et al. (ICIEA, 2017). “Traffic Sign Recognition Based on SVM and Convolutional Neural Network”. [1]

The technique in this research was broken down into three stages: a) HSV colour space conversion b) Localizing traffic signs using HOG and SVM; c) classifying using enhanced CNN. Conversion to HSV colour space is the first step since this colour model performs better under varying lighting conditions and is more in line with how humans understand data [2]. By adjusting the threshold values of the H and S components, the red, yellow, and blue are transformed into binary pictures. This makes it easier to obtain ROIs for the image by eliminating extraneous noise.

The HOG feature vector of the ROI is extracted in the following phase and sent to the SVM classifier, which will determine if it is a traffic sign. In this case, they employed a histogram of nine-bin-oriented gradients, which implies that the gradient direction of the cell, which is 360o, is divided into nine orientations. HOG of the cell is obtained by utilising weights of gradient direction in the histogram. Later, each cell unit is joined in series to form a large, continuous block to get a 9-dimensional feature vector of the corresponding cell. By doing this, the HOG feature vector of ROI is derived, which further categorises it as a probable traffic sign. In the last phase, the author suggests classifying data using an upgraded CNN model and provides four layers: Input, Convolution, Pooling, and Classifier.

The CNN receives an RGB image matrix as its input rather than a grayscale value of the pixel since doing otherwise would waste data. GTSDB and GTSRB are the datasets utilised in this study for the training and testing of 43 types of traffic signs. The experimental is suggested for indication signs, prohibition signs, and warning signs. The results obtained were 94.33%, 97.44%, and 96.47%, respectively. The proposed algorithm's overall accuracy rate is 99.21%, with an average picture processing time of fewer than 10 milliseconds.

B. A. Vennelakanti, et al. (ICCE, 2019), “Traffic sign detection and recognition using a CNN Ensemble”. [2]

In this study, the author suggested two-stage systems—detection and recognition—which were analysed and tested using publicly accessible Belgian Dataset and German dataset. Because Hue Saturation Value (HSV) processes images similar to how humans do and because it has a wide spectrum of colours, it is used in the detection phase. The following phase is locating the prospective traffic sign based on colour and determining its shape. Red signs are carefully reviewed in colour-based identification, and if they meet a particular threshold, that area is checked to see if they are traffic signs. The author concentrated on two shapes, namely the circle and triangle, for shape-based detection. It makes use of the Douglas-Peucker algorithms. This algorithm counts the edges and locates contour regions; if they meet the minimal requirement, bounding boxes are generated to demarcate the intended region. By using image thresholding and inversion filter, this sign is verified and moved on to the next phase. In the recognition phase, the author used the CNN feed-forward neural network which consists of six convolutional layers and has dropout layers in between the fully connected hidden layers to prevent overfitting. They employed a trio of CNNs in an ensemble to increase accuracy. Each of these CNNs is trained using weights that are initialised at random. The outputs of each of these three CNNs are eventually averaged to create a single ensemble model. To train this model effectively and save training time, it would be beneficial to train all these models simultaneously. Overall train sets for triangles and circles were 10582 and 16106 images. Similarly, the test sets were 3456 and 5277 images respectively When put to the test, they scored 98.11% accuracy for triangles and 99.18% accuracy for circles

C. Sudha S. K, et al. (IJERT, 2016). “A Real-Time System for Detection and Recognition of Traffic Signs”. [3]

The author presented the technique in this research in two stages, Detection and Recognition. It contains layers for image capture, pre-processing, detection, and classification, the author further clarifies. In the detection phase, 720p images are recorded from the moving car's roof at a predetermined frames-per-second rate. The visibility of the traffic sign is impacted by variations in lighting and weather. To reduce noise, the Weiner filter and Gaussian noise are utilised. Then, RGB is converted to YCbCr, where Y stands for chrominance information, the difference between the blue component and a reference value is represented by Cb, the difference between the red component and a reference value is represented by Cr. Red and blue colour objects are retrieved using the threshold approach, and all stray pixels are separated using an adjusted morphological process. In this case, the author concentrated mostly on four shapes: triangle, rectangle, circle, and octagon. The candidate object is chosen using a shape-based analysis. If there is a discrepancy in similarity between the colour-segmented sign and the template database greater than the threshold value, the sign is recognised as a traffic sign A pictogram threshold is used in the sign recognition stage. All pixels are divided into white and black pixels, depending on whether they are above or below the threshold value. Traffic signs are therefore retrieved from the pictograms inside and used for classification using neural networks. For classification, the model employs a multilayer feed-forward CNN, and the network applies a backpropagation learning technique to train multiple layers. The primary function of the classification module is to assign the appropriate category of traffic signs to the extracted regions of interest that are supplied as input.

D. Hasan, et al. (ICICCT, 2020). “Traffic Sign Recognition System (TSRS): SVM and Convolutional neural Network”. [4]

The author of this study explains in detail how to localise traffic signs using SVM and classify them using CNN. The real-time dataset was initially built by cropping the region around the traffic signs in the video frames. To propose CNN and SVM models, they split the entire dataset into training and validation, each with 1200 images. They have considered 12 different traffic signs and each class has 100 images. Grayscale images are created by first applying a colour transformation to RGB images.

The Region of Interest (ROI) is more competently refined in the second step based on the management of shape. The next stage is feature extraction using the HOG descriptor to train the SVM classifier. Here they rescaled the image to 32x32 pixels and employed a 9-bin histogram for the HOG feature vector. The SVM model will validate that it is a traffic sign.  If a traffic sign is present, it will be processed further by the CNN model, which will classify it and label the sign using a bounding box. Convolutional, pooling, and fully connected layers are listed by the author as the CNN model's layers. The Rectified linear activation function was employed in this study to train the CNN model since it outperforms the competition in a variety of scenarios. They achieved 96.40% validation accuracy in the CNN model and 98.33% accuracy for SVM classification during the training procedure with 80:20 data split ratio. They created a system with real-time video and tested it to assess the outcomes.

E. M. Swathi, et al. (RTEICT, 2017), “Automatic Traffic sign Detection and recognition in Video Sequences”. [5]

The detection and recognition stages of the system are divided into two fundamental categories by the author of this work. The detection step is separated into three sections: colour reformation from RGB to HSV, colour segmentation, and shape recognition. To separate the chromatic and achromatic components and make them resistant to changes in illumination, the image frame is first retrieved from the video sequence using ROI and transformed to HSV colour space. By removing the distracting backdrops, the search area in the image sequences is reduced and the traffic sign is segregated. The following phase was using the Douglass-Peucker method algorithm, a contour approximation technique frequently used for Shape identification. The author primarily focused on two shapes, triangles and circles. If the total number of lines obtained is 3 and 8 in the final curve, then it is categorized as triangle and circles respectively. Because it is based on lines and curves, it functions well for traffic signals with certain geometric aberrations. With the HOG feature extracted, a multi-layer feed-forward neural network is used to classify traffic signs during the recognition phase. The recognized traffic sign is rescaled to 24x20 pixels and by gradient computation HOG feature vector of length 1080 is obtained. The input, hidden, and output layers of a multi-layer perceptron design each include a set of nodes that are coupled to the preceding node with certain weights. The activation function in this design is nonlinear, except for the input layer. The sigmoid function described in (4) is the activation function utilised in this design.

By comparing the output error with the predicted outcomes using the backpropagation technique, the weights are modified each time the data is analysed. Seven circular and thirteen triangular classes out of a total of 21 classes were trained, with 70 photos for each class considered. The detection rate of the experiment's real-time data, which was recorded at three separate times during the day (morning, afternoon, and evening), was 82.94%, 90%, and 89.41%. They attained 97.14% and 95.71% accuracy for circular and triangular, respectively, at an average frame rate of 15 to 26 frames per second.

TABLE II
Reflection Note Of Literature Survey

VII. ACKNOWLEDGMENT

Researchers would like to thank their colleagues for their excellent contributions and discussions on the above-mentioned topics.

Conclusion

The main objective of this study is to categorize the prime approaches into sections to make the concept easily understandable and to give a comprehensive review of different methods available for traffic sign detection and recognition; including along with database and research trends associated with TSDR systems [6]. In this paper, the detection of traffic signs using colour and shape based on different techniques was explored. This study suggests that support vector machine and convolutional neural network-based methods were found most efficient and used because they have a high rate of detection, are very flexible, and are simple to adopt. [2] With the aid of new datasets from other nations, the total performance might be enhanced and tailored.

References

[1] T. Guofeng, C. Huairong, L. Yong and Z. Kai, \"Traffic sign recognition based on SVM and convolutional neural network,\" 2017 12th IEEE Conference on Industrial Electronics and Applications (ICIEA), 2017, pp. 2066-2071. [2] A. Vennelakanti, S. Shreya, R. Rajendran, D. Sarkar, D. Muddegowda and P. Hanagal, \"Traffic Sign Detection and Recognition using a CNN Ensemble,\" 2019 IEEE International Conference on Consumer Electronics (ICCE), 2019, pp. 1-4. [3] Sudha S. K., Mahathi L, 2016, A Real-Time System for Detection and Recognition of Traffic Signs, INTERNATIONAL JOURNAL OF ENGINEERING RESEARCH & TECHNOLOGY (IJERT) NCETET – 2016 (Volume 4 – Issue 17) [4] Hasan, N.; Anzum, T.; Jahan, N. Traf?c Sign Recognition System (TSRS): SVM and Convolutional Neural Network. In Inventive Communication and Computational Technologies (ICICCT); Springer: Singapore, 2021; pp. 69–79 [5] M. Swathi and K. V. Suresh, \"Automatic traffic sign detection and recognition in video sequences,\" 2017 2nd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT), 2017, pp. 476-480, doir: 10.1109/RTEICT.2017.8256642. [6] Wali, Safat B., Majid A. Abdullah, Mahammad A. Hannan, Aini Hussain, Salina A. Samad, Pin J. Ker, and Muhamad Bin Mansor. 2019. \"Vision-Based Traffic Sign Detection and Recognition Systems: Current Trends and Challenges\" Sensors 19, no. 9: 2093. [7] Glory Reuben Maxwell and Dr.Dinesh D.Patil, “A review on traffic sign detection and recognition system”, International Research Journal of Engineering and Technology (IRJET) – 2020 (Volume 7 – Issue 05) [8] https://morth.nic.in/sites/default/files/Road_Accidednts.pdf [9] S. Maldonado-Bascon, S. Lafuente-Arroyo, P. Gil-Jimenez, H. Gomez-Moreno and F. Lopez-Ferreras, \"Road-Sign Detection and Recognition Based on Support Vector Machines,\" in IEEE Transactions on Intelligent Transportation Systems, vol. 8, no. 2, pp. 264-278, June 2007. [10] Y. Sun, P. Ge and D. Liu, \"Traffic Sign Detection and Recognition Based on Convolutional Neural Network,\" 2019 Chinese Automation Congress (CAC), 2019, pp. 2851-2854.

Copyright

Copyright © 2022 Rajesh V, Rajashekar J. S.. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.