Traffic Sign Detection and Recognition System for Autonomous Vehicle

Abstract

Traffic Sign Recognition and detection system is specifically useful for drivers and driver-less cars. By identifying traffic signs accurately and effectively it improves the driving safety of autonomous cars. This research paper presents a comprehensive approach for the development of a traffic sign detection system using Convolution Neural Network, TensorFlow and OpenCV to classify the traffic signs in real-time effectively. TensorFlow and OpenCV play an important role in shaping effective traffic detection and recognition system. We have explained the process of data collection, preparation, model architecture and the integration of TensorFlow for training and interference. OpenCV for image processing and real time feed processing, ensuring seamless implementation on various hardware platform. The model uses German Traffic Sign Recognition dataset. The results show that the proposed system achieves high accuracy in detecting and recognizing traffic sign making it a valuable system for both autonomous vehicles and human drivers.

Introduction

I. INTRODUCTION

Traffic sign detection and recognition system play a vital role in enhancing road safety and the efficiency of transportation systems. These systems are instrumental in aiding drivers and autonomous vehicles in understanding and obeying traffic regulations. These systems rely on the integration of image processing, deep learning algorithms, and computer vision techniques to interpret road signs and provide crucial information to drivers or autonomous vehicles. In this project, we aim to develop a robust traffic sign detection and recognition system using key technologies such as Supervised Learning, Image Processing, Deep Learning Algorithms, Convolution Neural Networks (CNN), OpenCV, and TensorFlow

  Traffic signs are vital elements of road infrastructure which provides important information and warnings to drivers. Accurate and real-time detection and recognition of these signs are crucial for ensuring safe and efficient navigation on roadways. The conventional methods of traffic sign recognition often faced challenges related to variations in illumination, sign occlusion, and complex backgrounds. However, the integration of deep learning techniques like CNNs has revolutionized the field, allowing for highly accurate and robust recognition.

Techniques of Deep learning, example CNN, have shown growth in different computer vision tasks, including image classification, object detection, and segmentation. CNNs can learn features directly from the input images, eliminating the need for handcrafted features. Moreover, CNNs can handle complex images and achieve high accuracy rates. In this paper, we delve into the key aspects of traffic sign detection using CNNs, including data collection and preprocessing, model architecture, training, and evaluation. By harnessing the power of CNNs, we aim to improve the accuracy, reliability, and efficiency of traffic sign recognition systems, thereby fostering safer and smarter transportation networks.

II. LITERATURE REVIEW

The author SHOUHUI HE, LEI CHENI, of the paper entitled “Automatic Recognition of Traffic Signs Based on Visual Inspection” [1] The paper describes the main contributions of the study, including the design of a method to extract candidate regions from the input image through content analysis and key information recognition, the development of an HOG method to prevent projection distortion in actual images.

Author YI YAN , CHAO DENG, JUNJIE MA “A Traffic Sign Recognition Method Under Complex Illumination Conditions” [2]One of the main challenges in traffic sign recognition under complex illumination conditions is the large variation in lighting conditions. Traditional methods, such as color-based segmentation, are sensitive to lighting changes and may not be effective in these conditions.

The Author in the paper “Detection And Classification Of Speed Limit Traffic Signs”[3] Rubel Biswas & Hasan Fleyeh discribes about the the properties of speed limit traffic signs and They note that the proposed system is highly accurate, with a recognition rate of 98%, and can aid in the development of Intelligent Speed Adaptation.

The report written by Pramod Sai Kondamari & Anudeep Itha “A Deep Learning Application for Traffic Sign Classification” [4]The authors note that TensorFlow has become one of the most popular deep learning frameworks due to its flexibility, scalability, and ease of use.

The use of ReLU activation functions in CNNs has been shown to improve the performance of the models. In the paper, “Convolutional Neural Network (CNN) for Image Detection and Recognition”[5] the authors Rahul Chauhan mention the use of ReLU in their CNN models for image recognition.The ReLU activation function is used in the hidden layers of the CNN models to introduce non-linearity and improve the accuracy of the predictions.

The Author Jiudong Yang & Jianping Li provides an overview of the applications of deep convolutional neural networks (CNNs) in various fields such as image processing, speech recognition, and natural language processing. [6] In natural language processing, the author notes that CNNs have been used for tasks such as sentiment analysis, text classification, and language translation. The author cites research that shows that CNNs have achieved state-of-the-art performance in sentiment analysis tasks, where they can classify text into positive or negative sentiment.

The paper “Traffic Sign Recognition for Intelligent Vehicle/Driver Assistance System Using Neural Network on OpenCV” [7] illustrates and presents a study on recognizing traffic sign patterns using Neural Networks technique, which can significantly increase driving safety and comfort.  a real implementation using an intelligent vehicle to reduce the search space and indicate only potential regions for increasing the efficiency and speed of the system.

It describe the image processing techniques used in their study, including pre-processing, enhancement, and segmentation of the acquired images. They also explain how they used multilayer perceptron (MLP) with back propagation learning algorithm to recognize traffic sign patterns[8].

III. PROPOSED SYSTEM

The proposed system in the paper is a cost-sensitive CNN model with four convolution layers in traffic sign classification using the German Traffic Sign Detection and Recognition dataset. The system consists of three main components: data preprocessing, model training, and model evaluation. TensorFlow can be used at various stages in the system architecture of a traffic sign detection and recognition system, from data preprocessing and model development to real-time inference and deployment in the field. It plays a critical role in building and deploying deep learning solutions for this application. In the data preprocessing stage, the input images are resized to a fixed size and normalized to improve the performance of the model. In the model training stage, the CNN model is trained using the preprocessed images and their corresponding labels. In the model evaluation stage, the trained model is evaluated on a test set of images to measure its accuracy and performance.

A. System Architecture

1. Input Image: Taking the image from camera using OpenCV which contains background along with traffic sign which is to be detected. TensorFlow can be employed to process video feeds from cameras mounted on vehicles or at intersections. The model can continuously analyze the video stream and detect and recognize traffic signs in real-time.
2. Pre-Processing: Preprocessing allows to eliminate unwanted distortions and improve specific qualities. Its purpose is to raise the images quality so that we can analyze it more effectively. The images are used further for model training inferences. TensorFlow can be used for data preprocessing tasks, such as resizing, normalizing, and augmenting the traffic sign images. This helps in preparing the data for training and evaluation.
3. Color Segmentation: Color segmentation is a method used in machine learning to detect traffic signs. It uses color information from high-resolution images to select regions of interest (ROIs). The color information is used to reduce the search space.
4. Shape Detection: Based on the area of the counters, the shape of the traffic sign is detected with the threshold parameters defined by the user. Further, an edge detection feature is employed in shape segmentation to count the total contours of an object. 
5. Feature Extraction: Feature extraction in traffic sign detection using CNN involves automatically learning and capturing relevant visual patterns, such as edges, textures, and shapes, from input images to enable accurate classification and detection of traffic signs.
6. Model Evaluation: After training, TensorFlow is used to evaluate the performance of the trained models on validation and test datasets. Common evaluation metrics include accuracy, precision, recall, and F1-score.
7. Model Deployment: TensorFlow provides various options for deploying models, including TensorFlow Serving for serving models in a production environment or converting models to TensorFlow Lite for mobile and embedded applications.
8. Output: If Traffic sign matches with the given dataset then we display Detected Sign.

IV. FUTURE SCOPE

To enhance models’ efficiency, we can expand dataset further and include various country’s traffic sign dataset to study there traffic conditions and diversity. We can use predictions to access various features and vast set of images. In this way we can expand use of autonomous self-driving cars using Traffic Sign Detection System.

Conclusion

Traffic sign detection and recognition model helps drivers and autonomous vehicle provides safe and efficient navigation. For detection, a good range of camera is used to detect sign at a distance to help system recognize sign accurately and efficiently within expected time. In this experiment we use CNN algorithm as it is suitable so far and fits more accurately with our goal for this project. CNN detects traffic sign efficiently analyzes signs as it ensures road safety for fully automated driving systems and help prevent accidents. Such a system leverages advanced technologies, including computer vision and deep learning, to detect and interpret traffic signs accurately and in real-time.

References

[1] SHOUHUI HE , LEI CHEN “Automatic Recognition of Traffic Signs Based on Visual Inspection”, “Linyi University, Linyi 276000, China”, date of publication February 12, 2021,HONG LIU China. [2] YI YAN 1 , CHAO DENG “A Traffic Sign Recognition Method Under Complex Illumination Conditions”, “School of Automobile and Traffic Engineering, Wuhan University of Science and Technology, Hubei, Wuhan 430065”,China date of publication 13 April 2023 [3] R. Biswas, H. Fleyeh, M. Mostakim, “Detection and Classification of Speed Limit Traffic Signs,” IEEE World Congress on Computer Applications and Information System, pp. 1-6, January 2014. G. Antipov, SA Berrani, JL Dugelay, “Minimalistic CNN-based ensemble model for gender prediction from face images,” Elsevier, January 2016. [4] Pramod Sai Kondamari & Anudeep Itha, “A Deep Learning Application for Traffic Sign Classification” Bachelor of Science in Computer Science May 2021. [5] Rahul Chauhan & Kamal Kumar Ghanshala, “Convolutional Neural Network (CNN) for Image Detection and Recognition” Graphic Era University Dehradun India, 2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC). [6] Jiudong Yang and Jianping Li “Application Of Deep Convolution Neural Network” International Centre for Wavelet Analysis and Its Applications, School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, Sichuan, China. [7] Auranuch Larsakul, Jackrit Suthakorn “Traffic Sign Recognition for Intelligent Vehicle/Driver Assistance System Using Neural Network On Open CV” Department of Biomedical Engineering, Mahidol University. [8] Mrs. P. Shopa, Mrs. N. Sumitha & Dr. P.S.K Patra “Traffic Sign Detection and Recognition Using OpenCV” Agni College of Technology Chennai, Tamil Nadu,India. 2014.

Copyright

Copyright © 2023 Pratush Jadoun , Nikita Wanve, Namrata Khnadagale , Megha Shinde , Preeti Yadav. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.