Road Structure Quality Classification using Reinforcement Learning

Abstract

Owing to the exponentially growing amount of transportation on the road, the number of mishaps on daily basis is also growing at a shocking rate. There is one death every 4 minutes due to road mishaps in India. One of the foremost causes of these road mishaps to occur is the poor road environment. In the present work, Reinforcement learning is used to classify road quality. Reinforcement learning is a part of machine learning. Suitable actions are taken to maximize the reward. Reinforcement learning is different from supervised learning, in reinforcement learning, there is no trained dataset but the agent learns from its experience and decides what actions should be performed for the given task in order to achieve the maximum reward. We use Q-learning a Reinforcement learning algorithm that will find the best action, in the given current state of an environment. It chooses the action randomly and aims to maximize the reward. An environment is created in which policy, action, and rewards to classify road structure quality are defined. The dataset has 15 features and 5 classes based on which policy is defined that defines the next action to be taken based on the state. Training and Testing are done with 80:20 ratios. Accuracy for q-learning is 93.62% and enhanced q-learning is 93.74% obtained. A confusion matrix with precision, F1 score, recall, and support is calculated. The accuracy obtained for different training and testing model ratios 60:40, 70:30, 80:20, and 90:10 is 93.19, 91.06, 91.51, and 91.86 respectively. The average accuracy for the training and testing model is 91.86.

Introduction

I. INTRODUCTION

Owing to the exponentially growing amount of transportation on the road, the number of mishaps on daily basis is also growing at a shocking rate. There is one death every 4 minutes due to road mishaps in India. One of the foremost causes of these road mishaps to occur is the poor road environment. The geometry of the road shows a major part in road crash incidences as well as the crash ruthlessness level. Diverse features of road plans are essential.

A. Road Factors Affecting Road Welfare

1. Cross-section of the Roadway: The straight-down cross-section of the road factors consists of the width of the portable lane, the width and nature of the shoulder, and the skid conflict of the inner of the transportable way. The width of the transportable path does not only affect the ease of driving and operational features of a road but is also a significant factor affecting the roadway clatter occurrence as well as clatter severity. For any useful grouping of roadway, whether it is an arterial road or a local road, and for any location of the road, whether it is an urban road or a rural road, once the road width decreases, the likelihood of clangs increases severely. When the road traffic work is higher and the road width is less, the likelihood of outbreaks like head-on or run-off on the road is more.:
2. Roadside Condition: The protection of the road lane does not hang only on the features of the road but also hangs on the circumstance of the shoulder. The term “clear zone” is used to term the free, crossable area provided outside the edge of the portable way for the retrieval of the wayward vehicle. The clear zone includes shoulders, bicycle lanes, and any additional space, if obtainable. The larger the width of the clear zone, the added room is obtainable for a wayward driver to improve earlier thumping an object; thus a larger clear zone means a safer road lane. This type of location is more public in closely occupied urban areas. Seeing safety features, a 1 cross offset to vertical obstructions such as signs, utility poles, etc. is needed to avoid crashes.
3. The flat curvature of a road lane is significant as when a means of transportation moves in a circular path, it experiences a centripetal acceleration that performs toward the center of the curvature. In other words, centrifugal forces effort to change away from the means of transportation from its preferred line of drive which is the curved road lane. The straight-up grades or curving of vertical bends of the road lanes are also connected to road welfare. When vertical inclines are provided, it becomes more challenging for a motor vehicle to be controlled. This is a more vital problem for heavier vehicles like trucks.
4. Sight Distance: A sight distance of the necessary length is important so that a driver can switch the process of their vehicles to escape hitting an unpredicted entity on the lane. This is known as “Stopping Sight Distance (SSD). The Passing Sight Distance(PSD) relates to a two-lane roadway to allow drivers to use the opposite traffic path for overtaking new vehicles without interfering with other vehicles. The DSD is the distance required for a driver to notice an unpredicted information source or circumstance in a roadway to identify the situation to select a suitable speed and pathway.
5. Access management is the notion that related vehicular ideas and sizes can have severe disadvantages on the act of road traffic acts and road welfare. The assistance is significant, mostly in urban street localities where entry points are many and traffic bulks are high.

B. Environment

The system architecture uses the features that need to be considered from requirements and provides a skeleton of the system that needs to be prepared for execution. The system contains a road environment that is distributed as accident zones and junction roads. An agent will be placed in an environment, with the initial state and random actions will be performed by the agent to reach the destination. If there are any hotspots such as drainage pavement, anti-slip pavement, pedestrian crossing zone, high traffic zone,

C. Reinforcement learning

Reinforcement learning is a machine learning area. Suitable actions are taken by an agent in order to increase the reward in a given environment. Reinforcement learning is different from supervised learning, in reinforcement learning, there is no trained dataset but the agent learns from its experience and decides what actions should be performed for the given task in order to achieve the maximum reward.1. Agent: An individual that can explore the environment and act accordingly. 2. Environment: A state in which an agent is present or bounded by. 3. Action: Actions are the changes/movements taken by an agent within the environment. 4. State: State is a condition returned by the environment after each action taken by the agent. 5. Reward: A response returned to the agent from the environment to estimate the action of the agent. 6.Policy: Policy is an approach applied by the agent for the next action based on the current state.

II. PROBLEM STATEMENT

To consider different types of roads based on the road environment such as accident severity, road type, road surface, light conditions, and junction details and to classify the road structure quality based on the studied dataset from the system using the Q-learning algorithm from Reinforcement learning.

III. RELATED WORKS

Viswanath, Dhanya et.al (2021, [1]) in the proposed work have studied the relation between the condition of the road, road accidents, and the occurrence of an accident based on environmental factors was studied. They successfully created an application that would predict road accidents based on vehicle type, age of the driver, age of the vehicle, road structure. The dataset has a date, time, and location of accidents, the nature of the accident, whether it was a head-on collision or caused due to over-speed, skidding, or other causes, the type of the road – straight road or a curved road, how many lanes were there, whether it was a junction of multiple roads, the number of fatalities. They have used data mining techniques-Apriori algorithm and Support Vector Machines. The optimized models can be efficiently used by the government to reduce road accidents and to implement policies for road safety. Mariem ALLOUCH and Fedy OUNI (2020, [2]), their work aimed to highlight the way and extent through which the explanatory variables such as a human, vehicle, and environment affect the severity of a traffic accident. They chose a multinomial logit model to analyze the severity levels of road accidents at intersections.

They used the dataset from the National Road Safety Observatory in Tunisia from the period 2014-2016. The STATA 13.0 statistical software was used to estimate the multinomial logit model. The model developed was able to determine the variables that have a significant effect on the severity of accidents. The advantage of this model is that it allows calculating the probability that a victim is involved in an accident. Jayesh Patil et.al (2020, [3]), in their work they have used the k-means algorithm and machine learning to examine accident-prone or hotspot areas and their root causes. Government data was used to acquire weather and road conditions to execute their proposed system. Dataset worked with the help of the following attributes: Latitude, longitude, location name, Speed, Time, Alarm type, Weather, Peak hour timings, Potholes, Accident-prone areas. Once testing was completed, the model was made available to local authorities for use.

Hotspot regions were pinned in red where accidents usually occur and locations with high accident-prone areas were shown. Durgesh Kumar Yadav et.al (2020, [4]), in their work the dataset contained 620 positive videos containing the exact time of the accident in the last 15 frames, and 1130 negative videos containing no accidents. The dataset was collected from six major cities in Taiwan. The dataset includes: 42.6per  motorbike to car collision, 19.7per car to car collision, 15per motorbike to motorbike collision, and the remaining 22.7per are of other types. They used Long Short-Term Memory, Convolutional Neural Network, RNN in their project. A module was proposed that detects, and identifies accidents on the occurrence through means of vibrations and sends a notification to the given number.

IV. SYSTEM DESIGN

The System Design Document is for a base-level system, which will work as a proof of concept for the use of building a system that provides a base level of functionality to show feasibility for large-scale production use. The focus is on the generation of the documents and modification of the documents. The system will use in conjunction with other pre-existing systems and will consist largely of a document interaction faced that abstracts document interactions and handling of the document objects. This Document provides the Design specifications of Road Structure Quality Using Reinforcement Learning.

The working principle of road structure quality classification using Reinforcement learning is as shown in figure 3.1. An agent will be placed in an environment, that is to move from the initial state to the final state, by choosing maximum optimization. Based on the road features input will be provided to the agent at the initial state. The agent moves further to the next path, if the move taken by the agent is correct then the agent will be assigned a positive reward and the agent moves forward in order to reach the goal with maximum optimization. If the move taken by the agent is wrong, then a negative reward will be assigned.The goal of an agent is to travel from Path 1 to Path 5 with maximum optimization. From Path 2 the agent moves to Path 3 to reach the destination, if the move is correct the agent is rewarded with +1. From Path 3 the next move agent can take is Path 1, Path 2, or Path 4 to reach the destination. If the agent takes Path 2 it will be rewarded -1 because this path is already covered by the agent. If the agent takes Path 1 or Path 4 it will be rewarded +1. The agent can take either of the paths which has maximum optimization to reach Final state Path 5.The agent will classify the road based on the quality of the road structure.

A. Design Methodology

Figure 2.  shows the methodology design of the system. In Reinforcement learning we use Q-learning. The Road environment dataset is collected for which pre-processing of data is carried out. Once pre-processing is done the dataset will be classified as training and testing data. After train and test of data Environment Windygridworld is used in which we define policy, actions and reward for the system. Analysis of classification is done and we test Q-learning with enhanced-Q learning. Finally we will obtain accuracy and confusion matrix for the proposed system.

B. Dataset

1. The dataset consists of 29 features and 5 classes.
2. Dataset is a group of discrete and related items that may be accessed independently or as a whole unit.
3. The dataset for the proposed system is as follows shown in table 1.

C. Defining Policy

Table 2. describes the policy defined during the creation of an environment. We have considered 6 features to understand how policy is defined during creating an environment. The 6 features are Accident severity,Road type, Junction detail, Lightconditions, Weather conditions, Road surface. As we can see the integer values in the policy they are the road environment description.

V. ALGORITHM

The algorithm is the process to set rules to heed in the calculations and problem-solving solutions to the given requirement and design, which allows the developer to conduct the sequence of actions that need to be considered and help to develop the system in a logical way of representation. Input is the state and actions given to the agent at the starting point. Output is the classification of road quality from the given road environment dataset. In this system, the policy is defined as shown in table 2. The actions are the values of the features., Finally, the reward is the classification of the road quality by saying how good is the road to be used.

Conclusion

Classification of road structure quality is done using a Reinforcement learning algorithm in which Q-learning is used. The model will learn a dataset based on road structure analyze the data and classifies the road quality. Initially, the collection of a dataset and pre-processing are done after which training and testing are carried out. A wind grid environment is created in which policy, action, and rewards are defined. Q-learning and Enhanced Q-learning are used to analyze the dataset. Accuracy and confusion matrix is calculated, where accuracy for q-learning is 93.62% and accuracy for enhanced q-learning is 93.75% is obtained. Classification of a dataset is done based on the quality of road structure environment.

References

[1] Viswanath, D., Preethi, K., Nandini, R. and Bhuvaneshwari, R., “A Road Accident Prediction Model Using Data Mining Techniques”, In proceedings of the 5th International Conference on Computing Methodologies and Communication, PP:1618-1623, 2021. [2] Allouch, M. and Ouni, F., “Modeling the severity of road accidents at intersection”, In proceedings of the IEEE 13th International Colloquium of Logistics and Supply Chain Management, PP : 1-5, 2020. [3] Patil, J., Prabhu, M., Walavalkar, D. and Lobo, V. B., “Road Accident Analysis using Machine Learning”, In the proceedings of the IEEE Pune Section International Conference, PP: 108- 112, 2020. [4] Yadav, D. K. and Anjum, I., “Accident Detection Using Deep Learning”. In proceedings of the 2nd International Conference on Advances in Computing, Communication Control and Networking, PP: 232-23, 2020. [5] Afework, A. and Sipos, T., “Modelling of accidents for four lane non-urban highways using artificial neural networks technique”. In proceedings of the 14th International symposium on applied computational intelligence and informatics, PP: 000047-000052,2020. [6] Govada, K. A., Jonnalagadda, H. P., Kapavarapu, P., Alavala, S. and Vani, K. S. “ Road Deformation Detection”. In proceedings of the 7th International Conference on Smart Structures and Systems PP: 1-5, 2020. [7] Parveen, N., Ali, A. and Ali, A. “IOT Based Automatic Vehicle Accident Alert System”. In proceedings of the 5th International Conference on Computing Communication and Automation, PP: 330-333, 2020. [8] Yellamma, P., Chandra, N. S. N. S. P., Sukhesh, P., Shrunith, P. and Teja, S. S. “Arduino Based Vehicle Accident Alert System Using GPS, GSM and MEMS Accelerometer”. In proceedings of the 5th International Conference on Computing Methodologies and Communication, PP: 486-491, 2020. [9] Vijay, D. and Chikyal, N. “Vehicle Accident Alert System Using LABVIEW”. In proceedings of the International Conference on Emerging Smart Computing and Informatics PP: 24-27, 2021 [10] Rajesh, G., Benny, A. R., Harikrishnan, A., Abraham, J. J. and John, N. P. “A Deep Learning based Accident Detection System”. In proceedings of International Conference on Communication and Signal Processing, PP: 1322-1325, 2020. [11] Takanashi, M., Ishii, Y., Sato, S. I., Sano, N., and Sanda, K. “Road-Deterioration Detection using Road Vibration Data with Machine-Learning Approach”. In proceedings of the IEEE International Conference on Prognostics and Health Management, PP: 1-7, 2020. [12] Shivaanivarsha, N., Niththish, A., and Jagadeesh, U. “A Novel Approach for Monitoring Road Way Surface Using CNN Classifier”. In proceedings of the International Conference on Power, Energy, Control and Transmission Systems, PP: 1-5, 2020. [13] Chitale, P., Dhope, T., Akhelikar, A. and Dholay, S. “Smart Accident Recognition and Alerting System for Edge Devices”. In proceedings of the 5th International Conference on Computing Communication and Automation, PP: 486-491, 2020. [14] Kumar, N., Barthwal, A., Lohani, D. and Acharya, D. “Modeling IoT enabled automotive system for accident detection and classification”. In proceedings of the Sensors Applications Symposium, PP: 1-6, 2020. [15] Li, F., and Jiang, K. “Application of Random-parameter Negative Binomial Model to Examine the Relationship between the Severity of Traffic Accident”. In proceedings of the 5th International Conference on Intelligent Transportation Engineering PP: 351-354, 2020. [16] Kumar, M. N., Kumar, S. P., Premkumar, R., & Navaneethakrishnan, L. “Smart Characterization of Vehicle Impact and Accident Reporting System”. In proceedings of the 7th International Conference on Advanced Computing and Communication Systems, Vol: 1, PP: 964-968, 2021. [17] Sewwandi, A. K. T., Dissanayake, D. M. K. P., Navanjani, D. H. K. H., Shangavie, R., Rankothge, W. H., and Gamage, N. “SmartCop: An Automated Platform to Mitigate the Impact of Road Accidents”. In proceedings of the 8th R10 Humanitarian Technology Conference, PP: 1-6, 2020. [18] Rahman, W., Ruman, M. R., Jahan, K. R., Roni, M. J., Rahman, M. F. and Shahriar, M. A. H. “Vehicle Speed Control and Accident Avoidance System Based on Arm M4 Microprocessor”. In proceedings of the International Conference on Industry 4.0 Technology, PP: 154-158, 2020. [19] Kumar, N., Acharya, D. and Lohani, D, “An IoT-based vehicle accident detection and classification system using sensor fusion”, IEEE Internet of Things Journal, vol: 8, issue:2, PP:869-880, 2020 [20] Rana, S., Sengupta, S., Jana, S., Dan, R., Sultana, M. and Sengupta, D, “ Prototype Proposal for Quick Accident Detection and Response System”. In proceedings of the Fifth International Conference on Research in Computational Intelligence and Communication Networks, PP: 191-195, 2020. [21] Kapilan, K., Bandara, S. and Dammalage, T. “Vehicle Accident Detection and Warning System for Sri Lanka Using GNSS Technology”. In proceedings of the International Conference on Image Processing and Robotics, PP: 1-5, 2020. [22] Song, Y., Wu, P., Gilmore, D. and Li, Q. “A spatial heterogeneity-based segmentation model for analyzing road deterioration network data in multi-scale infrastructure systems”. IEEE Transactions on Intelligent Transportation Systems, 2020. [23] Hasib, K. M., Showrov, M. I. H. and Das, A. “Accidental prone area detection in bangladesh using machine learning model”. In proceedings of the 3rd International Conference on Computer and Informatics Engineering, PP: 58-62, 2020. [24] Patil, J., Patil, V., Walavalkar, D. and Lobo, V. B. “Road Accident Analysis and Hotspot Prediction using Clustering”. In proceedings of the 6th International Conference on Communication and Electronics Systems, PP: 763-768, 2021.

Copyright

Copyright © 2022 Shilpasree S. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.