Intelligent Road Condition Assessment and Pothole Detection

Abstract

Pavement surface monitoring is crucial in order to provide a secure and seamless road infrastructure. Road surface anomalies like potholes, cracks, speed bumps, manmade joints on the road surface that hamper the driving comfort must be accurately located in order to effectively monitor the state of roadways. Lately, sensing via smartphones has acquired more popularity since inbuilt sensors in smartphones are easily obtainable. Smartphone based sensing is starting to become more and more common. The area of research in locating surface irregularities in the pavement is quite massive. If device or smartphone have the capacity to spot abnormalities in the pavement surface. The comparatively low-frequency of today’s mobile device sensors and under sampled data collected from the sensors lead to unsatisfactory accuracy in detection. The most recent techniques for employing cellular devices to detect imperfections on the roadways are contrasted and investigated in this study. Further directions for pavement anomaly detection utilizing smartphones are also highlighted

Introduction

I. OBJECTIVE

The primary focus of this system is to employ deep learning to determine the presence of pothole on the pavement. This methodology automates the procedure for detecting pothole and therefore making it easier for automobile users to know when they are heading towards potholes present on the road.

II. INTRODUCTION

The World Health Organization (WHO) has released statistics showing that one of the top 10 causes of death worldwide is traffic accidents. Approximately 1.25 million people die in automobile accidents alone each year (2015). Studies indicate that poor road conditions are the primary cause of the most traffic accidents. Damage to roads result in high maintenance costs for both the broad and car themselves, which is a major issue for both drivers and automobiles. Therefore, monitoring systems for pavement surface condition are crucial for improving traffic safety, reducing the frequency of crashes, and protecting cars form harm caused by badly maintained roadways. Additionally, anyone interested in learning more about the quality (safety) of the road infrastructure, such as road guards and drivers.

Bad road conditions pose a serious threat to comfortable and safe travel but this ailment has to presumably cured first. The initial methods for identifying damage to roads is self-reporting to central authorities. If people are fair, this procedure is most accurate, but it also needs the greatest amount of human input. Based on the intensity of the usage, statistical analysis can be performed to estimate the damage possibilities of individual road sections. In order to identify road service monitoring, a new technique known as surface analysis method was developed. It makes use of variety of instruments, including GPR (Ground Penetrating Radar) and commercial items. Unfortunately, the aforesaid method’s availability is restricted by the expensive cost of the equipment required for it. As an alternative, participative sensing can expand the range and resolution of the information gathered. Taking photos on man-made road damage such as potholes, and sending them to a reputable authority is the simplest method. However, manual picture analysis and user participation are needed for this. It can take a lot of time and human resources because it calls for more interpersonal engagement. An automatic method of hole identification requires little to no human interaction is more promising in order to raise the energy and time consumption of this procedure. Compared to human intervention, this would help capture more accurate real-time data with fewer human error. It is crucial to make a hole tracking system more visually appealing, engaging, and user-friendly so that the users can access it quickly and easily. This will help the system work well and be embraced by a large user base.

III. LITERAURE REVIEW

[9] Smartphone are a useful tool for detecting the level of roughness of the pavement surface, consequently this could be helpful in planning for pavement inspection and maintenance. Smartphones are additionally affordable while being simple to use.  In this study, data was gathered inside vehicles travelling over uneven road segments using accelerometers and gyroscopes mounted on cell phones at various position. To determine the vibration magnitudes, the collected data is subjected to frequency domain processing. Each axis of accelerometers and gyroscope is measured, it has been found that there is a very substantial association for a specified range of 40-50 Hz between the average speed and the vibration magnitudes.

The affordability and ease of use feature of smartphones makes them a potentially helpful instrument to assess the speed of pavement surface.  When the road roughness condition is modelled as a linear function of vibration magnitudes and incorporates data from the accelerometer and gyroscope, the estimation performance is better than when the model only takes accelerometer’s magnitude and average speed into account. The finding could have a significant effect on developing a more precise model and an enhanced smartphone app for pavement roughness monitoring using versatile sensors. The discovery may have a big impact on the creation of a more accurate model and a better smartphone app for measuring pavement roughness with flexible sensors. 

[10] Nowadays, camera observation and vibration detection are the mainstays of most mobile device road conditioning monitoring techniques. Vision-based systems take pictures of the road surface using movable equipment mounted on moving vehicles. Afterwards the images are automatically processed using image analysis techniques in order to extract information about roads. The most widely used sensors in vibration-based approaches are accelerometers and gyroscopes, which have ability to record locations. DWT is a pattern matching approach used to determine the degree of similarity between two temporal sequences, which may differ from one another in both space and time.

In the realm of road inspection, machine learning techniques are increasingly being applied. Using GPS and accelerometers mounted on mobile sensors, Pothole Patrol traversed hundreds of kilometres over the city of US city of Boston, Massachusetts, gathering data along the way. Various filters, including speed, high pass, z-peak, xz-ratio, and speed vs. zratio, were used in their pothole-detection approach to eliminate various sorts of non-pothole occurrences. 

[5] On the other hand, the GPS receiver samples at a 1HZ frequency. This is the reason why the accelerometer needs to sample at the GPS frequency in order to receive georeferenced data. Using the suggested algorithm (az-min), the minimum is established. Five observations in a second are used to calculate the average (az-avg) and (az-maz) acceleration values since 5 Hz is the lowest accelerometer frequency. In order to accurately assess the state of the pavement surface and the probability of irregularities like potholes, bumps, and joints, only the vertical acceleration was examined.

To guarantee that the accelerometer captures the actual vertical vibrations even when the device is randomly thrown into the car, a partial reorientation of the mobile frame was done. According to Android Developers, SAE International (1976), this that the vertical axes of the vehicles and the device overlap since the orientation of the vehicle and the device ensures can both be represented using a 3D cartesian frame. By measuring the difference between the highest and minimum value of the vertical acceleration in a given unit of time, or the vertical acceleration impulse, the condition of the road surface was evaluated. The acceleration absolute amplitude in one second is the temporal derivative, or d(azmax-az-min).

[3] The technique for identifying potholes and other abnormalities in the road is based on analysing the acceleration signal in terms of high-energy events. Identification of these instances with high precision was the aim, as was the possibility of linking the observed acceleration impulses to a particularly road irregularity. Reorienting the z acceleration, processing filters were used to “purify” the signal and exclude one or more non-anomaly events.

An extra step entailed labelling the anomalies manually that were found during the during the field test in order to confirm that the automatic method can accurately replicate the actual surface state. At this detection phase, potholes and bumps were manually tallied and categorized based on their dimensions; georeferred picture were used to evaluate the degree of element matching

[8] The survey smartphone is mounted on the Tavera, an SUV, dashboard using strong, thin double-sided adhesive tapes. The smartphone’s head and phase were directed towards the front and roof of the car, respectively. The x, y and z axes of the smartphone accelerometer thus detect the motion of the car in the following directions: left to right, front to back, and up to down. The smartphone accelerometer, which has a 200 Hz capability, is utilized to gather data via the Andro sensor app. Information was collected from five distinct smartphones. Based on a variety of factors, such as the smartphone’s features, accuracy, and accelerometer type, one of the smartphone’s acceleration data sets was selected for additional analysis.

For the study roads, two rounds of concurrent IRI data collection, lane-by-lane smartphone-based vertical acceleration, and preliminary meter-based IRI values were carried out. The link between Power Spectral Density (PSD) and IRI was found using the quarter Car Simulation model concept.

A model that made use of the data was created using this relationship. Utilizing the second set of data, the model demonstrated accuracy. Correlation between the real (based on a roughmeter) and expected (based on a smartphone) IRI values was used to further verify the correctness.

[7] To interpret the data collected with cell phones and determine the bad road conditions, the researchers have used a variety of supervised and unsupervised machine learning approaches. The method of supervised learning has been used to find anomalies in the road. Neural networks or signal processing techniques have been to the strategies suggested by the authors in order to improve the accuracy measure. Using Dynamic Time Warping (DWT) as a signal processing technique, the system has an 88% accuracy rate in identifying potholes and bumps in the road. Using smartphone photos, the authors applied deep learning techniques for item detection. The suggested method greatly improves road user safety by detecting potholes about 100 meters ahead. A method for detecting potholes using convolutional neural networks (CCNs) has been suggested. Suggested system produces 97% accuracy and uses Google API to map the detected pothole on Google.

IV. IMPLEMENTATION

The implementation details of the project are divided into given following steps.

A. Data Acquisition 

The smartphone comprises of number of integrated sensors. Among these sensors, some utilize hardware, while others are software-based sensors. Physical sensors are another term for hardware-based accelerometers. Physical sensor includes temperature, light, magnetometers, gyroscopes, and accelerometers. These are built-in sensors that monitor orientation, motion, and environmental factors including the device’s physical location and acceleration force, among others. Conversely, virtual sensors, or software-based sensors, are another option. Based on desired results, the softwarebased sensors, are another-option. Based on desired results, the software-based sensors virtually calculate real-time values or more hardware-based sensors. In general, sensors can be categorized into three types: environmental, motion and position.

Combination of various physical and virtual sensors can be utilized by each smartphone by application depending on the intended application criteria. To detect potential tilt motion sensors can be tracked or tremor brought on by road surface anomalies in a moving vehicle, which can be used to design application for road surface anomaly identification. Accelerometer and gyroscopes are two common motion sensors that have been used in past research on the identification of road surface abnormalities using smartphone sensors.

Accelerometer sensors detects the three physical axes of gravity force (meter per second) applied to device. The speed at which a device rotates (radian per second) around each of three physical axes (meter per second). Gyroscopes sensors measure the speed at which a device rotates (radian per second) across each of the three physical axes. Previous study has often employed accelerometer sensor data to identify anomalies, as the influence of road surface abnormalities is greater than that of rotation rate produced by vehicle vibration.

The location API (Application Program Interface) uses the currently available location providers, such as GPS (Global Positioning System) and/or Wi-Fi, to determine the current location information of smartphones or mobile devices, including latitude, longitude, bearing of moving directions, and velocity of movement. Furthermore, the API offers accurate location data for all of its users.

Here are sensors used for pothole detection.

 B. Preprocessing

Typically, sounds or unrelated data were present in accelerometer readings. To improve the amount of noise contributed to these signals and minimize its presence, a preprocessing phase ought to be included. The mobile application continuously reads and save the data which is in the form of coordinate. The x, y and z axis are readings are retrieved from the accelerometer sensors from mobile device. Additionally, it displays the time that the gadget was shook if pothole occurred. Your Android device will have a positive ZAccel number from gravity if it is set face up on a tabletop. The XAccel value will become positive if the device is turns to the left. YAceel will receive a positive value if the device is held upright. We may observe this with an Android application. Gravity is 9.81 m/s/s. The shaking function is triggered if the value of the accelerometer changes drastically, and immediately after, the GPS will automatically determining the exact location of the jerk by determining the longitude and latitude of the occurrence. The coordinates will be found and added to a fire database that has the precise shaking locations of the potholes. A cloud-hosted NOSQL database called Firebase Realtime Database enables real-time data syncing and storing across users. Bit if the shaking function does not occur and it shows normal vibrations signal then the data is not sent to firebase database instead the data is stored in the user mobile device itself. Later the data is destroyed from the user device.

C. Model Building  

The model for detection of pothole is built using Python language. The data that is stored in firebase database is feed to the model that will detect if it is pothole or not. The DWT approach is used in time series signal processing to gauge how similar two signal patterns are to one another and to derive properties from the signal data. For example, a DTW approach to identifying abnormalities in the road surface using accelerometer sensor data. Using this method, accelerometer sensor data for each pothole and bump was recorded using time series values, which were subsequently saved as templates on a central server. Subsequently, the stored templates were matched with the incoming sensor data to identify commonalities. The quality of the reference template had a significant impact on this approach’s accuracy. Consequently, this method was unreliable and computationally demanding since it needed reference tables for every possible scenario (different cars, different speeds, etc.). A deep learning-based forecasting method is used by the model. By utilizing machine learning technologies and historical data, predictive modelling is a statistical technique that predicts future events. Predictive models use past and present assumptions to show the desired outcome. Predictive analytics models are regularly updated to take into account modifications to underlying data.

Additionally, most of these prediction models run faster and complete their calculations instantly. If the pothole detected by the model, the model marks it on the map. For example, if user1 detects a pothole the model marks it on the map and other users also detects the same pothole then the probability that there is pothole becomes stronger. But if for the same pothole data if the user 1 detect the pothole and other users does not then the mark will be removed from firebase database.

D. Training a Classifier Model 

After building the model we train the model. To detect road conditions, a classifier model is trained using machine learning techniques based on the tuples for all road situations. The input for the machine learning method for each road co4.5, Support Vector Machines, and naïve Bayes) to determine the highest classification accuracy before choosing it as the foundational classification method.

E. Prediction Phase 

Road conditions are detected and identified by the prediction phase which is placed on a smartphone and senses real-time vehicular dynamics. Our technique initially detects the gyroscope and accelerometer measurements that are integrated in smartphones. Preprocessing is done on the sensor measurements once real-time readings are obtained. The road location is then traced on a geographic map by our system, which first gathers features from patterns in the road conditions and uses the classifier model it built during the training stage to forecast the road quality. Eventually, a road of every reported road condition is preserved.

F. Creating mobile and web Application 

Using the React framework, build a mobile application. Using this platform, you may create mobile applications using web applications and programming languages such as HTML, CSS, JavaScript, Angular, and Typescript. The functionality of a mobile platform is provided by a group of components called Ionic. The smartphone application will display a map with marks on it that indicate potholes that are found in the road and an option for generating report of potholes. These reports can be sent to higher authorities for repairment of pothole. Leaflet library is used in the creation of the map. For mobile-friendly interactive maps. Leaflet library is the most popular open-source JavaScript library. You can store and sync data amongst your users in real-time using firebase realtime Database, a cloud-hosted NoSQL database. Users can more easily collaborated and access their data from any web or mobile device thanks to real-time syncing. Developers maybe benefit from simple and intuitive authentication procedure thanks to the combination of firebase Authentication and the real-time database. With our declarative security approach, you may match patterns in your data or provide access based on user identity. We are able to construct dynamic links and provide authentication thanks to Firebase. The user can click on the link, log in to the website, and the locations of the potholes. Through the use of .h files, these models are linked to the program. In order for the model to identify the pothole on the map.

V. FLOWCHART

VI. PERFORMANCE EVALUATION

Identifying the precision and efficacy with which our system can identifying road conditions is the main objective of this assessment. Serval classification methods are applied to the learning data-set containing the chosen characteristics in order to determine which model is best. Cross validation is the assessment technique used to determine how well various classifiers perform. The smartphone sensor dataset is split into training and test sets, to be more precise.

The purpose of cross-validation is to identifying a validation set or subset of data to test the model during the training phase, allowing for the close examination of issues like over-fitting. A 10-fold cross validation was carried out, using one-tenth of the two data sets exclusively for testing, to ascertain whether classifier performs better than the other. To assess the effectiveness of the employed classifiers, we have employed a number of metrics.

1) Classification accuracy: The overall categorization performance for each activity class can be explained by accuracy in the following ways:

Figure 4 depicts the classification accuracies of various machine learning algorithms both with and without feature selection. Notably, C4.5 emerges as the most accurate classifier, boasting an average accuracy of 76.53%when compare to Support Vector Machine (SVM) and Naïve Bayes. When employing the Correlation-based Features Selection (CFS) algorithm, the average accuracy of C4.5 is further by 0.1%, showing its robust performance. This heightened accuracy is attributed to the fusion of data originating from accelerometer and gyroscope sensors, which effectively mitigates false positives. In contrast, SVM exhibits subpar performance with an accuracy of 70.25%, while Navie Bayesian classifier closely trails behind with a modest accuracy of 67.80%.

Both precision and recall are rooted in the concept of relevance assessment. Precision quantifies the level of exactness or quality in classification, while recall gauges the degree of completeness or quantity in identifying relevant instances.VM>Naïve Bayes, as indicated in Table II. Regarding the recall ratio, C4.5 demonstrate superiority over Navie Bayes and SVM.F-measure represents the harmonic mean of precision and recall, comparing the F-measure values among the classifiers provides insight into their combined performance.

4) Confusion Matrix: A commonly employed method for assessing performance involves utilizing a confusion matrix analysis. This matrix provides comprehensive details regarding both actual and predicted classification made by classification system.

Each the confusion matrix for three algorithms. Attention should primarily be directed towards the values along the diagonal, as they signify the instances correctly classified by the algorithms. A higher value along the diagonal indicates a greater accuracy in classification. The classification results derived from the c4.5 Classifier are summarized in the provided table. Among the total instances, 1725 were correctly classified, while 31 instances were misclassified. Upon closer examination, it is evident that within the first row, which pertains to the “smooth” class consisting of 1500 instances, 1480 were accurately classified, with remaining 10 misclassified as “pothole”. These misclassifications are attributed to instances where rough road conditions were erroneously categorized as smooth during the learning phase. Conversely, the second row representing the “potholed” class, with 245 instances, indicates 15 instances were misclassified as “smooth”. These false positive attributed to the absence of certain pothole characteristics during the learning phase.

Our experimental findings demonstrate that the C4.5 Decision Tree Classifier supress SVM and Naïve Bayesian classifiers across all performance metrics, making it the preferred choice. Consequently, we advocate for the adoption of the c4.5 classifier to identify patterns in road conditions and predict the quality of unknown roads. The robustness of our analyses further strengthens the rationale for employing C4.5 in this context.

The C4.5 algorithm, a pruned decision tree model, is renowned for its effectiveness in recognition tasks, offering both low computational requirements and satisfactory accuracies. Its selection as a classification model stems from its capability to generate concise rules and eliminate insignificant parameters prior to initiating the tree induction process. Additionally, the C4.5 decision tree possesses the ability to manage datasets containing errors or missing values. Given these attributes, decision trees are deemed as superior solutions for integration into smartphone applications

???????

Conclusion

Many techniques that use information from smartphone sensor to identify irregularities in the road surface have been tried. These methods are still not without difficulties, though. People can now regularly observe and report instances of abnormalities on the pavement surface because more individuals are carrying smartphones. Sophisticated data crowd sourcing strategies have the potential to significantly boost traffic monitoring systems’ efficacy. Smartphones are perfect for crowdsourcing projects and changing drivers’ perception of incoming notifications since they come with built-in wireless interfaces like Bluetooth, Wi-Fi, and cellular networks. Indeed, via cellular or Wi-fi networks, abnormalities can be reported to the central server for additional analysis and driver/authority data. Through the execution of data classification operations and training of machine learning algorithms, data integration in the server would boost system accuracy. Its challenging to compare the performance and accuracy of different approaches with the accompanying algorithms because of limited availability of the aforementioned data sets and algorithms. The setting of the threshold is still unknown for many threshold-based approaches. Furthermore, a lot of data is required to train the detection model in systems that employ supervised or unsupervised learning techniques. Therefore, a hybrid system that can continuously identify and differentiate between various abnormalities in the pavement surface needs to be created. Realtime data streaming from smartphone sensors and other geographic data would be utilized by this system. For the purpose of improving detection accuracy and providing smartphone users with greater flexibility, sensor data values should be smoothed and reoriented. A self-learning and self-adapting system that could adjust to changing road conditions, different cars’ dynamic behaviours, and any platform would be ideal. It would also give law enforcement organizations and drivers more chances to alert them to possible abnormalities in the state and condition of the road surface before possible restoration and repair is undertaken. An open-source, multi-platform needs to be created that is free of cost. This fully automates program would free up the driver’s attention while driving. It is also necessary to reduce the power consumption of the suggested application, possibly by employing fewer GPS sensors. Various methods should also be looked into because different cell phones have differing degrees of sensor noise and data accessibility, such as accurate sensor calibration reading.

References

[1] Amusan Gideon[1], Arulogun Tayo[2], Oluwatobi Odunaiya[3], Ade-Akanfe Adebayo[4], Ilupeju Peter[5] “Design and implementation of road surface monitoring system using camera and global positioning system.” ScienceOpen Preprints 20 March 2022 [2] Dimpal V. Mahajan. “ESTIMATION OF ROAD ROUGHNESS CONDITION BY USING SENSORS IN SMARTPHONES.” International Journal of Computer Engineering & Technology (IJCET) Volume 6, Issue 7, Jul 2015, pp. 41-49, Article ID: IJCET_06_07_006 Available online at http://www.iaeme.com/IJCET/issues.asp?JTypeIJCET&VTy pe=6&IType=7 [3] Vittorio Astarita, Maria Vittoria Caruso, Guido Danieli, Demetrio Carmine Festa, Vincenzo Pasquale Giofrè, Teresa Iuele, Rosolino Vaiana. “A mobile application for road surface quality control: UNIquALroad.” Procedia - Social and Behavioral Sciences 54 ( 2012 ) 1135 – 1144. [4] Azza Allouch, Anis Koubaa, Tarek Abbes, and Adel Ammar. “RoadSense: Smartphone Application to Estimate Road Conditions using Accelerometer and Gyroscope.” n IEEE Sensors Journal May 2017 DOI: 10.1109/JSEN.2017.2702739. [5] Astarita Vittorioa , Vaiana Rosolinoa , Iuele Teresaa,*, Caruso Maria Vittoriaa , Giofrè Vincenzo P. a, De Masi Francescoa. “Automated sensing system for monitoring of road surface quality by mobile devices.” n Procedia - Social and Behavioral Sciences · February 2014 DOI: 10.1016/j.sbspro.2014.01.057 [6] Lars Forslöf [1], Hans Jones[2]. “Roadroid: Continuous Road Condition Monitoring with Smart Phones.” Journal of Civil Engineering and Architecture 9 (2015) 485-496 doi: 10.17265/19347359/2015.04.012 [7] Rishu Chhabra1 and Saravjeet Singh2. “A survey on smart phone-based road condition detection systems.” Chitkara University Institute of Engineering and Technology, Chitkara University, Punjab, India. [8] Dr. Janani L, Dr. Sunitha V. “Pavement Condition Assessment using Smartphone Accelerometers”. International Journal of Engineering Research & Technology (IJERT) ISSN: 2278-0181 Published by, www.ijert.org ICART - 2021 Conference Proceedings Volume 9, Issue 9 [9] Viengnam Douangphachanh[1], Hiroyuki Oneyama[2] “Exploring the Use of Smartphone Accelerometer and Gyroscope to Study on the Estimation of Road Surface Roughness Condition” DOI: 10.5220/0005117407830787 In Proceedings of the 11th International Conference on Informatics in Control, Automation and Robotics (ICINCO2014) [10] Chao Wu 1 , Zhen Wang 2 , Simon Hu 3,*, Julien Lepine 4 , Xiaoxiang Na 5 , Daniel Ainalis 5 and Marc Stettler 6. “An Automated Machine-Learning Approach for Road Pothole Detection Using Smartphone Sensor Data.” Sensors 28 September 2020, 20, 20, 5564; doi:10.3390/s20195564556doi:10.3390/s20195564

Copyright

Copyright © 2024 Dr. Gajendra Astukar, Prof. Ayeman Gani, Shreya Mendhe, Prachi Sahare, Samyak Waghade, Shashank Deshbhratar, Gouri Ikhar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.