Analysis and Improvement of the Current Pothole Detection System in Google Maps through Color Segmentation

Abstract

This paper presents algorithms for pothole detection on roads. We have collected a vast dataset from various sources and organized it into a structured format. Data preprocessing techniques were applied to reduce algorithm time and space complexity. The refined dataset was used for training, leading to the development of different models using color segmentation. The results obtained from these models demonstrated an impressive accuracy of 91% using color segmentation. The creation of this pothole detection system can significantly enhance safety for both you and others. Moreover, it has the potential to be widely integrated into Google Maps by extracting coordinates from Geographical Information System (GIS) data and incorporating them into the platform.

Introduction

I. INTRODUCTION

The inspection of roads and the assessment of their physical and functional conditions are of utmost importance for governments to ensure the safety of their citizens. In many countries, manual visual inspection by government authorities has been the primary method for assessing road conditions. However, in larger countries like India and China, this approach has proven to be ineffective. Subsequently, sensor-equipped vehicles were introduced to evaluate road surfaces. Unfortunately, these automatic inspection methods have limitations, including the high cost of sensors and the level of detail they provide.

One of the most common issues on road surfaces is the presence of potholes, which must be addressed to ensure safety and drivability. However, methods like 3D reconstruction, vibration analysis, and vision-based approaches often require expensive equipment and may be unreliable on surfaces such as bridges and flyovers. Currently, many are adopting a supervised learning method that relies on training and testing data, which can potentially lead to overfitting.

In this paper, we address pothole detection using the color segmentation method of image processing. We propose the integration of the Mean Shift Algorithm and the Normalized Cut Algorithm. Firstly, pothole images are collected and pre-processed to eliminate null values and redundant information. Subsequently, these images are all loaded into a dataset. Finally, the Mean Shift and Normalized Cut Algorithms are applied to the dataset.

The proposed method follows the procedure below:

1. The first step involves segmenting the image into regions using the Mean Shift Segmentation method.
2. This results in the division of an image into regions.
3. The second stage entails treating these regions as nodes in the image plane.
4. The third step involves calculating a similarity-weighted matrix by representing them as nodes and edges within a graph structure.
5. The final step is to partition these regions into two disjoint sets, thus enabling the detection of potholes within these sets.

II. RELATED WORK

A paper is presented that integrates the existing Digital Highway Data Vehicle (DHDV) with multiple high-resolution digital cameras to assess pavement conditions using stereo vision technology [1]. It builds a classifier by combining a new unsupervised vision-based algorithm that doesn't require any special equipment, training, or additional filtering. This method utilizes spectral clustering and image processing for the random estimation of potholes [2].

The paper [3] illustrates a Pothole Detector using the Hough Transformation method for lane recognition, while Color Segmentation is employed with the nearest neighbor classifier for road sign recognition and classification. Finally, pothole detection is performed by a K-means clustering-based algorithm. Another paper [4] elucidates Pothole Detection by developing a vehicle-mounted sensor system that automatically detects and analyzes potholes [4].

An in-depth review focuses on 3-D Imaging Technologies and is published, concentrating on automated visual surveys. Various types of three-dimensional imaging techniques are discussed, with laser imaging being widely used [5]. Another paper [6] is published that uses three key terms: disparity transformation efficiency, golden section search, and dynamic programming to estimate the transformation parameters of the road and extract the damaged road area through Otsu's thresholding.

The preliminary part of the paper [7] presents a wide-ranging array of ideas in computer vision-based defect detection and condition assessment related to concrete and asphalt civil infrastructure. The role of the paper [8] is to introduce a modified embedded device dedicated to checking road surfaces using microphone and accelerometer sensors, also involved in creating detailed road meteorology maps.

This article [9] proposes an innovative pothole detection algorithm based on unsupervised disparity map segmentation. Its contribution is to capture accurate 3D point-cloud points with their elevations through a grid-based processing approach [10].

The paper [11] introduces the fastest 3-D transverse scanning technique used to detect road distress features in real time and with a low-cost scanning system. This research has inspired a novel color segmentation algorithm in the RGB color space based on color saliency. In the first step, the R, G, and B color channels are separated from the acquired RGB color image. Then, the color information for each channel is calculated. Finally, the saliency of each pixel in its respective channel is determined [12].

The article [13] uses modern data collection hardware to deploy a vibration-based system for primary pavement condition evaluation. A paper [14] is proposed to monitor road surface conditions using a sensor network based on the public transport system. A network called BusNet is created to monitor environmental pollution [14].

The paper [15] examines an application of mobile sensing that detects and reports road conditions. By collecting data from sensor-equipped vehicles, a system is developed to monitor public infrastructure.

III. PROPOSED METHOD

This paper provides the validation of precise pothole detection. The model we've developed addresses the limitations of relying on sophisticated equipment, massive storage requirements, and complex algorithms. In many existing systems, classifiers struggle with the identification of small potholes and are not efficient. Among the current methods, many utilize kinetic sensors, SVM, lasers, accelerometers, and mobile sensors to detect potholes. In contrast, our proposed system detects potholes through image processing segmentation modules. We are also working on associating the coordinates from GIS data with potholes, which can then be integrated into Google Maps for the convenience of users.

IV. IMPLEMENTATION

Before initiating any operation, the pre-processed dataset must be loaded onto the drive and imported into the Pothole detection model. Since the Mean Shift Segmentation Algorithm is a clustering-based approach, it provides robust feature space analysis. Given the unknown data shape, we employ a region merging algorithm to attain it. Subsequently, the images are divided into segmented regions. Building a similarity-weighted matrix becomes more straightforward when we regard these regions as nodes in the image plane. After this process is completed, we segregate these regions into disjoint sets, which aids in detecting the potholes. The above process is illustrated in Figure 1.

V. ALGORITHM

A. Color Image Segmentation

Image segmentation is the process of dividing an image into distinct parts, allowing humans to perceive these parts as individual and independent elements. Since color segmentation relies on various aspects of an image, it is essential to extract key features. These features may include color statistics used to generate histograms or data related to pixels in the images, which represent nodes, vertices, edges, or texture information. Both color and texture are essential components for distinguishing content in images and videos.

The methods of Color Image Segmentation include:

1. Region-Based Techniques
2. Clustering Technique
3. Split and Merge Technique
4. Normalized Cuts
5. Region Growing
6. Thresholding
7. Edge-Based Techniques

B. Mean Shift Segmentation Algorithm

The clustering methodology of Color Image Segmentation utilizes the Mean Shift Segmentation Algorithm to perform robust feature space analysis. This algorithm can be applied to smoothen an image and identify discontinuity areas. In this approach, it is crucial to track the shape of data clusters, which can be accomplished using a region-merging technique. Reducing the number of pixels in the region is necessary to determine the count of segmented regions. The Mean Shift (MS) algorithm is grounded in probabilistic assumptions. To understand this approach, one should first grasp mixture models, then delve into the concept of mean-shift, and finally, comprehend kernel density estimation. Essentially, Mean Shift calculates the difference between weighted means. This method ensures the inclusion of points with zero gradients, and it computes the Region of Interest (ROI). The regions produced by the MS algorithm are referred to as segmented regions, and a merging process is carried out on these regions.

Normalized Cut Algorithm

In the Normalized Cut Algorithm, segmentation zones are represented as nodes or vertices within a network, where distances between two pixels are depicted as edges. Weighted edges are generated using the normalized cut methodology. By approaching segmentation as a graph partitioning problem, the Normalized Cuts technique assesses both the overall similarity between different groups and the total dissimilarity within these groups. The splitting points are calculated by solving a general eigenvalue problem to obtain the optimal solution. The graph, represented as G = (V, E, W), can be divided into two distinct sets, P and Q, and the degree of dissimilarity between these two sets is determined using a specific formula.

VII. FUTURE SCOPE

The primary objective of this paper is to reduce accidents caused by potholes and achieve accurate results. The future scope includes incorporating location data into the pothole images. Detecting the location in addition to the pothole presence is beneficial for people to avoid traveling on routes displaying signs of potholes. This can be accomplished by integrating GPS tracking into the project, which not only aids in project development but also contributes to minimizing the problem to the greatest extent.

Conclusion

This paper elaborates on convolutional neural networks, image processing, and classification techniques, along with various methods for pothole recognition. Image processing techniques were employed in color segmentation with the aim of creating a pothole classifier. Using the MSNC algorithm, the developed classifier achieves an accuracy of 91%. The dataset comprises pothole images and is used for training to identify roads with potholes. This detector can classify potholes by obtaining similar S-N curves in nearly all images within the dataset. The model for this detector is precise and computationally efficient.

References

[1] K. C. P. Wang: Challenges and feasibility for comprehensive automated survey of pavement conditions, In 8th International Conference on Applications of Advanced Technologies in Transportation Engineering (2004). [2] Buza, E., Omanovic, S. & Huseinovic, A., 2013, Pothole Detection with Image Processing and Spectral Clustering, Antalya, Turkey, 2nd International Conference on Information Technology and Computer Networks. [3] Danti, A., Kulkarni, J. & Hiremath, P., 2012, An Image Processing Approach to Detect Lanes, Pot Holes and Recognize Road Signs in Indian Roads, International Journal of Modeling and Optimization. [4] Joubert, D., Tyatyantsi, A., Mphahlehle, J. & Manchidi, V., 2011, Pothole Tagging System Pretoria, 4th Robotics and Mechatronics Conference of South Africa. [5] Mathavan, S., Kamal, K., Rahman, M.: A review of three-dimensional imaging technologies for pavement distress detection and measurements, IEEE Transactions on Intelligent Transportation Systems. [6] Fan, R., Ozgunalp, U., Hosking, B., Liu, M., Pitas, I.: Pothole detection based on disparity transformation and road surface modeling, IEEE Transactions on Image Processing. [7] Koch, C., Georgieva, K., Kasireddy, V., Akinci, B., Fieguth, P.: A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure, Advanced Engineering Informatics. [8] A. Mednis, G. Strazdins, R. Zviedris, G. Kanonirs, and L. Selavo, Real time pothole detection using Android smartphones with accelerometers, In Proceedings of the International Conference on Distributed Computing in Sensor Systems and Workshops (2001). [9] Fan, R., Liu, M.: Road damage detection based on unsupervised disparity map segmentation, IEEE Transactions on Intelligent Transportation Systems (2019). [10] K. T. Chang, J. R. Chang and J. K. Liu: Detection of Pavement Distresses Using 3D Laser Scanning Technology, International Conference on Computing in Civil Engineering, 2005. [11] Li, Q., Yao, M., Yao, X and Xu, B. (2009): A real-time 3D Scanning System for pavement distortion inspection, measurement science and technology. [12] Chen Zhang, Wenzhu Yang, Zhaohai Liu, Daoliang Li, Yingyi Chen, Zhenbo Li: Color Image Segmentation in RGB Color Space Based on Color Saliency, International Conference on Computer and Computing Technologies in Agriculture,2013. [13] B. X. Yu, and X. Yu, Vibration-based system for pavement condition evaluation, In Proceedings of the 9th International Conference on Applications of Advanced Technology in Transportation (2006). [14] K. De Zoysa, C. Keppitiyagama, G. P. Seneviratne, and W.W.A.T. Shihan, A public transport system based sensor network for road surface condition monitoring, In Proceedings of Workshop on Networked Systems for Developing Regions (2007). [15] J. Erikson, L. Girod, and B. Hull, The pothole patrol: using a mobile sensor network for road surface monitoring, In Proceedings of the 6th International Conference on Mobile Systems, Applications, and Services (2008).

Copyright

Copyright © 2023 Dr. Venkata Ramana Kaneti. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.