Vision-Based Asphalt Pavement Defect Detection Using Deep CNN with BIM Integration

Abstract

The major objective of the research is to predict the defects on the asphalt pavement which is the main concern for the smooth movement of vehicles by using deep CNN. CNN consists of different operations that is image classification, object detection, and image segmentation. We use detectron2 for labelling the defective part to detect the defect accurately. The data set which consists of images of defects of pavement is collected and used for training the algorithm and some of the collected images are used for the testing. The suggested approach makes use of deep learning to automatically detect and categorize a range of asphalt pavement defects, such as surface distress, potholes, and cracks. The training, validation, and testing of the model are conducted on an extensive dataset of annotated pavement photographs. The trained model shows resilience and the ability to generalize, which qualifies it for practical uses in infrastructure management and pavement upkeep. This study advances automated pavement inspection systems, which provide a practical and economical means of determining which maintenance tasks should be prioritized to maintain safer and smoother roads and this approach predicted the defects with an accuracy of 98%.

Introduction

I. INTRODUCTION

The foundation of the global transportation system is asphalt pavement, which makes it easier to move people and products. However, with time, several variables including weather, traffic volume, and shoddy construction result in the development of flaws in asphalt pavements. Road networks must be repaired and maintained to be safe, long-lasting, and economically viable. Pavement flaws have traditionally been found by manual inspection techniques, which are labour intensive, time-consuming, and prone to human mistakes. An important trend in asphalt pavement technology is the use of Convolutional Neural Networks (CNNs), or deep learning technologies, for automated flaw detection. This study investigates the use of deep CNNs in asphalt pavement defect identification. CNNs can accurately identify a variety of flaws, such as cracks, potholes, rutting, and surface distress, by using massive datasets of photographs of pavement. Deep learning improves the accuracy and dependability of problem detection while streamlining the inspection process, which results in more efficient maintenance and rehabilitation plans. we investigate the design and training strategies of deep CNN models designed for the identification of defects in asphalt pavement. We also go over the opportunities and difficulties that come with putting these technologies to use in practical settings, including issues with deployment, scalability, and interface with current pavement management systems. All things considered, the incorporation of deep CNNs has great potential to transform the way we identify and address flaws in asphalt pavement, ultimately leading to the development of safer and more environmentally friendly transportation infrastructure.

II. LITERATURE REVIEW AND METHODOLOGY

The text discusses a vision-based system for detecting distressed areas on pavements using low-cost technologies. Videos of pavement surfaces are recorded by a camera on a moving vehicle and an algorithm processes the data to identify pavement defects with high accuracy. This system offers a cost-effective and efficient method for pavement assessment, potentially saving time and resources for transportation departments [1]. The text discusses the importance of selecting optimal hyperparameters for deep convolutional neural networks (DCNNs) used in crack segmentation tasks. It introduces a Bayesian optimization framework to guide the selection of hyperparameters for improved segmentation accuracy and provides insights into the importance of different hyperparameters on the performance of DCNNs in crack segmentation tasks using real-world data. The study aims to enhance the performance and reproducibility of DCNN models by automating the hypermeter optimization process through Bayesian optimization [2]. The text discusses the use of deep learning frameworks for automatically detecting and classifying pavement distresses, which is crucial for, Maintaining and rehabilitating pavement surfaces.

The study aimed to assess the ability of these models to predict and classify distress by testing them on datasets from the IEEE Global Road Damage Detection Challenge, achieving an F1 score of 0.58 and 0.57 on two data sets [3]. The text discusses the use of deep learning techniques, specifically faster R-CNN and YOLO v3, for automatically detecting and classifying pavement distresses in 3D pavement surface images. These methods achieved high accuracy levels, around 89.6% to 89.8%, and were compared to manual quality control results, showing a high level of agreement. The study highlights the potential of using deep learning and 3D imaging for efficient and accurate assessment of pavement conditions, aiding in maintenance and decision-making processes for infrastructure management [4]. The text discusses a vision-based system for detecting distressed areas on pavements using low-cost technologies.  Videos of pavement surfaces are recorded by a camera on a moving vehicle and an algorithm processes the data to identify pavement defects with high accuracy. This system offers a cost-effective and efficient method for pavement assessment, potentially saving time and resources for transportation departments [5]. The text discusses the development of a system called Multi-Distress Detection System (MDDS) for asphalt pavements using artificial intelligence techniques. The system utilizes you only look once version 4 (YOLOv4) with transfer learning to detect various distresses like cracking, potholes, and patch deterioration from video images of pavement conditions The MDDS algorithm was trained on multiple datasets and achieved an average loss of 1.5123 during training, showing promise for real-time distress detection in pavement monitoring [6]. The text discusses the current advancements in defect detection using machine vision technology. It highlights the importance of optical illumination, image acquisition, image processing, and deep learning in improving defect detection efficiency and reliability in industrial settings. machine vision technology, which involves automatic image processing and analysis, is replacing traditional manual visual inspection methods to meet modern industrial production lines' efficiency and quality requirements [7]. The text discusses a method for automatically detecting cracks in asphalt pavement using deep learning technology, specifically a convolutional neural network (CNN).

The CNN is trained to identify crack features from images without the need for preprocessing. This approach shows promising results with about 96% accuracy in detecting pavement cracks, offering a potential automated solution for managing pavement distress effectively [8]. The text discusses a method for automatically detecting cracks in asphalt pavement using deep learning technology, specifically a convolutional neural network (CNN). The CNN is trained on a dataset of pavement images to identify cracks without the need for preprocessing, achieving around 96% accuracy. This automated approach offers a promising solution for efficiently and accurately detecting pavement distress, such as cracks, which is crucial for maintaining road safety and capacity [9]. The text refers to a study that utilizes computer vision technology to estimate the influence of lines for highway bridges. Influence lines help engineers understand how loads affect the structure of a bridge, and the Bayesian parametric estimation method is used to analyse the data obtained from computer vision measurements to determine these influence lines accurately. This research aims to improve the monitoring and maintenance of highway bridges by leveraging advanced technology for structural analysis and assessment [10].

III. METHODOLOGY

A. Data Collection and Preprocessing

The gathering of a varied dataset with pictures of asphalt pavements with different kinds of flaws like rutting, cracks, and potholes.
Labelling these photos with annotations that identify the kinds and locations of any faults. Image preprocessing, such as scaling, normalization, and augmentation methods, to improve the dataset's quality and variety. Select the best deep CNN architecture for the job. Popular options include Inception, VGG, ResNet, and custom architectures made for certain tasks. The chosen architecture ought to strike a suitable compromise between performance and computational complexity.

B. Model Architecture Selection

Select the best deep CNN architecture for the job. Popular options include Inception, VGG, ResNet, and custom architectures made for certain tasks. The chosen architecture ought to strike a suitable compromise between performance and computational complexity.

C. Transfer Learning

Provide pre-trained weights from a model trained on a sizable dataset, such as ImageNet, to the selected CNN architecture at startup. This facilitates the use of the features discovered from a variety of image sets. Adjust the pre-trained model to the particular task of defect detection by fine-tuning it using the dataset of asphalt pavement.

D. Training

Make training, validation, and test sets out of the dataset. Utilizing the training set, train the CNN model. The model gains the ability to link input photos to defect labels during training. Through the use of methods like Adam optimization or stochastic gradient descent (SGD) to propagate gradients both forward and backward. Check for overfitting by evaluating the model's performance on the validation set and modifying the hyperparameters as necessary.

E. Evaluation

Analyse the trained model's performance in identifying pavement flaws using the test set. Performance measures including F1-score, accuracy, precision, recall, and IoU (Intersection over Union) are frequently used to assess how well a model works.

F. Post-processing

To improve the detected faults, post-processing procedures (e.g., eliminating false positives, filling in gaps in detected cracks) might be applied to the model's predictions.

G. Deployment

The model can be used for real-world defect detection jobs once it performs well enough. The trained model may be deployed by incorporating it into a program or system that can instantly assess photos or video feeds of pavement surfaces.

H. Convolutional Neural Network (CNN)

1. We use different filters to extract different featured images.
2. Convolutional neural network is used for recognize characters such as digits and alphabets.
3. A convolutional neural network (CNN) is a type of artificial neural network that’s also used for image recognition and processing.
4. They are used for tasks that require object recognition, such as image classification, detection, and segmentation.
5. It is used in reducing multiplication operators to produce featured images.
6. Less processing power can be achieved in training up the model.

I. Detectron2:

1. Prepare the custom data set on our own and then we train the detectron2 by using LABELME software.
2. We have to keep all our required images in one folder and then type pip install LabelMe, and Open the images folder.
3. Create the polygon by drawing it into the required shape then name it and save it.
4. All image's height and width should be the same.
5. Train and test the detectron2, the training time should be less than 8-9 mins.
6. Each image has four keys-

• File Name
• Height
• Width
• Annotation.

Default predictor which predicts the data by running the algorithm it gives the majority.

Conclusion

In conclusion, the effective use of deep Convolutional Neural Networks (CNNs) in asphalt pavement fault detection marks a major advancement in infrastructure safety and maintenance. This technology demonstrates the transformative potential of artificial intelligence in addressing important difficulties faced by road maintenance agencies and transportation authorities, with an amazing accuracy rate of over 90%. Deep CNNs provide proactive maintenance techniques by effectively detecting a variety of pavement flaws, including cracks, potholes, and surface deformations. This allows for prompt repairs and reduces the accidents and vehicle damage. Furthermore, by optimizing resource allocation and prioritizing maintenance interventions based on the severity and location of problems, the integration of deep learning algorithms into pavement management systems not only improves operational efficiency but also lowers cost. Further development and incorporation of deep CNNs into current infrastructure management frameworks, along with research and technological advancements, has the potential to transform pavement maintenance practices and guarantee the robustness and lifespan of road networks globally. This joint venture between civil engineering and artificial intelligence highlights the value of innovation in preserving public infrastructure and improving community well-being worldwide.

References

[1] Sarker, M.M. et al. (2021) ‘A means of assessing deep learning-based detection of ICOS protein expression in colon cancer’, Cancers, 13(15), p. 3825. doi:10.3390/cancers13153825. [2] Canchila, C., Zhou, S. and Song, W. (2024a) ‘Hyperparameter optimization and importance ranking in deep learning–based crack segmentation’, Journal of Computing in Civil Engineering, 38(2). doi: 10.1061/jccee5.cpeng-5512. [3] Mandal, V., Mussah, A.R. and Adu-Gyamfi, Y. (2020) ‘Deep learning frameworks for pavement distress classification: A comparative analysis’, 2020 IEEE International Conference on Big Data (Big Data) [Preprint]. doi:10.1109/bigdata50022.2020.9378047. [4] Ghosh, R. and Smadi, O. (2021) ‘Automated detection and classification of pavement distresses using 3D pavement surface images and deep learning’, Transportation Research Record: Journal of the Transportation Research Board, 2675(9), pp. 1359–1374. doi:10.1177/03611981211007481. [5] Hadjidemetriou, G.M. and Christodoulou, S.E. (2019) ‘Vision- and entropy-based detection of distressed areas for Integrated Pavement Condition Assessment’, Journal of Computing in Civil Engineering, 33(3). doi:10.1061/(asce)cp.1943-5487.0000836. [6] Peraka, N.S., Biligiri, K.P. and Kalidindi, S.N. (2021) ‘Development of a multi-distress detection system for asphalt pavements: Transfer learning-based approach’, Transportation Research Record: Journal of the Transportation Research Board, 2675(10), pp. 538–553. doi:10.1177/03611981211012001. [7] Ren, Z. et al. (2021) ‘State of the art in defect detection based on machine vision’, International Journal of Precision Engineering and Manufacturing-Green Technology, 9(2), pp. 661–691. doi:10.1007/s40684-021-00343-6. [8] Han, Z. et al. (2021) ‘Vision-based crack detection of asphalt pavement using deep convolutional neural network’, Iranian Journal of Science and Technology, Transactions of Civil Engineering, 45(3), pp. 2047–2055. doi:10.1007/s40996-021-00668-x. [9] Ruseruka, C. et al. (2023) ‘Pavement distress identification based on computer vision and Controller Area Network (CAN) sensor models’, Sustainability, 15(8), p. 6438. doi:10.3390/su15086438. [10] Zhou, Y. et al. (2023) ‘Identification of influence lines for highway bridges using Bayesian parametric estimation based on computer vision measurements’, Journal of Bridge Engineering, 28(12). doi: 10.1061/jbenf2.beeng-6235.

Copyright

Copyright © 2024 P. Dimpul Kumar, P. Pavani, S. Yamini, S. Harshavardhini. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.