Point Cloud-based Deep Learning and UAV Path Planning for Surface Defect Detection of Concrete Bridges

Over the past decades, several bridges have collapsed, causing many losses due to the lack of proper monitoring and inspection. Although several new techniques have been developed to detect bridge defects, annual visual inspection remains the main approach. Visual inspection, using naked eyes, is time-consuming and subjective because of human errors. Light Detection and Ranging (LiDAR) scanning is a new technology to collect 3D point clouds. The main strength of point clouds over 2D images is collecting the third dimension of the scanned objects. Deep Learning (DL)-based methods have attracted the researchers’ attention for concrete surface defect detection. However, no point cloud-based DL method is currently available for semantic segmentation of bridge surface defects without converting the raw point cloud dataset into other representations, which results in increasing the size of the dataset and leads to some challenges regarding storage capacity, cost, and training time. Some promising point cloud-based semantic segmentation methods (i.e., PointNet and PointNet++) have been applied in segmenting bridge components (i.e., slabs, piers), but not for segmenting surface defects (i.e., cracks, spalls). Moreover, most of the current point cloud-based concrete surface defect detection methods focus on only one type of defects. On the other hand, in DL, a dataset plays a key role in terms of variety, diversity, accuracy, and size. The lack of publicly available point cloud datasets for bridge surface defects is one of the reasons of the lack of studies in the area of point cloud-based methods.
\nFurthermore, compared with terrestrial LiDAR scanning, LiDAR-equipped Unmanned Aerial Vehicle (UAV) is capable of scanning the inaccessible surfaces of the bridges at a closer distance with higher safety. Although the UAV flying path can be controlled using remote controllers, automating and optimizing UAV path planning is preferable for being able to trace a collision-free path with minimum flight time. To increase the efficiency and accuracy of this approach, it is crucial to scan all parts of the bridge with a near perpendicular view. However, in the case of obstacle existence (e.g., bridge piers), achieving full coverage with near perpendicular view may not be possible. To provide more accurate results, using overlapping views is recommended. However, this method could result in increasing the inspection cost and time. Therefore, overlapping views should be considered only for surface areas where defects are expected.
\nAddressing the above issues, this research aims to: (1) create a publicly available point cloud dataset for concrete bridge surface defect semantic segmentation, (2) develop a point cloud-based semantic segmentation DL method to detect different types of concrete surface defects, and (3) propose a novel near-optimal path planning method for LiDAR-equipped UAV with respect to the minimum path length and maximum coverage considering the potential locations of defects.
\nOn this premise, a point cloud-based DL method for semantic segmentation of concrete bridge surface defects (i.e., cracks and spalls), called SNEPointNet++, is developed. To have a network with high-performance, SNEPointNet++ focuses on two main characteristics related to surface defects (i.e., normal vector and depth) and takes into account the issues related to the point cloud dataset (i.e., small size and imbalanced dataset). Sensitivity analysis is applied to capture the best combination of hyperparameters and investigate their effects on network performance. The dataset, which was collected from four concrete bridges, was annotated, augmented, and classified into three classes: cracks, spalls, and no defect. This dataset is made available for other researchers. The model was trained and evaluated using 60% and 20% of the dataset, respectively. Testing on the remaining part of the dataset resulted in 93% recall (69% IoU) and 92% recall (82.5% IoU) for cracks and spalls, respectively. Moreover, the results show that the spalls of the segments deeper than 7 cm (severe spalls) can be detected with 99% recall. 
\nOn the other hand, this research proposes a 3D path planning method for using a UAV equipped with a LiDAR for bridge inspection to have efficient data collection. The method integrates a Genetic Algorithm (GA) and A* algorithm to solve the Traveling Salesman Problem (TSP), considering the potential locations of bridge surface defects such as cracks. The objective is to minimize the time of flight while achieving maximum visibility. The method provides the potential locations of surface defects to efficiently achieve perpendicular and overlapping views for sampling the viewpoints. Calculating the visibility with respect to the level of criticality leads to giving the priority to covering the areas with higher risk levels. Applying the proposed method on a 3-span bridge in Alberta, the results reveal that considering overlapping views based on the level of criticality of the zones and perpendicular views for all viewpoints leads to accurate and time-efficient data collection.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.