Integrating Hardware-in-the-loop simulation and BIM for planning UAV-based as-built MEP inspection with deep learning techniques

Unmanned Air Vehicles (UAVs) are increasingly used and have many potential applications in the architecture, engineering, and construction (AEC) industry. UAVs are now commonly controlled manually with an experienced survey engineer in field operation, but UAVs can also fly autonomously with the support of localization technologies such as simultaneous localization and mapping (SLAM). Autonomous flying of UAVs can reduce human effort and minimize human error. However, it requires careful flight path planning, active environment sensing, and effective obstacle avoidance techniques, especially where the built environment is complex and space is tight. Currently, algorithms flight path planning, object detection and obstacle avoidance are developed based on testing that involves actual Survey UAVs; this is not only time consuming but also expensive and hazardous in the sense that it may damage hardware and injury ground personnel if error-induced crashes occur. This paper presents a hardware-in-the-loop simulator which can simulate the behavior of a drone and generate synthetic images from the scene as datasets, detect and verify as-built MEP objects with a trained neural network, and generate point cloud data for validating as-designed BIM model with as-built BIM model in the future. The system architecture, operations, and performance of the developed simulator are discussed, followed by an illustrative example.