Real-time vehicle fleet monitoring on edge using deep learning technique and point cloud data

Abstract

Many of the ITS applications require real-time 3D information of the traffic conditions such as location, size, class and velocity of vehicles. From time criticality and scalability perspective, most of the computation processes such as 3D object detection algorithm, tracking, counting vehicles and estimating vehicle velocities should be done on edge due to bandwidth and latency constraints. When it comes to generating accurate, consistent 3D information, LiDAR is a better sensor compared to video cameras. However, the use of point cloud in vehicle fleet monitoring system has many challenges to overcome, such as the computation complexity of the 3D object detection algorithm, and the capability of performing fine-grain vehicle classification. In this thesis, a scalable vehicle fleet monitoring system that runs on edge in real-time using deep learning technique and point cloud data has been developed, fulfilling all the demands mentioned. Multi-beam flash LiDAR is specifically selected for its high point rate, and operating frequency. A GPU accelerated sensor interface module is developed to extract points of interest from the high density point cloud efficiently, through background filtering and point-in-polygonal-prism cropping. A 3D object detection model, DV-Det is then used to detect and classify on-road vehicles. The detected vehicle is assigned with unique ID and its velocity is estimated using a 3D multi-vehicle tracking algorithm, AB3DMoT. With the trajectory information of the tracked vehicle, a counting algorithm specialized for 3D scene extracts the vehicle count. The whole system can run at real-time speed of 10Hz with a latency of 62.71 milliseconds, while still achieving satisfying fine-grain vehicle classification performance of at least 80mAP at 3D IoU threshold of 0.5, vehicle counting with 100% recall and at least 80% precision for most of the classes. Example of application of the systems on emission inventory estimation and traffic pattern visualization are conducted.

Date of Award	2022
Original language	English
Awarding Institution	The Hong Kong University of Science and Technology
Supervisor	Yu-Hsing WANG (Supervisor)