AI-Based Digital Twinning for Automated Joint 3D Scene Reconstruction and Semantic Enrichment

Digital twin, with captured as-is information, provides potential for condition monitoring and inspection of the built environment. However, the state-of-the-art digital twin lacks a systematic approach to observing the environment and communicating information to the facility manager due to the incomplete data description capabilities. In addition, capturing as-built geometric information would enhance the digital twin towards smart facility management, but the current employment of 2D computer vision provides limited support to reflect the building conditions for maintenance management. Therefore, this paper presents an AI-based digital twinning approach for automated joint 3D scene reconstruction and semantic enrichment to incorporate the as-is information of the built environment. Two works are researched sequentially: the first concerns integrating BIM data schema with Sensor Model Language (SensorML) to enhance sensor description capability for assorted information queries, and the second focuses on an automated 3D reconstruction and defect detection to enrich digital twin with the as-is condition of the built environment. It is envisaged that the research will contribute to a new method to enhance the digital twin for the built environment including a scientific approach for data mapping between the BIM domain and sensing domain and a new framework for capturing accurate as-built geometric information on the digital twin.