Multimodal Data Fusion-based Ergonomic Assessment Method for Construction Workers

Abstract

Work-related musculoskeletal disorders (WMSDs) are the leading cause of non-fatal injuries in the construction industry. Due to the high-intensity and repetitive tasks, construction workers are at a significantly higher risk of developing WMSDs compared to other sectors. These disorders often start with reversible symptoms but can progress to chronic conditions, resulting in long-term disabilities that adversely affect workers' health and the economy. Accurate ergonomic assessment methods can effectively prevent these diseases. Traditional ergonomic assessments typically depend on expert observations utilizing evaluation scales. It is often costly, time-consuming, and subjective, which limits its practical applications in construction sites. Vision-based methods are widely used in construction sites because of their non-invasiveness and cost-effectiveness. However, these methods have difficulty obtaining accurate assessment data in the presence of visual obstructions, such as low lighting, object occlusions, and body parts not being visible. Additionally, these methods do not account for external load factors in ergonomics and lack automated approaches for assessing repetitive motions. As a result, existing methods in complex construction sites often lead to significant errors in risk assessment. This study proposes a multimodal ergonomic assessment method that combines visual data and pressure signals to evaluate ergonomic risk factors for workers, including fatigue postures, external loads, and repetitive motions. The multimodal approach integrates pressure and visual data into a unified posture feature space, enhancing posture estimation results and providing ergonomic risk assessment data for external loads and repetitive motions. The proposed method is validated on challenging real-world construction datasets, showing an average risk assessment accuracy 16.9% higher than existing methods on RULA, REBA, and OWAS evaluation criteria. The proposed multimodal approach enhances health and safety standards in the construction industry by providing precise ergonomic risk assessments for workers and relevant recommendations, ultimately contributing to the industry's long-term sustainability.

Date of Award	2025
Original language	English
Awarding Institution	The Hong Kong University of Science and Technology
Supervisor	Yantao YU (Supervisor)