Monitoring big process data of industrial plants with multiple operating modes based on Hadoop

For modeling and monitoring large-scale plant-wide processes with big data from multiple operating conditions, a novel distributed parallel Gaussian mixture model is proposed based on the Hadoop MapReduce framework. To deal with high-dimensional process variables, a multiblock method is adopted. For big data chunks in each divided block, an analytical procedure is carried out with three key procedures. First, the fundamental data statistics are obtained with the designed distributed and parallel manners for data standardization. Second, conventional Gaussian mixture model learning steps are accommodated in the parallel paradigm of the MapReduce platform. Finally, multilevel fault detection and diagnosis schemes are developed to conduct hierarchical monitoring from plant-wide, unit block, and variable levels. The feasibility and effectiveness of the proposed method are demonstrated on two study cases.