Chip-Specific Power Delivery and Consumption Co-Management for Process-Variation-Aware Manycore Systems Using Reinforcement Learning

Energy efficiency has become a critical design metric for high-performance systems. Various power management techniques have been proposed for the processor cores such as dynamic voltage and frequency scaling (DVFS), whereas few solutions consider the power losses suffered on the power delivery system (PDS), despite the fact that they have a significant impact on the overall energy efficiency of the system. With the explosive growth of system complexity and highly dynamic workloads variations, it is also challenging to find the optimal power management policies which can effectively match the power delivery with the power consumption. In addition, process variations (PVs) add heterogeneity to systems and make traditional power management methods less effective. To tackle the above problems, we propose a reinforcement-learning-based Chip-Specific Power co-Management (CSPM) scheme for PV-aware manycore systems. Both PDS and processor cores are jointly adjusted by distributed agents with modular Q-learning to improve the overall energy efficiency of the system. System characteristics are naturally included in the learning process to obtain chip-specific policies. Experimental results show that when applied to PV-aware manycore systems with a hybrid PDS constructed by both on- and off-chip voltage regulators, the proposed method achieves a 60.1% reduction of the overall energy delay product (EDP) of the system, on average, compared to a traditional DVFS approach.