An Efficient Bayesian Policy Exploration Approach for Reinforcement Learning Model Predictive Control

Reinforcement Learning Model Predictive Control (RL-MPC) has achieved significant progress in recent years. However, existing approaches still have some limitations. This paper proposes a Bayesian policy exploration method for RLMPC that substantially enhances its performance. Specifically, we implement Bayesian posterior estimation of value functions and introduce an optimistic exploration strategy tailored for efficient exploration of RL-MPC, which improves the sample efficiency of RL policy exploration. Then an optimistic Bayesian exploration strategy is proposed, which encourages the agent to leverage existing model information to achieve superior control performance. The soundness and effectiveness of our method are evaluated through an empirical study of controlling a drone to reach targets subject to uncertain model parameters and environmental perturbations. The results validate that our approach has superior performance compared with benchmarks.