Abstract
Addressing the urgent transition to low-carbon energy systems requires microgrids capable of locally coordinating electricity generation, storage, and flexible consumption. Their efficient integration calls for control strategies that are scalable, privacy-preserving, and robust to uncertainty. To address such a challenging control problem, this work proposes a decentralised Multi-Agent Reinforcement Learning (MARL) approach based on the Cross-Entropy Method (CEM) for the coordination of prosumers, equipped with renewable generation and vehicle-to-grid capabilities. To improve sample efficiency and robustness, the policy is warm-started using Behaviour Cloning (BC) from a classical Proportional-Integral-Derivative (PID) controller, resulting in a hybrid BC-CEM framework. The proposed method is evaluated in a realistic microgrid simulation with stochastic demand and real weather and generation profiles. Results show that BC-CEM accelerates convergence and achieves lower energy costs compared to both PID control and randomly initialized CEM, without sacrificing comfort or mobility requirements. The findings highlight the effectiveness of combining derivative-free optimization with imitation learning in complex MARL tasks, such as energy flexibility coordination.