Multi-Agent Contextual Combinatorial Multi-Armed Bandits with Linear Structured Super Arm: application to energy management optimization in Smart Grids

The integration of electric vehicles (EVs) and renewable energy sources (RES) into future electrical gridspresents both significant challenges and opportunities for energy management. While RES production isan incentive for increased demand at specific times of the day (e.g., at noon for photovoltaic production),the distribution system operator (DSO) should also prevent grid congestion. The optimal strategy forintegrating each EV with regard to the DSO constraints in a manner that maximizes the local useof RES is an NP-hard problem, as it requires the resolution of a mixed-integer linear programmingproblem (MILP) [1]. Moreover, due to weather and human behaviors, this optimization should be doneunder uncertainty. In particular, learning algorithms from the family of Bandit algorithms have beenstudied in [2], [3] to handle uncertainty. In light of the decentralized algorithms developed in [2], wepropose a Multi-Agent Contextual Combinatorial Multi-Armed Bandits (MA-CC-MAB) approach withlinearly structured super arms. By leveraging contextual information such as time-varying renewableenergy generation, grid load conditions and day of the week, each agent will selfishly and dynamicallyschedule its charging intervals while considering grid constraints. The use of linearly structured superarms enables efficient exploration and exploitation in a high-dimensional combinatorial action space,while decentralization will address scalability issues inherent to large systems. The use of contextualinformation allows adapting to different conditions of the environment, ensuring efficient decision-makingunder uncertainty. We show performance of this algorithm on the IEEE LVTF network model (55agents) with PV production, using pandapower python library for the load flow simulation.