Grid2Op is an open-source framework from Réseau de Transport d'Électricité (RTE), the operator of Europe’s largest high-voltage grid.

It is designed explicitly for training and evaluating reinforcement learning (RL) agents tasked with managing real-world power grids. By simulating realistic grid management scenarios, Grid2Op supports innovative research to stabilize power distribution, minimize outages, and integrate renewable energy sources efficiently.

• Replace inefficient, manually-defined clusters in Multi-Agent Reinforcement Learning (MARL) within Grid2Op.

• Identify and implement an optimal clustering solution that dynamically groups substations based on real grid connectivity.

• Enhance agent performance, improving decision-making efficiency and adaptability in diverse grid scenarios.

• Validate the solution by comparing performance against previous hard-coded clustering methods.

The Rootcode AI team addressed the limitations of manual substation clustering by implementing the Louvain algorithm, a powerful graph-based clustering method known for efficiently detecting meaningful community structures within large-scale networks. The team generated a comprehensive connectivity matrix, accurately representing the direct interconnections between substations. Using this connectivity data, the Louvain algorithm dynamically clustered substations, maximizing network modularity, ensuring substations grouped had dense internal connections and minimal external links. Subsequently, these clustering outcomes were translated into an actionable format, clearly mapping clusters to specific MARL agents within Grid2Op.

Upon validation, the Louvain clustering approach demonstrated substantial performance improvements over the traditional, manually-defined clustering methods. Specifically, maximum episode rewards dramatically increased from approximately 4,000 to over 30,000, accompanied by consistently higher mean episode rewards. The method also showed enhanced exploration and stability, highlighting its superior adaptability across diverse operational scenarios. This implementation empowers Grid2Op users worldwide, providing a more dynamic, intelligent, and scalable solution for effective power grid management and increased grid reliability.