数据与计算发展前沿 ›› 2026, Vol. 8 ›› Issue (3): 166-180.

doi: 10.11871/jfdc.issn.2096-742X.2026.03.014

• 技术与应用 • 上一篇    下一篇

基于强化学习的电力供应链多层博弈系统

牛欣欣1(),刘宇轩2,王艺憬3,游博4,*(),李学恩4   

  1. 1 国家能源集团新能源技术研究院有限公司， 北京 102209
    2 中国科学院自动化研究所， 北京 100190
    3 哈尔滨工业大学， 黑龙江 哈尔滨 150001
    4 天津中科智能识别有限公司， 天津 300457
  • 收稿日期:2025-10-29 出版日期:2026-06-20 发布日期:2026-06-18
  • 通讯作者: 游博
  • 作者简介:牛欣欣，国家能源集团新能源技术研究院有限公司，基石运营技术研究中心，总工程师，主要研究方向为人工智能、精益调度等。
    本文主要工作为方法的提出和实现。
    NIU Xinxin, is the chief engineer of the Cornerstone Operation Technology Research Center, CHN Energy New Energy Technology Research Institute Co., Ltd. His research interests include artificial intelligence and lean scheduling.
    In this paper, he is mainly responsible for proposing and implementing the method.
    E-mail: 16810093@ceic.com|游博，天津中科智能识别有限公司，工程师，研究方向为计算机视觉、智能体等。
    本文中负责写作指导以及论文最终审定。
    YOU Bo is currently an engineer at Tianjin Zhongke Intelligent Identification Co., Ltd. Her research interests include computer vision and intelligent agents.
    In this paper, she is responsible for paper writing instruction and manuscript reviewing.
    E-mail: youbo2019@ia.ac.cn
  • 基金资助:
    国家能源集团科技项目“基于数字孪生技术的全产业链生产运营一体化智能仿真平台研发与应用”(GJNY-23-176)

Multi-Layer Game System of Power Supply Chain Based on Reinforcement Learning

NIU Xinxin1(),LIU Yuxuan2,WANG Yijing3,YOU Bo4,*(),LI Xueen4   

  1. 1 CHN Energy New Energy Technology Research Institute Co., Ltd, Beijing 102209, China
    2 Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
    3 Harbin Institute of Technology, Harbin, Heilongjiang 150001, China
    4 Tianjin Zhongke Intelligent Identification Co., Ltd, Tianjin 300457, China
  • Received:2025-10-29 Online:2026-06-20 Published:2026-06-18
  • Contact: YOU Bo

RichHTML

1

PDF

27

摘要：

【目的】为解决电力与煤炭供应链多层级博弈优化问题，本文以中国省级煤电供应链为背景，针对传统规则决策与博弈论均衡求解难以处理动态高维动作空间及参与者学习适应性的局限，构建了包含省级发电公司、市级电厂、煤矿单位的多智能体模型。 【方法】该模型采用Stackelberg博弈框架实现层级协调，结合纳什均衡模拟同级竞争，并集成TD3BC强化学习算法优化智能体决策，配套统一价格拍卖的市场清算机制保障供需匹配。 【结果】通过对比利润保护型、纯成本最优型、市场化竞价型三种电厂博弈目标的运行效果，发现市场化竞价型在系统整体效益与供需平衡上表现最优，且启用TD3BC算法后，系统总利润、市场效率及稳定性较传统规则决策显著提升。 【局限】研究局限在于采用简化的市场参数，未考虑运输拓扑结构及真实市场中的长期合同、发电机组约束等因素。 【结论】强化学习与多层博弈结合的方法可有效优化电力供应链决策，为煤电一体化运营提供理论支撑，市场化竞价策略更适用于追求系统高效与利润增长的场景。

关键词: 电力供应链, 多层博弈, 强化学习, 市场清算机制, TD3BC算法, Stackelberg博弈, 供需平衡

Abstract:

[Objective] To address the multi-level game optimization problem in the power and coal supply chain, this study uses China’s provincial coal-fired power supply chain as a context. Addressing the limitations of traditional rule-based decision-making and game-theoretic equilibrium solutions in handling dynamic, high-dimensional action spaces and the learning adaptability of participants, this study constructs a multi-agent model encompassing provincial operators, municipal power plants, and coal mines. [Methods] This model employs a Stackelberg game framework for hierarchical coordination, incorporates Nash equilibrium to simulate intra-level competition, and integrates the TD3BC reinforcement learning algorithm to optimize agent decision-making. A unified price auction market clearing mechanism ensures supply and demand matching. [Results] By comparing the performance of three power plant game objectives—profit protection, pure cost optimization, and market-based bidding—the market-based bidding model demonstrates the best overall system efficiency and supply-demand balance. Furthermore, the implementation of the TD3BC algorithm significantly improves system total profit, market efficiency, and stability compared to traditional rule-based decision-making. [Limitations] This study is limited by the use of simplified market parameters and the lack of consideration of factors such as transportation topology, long-term contracts, and unit constraints in real markets. [Conclusions] The method combining reinforcement learning with multi-layer game theory can effectively optimize the decision-making of the power supply chain and provide theoretical support for the integrated operation of coal and electricity. The market-based bidding strategy is more suitable for the scenario of pursuing system efficiency and profit growth.

Key words: power supply chain, multi-layer game, reinforcement learning, market clearing mechanism, TD3BC algorithm, Stackelberg game, supply and demand balance