Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2305.06921

How to Use Reinforcement Learning to Facilitate Future Electricity Market Design? Part 2: Method and Applications

Ziqing Zhu·Siqi Bu·Ka Wing Chan·Bin Zhou·Shiwei Xia
View original source ↗Read source document ↗Export metadata ↓

Abstract

This two-part paper develops a paradigmatic theory and detailed methods of the joint electricity market design using reinforcement-learning (RL)-based simulation. In Part 2, this theory is further demonstrated by elaborating detailed methods of designing an electricity spot market (ESM), together with a reserved capacity product (RC) in the ancillary service market (ASM) and a virtual bidding (VB) product in the financial market (FM). Following the theory proposed in Part 1, firstly, market design options in the joint market are specified. Then, the Markov game model is developed, in which we show how to incorporate market design options and uncertain risks in model formulation. A multi-agent policy proximal optimization (MAPPO) algorithm is elaborated, as a practical implementation of the generalized market simulation method developed in Part 1. Finally, the case study demonstrates how to pick the best market design options by using some of the market operation performance indicators proposed in Part 1, based on the simulation results generated by implementing the MAPPO algorithm. The impacts of different market design options on market participants' bidding strategy preference are also discussed.

Explore related subjects

cs.GTcs.AIcs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Ziqing Zhu, Siqi Bu, Ka Wing Chan, Bin Zhou, Shiwei Xia. 2023-05-12. How to Use Reinforcement Learning to Facilitate Future Electricity Market Design? Part 2: Method and Applications. https://arxiv.org/abs/2305.06921

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Coverage Games

We introduce and study coverage games - a novel framework for multi-agent planning in settings in which a system operates several agents but does not have full control on them, or interacts with an environment that consists of several agents. The game is played between a coverer, who has a set of objectives, and a disruptor. The coverer operates several agents that interact with the adversarial disruptor. The coverer wins if every objective is satisfied by at least one agent. Otherwise, the disruptor wins. Coverage games thus extend traditional two-player games with multiple objectives by allowing a (possibly dynamic) decomposition of the objectives among the different agents. They have many applications, both in settings where the system is the coverer (e.g., multi-robot surveillance, coverage in multi-threaded systems) and settings where it is the disruptor (e.g., prevention of resource exhaustion, ensuring non-congestion). We first study the theoretical properties of coverage games, including determinacy, and the ability to a priori decompose the objectives among the agents. We then study the problems of deciding whether the coverer or the disruptor wins. Besides a comprehensive analysis of the tight complexity of the problems, we consider interesting special cases, such as the one-player cases and settings with a fixed number of agents or objectives.

cs.GT↗
arXiv2026

Trading Proportionality for Strategic Robustness in Multi-Winner Approval Voting

Classical strategyproofness assumes a manipulator either knows how everyone else votes or is willing to gamble as if they did. Real voters rarely do. The recently introduced RAT-degree measures how many other participants' reports an agent must actually observe before a manipulation becomes strictly safe, interpolating between full truthfulness and immunity to blind manipulation. While previously explored in auctions and single-winner settings, we bring this measure to multi-winner elections. We apply it to Approval-Based Committee (ABC) rules under free-riding: a voter drops approved candidates from their truthful ballot to concentrate weight on marginal ones. We first analyze Proportional Approval Voting (PAV). Knowledge of $\lceil n/k \rceil$ ballots already enables a safe and strictly profitable drop, whereas knowledge of at most $\lfloor n/(k+1) \rfloor - 1$ ballots leaves the rule completely immune; an explicit instance shows the latter bound cannot be raised in general. Since a manipulator informed about roughly a $1/k$ fraction of the electorate therefore suffices, we ask how much proportionality must be surrendered to buy strategic robustness. We introduce $d$-RPAV, a parameterized family of Thiele rules with weights $d/(j+d-1)$ that recovers PAV at $d = 1$ and approaches Approval Voting (AV) as $d$ grows. We prove that $d$-RPAV satisfies $α$-Justified Representation ($α$-JR) for $α= d$, and is immune to safe free-riding given up to $\lfloor dn/(k+2d-1) \rfloor - 1$ known ballots, yielding a clean and tunable trade-off between proportional representation and strategic robustness.

cs.GT↗
arXiv2026

Online Fair Division: Pushing the Frontier of Approximate Proportionality

Online fair division captures allocation problems in which indivisible resources arrive over time and must be assigned before future resources are known. Understanding what fairness remains achievable when allocation decisions are immediate and irrevocable is a fundamental question in this setting. We study deterministic online allocation among $n$ agents with nonnegative additive valuations, where the number of goods is unknown and the adversary can adapt to previous allocation decisions. We focus on proportionality up to one good (PROP1) and examine how advance information affects the achievable guarantees. Without additional future information, we give a deterministic algorithm that guarantees $Ω(\frac{1}{\log(nm)})$-PROP1 after every round, where $m$ is the number of goods at termination. We complement this result by showing that, for every $n$ and sufficiently large $m$, every deterministic algorithm has an adaptive instance with $m$ goods on which its allocation has PROP1 approximation guarantee $O(\frac{\log \log m}{\log m})$. Thus, when the number of agents is fixed, our upper and lower bounds on the competitive ratio differ by at most an $O(\log\log m)$ factor. These results answer an open question proposed by Choo et al. on whether a nontrivial deterministic approximation for PROP1 can be obtained. We also study the setting where the algorithm knows the predictions of the maximum item value for every agent. When the predictions are accurate, we give a deterministic $\frac{1}{2}$-PROP1 algorithm, improving the $\frac{1}{n}$ guarantee of Choo et al. to a constant. We further establish an explicit upper bound below one on the competitive ratio, even for two agents with accurate predictions. Finally, we give a single deterministic algorithm that guarantees $\frac{1}{2}$-PROP1 when predictions are accurate and $Ω(\frac{1}{\log (nm)})$-PROP1 for arbitrary predictions.

cs.GT↗