A learning method to evaluate a generation company's bidding strategy in the electricity market

In the electricity market, generation companies (GenCos) are usually faced with the problem of choosing a better bidding strategy. They often have to evaluate each possible strategy according to its potential reward. In a competitive market environment, the electricity price is stochastic and volatile, and the GenCo's mixed strategies also make the problem more complicated. In this paper, we model the market price with a Markov regime-switching model and propose the temporal difference learning method in the Markov decision process to approximate the expected reward over an infinite horizon. The simulations based on this method have achieved the evaluation of 2 mixed strategies. The results show the difference of the expected rewards between the strategies, which could be important evidence for choosing a better strategy.

Anahtar Kelimeler:

Bidding strategy, Markov decision process, temporal difference learning, Markov regime-switching

A learning method to evaluate a generation company's bidding strategy in the electricity market

Keywords:

Bidding strategy, Markov decision process, temporal difference learning, Markov regime-switching,

PDF

___

A.R. Kian, J.B. Cruz, “Bidding strategies in dynamic electricity markets”, Decision Support Systems, Vol. 40, pp. 543–551, 2005.
F.D. Galiana, I. Kockar, P.C. Franco, “Combined pool/bilateral dispatch—Part I: performance of trading strategies”, IEEE Transactions on Power Systems, Vol. 17, pp. 92–99, 2002.
H. Song, C. Liu, J. Lawarr´ ee, R.W. Dahlgren, “Optimal electricity supply bidding by Markov decision process”, IEEE Transactions on Power Systems, Vol. 15, pp. 618–624, 2000.
A. Bagnall, G. Smith, “A multi-agent model of the UK market in electricity generation”, IEEE Transactions on Evolutionary Computation, Vol. 9, pp. 522–536, 2005.
E. Guerci, S. Ivaldi, S. Cincotti, “Learning agents in an artificial power exchange: tacit collusion, market power and efficiency of two double-auction mechanisms”, Computational Economics, Vol. 32, pp. 73–98, 2008.
D. Huang, X. Han, “Study on generation companies’ bidding strategy based on hybrid intelligent method”, Ninth International Conference on Hybrid Intelligent Systems. Vol. 3, pp. 409–412, 2009.
J. Contreras, R. Esp´ınola, F.J. Nogales, A.J. Conejo, “ARIMA models to predict next-day electricity prices”, IEEE Transactions on Power Systems, Vol. 18, pp. 1014–1020, 2003.
A.J. Conejo, M.A. Plazas, R. Espinola, A.B. Molina, “Day-ahead electricity price forecasting using the wavelet transform and ARIMA models”, IEEE Transactions on Power Systems, Vol. 20, pp. 1035–1042, 2005.
F.J. Nogales, J. Contreras, A.J. Conejo, R. Esp´ınola, “Forecasting next-day electricity prices by time series models”, IEEE Transactions on Power Systems, Vol. 17, pp. 342–348, 2002.
D.C. Sansom, T. Downs, T.K. Saha, “Evaluation of support vector machine based forecasting tool in electricity price forecasting for Australian national electricity market participants”, Journal of Electrical and Electronics Engineering, Vol. 22, pp. 227–234, 2003.
B.R. Szkuta, L.A. Sanavria, T.S. Dillon, “Electricity price short-term forecasting using artificial neural networks”, IEEE Transactions on Power Systems, Vol. 14, pp. 851–857, 1999.
C.P. Rodriguez, G.J. Anders, “Energy price forecasting in the Ontario competitive power system market”, IEEE Transactions on Power Systems, Vol. 19, pp. 366–374, 2004.
R. Weron, “Heavy-tails and regime-switching in electricity prices”, Mathematical Methods of Operations Research, Vol. 69, pp. 457–473, 2008.
I. Simonsen, “Volatility of power markets”, Physica A: Statistical Mechanics and its Applications, Vol. 355, pp. 10–20, 2005.
S. Vucetic, K. Tomsovic, Z. Obradovic, “Discovering price-load relationships in California’s electricity market”, IEEE Transactions on Power Systems, Vol. 16, pp. 280–286, 2001.
A. Alizadeh, N. Nomikos, P. Pouliasis, “A Markov regime switching approach for hedging energy commodities”, Journal of Banking & Finance, Vol. 32, pp. 1970–1983, 2008.
T. Mount, Y. Ning, X. Cai, “Predicting price spikes in electricity markets using a regime-switching model with time-varying parameters”, Energy Economics, Vol. 28, pp. 62–80, 2006.
J. Janczura, R. Weron, “An empirical comparison of alternate regime-switching models for electricity spot prices”, Energy Economics, Vol. 32, pp. 1059–1073, 2010.
R. Huisman, R. Mahieu, “Regime jumps in electricity prices”, Energy Economics, Vol. 25, pp. 425–434, 2003. H. Higgs, A. Worthington, “Stochastic price modeling of high volatility, mean-reverting, spike-prone commodities: the Australian wholesale spot electricity market”, Energy Economics, Vol. 30, pp. 3172–3185, 2008.
R. Becker, S. Hurn, V. Pavlov, “Modelling spikes in electricity prices”, Economic Record, Vol. 83, pp. 371–382, 200 R. Sutton, A. Barto, Reinforcement Learning: An Introduction, Cambridge, MIT Press, 1998.