Discrete-time average-cost mean-field games on Polish spaces

In stochastic dynamic games, when the number of players is sufficiently large and the interactions between agents depend on empirical state distribution, one way to approximate the original game is to introduce infinitepopulation limit of the problem. In the infinite population limit, a generic agent is faced with a so-called mean-field game. In this paper, we study discrete-time mean-field games with average-cost criteria. Using average cost optimality equation and Kakutani’s fixed point theorem, we establish the existence of Nash equilibria for mean-field games under drift and minorization conditions on the dynamics of each agent. Then, we show that the equilibrium policy in the mean-field game, when adopted by each agent, is an approximate Nash equilibrium for the corresponding finite-agent game with sufficiently many agents.

Keywords:

Mean-field games, average cost approximate Nash equilibrium,

PDF

___

[1] Adlakha S, Johari R, Weintraub GY. Equilibria of dynamic games with many players: Existence, approximation, and market structure. Journal of Economic Theory 2015; 156: 269-316.
[2] Aliprantis CD, Border KC. Infinite Dimensional Analysis. 3rd ed. Berlin, Germany: Springer, 2006.
[3] Bensoussan A, Frehse J, Yam P. Mean Field Games and Mean Field Type Control Theory. Springer, New York, NY, USA, 2013.
[4] Billingsley P. Convergence of Probability Measures. 2nd ed. New York, NY, USA: Wiley, 1999.
[5] Biswas A. Mean field games with ergodic cost for discrete time Markov processes; 2015. arXiv:1510.08968.
[6] Bogachev VI. Measure Theory: Volume II. Springer; 2007.
[7] Cardaliaguet P. Notes on mean-field games, 2011.
[8] Carmona R, Delarue F. Probabilistic analysis of mean-field games. SIAM Journal on Control and Optimization 2013; 51 (4): 2705-2734.
[9] Elliot R, Li X, Ni Y. Discrete time mean-field stochastic linear-quadratic optimal control problems. Automatica 2013; 49: 3222-3233.
[10] Gomes DA, Mohr J, Souza RR. Discrete time, finite state space mean field games. Journal de Mathematiques Pures et Appliquees 2010; 93: 308-328.
[11] Gomes DA, Saúde J. Mean field games models - a brief survey. Dynamic Games and Applications 2014; 4 (2): 110-154.
[12] Gordienko E, Hernandez-Lerma O. Average Cost Markov Control Processes with Weighted Norms: Existence of Canonical Policies. Applicationes Mathematicae 1995; 23 (2): 199-218.
[13] Hernández-Lerma O, Lasserre JB. Discrete-Time Markov Control Processes: Basic Optimality Criteria. New York, NY, USA: Springer, 1996.
[14] Hernández-Lerma O, Lasserre JB. Further Topics on Discrete-Time Markov Control Processes. Springer; 1999.
[15] Hernández-Lerma O, Montes-De-Oca R, Cavazos-Cadena R. Recurrence conditions for Markov decision processes with Borel state space: a survey. Annals of Operations Research 1991; 28 (1): 29-46.
[16] Huang M. Large-population LQG games involving major player: The Nash certainity equivalence principle. SIAM Journal on Control Optimization 2010; 48 (5): 3318-3353.
[17] Huang M, Caines PE, Malhamé RP. Large-population cost coupled LQG problems with nonuniform agents: Individual-mass behavior and decentralized ϵ -Nash equilibria. IEEE Transactions on Automatic Control 2007; 52 (9): 1560-1571.
[18] Huang M, Malhamé RP, Caines PE. Large population stochastic dynamic games: Closed loop McKean-Vlasov sysyems and the Nash certainity equivalence principle. Communications in Information Systems 2006; 6: 221-252.
[19] Langen HJ. Convergence of dynamic programming models. Mathematics of Operations Research 1981; 6 (4): 493-512.
[20] Lasry J, Lions P. Mean field games. Japanese Journal of Mathematics 2007; 2: 229-260.
[21] Moon J, Başar T. Discrete-time decentralized control using the risk-sensitive performance criterion in the large population regime: a mean field approach. In: ACC 2015. Chicago; 2015.
[22] Moon J, Başar T. Discrete-time mean field Stackelberg games with a large number of followers. In: CDC 2016. Las Vegas, USA; 2016.
[23] Moon J, Başar T. Robust mean field games for coupled Markov jump linear systems. International Journal of Control 2016; 89 (7): 1367-1381.
[24] Nourian M, Nair GN. Linear-quadratic-Gaussian mean field games under high rate quantization. In: CDC 2013. Florence, Italy; 2013.
[25] Parthasarathy KR. Probability Measures on Metric Spaces. AMS Bookstore; 1967.
[26] Saldi N, Başar T, Raginsky M. Markov–Nash Equilibria in Mean-Field Games with Discounted Cost. SIAM Journal on Control and Optimization 2018 ;56 (6): 4256-4287.
[27] Saldi N, Linder T, Yüksel S. Finite approximations in discrete-time stochastic control: Quantized models and asymptotic optimality. Cham, Switzerland: Springer, 2018.
[28] Serfozo R. Convergence of Lebesgue integrals with varying measures. Sankhya SerA 1982; p. 380-402.
[29] Tembine H, Zhu Q, Başar T. Risk-sensitive mean field games. IEEE Transactions on Automatic Control 2014; 59 (4): 835-850.
[30] Vega-Amaya O. The average cost optimality equation: a fixed point approach. Boletin de la Sociedad Matematica Mexicana 2003; 9 (3): 185-195.
[31] Wiecek P. Discrete-time ergodic mean-field games with average reward on compact spaces. Dynamic Games and Applications 2019; p. 1-35.
[32] Wiecek P, Altman E. Stationary anonymous sequential games with undiscounted rewards. Journal of Optimization Theory and Applications 2015; 166 (2): 686-710.