Markov Jump Linear Systems (MJLS) are well-studied in Stochastic Control literature. MJLS are linear dynamic systems, the matrices of which may undergo abrupt changes, according to the state of a Markov chain. We studied the case where the Markov chain may have a general state space, generalizing previous research about Markov chains with discrete state space. (See the IEEE TAC paper 'On stability and LQ control of MJLS with a Markov chain with general state space'). The motivation came from the discretization of systems with random delays and the study of games with a large number of players in the continuum limit. This work focused on stochastic stability and Linear Quadratic control and has many potential applications, one of which is the optimal control of cyber-physical systems, modeled as LTI continuous-time systems, where there are sampled measurements and random dependent communication delays between the sensor and the controller. A potential extension of this line of research is the study of optimal (or sub-optimal) control of distributed systems, where there are dependent time delays of the measurement transmission among the several sub-systems (agents).
This research was part of my Ph.D. at NTUA, under the supervision of Professor G.P. Papavassilopoulos.