Stability and optimal control of polytopic time-inhomogeneous Markov jump linear system

Linear systems subject to abrupt parameter changes due, for instance, to environmental disturbances, component failures, changes in subsystems interconnections, etc., can be modeled as a set of discrete-time linear systems with modal transition given by a discrete-time finite-state Markov chain. This family of systems is known as discrete-time Markov(ian) jump linear systems (from now on MJLSs). MJLSs represent a promising mathematical model of cyber-physical systems, the applications of which arguably have the potential to dwarf the fourth industrial revolution. The bulk of the existing research on MJLSs is based on the fundamental assumption that parameters of the Markov chain are known and static. However, in several cyber-physical systems’ applications the MJLSs’ model is affected by abrupt and unpredictable perturbations on the underlying Markov chain. For instance, Markov chain models of slow fading channels are derived via measurements on real channels or via numerical reasoning, which always introduces errors. Furthermore, fading channels can partially be compensated for by adjusting the transmission power levels, with higher transmission power giving less packet errors, but increasing the energy consumption and interference with other systems. Another example can be found in the vertical take-off landing helicopter systems, where the airspeed variation is ideally modeled as homogeneous Markov process, but because of the external environment (like wind) the transition probabilities of the jumps are time-varying. We take into account the intrinsic to the real world systems uncertainty and time-variance of the jump parameters by considering MJLSs where the underlying Markov chain is polytopic and time-inhomogeneous, i.e., its transition probability matrix is varying over time with variations that are arbitrary within a polytopic set of stochastic matrices. We show that the conditions used for time-homogeneous MJLSs are not enough to ensure the stability of the time-inhomogeneous system, and that perturbations on values of the transition probability matrix can make a stable system unstable. We present necessary and sufficient conditions for mean square stability (hereupon, MSS) of polytopic time-inhomogeneous MJLSs. We prove that deciding MSS on such systems is NP-hard and that MSS is equivalent to exponential mean square stability and to stochastic stability. We also derive necessary and sufficient conditions for robust MSS of MJLSs affected by polytopic uncertainties on transition probabilities and bounded disturbances. Then, we address and solve for this class of systems the finite horizon optimal control and filtering problems. In particular, we show that the optimal controller having only partial information on the continuous state can be obtained from two types of coupled Riccati difference equations, one associated to the control problem, and the other one associated to the filtering problem. Finally, we present and solve the finite horizon optimal control problem also for switched linear systems, where a switching signal is governed by a Markov decision process, in polytopic time-inhomogeneous setting. We call this type of systems Markov jump switched linear system. It generalizes the MJLSs’ framework to optimal decision problems, with applications for example to optimal power management of wireless networked control systems. These results construct a solid basis for the future development of novel (correct-by-design) fault and intrusion detection, isolation and reconfiguration techniques for cyber-physical systems modeled by discrete-time polytopic time-inhomogeneous Markov jump (switched) linear systems.