2 Propagation of states’ mean and covariances
This chapter describes on how the mean and covariance of a state propagates through time. The goverining equations of its propagation under the following different systems were derived.
- Discrete-time systems
- Sampled-data systems
- Continuous-time systems
This is fundamental for the Kalman filter method.
The main reference for this chapter is Simon (2006).
2.1 Discrete-time systems
Consider the continuous-time linear stochastic system
\[ \dot{\mathbf{x}}(t) = \mathbf{A}\mathbf{x}(t) + \mathbf{B}\mathbf{u}(t) + \mathbf{w}(t) \]
where \(\mathbf{x}(t) \in \mathbb{R}^n\) is the state vector, \(\mathbf{u}(t)\) is the control input, and \(\mathbf{w}(t)\) is a continuous-time Gaussian process noise. The noise is commonly modeled as a zero-mean white process with covariance
\[ \mathbb{E}[\mathbf{w}(t)\mathbf{w}(\tau)^T] = \mathbf{Q}_c \delta(t - \tau) \]
The objective is to obtain an equivalent discrete-time representation suitable for recursive state estimation. Let the system be sampled with a uniform sampling interval \(\Delta t\), and define \(\mathbf{x}_k = \mathbf{x}(t_k)\) with \(t_k = k\Delta t\). The desired discrete-time model is
\[ \mathbf{x}_k = \mathbf{F}_{k-1}\mathbf{x}_{k-1} + \mathbf{G}_{k-1}\mathbf{u}_{k-1} + \mathbf{w}_{k-1} \]
The exact solution of the continuous-time system over one sampling interval \([t_{k-1}, t_k]\) is given by
\[ \mathbf{x}(t_k) = e^{\mathbf{A}\Delta t}\mathbf{x}(t_{k-1}) + \int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k - \tau)}\mathbf{B}\mathbf{u}(\tau)\, d\tau + \int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k - \tau)}\mathbf{w}(\tau)\, d\tau \]
To proceed with discretization, the control input is assumed to be held constant over each sampling interval according to a zero-order hold assumption, such that \(\mathbf{u}(\tau) = \mathbf{u}_{k-1}\) for \(\tau \in [t_{k-1}, t_k)\). Under this assumption, the state transition matrix is defined as
\[ \mathbf{F}_{k-1} = e^{\mathbf{A}\Delta t} \]
The discrete-time input matrix is obtained by evaluating the input integral as
\[ \mathbf{G}_{k-1} = \int_{0}^{\Delta t} e^{\mathbf{A}s}\mathbf{B}\, ds \]
If the matrix \(\mathbf{A}\) is nonsingular, this expression can be written in closed form as
\[ \mathbf{G}_{k-1} = \mathbf{A}^{-1}(e^{\mathbf{A}\Delta t} - \mathbf{I})\mathbf{B} \]
The discrete-time process noise is defined as the accumulated effect of the continuous-time noise over one sampling interval, given by
\[ \mathbf{w}_{k-1} = \int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k - \tau)}\mathbf{w}(\tau)\, d\tau \]
This noise term is zero-mean and Gaussian, and it captures the influence of modeling errors, disturbances, and unmodeled dynamics acting over the finite sampling interval. Combining the above results yields the discrete-time state-space model
\[ \mathbf{x}_k = \mathbf{F}_{k-1}\mathbf{x}_{k-1} + \mathbf{G}_{k-1}\mathbf{u}_{k-1} + \mathbf{w}_{k-1} \tag{2.1}\]
The covariance of the discrete-time process noise is obtained as
\[ \mathbf{Q}_d = \mathbb{E}[\mathbf{w}_{k-1}\mathbf{w}_{k-1}^T] = \int_{0}^{\Delta t} e^{\mathbf{A}\tau}\mathbf{Q}_c e^{\mathbf{A}^T\tau}\, d\tau \]
Although the continuous-time noise \(\mathbf{w}(t)\) is modeled as white, the discrete-time noise \(\mathbf{w}_{k-1}\) represents the integrated effect of uncertainty over a finite sampling interval. This formulation justifies the use of an additive Gaussian process noise term in discrete-time Kalman filtering.
In this system given by Eq. 2.1, both the governing state equations and the measurements are discrete in time.
Now, taking expectation on Eq. 2.1 to get an expression for mean state \(\bar{\mathbf{x}}_k\) propagation through time
\[ \bar{\mathbf{x}}_k = E(\mathbf{x}_k) = \mathbf{F}_{k-1}\bar{\mathbf{x}}_{k-1} + \mathbf{G}_{k-1}\mathbf{u}_{k-1} + \mathbf{0} . \tag{2.2}\]
Now to find an expression for the covariance of \(\mathbf{x}_k\) propagation through time, taking Eq. 2.1 and Eq. 2.2 to obtain \((\mathbf{x}_k - \bar{\mathbf{x}}_k)(\mathbf{x}_k - \bar{\mathbf{x}}_k)^T\) as
\[ \begin{align} (\mathbf{x}_k - \bar{\mathbf{x}}_k)(\mathbf{x}_k - \bar{\mathbf{x}}_k)^T &= (\mathbf{F}_{k-1}\mathbf{x}_{k-1} + \mathbf{G}_{k-1}\mathbf{u}_{k-1} +\mathbf{w}_{k-1} - \bar{\mathbf{x}}_k)(\mathbf{F}_{k-1}\mathbf{x}_{k-1} + \mathbf{G}_{k-1}\mathbf{u}_{k-1}+\mathbf{w}_{k-1} - \bar{\mathbf{x}}_k)^T \\ &= (\mathbf{F}_{k-1}(\mathbf{x}_{k-1}-\bar{\mathbf{x}}_{k-1})+\mathbf{w}_{k-1})(\mathbf{F}_{k-1}(\mathbf{x}_{k-1}-\bar{\mathbf{x}}_{k-1})+\mathbf{w}_{k-1})^T \\ &= \mathbf{F}_{k-1}\left(\mathbf{x}_{k-1}-\bar{\mathbf{x}}_{k-1}\right)\left(\mathbf{x}_{k-1}-\bar{\mathbf{x}}_{k-1}\right)^T \mathbf{F}_{k-1}^T + \mathbf{w}_{k-1}\mathbf{w}_{k-1}^T + \mathbf{F}_{k-1}\left(\mathbf{x}_{k-1}\bar{\mathbf{x}}_{k-1}\right)\mathbf{w}_{k-1}^T+ \mathbf{w}_{k-1}\left(\mathbf{x}_{k-1}\bar{\mathbf{x}}_{k-1}\right)^T\mathbf{F}_{k-1}^T \end{align} \]
Now taking expectation on above equation, note that the terms \((\mathbf{x}_{k-1}-\bar{\mathbf{x}}_{k-1})\) and \(\mathbf{w}\mathbf{w}^T\) are uncorrelated. So,
\[ \begin{align} \mathbf{P}_{k} &= E\left[\left(\mathbf{x}_{k}-\bar{\mathbf{x}}_{k}\right)\left(\mathbf{x}_{k}-\bar{\mathbf{x}}_{k}\right)^T\right] \\ &= \mathbf{F}_{k-1}\mathbf{P}_{k-1} \mathbf{F}_{k-1}^T + \mathbf{Q}_{k-1} . \end{align} \tag{2.3}\]
The Eq. 2.3 is called the discrete-time Lyapunov equation.
The Eq. 2.2 and Eq. 2.3 are fundamental in the derivation of Kalman filter. Therefore,
\[ \mathbf{x}_k \sim \mathcal{N}(\mathbf{x}_k,\mathbf{P}_k) \]
A theorem (Theorem 21) was given in the book that states the conditions under which the discrete-time Lyapunov equation (Eq. 2.3) has a steady-state solution. It can be referred in the book if needed.
2.2 Sampled-data systems
This is the most frequently encountered systems in practice. It is a system whose dynamics are described by a continuous-time differential equation, but the input only changes at discrete time instants.
The continuous-time dynamics are described as
\[ \dot{\mathbf{x}} = \mathbf{A}\mathbf{x} + \mathbf{B}\mathbf{u} + \mathbf{w}. \tag{2.4}\]
The general form of solution to the above equation, that was derived in the introductory chapters in Simon (2006), at some arbitrary time \(t_k\) is given as
\[ \mathbf{x}(t_k) = e^{\mathbf{A}(t_k-t_{k-1})}\mathbf{x}(t_{k-1})+\int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k-\tau)}\left[\mathbf{B}(\tau)\mathbf{u}(\tau)+\mathbf{w}(\tau)\right]d\tau . \tag{2.5}\]
Taking the control \(\mathbf{u}(t_k)\) to be piecewise constant i.e. \(\mathbf{u}(t) = \mathbf{u}_{k}, \ t \in [t_k,t_{k+1})\). (Technically, it is called 0th order hold, piecewise linear is called 1th order hold etc…). Making few definitions as
\[ \begin{align} \Delta t &= t_k - t_{k-1} \\ \mathbf{x}_k &= \mathbf{x}(t_k) \\ \mathbf{u}_k &= \mathbf{u}(t_k) \end{align} \]
and substituting them in the Eq. 2.5, which becomes
\[ \mathbf{x}_k = e^{\mathbf{A}\Delta t} \mathbf{x}_{k-1} + \int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k - \tau)} \mathbf{B}(\tau) d\tau \mathbf{u}_{k-1} + \int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k-\tau)} \mathbf{w}(\tau) dt\tau . \tag{2.6}\]
Let \(\mathbf{F}_k\) and \(\mathbf{G}_k\) be defined as
\[ \begin{align} \mathbf{F}_k &= e^{\mathbf{A} \Delta t} \\ \mathbf{G}_k &= \int_{t_k}^{t_{k+1}} e^{\mathbf{A}(t_{k+1}-\tau)} \mathbf{B}(\tau)d\tau . \end{align} \]
Substituting above equations in Eq. 2.6
\[ \mathbf{x}_k = \mathbf{F}_{k-1} \mathbf{x}_{k-1} + \mathbf{G}_{k-1}\mathbf{u}_{k-1} + \int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k-\tau)} \mathbf{w}(\tau) d\tau . \]
And taking expectation of above equation gives the expression for state mean propagation as shown below.
\[ \begin{align} \bar{\mathbf{x}}_k &= E(\mathbf{x}_k) \\ &= \mathbf{F}_{k-1}\bar{\mathbf{x}}_{k-1} + \mathbf{G}_{k-1}\mathbf{u}_{k-1} . \end{align} \tag{2.7}\]
The above equation can be used to compute the covariance of the state as
\[ \begin{align} \mathbf{P}_k &= E\left[\left(\mathbf{x}_k - \bar{\mathbf{x}}_k\right)\left(\mathbf{x}_k - \bar{\mathbf{x}}_k\right)^T\right] \\ &= E\left[\left(\mathbf{F}_{k-1} \mathbf{x}_{k-1} + \mathbf{G}_{k-1}\mathbf{u}_{k-1} + \int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k-\tau)} \mathbf{w}(\tau) d\tau - \bar{\mathbf{x}}_k\right)\left(\mathbf{F}_{k-1} \mathbf{x}_{k-1} + \mathbf{G}_{k-1}\mathbf{u}_{k-1} + \int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k-\tau)} \mathbf{w}(\tau) d\tau - \bar{\mathbf{x}}_k\right)^T\right] \\ &= \mathbf{F}_{k-1}\mathbf{P}_{k-1}\mathbf{F}_{k-1}^T + E\left[\left(\int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k-\tau)} \mathbf{w}(\tau) d\tau\right)\left(\int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k-\tau)} \mathbf{w}(\tau) d\tau\right)^T\right] \\ &= \mathbf{F}_{k-1}\mathbf{P}_{k-1}\mathbf{F}_{k-1}^T + \int \int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k-\tau)} E\left[\mathbf{w}(\tau)\mathbf{w}^T(\alpha)\right] e^{\mathbf{A}^T(t_k-\alpha)} d\tau d\alpha \end{align} \tag{2.8}\]
If \(\mathbf{w}(t)\) is assumed as a continuous-time white noise with covariance \(\mathbf{Q}_c(t)\), then
\[ E\left[\mathbf{w}(\tau)\mathbf{w}^T(\alpha)\right] = \mathbf{Q}_c(\tau) \delta(t-\tau). \]
Substituting above expression in Eq. 2.8 gives
\[ \begin{align} \mathbf{P}_k &= \mathbf{F}_{k-1}\mathbf{P}_{k-1}\mathbf{F}_{k-1}^T + \int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k-\tau)} \mathbf{Q}_c(\tau) e^{\mathbf{A}^T(t_k - \tau)} d\tau \\ &= \mathbf{F}_{k-1}\mathbf{P}_{k-1}\mathbf{F}_{k-1}^T + \mathbf{Q}_{k-1} . \end{align} \tag{2.9}\]
Where, \[ \mathbf{Q}_{k-1} = \int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k-\tau)} \mathbf{Q}_c(\tau) e^{\mathbf{A}^T(t_k - \tau)} d\tau . \]
Computing \(\mathbf{Q}_{k-1}\) can be tedious, but for small values of \(\Delta t\) the above expression can be approximated as
\[ \begin{align} e^{\mathbf{A}(t_k-\tau)} &\approx \mathbf{I}, \ \forall \ \tau \in [t_{k-1},t_k) \\ \mathbf{Q}_{k-1} &\approx \mathbf{Q}_c(t_k)\Delta t \end{align} \tag{2.10}\]
2.3 Continuous-time systems
Here, both the dynamics and the input are assumed to be continuous. Consider the continuous-time system
\[ \dot{\mathbf{x}} = \mathbf{A}\mathbf{x}+\mathbf{B}\mathbf{u}+\mathbf{w} \tag{2.11}\]
with \(\mathbf{w}(t)\) is zero-mean white noise with covariance as
\[ E\left[\mathbf{w}(t)\mathbf{w}(t)^T\right] = \mathbf{Q}_c \delta(t-\tau) . \]
Taking expectation on Eq. 2.11 gives
\[ \dot{\bar{\mathbf{x}}} = \mathbf{A}\bar{\mathbf{x}} + \mathbf{B}\mathbf{u} \tag{2.12}\]
Above equation defines the state mean propagation through time. An alternate way of deriving above equation is as follows. Taking expectation on Eq. 2.5 gives
\[ \bar{\mathbf{x}}_k = e^{\mathbf{A}\Delta t} \bar{\mathbf{x}}_{k-1} + \int_{t_{k-1}}^{t_k}e^{\mathbf{A}(t_k-\tau)}\mathbf{B}(\tau)\mathbf{u}(\tau)d\tau \tag{2.13}\]
The state transition matrix \(\mathbf{F}\) can be written as \[ \begin{align} \mathbf{F} &= e^{\mathbf{A} \Delta t} \\ &= \mathbf{I} + \mathbf{A}\Delta t+ \frac{\left(\mathbf{A}\Delta t\right)^2}{2!} + \dots \end{align} \]
For small values of \(\Delta t\), this can be approximated as \[ \mathbf{F} \approx \mathbf{I} + \mathbf{A}\Delta t \tag{2.14}\]
Substituting above expression in Eq. 2.13 gives
\[ \bar{\mathbf{x}}_k = \left(\mathbf{I}+\mathbf{A}\Delta t\right)\bar{\mathbf{x}}_{k-1} + \int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k-\tau)} \mathbf{B}(\tau) \mathbf{u}(\tau)d\tau \]
Rearranging above equation gives
\[ \frac{\bar{\mathbf{x}}_k -\bar{\mathbf{x}}_k}{\Delta t} = \mathbf{A}\bar{\mathbf{x}}_{k-1} + \frac{1}{\Delta t}\int_{t_{k-1}}^{t_k} e^{\mathbf{A}(t_k-\tau)} \mathbf{B}(\tau) \mathbf{u}(\tau)d\tau \tag{2.15}\]
Taking limits on \(\Delta t\) gives
\[ \begin{align} \lim_{\Delta t \to 0} \frac{\bar{\mathbf{x}}_k - \bar{\mathbf{x}}_{k-1}}{\Delta t} &= \dot{\bar{\mathbf{x}}} \\ \lim_{\Delta t \to 0} e^{\mathbf{A}(t_k-\tau)} &= \mathbf{I}, \forall \ \tau \in [t_{k-1},t_k) \end{align} \]
Substituting these in Eq. 2.15 gives
\[ \dot{\bar{\mathbf{x}}} = \mathbf{A}\bar{\mathbf{x}} + \mathbf{B}\mathbf{u} \tag{2.16}\]
which is same as Eq. 2.12 by a more direct method.
Now, taking Eq. 2.9 for deriving covariance propagation equation
\[ \mathbf{P}_k = \mathbf{F}_{k-1}\mathbf{P}_{k-1}\mathbf{F}_{k-1}^T+\mathbf{Q}_{k-1} \]
Substituting Eq. 2.14 in above expression gives
\[ \begin{align} \mathbf{P}_k &= (\mathbf{I} + \mathbf{A}\Delta t)\mathbf{P}_{k-1}(\mathbf{I}+\mathbf{A}\Delta t)^T + \mathbf{Q}_{k-1} \\ \frac{\mathbf{P}_k - \mathbf{P}_{k-1}}{\Delta t} &= \mathbf{A}\mathbf{P}_{k-1} + \mathbf{P}_{k-1}\mathbf{A}^T + \mathbf{A}\mathbf{P}_{k-1}\mathbf{A}^T\Delta t + \frac{Q_{k-1}}{\Delta t} \end{align} \tag{2.17}\]
From Eq. 2.10, it can be written as
\[ \frac{\mathbf{Q}_{k-1}}{\Delta t} \approx \mathbf{Q}_c(t_k) \]
Substituting above equation in Eq. 2.17 and taking limit yields
\[ \dot{\mathbf{P}} = \mathbf{A}\mathbf{P} + \mathbf{P}\mathbf{A}^T + \mathbf{Q}_c \tag{2.18}\]
The above equation is called the continuous-time Lyapunov equation, or Sylvester equation.
2.4 Summary
The derived equations Eq. 2.2,Eq. 2.3, Eq. 2.7, Eq. 2.9, Eq. 2.12 and Eq. 2.18 define the propagation of states’ mean and covariances through time for discrete-time, sample-data and continuous-time systems. This is fundamental for the Kalman filter derivation.