We can classify random processes based on many different criteria. One of the important questions that we can ask about a random process is whether it is a stationary process. Intuitively, a random process {X(t),t∈J} is stationary if its statistical properties do not change by time. For example, for a stationary process, X(t) and X(t+Δ) have the same probability distributions. In particular, we have
FX(t)(x)=FX(t+Δ)(x), for all t,t+Δ∈J.
More generally, for a stationary process, the joint distribution of X(t1) and X(t2) is the same as the joint distribution of X(t1+Δ) and X(t2+Δ). For example, if you have a stationary process X(t), then
P((X(t1),X(t2))∈A)=P((X(t1+Δ),X(t2+Δ))∈A),
for any set A∈R2. In sum, a random process is stationary if a time shift does not change its statistical properties. Here is a formal definition of stationarity of continuous-time processes.
A continuous-time random process {X(t),t∈R} is strict-sense stationary or simply stationary if, for all t1,t2,⋯,tr∈R and all Δ∈R, the joint CDF of
Example
Consider the discrete-time random process {X(n),n∈Z⋯}, in which the X(n)'s are i.i.d. with CDF FX(n)(x)=F(x). Show that this is a (strict-sense) stationary process.
Intuitively, since X(n)'s are i.i.d., we expect that as time evolves the probabilistic behavior of the process does not change. Therefore, this must be a stationary process. To show this rigorously, we can argue as follows. For all real numbers x1,x2,⋯,xr and all distinct integers n1, n2,⋯, nr, we have
FX(n1)X(n2)⋯X(nr)(x1,x2,⋯,xr)=FX(n1)(x1)FX(n2)(x2)⋯FX(nr)(xr) (since the X(ni)'s are independent)=F(x1)F(x2)⋯F(xr) (since FX(ti)(x)=F(x)).
We also have
FX(n1+D)X(n2+D)⋯X(nr+D)(x1,x2,⋯,xr)=FX(n1+D)(x1)FX(n2+D)(x2)⋯FX(nr+D)(xr) (since the X(ni+D)'s are independent)=F(x1)F(x2)⋯F(xn) (since FX(ni+D)(x)=F(x)).
In practice, it is desirable if a random process X(t) is stationary. In particular, if a process is stationary, then its analysis is usually simpler as the probabilistic properties do not change by time. For example, suppose that you need to do forecasting about the future of a process X(t). If you know the process is stationary, you can observe the past, which will normally give you a lot of information about how the process will behave in the future.
However, it turns out that many real-life processes are not strict-sense stationary. Even if a process is strict-sense stationary, it might be difficult to prove it. Fortunately, it is often enough to show a "weaker" form of stationarity than the one defined above.
Weak-Sense Stationary Processes:
Here, we define one of the most common forms of stationarity that is widely used in practice. A random process is called weak-sense stationary or wide-sense stationary (WSS) if its mean function and its correlation function do not change by shifts in time. More precisely, X(t) is WSS if, for all t1,t2∈R and all Δ∈R,
E[X(t1)]=E[X(t2)],
E[X(t1)X(t2)]=E[X(t1+Δ)X(t2+Δ)].
Note that the first condition states that the mean function μX(t) is not a function of time, t, thus we can write μX(t)=μX. The second condition states that the correlation function RX(t1,t2) is only a function of τ=t1−t2, and not t1 and t2 individually. Thus, we can write RX(t1,t2)=RX(t1−t2)=RX(τ). Therefore, we can provide the following definition.
A continuous-time random process {X(t),t∈R} is weak-sense stationary or wide-sense stationary (WSS) if
μX(t)=μX, for all t∈R,
RX(t1,t2)=RX(t1−t2), for all t1,t2∈R.
We can provide a similar definition for discrete-time WSS processes.
A discrete-time random process {X(n),n∈Z} is weak-sense stationary or wide-sense stationary (WSS) if
μX(n)=μX, for all n∈Z,
RX(n1,n2)=RX(n1−n2), for all n1,n2∈Z.
Example
Consider the random process {X(t),t∈R} defined as
X(t)=cos(t+U),
where U∼Uniform(0,2π). Show that X(t) is a WSS process.
μX(t)=E[X(t)]=E[cos(t+U)]=∫2π0cos(t+u)12πdu=0, for all t∈R.
We can also find RX(t1,t2) as follows
RX(t1,t2)=E[X(t1)X(t2)]=E[cos(t1+U)cos(t2+U)]=E[12cos(t1+t2+2U)+12cos(t1−t2)]=E[12cos(t1+t2+2U)]+E[12cos(t1−t2)]=∫2π0cos(t1+t2+u)12πdu+12cos(t1−t2)=0+12cos(t1−t2)=12cos(t1−t2), for all t1,t2∈R.
As we see, both conditions are satisfied, thus X(t) is a WSS process.
Since for WSS random processes, RX(t1,t2)=RX(t1−t2), we usually denote the correlation function by RX(τ), where τ=t1−t2. Thus,
for a WSS process, we can write
RX(τ)=E[X(t)X(t−τ)]=E[X(t+τ)X(t)](10.1)
As we will see in Section 10.2, RX(τ) is a very useful tool when we do frequency domain analysis. Here, we would like to study some properties of RX(τ) for WSS signals. Let {X(t),t∈R} be a WSS process with correlation function RX(τ). Then, we can write
RX(0)=E[X(t)2].
The quantity E[X(t)2] is called the expected (average) power in X(t) at time t. For a WSS process, the expected power is not a function of time. Since X(t)2≥0, we conclude that RX(0)≥0.
Therefore, we conclude that |RX(τ)|≤RX(0). Considering these properties, Figure 10.4 shows some possible shapes for RX(τ).
Figure 10.4 - Some possible shapes for RX(τ).
Jointly Wide-Sense Stationary Processes:
We often work with multiple random processes, so we extend the concept of wide-sense stationarity to more than one process. More specifically, we can talk about jointly wide-sense stationary processes.
Two random processes {X(t),t∈R} and {Y(t),t∈R} are said to be jointly wide-sense stationary if
X(t) and Y(t) are each wide-sense stationary.
RXY(t1,t2)=RXY(t1−t2).
Example
Let X(t) and Y(t) be two jointly WSS random processes. Consider the random process Z(t) defined as
Some practical random processes have a periodic structure. That is, the statistical properties are repeated every T units of time (e.g., every T seconds). In other words, the random variables
X(t1),X(t2),⋯,X(tr)
have the same joint CDF as the random variables
X(t1+T),X(t2+T),⋯,X(tr+T).
Such random variables are called cyclostationary. For example, consider the random process {X(t),t∈R} defined as
X(t)=Acos(ωt),
where A is a random variable. Here, we have
X(t+2πω)=Acos(ωt+2π)=Acos(ωt)=X(t).
We conclude X(t) is in fact a periodic signal with period T=2πω. Therefore, the statistical properties of X(t) do not change by shifting the time by T units, so X(t) is a cyclostationary random process with period T=2πω. Similarly, we can define wide-sense cyclostationary random processes.
A continuous-time random process {X(t),t∈R} is cyclostationary if there exists a positive real number T such that, for all t1,t2,⋯,tr∈R, the joint CDF of
X(t1),X(t2),⋯,X(tr)
is the same as the joint CDF of
X(t1+T),X(t2+T),⋯,X(tr+T).
A continuous-time random process {X(t),t∈R} is weak-sense cyclostationary or wide-sense cyclostationary if there exists a positive real number T such that
μX(t+T)=μX(t), for all t∈R;
RX(t1+T,t2+T)=RX(t1,t2), for all t1,t2∈R.
Similarly, you can define cyclostationary discrete-time processes. For example, a discrete-time random process {X(n),n∈Z} is wide-sense cyclostationary if there exists M∈N such that
μX(n+M)=μX, for all n∈Z;
RX(n1+M,n2+M)=RX(n1,n2), for all n1,n2∈Z.
Derivatives and Integrals of Random Processes:
Many real-life systems are described by differential equations. To analyze such systems when randomness is involved, we often need to differentiate or integrate the random processes that are present in the system. You have seen concepts such as continuity, differentiability, and integrability in calculus for deterministic signals (deterministic functions). Here, we need to extend those concepts to random processes. Without going much into mathematical technicalities, here we would like to provide some guidelines on how to deal with derivatives and integrals of random processes.
Let X(t) be a continuous-time random process. We say that X(t) is mean-square continuous at time t if
limδ→0E[∣∣X(t+δ)−X(t)∣∣2]=0.
Note that mean-square continuity does not mean that every possible realization of X(t) is a continuous function. It roughly means that the difference X(t+δ)−X(t) is small on average.
Example
The Poisson process is discussed in detail in Chapter 11. If X(t) is a Poisson process with intensity λ, then for all t>s≥0, we have
X(t)−X(s)∼Poisson(λ(t−s)).
Show that X(t) is mean-square continuous at any time t≥0.
It is worth noting that there are jumps in a Poisson process; however, those jumps are not very "dense" in time, so the random process is still continuous in the mean-square sense. Figure 10.5 shows a possible realization of a Poisson process.
Figure 10.5 - A possible sample function of a Poisson process.
We can similarly talk about mean-square differentiability and mean-square integrability. If X(t) is a random process, the derivative of X(t),
Y(t)=ddtX(t),
is also a random process. For nice and smooth processes, the derivative can be obtained in a natural way. For example, if you have a random process defined as
X(t)=A+Bt+Ct2, for all t∈[0,∞),
where A, B, and C are random variables, then the derivative of X(t) can be written as
X′(t)=B+2Ct, for all t∈[0,∞).
Without trying to go much into mathematical technicalities, here we would like to provide some guidelines on how to deal with derivatives and integrals of random processes (assuming some mild regularity conditions are satisfied). A key point to note is that differentiation and integration are linear operations. This, for example, means that you can often interchange integration and expectation. More specifically, you can write
E[∫t0X(u)du]=∫t0E[X(u)]du.
Similarly, if the derivative of X(t) is well-defined, we can write
E[ddtX(t)]=ddtE[X(t)].
Example
Consider a random process X(t) and its derivative, X′(t)=ddtX(t). Assuming that the derivatives are well-defined, show that