- clc1
-
根据平稳随机过程的定义,时间平移不会改变其统计性质,即对于任意整数 k,随机变量序列 (X_i) 和 (X_i+k) 具有相同的统计性质。因此,我们可以将时间箭头移到时刻 0,即假设 (X_i) 平稳,等价于假设 (X_i+1) 平稳。
根据条件熵的定义,有:
H(X_0+|+X_{-1},X_{-2},...,X_{-n}) = H(X_0,X_{-1},X_{-2},...,X_{-n}) - H(X_{-1},X_{-2},...,X_{-n})
根据平稳性质,有:
H(X_0,X_{-1},X_{-2},...,X_{-n}) = H(X_1,X_0,X_{-1},...,X_{-n+1})
将上式代入条件熵的定义式中,有:
H(X_0+|+X_{-1},X_{-2},...,X_{-n}) = H(X_1,X_0,X_{-1},...,X_{-n+1}) - H(X_{-1},X_{-2},...,X_{-n})
根据链式规则,有:
H(X_1,X_0,X_{-1},...,X_{-n+1}) = H(X_1|X_0,X_{-1},...,X_{-n+1}) + H(X_0,X_{-1},...,X_{-n+1})
将上式代入上式中,有:
H(X_0+|+X_{-1},X_{-2},...,X_{-n}) = H(X_1|X_0,X_{-1},...,X_{-n+1}) + H(X_0,X_{-1},...,X_{-n+1}) - H(X_{-1},X_{-2},...,X_{-n})
根据马尔可夫性质,有:
H(X_1|X_0,X_{-1},...,X_{-n+1}) = H(X_1|X_0)
将上式代入上式中,有:
H(X_0+|+X_{-1},X_{-2},...,X_{-n}) = H(X_1|X_0) + H(X_0,X_{-1},...,X_{-n+1}) - H(X_{-1},X_{-2},...,X_{-n})
根据平稳性质,有:
H(X_1|X_0) = H(X_0|X_{-1})
将上式代入上式中,有:
H(X_0+|+X_{-1},X_{-2},...,X_{-n}) = H(X_0|X_{-1}) + H(X_0,X_{-1},...,X_{-n+1}) - H(X_{-1},X_{-2},...,X_{-n})
根据链式规则,有:
H(X_0,X_{-1},...,X_{-n+1}) = H(X_0|X_{-1},X_{-2},...,X_{-n+1}) + H(X_{-1},X_{-2},...,X_{-n+1})
将上式代入上式中,有:
H(X_0+|+X_{-1},X_{-2},...,X_{-n}) = H(X_0|X_{-1}) + H(X_0|X_{-1},X_{-2},...,X_{-n+1}) + H(X_{-1},X_{-2},...,X_{-n+1}) - H(X_{-1},X_{-2},...,X_{-n})
根据马尔可夫性质,有:
H(X_0|X_{-1},X_{-2},...,X_{-n+1}) = H(X_0|X_{-1})
将上式代入上式中,有:
H(X_0+|+X_{-1},X_{-2},...,X_{-n}) = H(X_0|X_{-1}) + H(X_0|X_{-1}) + H(X_{-1},X_{-2},...,X_{-n+1}) - H(X_{-1},X_{-2},...,X_{-n})
根据链式规则,有:
H(X_{-1},X_{-2},...,X_{-n+1}) = H(X_{-1}|X_{-2},...,X_{-n+1}) + H(X_{-2},...,X_{-n+1})
将上式代入上式中,有:
H(X_0+|+X_{-1},X_{-2},...,X_{-n}) = H(X_0|X_{-1}) + H(X_0|X_{-1}) + H(X_{-1}|X_{-2},...,X_{-n+1}) + H(X_{-2},...,X_{-n+1}) - H(X_{-1},X_{-2},...,X_{-n+1})
根据马尔可夫性质,有:
H(X_{-1}|X_{-2},...,X_{-n+1}) = H(X_{-1}|X_{-2})
将上式代入上式中,有:
H(X_0+|+X_{-1},X_{-2},...,X_{-n}) = H(X_0|X_{-1}) + H(X_0|X_{-1}) + H(X_{-1}|X_{-2}) + H(X_{-2},...,X_{-n+1}) - H(X_{-1},X_{-2},...,X_{-n})
根据马尔可夫性质,有:
H(X_{-1}|X_{-2}) = H(X_{-1})
将上式代入上式中,有:
H(X_0+|+X_{-1},X_{-2},...,X_{-n}) = H(X_0|X_{-1}) + H(X_0|X_{-1}) + H(X_{-1}) + H(X_{-2},...,X_{-n+1}) - H(X_{-1},X_{-2},...,X_{-n})
根据熵的定义,有:
H(X_0|X_{-1}) <= H(X_0)
将上式代入上式中,有:
H(X_0+|+X_{-1},X_{-2},...,X_{-n}) <= 2H(X_0) + H(X_{-1}) + H(X_{-2},...,X_{-n+1}) - H(X_{-1},X_{-2},...,X_{-n})
由于熵是非负的,因此有:
H(X_0+|+X_{-1},X_{-2},...,X_{-n}) <= 2H(X_0)
又根据对称性,有:
H(X_0+|+X_{-1},X_{-2},...,X_{-n}) = H(X_0-|+X_{-1},X_{-2},...,X_{-n})
将上式代入上式中,有:
H(X_0-|+X_{-1},X_{-2},...,X_{-n}) <= 2H(X_0)
因此,有:
H(X_0?) <= 2H(X_0)
最后,根据熵的定义,有:
H(X_0?) = H(X_0) - H(X_0|?)
因为 H(X_0|?) >= 0,所以有:
H(X_0?) <= H(X_0)
综上所述,我们证明了 H(X_0+|+X_{-1},X_{-2}...,X_{-n}) <= H(X_0?),即条件熵不大于熵。此外,由于条件熵和熵的差是非负的,因此在本题中,当且仅当条件熵等于熵时,条件熵的下界为 0,即 H(X_0+|+X_{-1},X_{-2}...,X_{-n}) = H(X_0?) = H(X_0)。因此,我们证明了 H(X_0+|+X_{-1},X_{-2}...,X_{-n}) = H(X_0?),即在平稳随机过程的条件下,X_0 与 X_{-1},X_{-2},...,X_{-n} 的条件熵等于 X_0 的熵。