Bayesian estimation[9]

Define a risk (or average/expected cost)

$$\begin{array}{lll} \mathcal{R}&=&\mathcal{E}[C(x,\hat{x})]... ...ty} C(x,\hat{x}) f_{X\vert Y}(x\vert y) dx dy\\ \end{array}$$

The Bayes estimate minimizes the risk with respect to $f_{X,Y}(x,y)$, the joint probability density function of $X$ and $Y$. As $C(x,\hat{x})$ is non-negative, it is sufficient to minimize only the inner integral, giving

$$\hat{x}_{opt}=argmin_{\hat{x}} \int_{- \infty}^{\infty} C(x,\hat{x}) f_{X\vert Y}(x\vert y) dx dy$$

The optimal solution in the Minimum Mean Square Error sense is given by the mean of the posterior density, i.e. the conditional mean $\mathcal{E}[X\vert Y]$ of r.v. $X$ given r.v. $Y$ is the MMSE estimate of $X$ given $Y$.

$$\begin{array}{lll} \mathcal{E}[(X-f(Y))^2] & = & \mathcal{... ...ge & \mathcal{E}[(X-\mathcal{E}[X\vert Y])^2]\\ \end{array}$$