3.3 Parameter estimation

Very often we know the waveform of the signal that we are searching for in the data in terms of a finite number of unknown parameters. We would like to find optimal procedures of estimating these parameters. An estimator of a parameter 𝜃 is a function ˆðœƒ(x) that assigns to each data set the “best” guess of the true value of 𝜃. Note that because 𝜃ˆ(x ) depends on the random data it is a random variable. Ideally we would like our estimator to be (i) unbiased, i.e., its expectation value to be equal to the true value of the parameter, and (ii) of minimum variance. Such estimators are rare and in general difficult to find. As in the signal detection there are several approaches to the parameter estimation problem. The subject is exposed in detail in [81, 82]. See also [149] for a concise account.

3.3.1 Bayesian estimation

We assign a cost function ′ C (𝜃 ,𝜃) of estimating the true value of 𝜃 as ′ 𝜃. We then associate with an estimator ˆ 𝜃 a conditional risk or cost averaged over all realizations of data x for each value of the parameter 𝜃:

∫ ˆ ˆ ( ˆ ) R𝜃(𝜃) = E 𝜃[C (𝜃,𝜃)] = C 𝜃 (x ),𝜃 p(x,𝜃)dx, (49 ) X
where X is the set of observations and p(x,𝜃) is the joint probability distribution of data x and parameter 𝜃. We further assume that there is a certain a priori probability distribution π(𝜃) of the parameter 𝜃. We then define the Bayes estimator as the estimator that minimizes the average risk defined as
∫ ∫ ˆ ˆ (ˆ ) r(𝜃) = E[R𝜃(𝜃)] = C 𝜃(x),𝜃 p(x,𝜃)π (𝜃)d𝜃dx, (50 ) X Θ
where E is the expectation value with respect to an a priori distribution π, and Θ is the set of observations of the parameter 𝜃. It is not difficult to show that for a commonly used cost function
′ ′ 2 C (𝜃 ,𝜃) = (𝜃 − 𝜃) , (51 )
the Bayesian estimator is the conditional mean of the parameter 𝜃 given data x, i.e.,
∫ 𝜃ˆ(x ) = E[𝜃|x ] = 𝜃p (𝜃 |x)d 𝜃, (52 ) Θ
where p(𝜃|x ) is the conditional probability density of parameter 𝜃 given the data x.

3.3.2 Maximum a posteriori probability estimation

Suppose that in a given estimation problem we are not able to assign a particular cost function C (𝜃′,𝜃). Then a natural choice is a uniform cost function equal to 0 over a certain interval I𝜃 of the parameter 𝜃. From Bayes theorem [28] we have

p(𝜃|x ) = p(x,𝜃)π(𝜃), (53 ) p(x)
where p(x) is the probability distribution of data x. Then, from Eq. (50View Equation) one can deduce that for each data point x the Bayes estimate is any value of 𝜃 that maximizes the conditional probability p(𝜃|x ). The density p(𝜃|x ) is also called the a posteriori probability density of parameter 𝜃 and the estimator that maximizes p(𝜃|x) is called the maximum a posteriori (MAP) estimator. It is denoted by ˆðœƒMAP. We find that the MAP estimators are solutions of the following equation
∂ log-p(x,𝜃)-= − ∂-log-π(𝜃)-, (54 ) ∂𝜃 ∂𝜃
which is called the MAP equation.

3.3.3 Maximum likelihood estimation

Often we do not know the a priori probability density of a given parameter and we simply assign to it a uniform probability. In such a case maximization of the a posteriori probability is equivalent to maximization of the probability density p(x,𝜃) treated as a function of 𝜃. We call the function l(𝜃,x) := p(x,𝜃) the likelihood function and the value of the parameter 𝜃 that maximizes l(𝜃,x) the maximum likelihood (ML) estimator. Instead of the function l we can use the function Λ (𝜃,x) = l(𝜃,x)∕p(x) (assuming that p (x ) > 0). Λ is then equivalent to the likelihood ratio [see Eq. (37View Equation)] when the parameters of the signal are known. Then the ML estimators are obtained by solving the equation

∂ log-Λ(𝜃,x)- ∂ 𝜃 = 0, (55 )
which is called the ML equation.
  Go to previous page Go up Go to next page