### 3.3 Parameter estimation

Very often we know the waveform of the signal that we are searching for in the data in terms of a finite
number of unknown parameters. We would like to find optimal procedures of estimating these parameters.
An estimator of a parameter is a function that assigns to each data set the “best” guess of the
true value of . Note that because depends on the random data it is a random variable. Ideally
we would like our estimator to be (i) unbiased, i.e., its expectation value to be equal to the
true value of the parameter, and (ii) of minimum variance. Such estimators are rare and in
general difficult to find. As in the signal detection there are several approaches to the parameter
estimation problem. The subject is exposed in detail in [81, 82]. See also [149] for a concise
account.

#### 3.3.1 Bayesian estimation

We assign a cost function of estimating the true value of as . We then associate with an
estimator a conditional risk or cost averaged over all realizations of data for each value of the
parameter :

where is the set of observations and is the joint probability distribution of data and
parameter . We further assume that there is a certain a priori probability distribution of the
parameter . We then define the Bayes estimator as the estimator that minimizes the average risk defined
as
where E is the expectation value with respect to an a priori distribution , and is the set of
observations of the parameter . It is not difficult to show that for a commonly used cost function
the Bayesian estimator is the conditional mean of the parameter given data , i.e.,
where is the conditional probability density of parameter given the data .

#### 3.3.2 Maximum a posteriori probability estimation

Suppose that in a given estimation problem we are not able to assign a particular cost function .
Then a natural choice is a uniform cost function equal to 0 over a certain interval of the parameter .
From Bayes theorem [28] we have

where is the probability distribution of data . Then, from Eq. (50) one can deduce that for each
data point the Bayes estimate is any value of that maximizes the conditional probability .
The density is also called the a posteriori probability density of parameter and the
estimator that maximizes is called the maximum a posteriori (MAP) estimator. It is
denoted by . We find that the MAP estimators are solutions of the following equation
which is called the MAP equation.

#### 3.3.3 Maximum likelihood estimation

Often we do not know the a priori probability density of a given parameter and we simply assign to it a
uniform probability. In such a case maximization of the a posteriori probability is equivalent to
maximization of the probability density treated as a function of . We call the function
the likelihood function and the value of the parameter that maximizes
the maximum likelihood (ML) estimator. Instead of the function we can use the function
(assuming that ). is then equivalent to the likelihood ratio [see
Eq. (37)] when the parameters of the signal are known. Then the ML estimators are obtained by solving
the equation

which is called the ML equation.