Minimum Variance Unbiased Estimator (MVUE): A Deep Dive

In the realm of estimation theory, the primary objective is to develop algorithms that can accurately estimate random variables. Ideally, these estimations should be both unbiased and possess the minimum possible variance. This article delves into the concept of the minimum variance unbiased estimator (MVUE), a cornerstone in statistical estimation, exploring its definition, conditions for existence, and methodologies for its determination.

To recap from the introduction to estimation theory, the ideal estimator, denoted as for a true parameter f₀, should satisfy two crucial criteria:

Unbiasedness: The expected value of the estimator should be equal to the true parameter value. Mathematically, this is represented as:

This equation signifies that, on average, the estimator correctly identifies the parameter value, without systematic over or underestimation.
Minimum Variance: Among all unbiased estimators, the MVUE is the one that minimizes the variance of the estimates. Variance, in this context, quantifies the spread or dispersion of the estimator’s values around its expected value. Lower variance indicates more precise and reliable estimates. The variance is given by:

Here, f₀ might represent a transmitted carrier frequency, and is its estimate derived from observed data. For a comprehensive understanding of the foundations, referring to an introductory article on estimation theory is recommended.

The Existence Question: Is a Minimum Variance Unbiased Estimator Always Possible?

The concept of a minimum-variance unbiased estimator (MVUE) is central to estimation theory because it represents the best-case scenario for unbiased estimation – achieving the lowest possible variance. However, it’s crucial to understand that an MVUE does not always exist for every estimation problem. The non-existence can arise in a couple of ways:

Absence of Unbiased Estimators: In some scenarios, it may be mathematically impossible to construct any estimator that is unbiased. The nature of the data or the parameter being estimated might inherently preclude the existence of an estimator that satisfies the unbiasedness condition.
No Uniform Minimum Variance Among Unbiased Estimators: Even when multiple unbiased estimators exist, there might not be one that consistently offers the minimum variance across all possible values of the parameter being estimated. Different unbiased estimators might exhibit lower variance in different ranges of the parameter values.

To illustrate this, consider Figure 1, which depicts two scenarios with three unbiased estimators (g1, g2, and g3) for a deterministic parameter θ, yielding estimates , , and respectively.

Figure 1: Illustration of existence of Minimum Variable Unbiased Estimator (MVUE)

In Figure 1a, estimator g3 clearly stands out by providing a uniformly minimum variance. Across the entire range of θ, g3’s variance is lower than both g1 and g2, making it the MVUE in this case.

However, Figure 1b presents a different situation. Here, none of the estimators achieves uniform minimum variance. For certain ranges of θ, g1 might have the lowest variance, while in other ranges, g2 or g3 could be superior in terms of variance. In this scenario, an MVUE, in the strictest sense of uniform minimum variance, does not exist. Choosing the “best” estimator would then depend on the specific range of θ of interest or other performance criteria beyond just variance.

Methods for Finding the Minimum Variance Unbiased Estimator

When an MVUE exists, several methods can be employed to identify it. Here are three primary approaches:

Cramer-Rao Lower Bound (CRLB) Approach:

The Cramer-Rao Lower Bound (CRLB) is a fundamental concept in estimation theory. It provides a lower bound on the variance of any unbiased estimator for a deterministic parameter. In essence, the CRLB sets a theoretical limit on how precise an unbiased estimator can be. The crucial insight for finding MVUEs is: if an unbiased estimator’s variance attains the CRLB for all possible values of the parameter, then that estimator is guaranteed to be the MVUE.

To use this method:

a. Determine the CRLB for the parameter of interest. Resources are available to guide you through calculating the CRLB.
b. Derive an unbiased estimator using any suitable method.
c. Calculate the variance of the derived estimator.
d. Compare the estimator’s variance to the CRLB. If the variance equals the CRLB, you have found the MVUE.

It’s important to note that while achieving the CRLB is a sufficient condition for an estimator to be MVUE, it’s not always guaranteed that such an estimator exists. The CRLB serves as a benchmark, and estimators that reach it are considered highly efficient. Further reading on applying the CRLB to find MVUEs is available.
Rao-Blackwell-Lehmann-Scheffe (RBLS) Theorem:

The Rao-Blackwell-Lehmann-Scheffe (RBLS) Theorem provides a powerful method for improving unbiased estimators. It states that if you have an unbiased estimator and a sufficient statistic for the parameter being estimated, you can always find another unbiased estimator that is a function of the sufficient statistic and has a variance no larger than the original estimator. If the sufficient statistic is also complete, then this improved estimator is the unique MVUE.

In practice, applying the RBLS Theorem involves:

a. Identifying a sufficient statistic for the parameter. A sufficient statistic is one that captures all the information in the data relevant to the parameter.
b. Finding an unbiased estimator (even a crude one).
c. Conditioning the initial unbiased estimator on the sufficient statistic. This process yields a new estimator that is a function of the sufficient statistic and is guaranteed to be an MVUE if the sufficient statistic is complete.

While theoretically significant, the RBLS theorem is less frequently used directly in practical applications for directly deriving MVUEs compared to methods based on CRLB. However, it provides a valuable theoretical framework and justification for focusing on sufficient statistics in estimation problems. More details on the RBLS Theorem can be found in dedicated notes.
Minimum Variance Linear Unbiased Estimator (MVLUE):

In situations where finding a general MVUE is challenging, or when we have reasons to restrict our search to linear estimators, the concept of the Minimum Variance Linear Unbiased Estimator (MVLUE) becomes relevant. MVLUE seeks the best linear estimator within the class of unbiased estimators. Linear estimators are simpler to work with and often have desirable properties.

The MVLUE approach involves:

a. Restricting the estimator to be a linear function of the observations.
b. Imposing the unbiasedness constraint.
c. Minimizing the variance among all linear unbiased estimators.

The resulting estimator is the MVLUE. However, it is crucial to remember that MVLUE is only the MVUE if the true optimal estimator is indeed linear. If the optimal estimator is non-linear, the MVLUE, while being the best linear unbiased estimator, will not be the overall MVUE. This method is particularly useful when the underlying problem is known or assumed to be linear or approximately linear.

In Summary

The minimum variance unbiased estimator (MVUE) is a gold standard in estimation theory, representing an estimator that is both unbiased and optimally precise in terms of variance. While not always guaranteed to exist, understanding MVUE and the methods to find it (CRLB, RBLS Theorem, MVLUE) is crucial for developing effective estimation algorithms. The choice of method and the feasibility of finding an MVUE depend on the specifics of the estimation problem at hand.