Uniformly Minimum-Variance Unbiased Estimator

Question

1. Let X1, . . . , Xn ∼ iid Gamma(α, θ) with α known.

(a) Find the uniformly minimum-variance unbiased estimator (UMVUE) for θ.

(b) Determine if the UMVUE is efficient.

(c) Determine if the UMVUE is consistent.

(d) Determine if the UMVUE is asymptotically efficient.

Answer 1

Hi Karl!

(a)

If X₁, X₂, ... , X_n are iid Gamma(α,θ) then their common density is:

ƒ(x) = 1/(θ^αΓ(α)) x^α-1e^-x/θ.

Therefore, the joint likelihood is

L(x) = 1/(θ^nαΓ(α)ⁿ) Πx^α-1e^-Σx/θ = 1/θ^nαe^-Σx/θ * Γ(α)^-nΠx^α-1

By The Factorization Theorem, T=Σx is sufficient for θ . This is because L(x) has been factored into the product of:

1. g(T(x),θ) = 1/θ^nαe^-Σx/θ which relies on x only through the sufficient statistic T, and
2. h(x) = Γ(α)^-nΠx^α-1 which is a function of the data and known parameters.

To show T is complete we must show that there is no function of T which is an unbiased estimator of 0 except the constant function 0, by the definition of completeness. Note that because the X_i are all iid Gamma(α,θ), their sum T is Gamma(nα,θ). Consider the expectation of f(T), a function of T:

E(f(T)) = ∫ f(t) * 1/(θ^nαΓ(nα)) t^nα-1e^-t/θ dt, where t ranges from 0 to infinity, nα and θ positive.

Factoring out terms which do not rely on t yields:

∫ f(t) * t^nα-1e^-t/θ dt

which is the integral of f(t) and the product of two functions which are positive for all t>0: t^nα-1 and e^-t/θ.

Therefore, if this expectation is 0, then f(t) must be precisely equal to 0 almost surely, and by the definition of completeness, T(x) is a complete statistic.

Note that because we know T is Gamma(nα,θ), we know E(T) = nαθ, and so T/nα must be an unbiased estimator of θ. Lehmann-Scheffe tells us that unbiased estimators based on complete sufficient statistics must be UMVUE, and so T/nα is UMVU for θ.

(b)

An estimator is efficient if it meets the Cramer-Rao Lower Bound, which is a theoretical lower bound for any unbiased estimator of any parameter, equal to the inverse of the Fisher Information. The Fisher Information has many equivalent definitions, but it's probably easiest in this scenario to define it as negative the expected value of the derivative of the score function with respect to θ.

To answer this question, first we must find the variance of T/nα. We know the distribution of T is Gamma(nα,θ), so its variance is nαθ², and therefore the variance of T/nα is θ²/nα.

Next, we must find the log-likelihood function for a single observation X:

L(x) = lnΓ(α) - αlnθ + (α-1)ln(x) - x/θ.

The derivative of this with respect to θ is the score function:

l(x) = -α/θ + x/θ²

And therefore, the derivative of the score function is

l`(x) = α/θ² - 2x/θ³

Negative the expected value of l` is

-E(l`(x)) = -α/θ² + 2/θ³E(X) = -α/θ² + 2αθ/θ³ = α/θ².

Because the Fisher Information scales with the sample size, the Fisher Information for a sample of size n is nα/θ².

And the inverse of this is θ²/nα, which is precisely the variance of T/nα.

Therefore, our estimator T/nα is consistent.

(c)

An estimator is consistent if it converges in probability to the parameter it is used to estimate.

Notice that T/n is exactly the sample mean, and we have by the Weak law of Large Numbers:

T/n → E(x) = αθ as n → infinity

where "→" symbolizes convergence in probability. Dividing through by α yields the desired result.

(d)

An estimator is asymptotically efficient if it asymptotically achieves the Cramer-Rao Lower Bound. That is,

Var(T/nα) → θ²/nα as n → infinity.

Because we've already shown that Var(T/nα) = θ²/nα, T/nα is asymptotically efficient.