Suppose that we have a random sample X1, ..., Xn from a Bernoulli(p) distribution. What is the maximum likelihood estimator (MLE) of p?

Answer 1

We have a random sample of variates from a Bernoulli distribution (a simple success-fail, 1-0 experiment for one trial, and with probability of success p). To know how the sample overall behaves, we need its joint density function which will be "re-branded" here as a likelihood function of p given the data we're seeing. (For this, we'll need to take a product from i = 1 to n, hence the Π symbol shown in the work below.) Note that the Bernoulli(p) density function is p^x(1-p)^1-x.

L(p) = f( x | p) = Π p^x_i(1-p)^1-x_i = p^Σx_i (1-p)^n-Σx_i

Now with the joint density function, let's take the logarithm of it. This gives the log-likelihood function:

l(p) = ln(L(p)) = ln (p^Σx_i (1-p)^n-Σx_i )

l(p) = (Σx_i )ln(p) + (n-Σx_i )ln(1-p)

Now we take the (partial) derivative of l(p) with respect to p and set this derivative equal to zero. From there, solve for p to get the MLE estimate for p.

(δl/δp)l(p) = ((Σx_i ) / p ) - ((n-Σx_i ) / (1-p)) = 0

((Σx_i ) / p ) = ((n-Σx_i ) / (1-p))

(Σx_i ) (1-p) = (n-Σx_i ) (p)

Σx_i - pΣx_i = pn - pΣx_i

Σx_i = pn

(Σx_i / n) = p

Doesn't that look a lot like the definition of x-bar? This is precisely that!

The MLE estimate for p is x-bar = (Σx_i / n).