Probability and statistics

Question

A binary message is transmitted as a signal that is either -1 with probability of 1-p or +1 with probability of p. The communication channel corrupts the transmission with additive Gaussian noise with mean u= 0 and variance o². The receiver concludes that the signal -1 (or +1) was transmitted if the value received is < T (or >T, respectively). Normal zero-mean with variance o² Transmitter(S=+1 or -1) --->Noisy Channel(N+S)--> Receiver = +1 if S+N>T or -1 if N +S<T

a) Find the value of T that minimizes the probability of total error? Find the solution in general without substituting values for o and p.

b) Find the value of threshold T for o =1 and p=0.5. c) Find the value of threshold T for o =1 and p=2/3. d) Interpret the results you found in "b" and "c".

Answer 1

For Z ~ N(0, 1), we write P(Z <= z) = P(z) = Φ(z) and f(z) = 1/sqrt(2 π) e^(-z^2/2)

Then, let E(T) = E(T|p,σ) be the total error at T.

The misclassification of the 1 signal will be Φ((T-1)/σ)

The misclassification of the -1 signal will be 1 - Φ((T+1)/σ) = Φ((-T-1)/σ)

The total error will be p Φ((T-1)/σ) + (1-p) Φ((-T-1)/σ)

To minimize the error, we take the derivative with respect to T. This is

p/σ f(T-1)/σ)) - (1-p)/σ f((-T-1)/σ) =

1/σ (p e-(T-1)^2/2σ^2 - (1-p)e-(T+1)^2/2σ^2))

Setting this equal to 0, we get p/(1-p) = exp((T-1)^2/2σ^2 - (T+1)^2/2σ^2)) = exp(-2T/σ^2)

ln(p/1-p) = -2T/σ^2

T = σ^2/2 ln((1-p)/p) = σ^2/2 ln(1/p - 1)

b) For σ = 1 and p = 1/2 (and, actually, for any σ)

T = 1/2 ln(1/(1/2) - 1) = 1/2 ln(0) = 0

For σ = 1 and p = 2/3

T = σ^2/2 ln((1-p)/p) = 1/2 ln(1/(2/3) - 1) = 1/2 ln(3/2 - 1) = 1/2 ln(1/2) = - 1/2 ln(2) = -0.346573590279973

The larger that p is, the smaller 1/p, ln(1/p - 1), and T = σ^2/2 ln(1/p - 1) is, so the smaller Tis.

This makes sense, the more likely that the signal is 1, the smaller the value must be before we think the signal is -1.