Applied Probability & Information Theory - Joint Entropies

Question

A child’s toy contains two windows in each of which a clown’s face appears when a button is pressed. The clown may be either smiling or crying. The design is such that the probability of each expression in each window is 0.5.

Define two binary random variables X_j (j = 1, 2) by,

X_j = 1 ;if clown in window j is smiling

X_j = 0 ;if clown in window j is crying (j=1,2).

(i) Assuming that there is no mechanism linking the expression on the face of clown 1 to that of clown 2, show that X₁ and X₂ are independent.

(ii) If there is a mechanism such that clown 1 smiles whenever clown 2 cries (and vice versa), show that X₁ and X₂ are not independent.

Answer 1

The definition of independence of two random events A and B is

P(A∩B) = P(A)P(B)

In our case

P(X₁ = 0 ∩ X₂ = 0) = P(X₁ = 0)P(X₂ = 0)

P(X₁ = 0 ∩ X₂ = 1) = P(X₁ = 0)P(X₂ = 1)

P(X₁ = 1 ∩ X₂ = 0) = P(X₁ = 1)P(X₂ = 0)

P(X₁ = 1 ∩ X₂ = 1) = P(X₁ = 1)P(X₂ = 1)

We are given that P(X_j = 0) = P(X_j = 1) = 0.5

So the right hand sides of all four equations are 0.25.

(i)

The information about "no mechanism ..." is probably meant to imply that

all the four possibilities on the left of our equations are equally likely,

which would given them the probability of 0.25.

(ii)

P(X₁ = 0 ∩ X₂ = 0) = 0 ≠ P(X₁ = 0)P(X₂ = 0)

Therefore X₁ and X₂ are not independent.