# HW2 (Due 9/19)

1. Continue with tropical island question in HW1. Let's denote $$W_i$$ as the weather condition (either sunny or rainy) and $$T_i$$ as the temperature on day $$i$$.

1. What is $$H(T_0)$$?

2. What is $$H(T_0|W_0)$$?

3. What is $$H(T_1|W_0)$$?

4. What is $$I(T_1;T_0|W_1)$$?

2. Let $$X_1$$ and $$X_2$$ be two discrete random variables whose outcomes do not overlap. And let $$X=X_1$$ with prob $$p$$ and $$X=X_2$$ with prob $$(1-p)$$. Show that $$H(X) = p H(X_1) + (1-p) H(X_2) + H(p)$$, where $$H(p) \triangleq -p \log_2(p) - (1-p) \log_2 (1-p)$$.

3. (Typical sequences of coin-flips) Consider sequences of $$10,000$$ coin-flips with the probability of head equal to $$0.7$$. Show that a sequence with $$7,000$$ heads and $$3,000$$ tails is typical.