# HW3 (Due 10/31)

1. (5 points) For a discrete variable with more than two outcomes, the Beta prior can be generalized to the Dirchlet prior (please also see slides 87-91). Assuming a Dirchlet prior of $$Dir(\alpha_1=3,\alpha_2=3,\alpha_3=3,\alpha_4=1,\alpha_5=1,\alpha_6=1)$$, compute the a) ML, b) MAP, and c) Bayesian estimate of the probability of the third toss of a dice being one when the first two tosses are also ones.

2. (5 points) Consider two distributions $$p=[0.199, 0.599, 0.199, 0.001, 0.001, 0.001]$$ and $$q=[0.1, 0.3, 0.1, 0.1, 0.3, 0.1]$$. Compute $$KL(p||q)$$ and $$KL(q||p)$$. Note that they are significantly different from one another. You may also want to check out this tweet to get some more insight.

3. (5 points) For a signal with maximum variance of $$5^2$$, if we want to store the signal with the precision of 1 decimal place, how many bits will be needed (without any other information)?

4. (5 points) Use the tropical island example in HW1 again. Let's denote $$Q$$ as a random variable that $$Q=1$$ if $$T > T_{th}$$ and $$Q=0$$ otherwise.

1. Compute $$H(W|Q)$$ when $$T_{th}=25.5$$

2. Compute $$H(W|Q)$$ when $$T_{th}=26.5$$

5. Extra credit. Write a program to compute the best threshold $$T_{th}$$ so as to design a decision tree for the tropical island problem.

1. (10 points) Assume that we sampled the tropical island for 10 days and it is sunny for the first 8 days and rainy for the last 2 days. And the temperatures are 28,27,28,29,33,34,27,26,27,22. Find the best $$T_{th}$$ that minimizes $$H(W|Q)$$ for the given samples. Please submit your source code in pdf and a screenshot running your code.

2. (5 points) Simulate the tropical island by generating 10,000 samples. Estimate $$H(W|Q)$$ with the given samples for $$T_{th}=25.5$$ and $$T_{th}=26.5$$

3. (5 points) Find the optimum threshold for your samples.