HW3 (Due 10/31)
(5 points) For a discrete variable with more than two outcomes, the Beta prior can be generalized to the Dirchlet prior (please also see slides 8791). Assuming a Dirchlet prior of \(Dir(\alpha_1=3,\alpha_2=3,\alpha_3=3,\alpha_4=1,\alpha_5=1,\alpha_6=1)\), compute the a) ML, b) MAP, and c) Bayesian estimate of the probability of the third toss of a dice being one when the first two tosses are also ones.
(5 points) Consider two distributions \(p=[0.199, 0.599, 0.199, 0.001, 0.001, 0.001]\) and \(q=[0.1, 0.3, 0.1, 0.1, 0.3, 0.1]\). Compute \(KL(pq)\) and \(KL(qp)\). Note that they are significantly different from one another. You may also want to check out this tweet to get some more insight.
(5 points) For a signal with maximum variance of \(5^2\), if we want to store the signal with the precision of 1 decimal place, how many bits will be needed (without any other information)?
(5 points) Use the tropical island example in HW1 again. Let's denote \(Q\) as a random variable that \(Q=1\) if \(T > T_{th}\) and \(Q=0\) otherwise.
Compute \(H(WQ)\) when \(T_{th}=25.5\)
Compute \(H(WQ)\) when \(T_{th}=26.5\)
Extra credit. Write a program to compute the best threshold \(T_{th}\) so as to design a decision tree for the tropical island problem.
(10 points) Assume that we sampled the tropical island for 10 days and it is sunny for the first 8 days and rainy for the last 2 days. And the temperatures are 28,27,28,29,33,34,27,26,27,22. Find the best \(T_{th}\) that minimizes \(H(WQ)\) for the given samples. Please submit your source code in pdf and a screenshot running your code.
(5 points) Simulate the tropical island by generating 10,000 samples. Estimate \(H(WQ)\) with the given samples for \(T_{th}=25.5\) and \(T_{th}=26.5\)
(5 points) Find the optimum threshold for your samples.
