Let’s recall the eight-school example– the posterior distribution of the school effect is essentially indistinguishable from 0, which by and large says there is little school-specific effect.

In spite of that, a parent seeing this result would still have to pick one school to which deliver their kids the next morning. They could row a dice and pick one at random given they are so sure these schools are just the same and do not bother to even select. But it is also rational to stick to the one with the largest posterior mean– more sophisticatedly model the “picking which school” in a decision theory framework.

The order of posterior mean may not be the same of empirical mean, which suggests that the most intuitive choice of picking the school with the empirically largest school may not always be optimal.

Practically however we have to make decisions anyway even without a reasonable model. I am thinking about this question in the Iowa context. Suppose in year 2800 AD and some archaeologist finds only half page of New York Times from 2020 which writes Mayor Pete won Iowa caucus by a solid one more delegates than Sanders. The archaeologist is trying to uncover the whole story but remaining details are missing from the unearthed paper. With information at hand, if the archaeologist is enforced to make inference, he has to guess Mayor Pete also won the popular vote.

Another example is the coronavirus. Suppose the same archaeologist reads about the daily updated number of new confirmed cases of coronavirus from the back of the page of that half unearthed New York Times, he finds the number spiked to 15,152 on FEB 12 2020, almost 5 times compared with the day before. We actually know it is because China changed its reporting rule and started to include “clinically diagnosed” cases in its figures and that 13,332 of the new cases fall under that classification. But with a much smaller sigma filed at hand, the archaeologist has to guess that day FEB 12 2020 AD was probably the worst single day, no matter what change-point-detection model he is willing to try.

All these examples are the consequence of lack of modeling– how the state delegates are counted, how the confirmed cases are collected, etc. To be fair, that archaeologist would make the correct inference in average– I suppose the probability of wining the popular vote but losing the overall election can be calculated explicitly and should be a small number.

The point is, even in a Bayesian decision theory framework, the decision is often chosen through an optimization procedure – it is suggested not to make binary decision, but if you has to eventually, you do make a binary decision. In many cases the uncertainty of the last step inherited from the last-step point optimization could be understated.