non-parametric MLE

Posted by Yuling Yao on Jun 17, 2019.       Tag: zombies  

When I talked to someone about the old proof the invariance of odds ratio in a respropective sampling, I mentioned the estimation of $q(x)$ is achieved by its non-parametric MLE or its empirical distribution (see the third display equation in https://www.yulingyao.com/blog/2019/Logistic-reg-case/).

Apparently not everyone if happy with the notation non-parametric MLE as I was immediately questioned by a fairly strong argument “it makes no sense to talk about the likelihood without a parametric model”.

Fine, a non-parametric MLE is indeed parametric, with the form $q(x)= \sum_{i=1}^n \pi_i 1(x=x_i).$ The log likelihood of which reads $\log q_\pi(x_1, \dots, x_n)= \sum_{i=1}^n \log \pi_i.$ To maximize the log likelihood subject to the constraint $\sum_{i=1}^n \pi_i=1$ yields $\pi_i=1/n$ through Jensen.

If I think it deeper, the likelihood principle also tells us that, we can never distinguish the true data generating mechanism $x\sim q(x)$ and $x \sim 1/n \sum_{i=1}^n 1(x_i)$ given observed values $x_{1:n}$. Then we could also derive the same result by noticing that the log score is strictly proper.

It is just the convention to name it non-parametric. I understand rigorous reasoning relies on rigorous language, but I still feel bad why someone immediately turns unhappy just because I mentioned a well-established and conventional notation.


Comments are disabled for this post. Feedbacks via email are more than welcome.