[D] How to compute the “true” posterior for a generative model?
I have seen a few papers that show plots of the “true posterior” for a generative model on a toy problem.
Before I did not understand how this could be computed, but now maybe I am closer. I’m sure you can help me
Is this the way to do it? (See below)
Definitions and setup:
The generative model is p(x|z)*p(z). The posterior is p(z|x).
A particular x is given and we want to know the distribution p(z|x) for that x,
using p(z|x) ~ p(x|z)*p(x) ,
i.e. without evaluate the Bayes denominator.
- Sample z from p(z),
- evaluate p(x|z) for the given x, and multiply this by p(z) for the z that was sampled in step 1. This gives an “unnormalized” probability for this particular z.
- Accept this new z as a sample using MCMC, e.g. metropolis-hasting.
- Repeat 1-3, and then eventually make a kernel density plot of the resulting z sample locations.