My question is about what constitutes a perfect Variational Autoencoders.

I am a bit confused by Table 1 on the paper about IAF Variational Autoencoders (https://arxiv.org/abs/1606.04934) however my question is about Variational Autoencoders in general.

The table looks somewhat like below (numbers different to the one in the paper):

​

​

In the typical derivation of the Variational Autoencoder (VAE) we find that we get the optimal model when our approximate posterior q(z|x; theta) approaches the true posterior p(z|x) and the difference between our VLB and log p(x) is KL[q(z|x) || p(z|x)].

On table 1, it shows values of both the VLB and log p(x) for different VAE models. It is clear why the VLB is less than log p(x) because of the gap between the approximate and true posterior, however why are the true marginals log p(x) different for different models?

After you’ve closed the gap between the approximate and true posterior – shouldn’t you achieve the perfect model? so why is it the case that the true marginals differ for different VAE models?