My understanding of VAE is that unlike Autoencoders, it does not directly give you a discrete encoding (latent code vectors n-dim) instead, it gives you both mu and sigma (n- dim mean vectors and n-dim standard deviation vectors). Then you have epsilon which you use to sample from a normal distribution with mu and sigma to create z. When combining mu, sigma and epsilon, you get z which is the one decoded by the VAE’s decoder. z is basically the main encoding.

Say my z, mu, sigma are of n-dimension like 10 (10-dim z, mu, sigma). I enable the user to have a free picking/giving me numbers 10 vectors [-a, a], say a = 5. So the user is free to pick 10 vectors between -5, 5.

This becomes my z that is decoded by my decoder to generate a new image.

[Main problem]

My VAE is trained on a dataset of apparel. Now, if I run my VAE’s encoder on each of the data in the dataset, I’d get a mu and sigma for each (not sure if this is still correct).

Using the z given by the user, how do I find the most similar from the dataset using VAE’s encoding of only mu and sigma?

My thinking is to generate z using mu and sigma generated by VAE’S encoder but in order to generate z, I still need to sample from a distribution using epsilon in which makes it non-discrete w.r.t user-generated z. This adds randomness to it so I am not sure how would I use the encoded z to find the nearest to user-generated z.