I have a project that uses Variational Autoencoder for an apparel dataset that is grouped into five categories (say, (A B C D E).

My plan is the following.

1. Train a VAE model using the apparel dataset.
2. Use encoder on each data to produce latent code (e.g. my bottleneck/latent representation is of size 10 for example). Store to database
3. Use K-Means to cluster the data (using the latent code) in the database with n categories (five for example). Store cluster labels for each data in database.
4. Store cluster centroids in the database created from #3.
5. User interacts with a GUI that lets him use sliders to generate its own latent code (ten slides because #2 is 10). Decoder generates an image from the latent code given.
6. Click Recommend – enables the user two parts
   1. Get the product/apparel from database that is most similar. (1. Predict cluster. 2. Find the most similar in the cluster using distance metric on the latent codes stored).
   2. Recommend from other cluster. The idea is that if the user generates a topwear (e.g. a shirt), I would also generate from other clusters (for example, other cluster have bottomwear, shoes, etc.). This is my problem right here.

For clustering, I could just use a simple K-Means. I can get the cluster labels and the cluster centroids.

My idea for 6.2 (Recommendation):

• I’m not really sure but for sure, there is a relationship between the cluster centroid (cluster mean) and the most similar/generated latent code. Is dot product applicable to this?
• My idea is that if my user generated code (vector) is called X, the most similar as A1, cluster centroid for the predicted cluster as A0, cluster centroid for another cluster as B0.
• I could know the projection of X w.r.t A0 and then use this amount of projection (idk what it is called, or if there is such a concept), to B0 to find the most similar in cluster B which is B1.

IS this even possible? If yes, what is this called? If not, could you recommend a better recommendation system that revolves around the same concept?