Paper: https://arxiv.org/abs/1910.03524 (NeurIPS 2019)

Code: https://github.com/stanis-morozov/prodige

The paper proposes an embedding layer based on weighted graph instead of vectors. Intuitively, this layer learns to represent concepts/words by their relation to other. Trains by backprop w.r.t. graph edges.

(Left) PRODIGE learned on a subset of MNIST. (Right) zoom-in of some clusters.

• + Learns interpretable hierarchies from raw objects;
• + The model is much smaller than typical vector embeddings;
• – The official code is CPU-only, it aint too fast

Interactive version of the plot above: https://neurips-anonymous.github.io/index.html