[Project] Multilingual Neural Machine Translation using Transformers with Conditional Normalization.
Implemented transformers with One-to-Many, Many-to-One, Many-to-Many translation, with another demo for fine-tuning, by replacing regular layer norm in both encoder and decoder with Conditional Norm.
Conditional Norm is a technique from Style Transfer, used to train a single network with multiple styles, as well as mix styles, for example
Broader set of demos at https://github.com/suyash/stylizer/blob/master/image_stylization_demo.ipynb
The goal here is similar, make the rest of the network learn a common representation, while making the normalization parameters learn language specific semantics.
The One-to-Many and Many-to-One models are trained for English to French, German, Italian and Spanish Translation and Vice Versa.
The Many to Many model is trained on English-French, French-English, English-German and German-English.
The image stylization paper specifies how a N-style network can pick up an N+1th style through fine-tuning an existing model. Similarly, I fine-tune my Many-to-Many model to pick up Portuguese.
I also provide demos for Few-Shot and Zero-Shot translation.
SavedModels, as well as standalone notebooks that run in Google Colaboratory.
Many-to-Many fine-tuned: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_many_fine_tune_demo.ipynb