Code: https://github.com/suyash/mlt

Implemented transformers with One-to-Many, Many-to-One, Many-to-Many translation, with another demo for fine-tuning, by replacing regular layer norm in both encoder and decoder with Conditional Norm.

Conditional Norm is a technique from Style Transfer, used to train a single network with multiple styles, as well as mix styles, for example

https://i.imgur.com/U0L1x8u.png

Broader set of demos at https://github.com/suyash/stylizer/blob/master/image_stylization_demo.ipynb

The goal here is similar, make the rest of the network learn a common representation, while making the normalization parameters learn language specific semantics.

The One-to-Many and Many-to-One models are trained for English to French, German, Italian and Spanish Translation and Vice Versa.

The Many to Many model is trained on English-French, French-English, English-German and German-English.

The image stylization paper specifies how a N-style network can pick up an N+1th style through fine-tuning an existing model. Similarly, I fine-tune my Many-to-Many model to pick up Portuguese.

I also provide demos for Few-Shot and Zero-Shot translation.

Have provided SavedModels, as well as standalone notebooks that run in Google Colaboratory.

One-to-Many: https://colab.research.google.com/github/suyash/mlt/blob/master/one_to_many_demo.ipynb

Many-to-One: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_one_demo.ipynb

Many-to-Many: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_many_demo.ipynb

Many-to-Many fine-tuned: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_many_fine_tune_demo.ipynb