Skip to main content

Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[Project] Multilingual Neural Machine Translation using Transformers with Conditional Normalization.

Code: https://github.com/suyash/mlt

Implemented transformers with One-to-Many, Many-to-One, Many-to-Many translation, with another demo for fine-tuning, by replacing regular layer norm in both encoder and decoder with Conditional Norm.

Conditional Norm is a technique from Style Transfer, used to train a single network with multiple styles, as well as mix styles, for example

https://i.imgur.com/U0L1x8u.png

Broader set of demos at https://github.com/suyash/stylizer/blob/master/image_stylization_demo.ipynb

The goal here is similar, make the rest of the network learn a common representation, while making the normalization parameters learn language specific semantics.

The One-to-Many and Many-to-One models are trained for English to French, German, Italian and Spanish Translation and Vice Versa.

The Many to Many model is trained on English-French, French-English, English-German and German-English.

The image stylization paper specifies how a N-style network can pick up an N+1th style through fine-tuning an existing model. Similarly, I fine-tune my Many-to-Many model to pick up Portuguese.

I also provide demos for Few-Shot and Zero-Shot translation.

Have provided SavedModels, as well as standalone notebooks that run in Google Colaboratory.

One-to-Many: https://colab.research.google.com/github/suyash/mlt/blob/master/one_to_many_demo.ipynb

Many-to-One: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_one_demo.ipynb

Many-to-Many: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_many_demo.ipynb

Many-to-Many fine-tuned: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_many_fine_tune_demo.ipynb

submitted by /u/suyash93
[link] [comments]