Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[P] Dilated Convolution Seq2Seq

I implemented dilated convolution Seq2Seq, based architecture from Convolution Seq2Seq, tested on 100k English-Malay translation dataset, and I beat that model in term of word position. 80% to train, 20% to test.

This result after 20 epochs only,

  1. Attention is All you need, train accuracy 19.09% test accuracy 20.38%

  2. BiRNN Seq2Seq Luong Attention, Beam decoder, train accuracy 45.2% test accuracy 37.26%

  3. Convolution Encoder Decoder, train accuracy 35.89% test accuracy 30.65%

  4. Dilated Convolution Encoder Decoder, train accuracy 82.3% test accuracy 56.72%

  5. Dilated Convolution Encoder Decoder Self-Attention, train accuracy 60.76% test accuracy 36.59%

Source code here

To make sure the translation work, I implemented beamsearch from tensor2tensor on no 4.

Feel free to use it for future research, and let me know if got better or bad results!

submitted by /u/huseinzol05
[link] [comments]

Next Meetup

 

Days
:
Hours
:
Minutes
:
Seconds

 

Plug yourself into AI and don't miss a beat

 


Toronto AI is a social and collaborative hub to unite AI innovators of Toronto and surrounding areas. We explore AI technologies in digital art and music, healthcare, marketing, fintech, vr, robotics and more. Toronto AI was founded by Dave MacDonald and Patrick O'Mara.