Learn About Our Meetup

4500+ Members

[D] GPT2 as seq2seq decoder

Hello! Not having the computational resources to train a seq2seq transformer-based model, I’m trying to do that by fine-tuning BERT as an encoder and GPT2 as a decoder. Has anyone tried something similar? How can I condition GPT2 on the encoder’s output?

submitted by /u/Viecce
[link] [comments]

Next Meetup




Plug yourself into AI and don't miss a beat


Toronto AI is a social and collaborative hub to unite AI innovators of Toronto and surrounding areas. We explore AI technologies in digital art and music, healthcare, marketing, fintech, vr, robotics and more. Toronto AI was founded by Dave MacDonald and Patrick O'Mara.