Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[P] The Illustrated GPT-2 (Visualizing Transformer Language Models)

Hello r/MachineLearning,

This is a new post in which I try to visualize the majority of what happens inside a trained GPT-2. We follow the journey of an input word from embedding, all the way up to the output of the model. I’ve also included a crude analogy for the query/key/value vectors of self-attention that I hope makes it easier for people starting out with transformer architectures. By the end of the post, we’d have looked at the major weight matrices of a single block, as well as the major weight matrices of the entire model. All feedback and corrections are welcomed!

The post: https://jalammar.github.io/illustrated-gpt2/

submitted by /u/nortab
[link] [comments]

Next Meetup

 

Days
:
Hours
:
Minutes
:
Seconds

 

Plug yourself into AI and don't miss a beat

 


Toronto AI is a social and collaborative hub to unite AI innovators of Toronto and surrounding areas. We explore AI technologies in digital art and music, healthcare, marketing, fintech, vr, robotics and more. Toronto AI was founded by Dave MacDonald and Patrick O'Mara.