Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[R] Understanding the generalization of “lottery tickets” in neural networks

Sharing our recent blog post summarizing some of our recent work understanding the boundaries of the lottery ticket hypothesis. In particular, we make some progress towards the following questions:

  • Do winning ticket initializations contain generic inductive biases or are they overfit to the particular dataset and optimizer used to generate them?
  • Is the lottery ticket phenomenon limited to supervised image classification, or is it also present in other domains like RL and NLP?
  • Can we begin to explain lottery tickets theoretically?

The blog post is below:

Understanding the generalization of “lottery tickets” in neural networks

And the papers covered can be found here:

One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers

Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP

Luck Matters: Understanding Training Dynamics of Deep ReLU Networks

Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension

submitted by /u/arimorcos
[link] [comments]

Next Meetup

 

Days
:
Hours
:
Minutes
:
Seconds

 

Plug yourself into AI and don't miss a beat

 


Toronto AI is a social and collaborative hub to unite AI innovators of Toronto and surrounding areas. We explore AI technologies in digital art and music, healthcare, marketing, fintech, vr, robotics and more. Toronto AI was founded by Dave MacDonald and Patrick O'Mara.