Learn About Our Meetup

5000+ Members



Join our meetup, learn, connect, share, and get to know your Toronto AI community. 



Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.



Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[D] Regarding the ability of neural networks to learn “simple” examples first

So I’ve been pretty interested in this paper A Closer Look at Memorization in Deep Networks and particularly the first experiment they did where they showed that certain data points are consistently fit in the first epoch of training whereas other data points consistently take longer epochs to fit.

But I haven’t seen any discussions anywhere about why that would be the case? Like what is it about these data points that allows them to be easily fit in the first epoch? How can we formalize this notion of “simpleness”?

My first thought is that the “simple” data are just the ones which have a gradient direction that is close to the averaged gradient direction for a given minibatch?

Anyone aware of any work specifically expanding on these questions?

Unfortunately I don’t have anyone in my lab to discuss these things with so I just resort to the next best place lol.

submitted by /u/Minimum_Zucchini
[link] [comments]

Toronto AI is a social and collaborative hub to unite AI innovators of Toronto and surrounding areas. We explore AI technologies in digital art and music, healthcare, marketing, fintech, vr, robotics and more. Toronto AI was founded by Dave MacDonald and Patrick O'Mara.