Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[D] NeurIPS 2019 Bengio Schmidhuber Meta-Learning Fiasco

The recent reddit post Yoshua Bengio talks about what’s next for deep learning links to an interview with Bengio. User u/panties_in_my_ass got many upvotes for this comment:

Spectrum: What’s the key to that kind of adaptability?***

Bengio: Meta-learning is a very hot topic these days: Learning to learn. I wrote an early paper on this in 1991, but only recently did we get the computational power to implement this kind of thing.

Somewhere, on some laptop, Schmidhuber is screaming at his monitor right now.

because he introduced meta-learning 4 years before Bengio:

Jürgen Schmidhuber. Evolutionary principles in self-referential learning, or on learning how to learn: The meta-meta-… hook. Diploma thesis, Tech Univ. Munich, 1987.

Then Bengio gave his NeurIPS 2019 talk. Slide 71 says:

Meta-learning or learning to learn (Bengio et al 1991; Schmidhuber 1992)

u/y0hun commented:

What a childish slight… The Schmidhuber 1987 paper is clearly labeled and established and as a nasty slight he juxtaposes his paper against Schmidhuber with his preceding it by a year almost doing the opposite of giving him credit.

I detect a broader pattern here. Look at this highly upvoted post: Jürgen Schmidhuber really had GANs in 1990, 25 years before Bengio. u/siddarth2947 commented that

GANs were actually mentioned in the Turing laudation, it’s both funny and sad that Yoshua Bengio got a Turing award for a principle that Jurgen invented decades before him

and that section 3 of Schmidhuber’s post on their miraculous year 1990-1991 is actually about his former student Sepp Hochreiter and Bengio:

(In 1994, others published results [VAN2] essentially identical to the 1991 vanishing gradient results of Sepp [VAN1]. Even after a common publication [VAN3], the first author of reference [VAN2] published papers (e.g., [VAN4]) that cited only his own 1994 paper but not Sepp’s original work.)

So Bengio republished at least 3 important ideas from Schmidhuber’s lab without giving credit: meta-learning, vanishing gradients, GANs. What’s going on?

submitted by /u/posteriorprior
[link] [comments]

Next Meetup

 

Days
:
Hours
:
Minutes
:
Seconds

 

Plug yourself into AI and don't miss a beat

 


Toronto AI is a social and collaborative hub to unite AI innovators of Toronto and surrounding areas. We explore AI technologies in digital art and music, healthcare, marketing, fintech, vr, robotics and more. Toronto AI was founded by Dave MacDonald and Patrick O'Mara.