probably many do not know this, I learned it by studying the references in section 19 of Jurgen’s very dense inaugural tweet

I knew AlexNet, the CUDA CNN by Alex Krizhevsky and Ilya Sutskever and Geoff Hinton which won ImageNet 2012, but prior to AlexNet, Jurgen’s team with his “outstanding Romanian postdoc Dan Ciresan … won 4 important computer vision competitions in a row between May 15, 2011, and September 10, 2012” with an earlier CUDA CNN, let me call this DanNet, the blog post on their miraculous year links to a summary of these contests

I saw a news article claiming that AlexNet started a deep learning revolution in 2012, but actually the references show that DanNet was the first superhuman CNN in 2011 and also won a medical imaging contest on images way bigger than AlexNet’s

the most cited DanNet paper is CVPR July 2012, 5 months before AlexNet at NIPS 2012, but earlier descriptions of DanNet appeared at IJCAI 2011 and IJCNN 2011

in his blog, Jurgen also cites CNN pioneers since Fukushima 1979, and GPU implementations of neural networks since Jung and Oh 2004

to be fair, AlexNet cites DanNet and admits that it is similar, however, it does not mention that DanNet won all those earlier challenges

ResNet beat AlexNet on ImageNet in 2015, but ResNet is actually a special case of the earlier highway networks, also invented in Jurgen’s lab, the “First Working Feedforward Networks With Over 100 Layers,” section 4 of The Blog links to an overview, he credits his students Rupesh Kumar Srivastava and Klaus Greff

there was a big reddit thread on section 5 of his blog, Jurgen’s GAN of 1990, and everybody knows LSTM, which won contests already in 2009, section 4 of The Blog, but I think many don’t know yet that his team also was first in the CUDA CNN game