The field of machine learning underwent massive changes in the 2010’s. At the beginning, the field saw diverse approaches applied to a variety of topics and data structures. Then Alexnet blew away the competition for the Imagenet challenge with his CNN, and the field was forever changed. However, there was a warming up phase. Caffe’s first release was in 2013. Tensorflow and Keras were first released in 2015. Pytorch in 2017. Before Caffe’s first release, groups were largely ill-equipped to do convolutional neural network research as they did not possess the resources to develop an efficient library that would run on the GPU. Research continued mostly the same from 2010 – 2014 while these libraries were being developed. Then the real explosion happened from 2014-2016, where paper submissions (and subsequent accepted papers) exploded at various conferences. And of course, as more people explored CNN based machine learning, the better the community got at designing them, so naturally, the scores on ILSVRC get better and better. Various other empirical benchmarks have been used in a standardized test fashion to explicitly compare methods (models).

Prior to the neural network revolution, and even a little after its beginning, it was still believed that there were some problems that are still unsolvable for learning machines, namely Go. The anomaly to these statistics is of course Deepmind and the like. They showed approximating the value function via a CNN is able to master seven classic Atari games. This was in 2013. In March of 2016, they beat the world champion of Go with a match score of 4-1. AlphaGo showcased the power of combining neural networks with traditional algorithms, which largely went against the tide of end-to-end approaches. In April of 2019, OpenAI 5 stomped the world champions in Dota 2. Deepmind showcased their StarCraft II AI at Blizzcon in November 2019, beating everyone, including the last world champion 4-0.

As the 2010’s wrap up, I encourage you to take a step back from your work and appreciate how far we’ve come. What was your favorite moment of this decade?