[R] You can find a lot of interesting things in the loss landscape of your neural network
![[R] You can find a lot of interesting things in the loss landscape of your neural network](https://b.thumbs.redditmedia.com/QE9v4YcNLKh2wRKVzC0dlq8BHkn6DF9AAO_NnbN-vZw.jpg "[R] You can find a lot of interesting things in the loss landscape of your neural network") |
Just sharing with you a small (and somewhat fun) project I was recently working on, which is about finding different patterns in the loss surface of neural networks. Usually, a landscape around a minimum looks like a pit with random hills and mountains surrounding it, but there exist more meaningful ones, like in the picture below (check the paper for more results). We have discovered that you can find a minimum with (almost) any landscape you like. An interesting thing is that the found landscape pattern remains valid even for a test set, i.e. it is a property that (most likely) remains valid for the whole data distribution. Paper: https://arxiv.org/abs/1910.03867 submitted by /u/universome |
