Convolution layers are understood to encode an infinitely strong prior on spatial locality. (Ian Goodfellow’s Deep Learning book, summary here https://medium.com/inveterate-learner/deep-learning-book-chapter-9-convolutional-networks-45e43bfc718d). From my understanding, this means that conv kernels are good at capturing regional patterns (e.g. a point here, a stroke there, etc).

Do strided transpose convolutions encode a prior of some kind as well? (an uneducated guess from my end – a preference for texture and pattern at a larger scale?)

Based on the very handy arXiv Sanity Preserver tool, this incarnation from the Semantic Scholar team (at the allenai.org non-profit) has a few notable features to call out:

• Instantly updated recommendations, as you annotate papers you can refresh and immediately see updated recommendations from the model
• Currently covers all of arXiv CS + stat.ML
• You can create multiple, independent feeds to track different topics/areas

​

You can try Semantic Sanity out today at: s2-sanity.apps.allenai.org/ and let us know what you think!

What would happen if I created a NN to create better mazes(with restrictions like size) but a separate NN to complete the mazes as fast as possible. Who wins lol, or is this an actual method of training? Would this be applicable anywhere?

Hello everyone, my team and I have been working on a project about CNN inference acceleration on FPGA and during this process I ended up writing a mini Convolutional Neural Network inference library in C so that we have everything laid out. I think it could help anyone wanting to learn how CNN inference works on a “low” level. My primary goal in writing this library was clarity and readability and I have inserted lots of comments to guide the reader. Any feedback would be much appreciated, here is a link to the repository:

CNN-Inference-Didactic

Just came back from CVPR and wanted to offer some personal observations.

TLDR humangous conferences pack lots of science, but it feels overwhelming and may affect how science is ‘shared’. For sure it affects the planet.

The good : * overall a smooth organization without major hickups * lots of nice and interesting papers * excellent presence of work across the globe but China had an impressive showing * lots of industry presence * expo full of companies ready to hire the best * large venue that managed to accommodate the almost 9000 people that didn’t feel that we were that many * good catering considering the size * hundreds of people worked behind the scenes to make it happen (catering, waiters, cleaners, security) and I am so happy to see people have work and take care of us

The bad : * too many people made it harder to bump into people and network. The super sized venue spread across didn’t help either. Time from one to the other could easily take 5 minutes. * posters the first day it was impossible due to crowding. The second day and after became much better but still impossible to go through all 200 posters per session. * 5 min per oral was an interesting concept but it has some problems. It keeps every one on time via the video delivery but it makes for a dry non interactive delivery. The short time also tweet-sizes research. You get to focus on how amazing are the slides rather than the science. It was evident that those that broke down the message to the simplest possible and made a good presentation about it gave the best presentations. Some presentations appeared professionally made by a company. Some rethinking of conference format is necessary. * questions after orals are obsolete. Remove them completely. * I found the herding by local security / catering people annoying. We were being herded always and everywhere. Too many rules… * THE WORST : the carbon footprint. I am not talking about travel. I am talking about the amount of plastic and trash produced. By a small calculation I expect more than 100000 paper cups and plastic lids. Every lunch was in a plastic box and plastic spoon: about 50000 plus were used. Hundreds of thousands of plastic cups were used for water and drinks. Come on… Seriously. Have someone sponsor a reusable mug and hand it out. People can bring it back as souvenir. There are nice ones made by bamboo. (By the way it is possible that we can recycle some of the plastic but you all know that we are past beyond that now in the planet.) As conference sizes sky rocket we must work together to help reduce impact to the planet.

If any of you went there or not, please share your thoughts.

How different is Self-supervised learning to Denoising Autoencoder? Is it like 2019 rebrand?

• Both need cleverly hand-engineered noise as input. This is the meta-label.
• Labels are free, because labels are the original inputs themselves.
• Unsupervised-as-supervised learning
  • Remove, change part of inputs (colors, rotation, pixels, words), then predict what is missing, what is original.
• Learnt embeddings become free feature extractor.

How about minimax game to noise and denoise? Can we do something with training frameworks like Actor-Critic, GAN, and PowerPlay ?

Denoising autoencoder 2019.

Noise the inputs and put original input as labels.

Noise is unfortunately must be cleverly highly hand-engineered.

Like crop image uncrop image, remove color channels predict colors, remove positions predict positions, rotate image reorient image, remove last word predict last word. Then take the learnt weights as feature extractor.

Now we wait for something like GANs for denoising, where generator make new denoising tasks, and discriminator must solve the new tasks