https://www.biorxiv.org/content/10.1101/582643v1

​

r/compmathneuro discussion here.

​

Abstract:

Over the last decade, artificial neural networks (ANNs), have undergone a revolution, catalyzed in large part by better tools for supervised learning. However, training such networks requires enormous data sets of labeled examples, whereas young animals (including humans) typically learn with few or no labeled examples. This stark contrast with biological learning has led many in the ANN community posit that instead of supervised paradigms, animals must rely instead primarily on unsupervised learning, leading the search for better unsupervised algorithms. Here we argue that much of an animal’s behavioral repertoire is not the result of clever learning algorithms—supervised or unsupervised—but arises instead from behavior programs already present at birth. These programs arise through evolution, are encoded in the genome, and emerge as a consequence of wiring up the brain. Specifically, animals are born with highly structured brain connectivity, which enables them learn very rapidly. Recognizing the importance of the highly structured connectivity suggests a path toward building ANNs capable of rapid learning.

How would you approach this concept: I am interested in making a bot that creates tattoos. I have a tattoo artist who is willing to potentially sacrifice his artistic integrity and feed the bot his simple designs to serve as a dataset. Which framework do you reckon would be best? Let’s say we have 1000 individual high res scans of black tattoos on white background and we want the bot to spit new, artificially synthesized designs (even if abstract and nonsensical). Any help would be much appreciated. This is for a short film. We have backing behind this so if there is anyone interested in being part of a creative collaboration and potentially get some small buck for your time, hit me in the DMs.

Do softmax probability outputs in classification actually mean anything? Given that NNs aren’t generative models, how can they give a probabilistic estimate of the class given a sample?

Should they be trusted if they don’t actually mean anything?

If I want to perform some computer vision task using CNN (let’s say segmentation), how can I ensure that I can get generalization across images that were captured using another camera(with different intrinsic matrix)? I tried using a learned semantic segmentation model on my own camera images and it failed terribly. I suspect this is because the convolution filters are overfitted to the particular resolution (fx, fy) we trained our model on.

I’m sorry if the question is too vague (or too stupid). I’m trying to build a very basic speech recognition (for my own learning). I’m using pyaudio to record from the microphone and I’ve managed to convert the bytes to 16bit representations. But I do not know how to move forward with my little project.

I did find bits and pieces of code that seems to do what I’m trying to achieve but doesn’t explain clearly why it does what it does.

For context I’ve previously worked on Image recognition, object detection and similar computer vision projects, but I’m a newb when it comes to audio.

Any help is appreciated.

Is there any good papers / surveys on transfer learning / domain adaptation for time series forecasting?

I am mostly working with LSTM recurrent neural networks, but any other ML-based approach should be an interesting read.

One of my favorite papers is Generalized End-to-End Loss for Speaker Verification. It describes a way to learn a model that can derive embeddings that are highly representative of the characteristics of the voice from speech segments. It does so with only the identity of the speakers as labels. It’s also an approach that can be applied to any kind of data beyond just voice, provided that the data is grouped by source (e.g. for speech it is grouped by speaker, for faces it is grouped by identity, …).

A classical approach that will work without any labels is using an autoencoder. Not being up to speed in that domain, are there autoencoder-based frameworks that have proved to extract powerful features, more so that the classical auto-encoder pipeline?

Do you also know of approaches beyond these that achieve this goal?

https://arxiv.org/abs/1907.09019

TL;DR we consider images that show illusion in Human, represent the images in ImageNet-trained VGG-19, and find that representational distances around an illusion image are “weird” (non-monotonic). This is mostly unlike distances around control images, at least to the extend that we checked.

What do you think? We are now considering extensions, and would very much appreciate ideas, even negative.

DeepMind’s self-supervised (a.k.a. unsupervised) network, CPC, surpassed AlexNet’s performance on ImageNet. If I understand correctly, both CPC and AlexNet used the same set of training images. CPC just didn’t use labels, while AlexNet did. So, what about instances where a self-supervised network can be trained on 10,000x as much data as would be economically feasible to label? In these cases, are supervised learning’s days numbered? Or not so fast?

The application I’m personally most interested in is self-driving cars. By putting cameras on consumer cars, you are limited really only by your fleet size, your data centre costs, and your customers’ monthly bandwidth limits for their home wifi in terms of how much data you can collect. Tesla, for instance, has over 500,000 cars with 360-degree cameras, GPUs or ASICs to run neural networks, and the ability to connect to wifi and upload training data. Elon Musk recently mentioned Tesla’s plans to “do unsupervised massive training of vast amounts of video”. Tesla’s Director of AI, Andrej Karpathy, also recently tweeted his strong support for self-supervised learning. So this question is more than hypothetical.