When iOS 13 released, the music app had a redesign on the lyrics player, showing the lyrics like a slide show as the music played with incredible time accuracy. Plus, all the phrases were revealed in perfectly grouped of text. The lyrics player even detected when the song was in an instrumental without any singing. It was even enable to detect the quiet mumbles and unintelligible words that are covered in the background music.

So the main question is… can this be done without ML and if it is (most likely), how can you get or train a model that can reproduce this intelligent system?

I’d love to hear your thoughts!

These days, the high fidelity GAN papers are often focused on reproducing class-conditional datasets such as ImageNet. But for use cases where it is important to not only generate “dog” but generate something like “dog on grass” or “dog on concrete”, has any work been done to independently encode notions of object and background with a generator?

A naive approach could be to explode the amount of potential classes by separating the data into a lot more classes. But that’s not very practical or interesting 😛

One thing I was thinking was perhaps to push this to the image translation domain (e.g. CycleGAN) – instead of doing something like trying to convert an object to another object (e.g. horse to zebra), the goal would be to convert a background to different sort of background. Any thoughts on this approach?

Another more experimental/interesting thing I was thinking was perhaps to use a multi-generator approach, one of which in theory could generate “object”, another which could do “background/context” – and you’d add the tensors together and backpropagate the loss through both generators. But as far as I can tell, nobody’s done anything like this, and I’ve no idea how to encode a prior on a specific type of generation so I’m just spitballing here. (miniature hypothesis that I have – you could use different architectures which have priors on creating different types of things, e.g. convGANs tend to do well with texture, while self attention GAN does better with structures/objects. there are certainly better ways to get networks to do what you want via different loss functions or something, but not sure how that’d all work together…).

I was wondering if anyone could help me getting started on the subject, indicating good resources to study from and interesting applications of this technique.

Thanks!

Link to the project page: https://pytorch.org/hub/facebookresearch_WSL-Images_resnext/

GitHub: https://github.com/facebookresearch/WSL-Images/blob/master/hubconf.py

Colab Notebook demo: https://colab.research.google.com/github/pytorch/pytorch.github.io/blob/master/assets/hub/facebookresearch_WSL-Images_resnext.ipynb

The released model is based on their ECCV paper from earlier:

Exploring the Limits of Weakly Supervised Pretraining

Abstract State-of-the-art visual perception models for a wide range of tasks rely on supervised pretraining. ImageNet classification is the de facto pretraining task for these models. Yet, ImageNet is now nearly ten years old and is by modern standards “small”. Even so, relatively little is known about the behavior of pretraining with datasets that are multiple orders of magnitude larger. The reasons are obvious: such datasets are difficult to collect and annotate. In this paper, we present a unique study of transfer learning with large convolutional networks trained to predict hashtags on billions of social media images. Our experiments demonstrate that training for large-scale hashtag prediction leads to excellent results. We show improvements on several image classification and object detection tasks, and report the highest ImageNet-1k single-crop, top-1 accuracy to date: 85.4% (97.6% top-5). We also perform extensive experiments that provide novel empirical data on the relationship between large-scale pretraining and transfer learning performance.

Will our image models will become biased like self-centered millennial influencers!?

Back in February 2018 there a blog article called Deep Reinforcement Learning Doesn’t Work Yet that basically argued that there are almost no cases of Deep RL being used in production systems (as opposed to game or research domains like AlphaGo, OpenAI Five, etc).

​

Over a year later, has anything changed? Are there any new Deep RL success stories that are in production and making a big impact on business? I would be particularly interested in applications outside of the FAANG companies (although in-FAANG would be interesting too).

MelNet seems to be one of the best audio generators I’ve seen yet.

It can even generate unconditional music. I’m super interested in playing around with it but I lack the intellect to build it from scratch based off of the paper.

https://audio-samples.github.io/

Hi all,
I just saw these two video demonstrations about audio noise cancelation/reduction. It looks much like those noise cancelation features from Bose and Sony headphones, but all through the use of machine learning.

1. Stationary vs non-stationary noises with Krisp App
2. 2Hz demo

I want to do something like this on my own. Can someone give me more information on the (most likely) used neural network and overall setup to achieve audio noise cancelation/reduction like in the videos?
Thanks in advance!

Towards Understanding Linear Word Analogies

Kawin Ethayarajh, David Duvenaud, Graeme Hirst

Abstract: A surprising property of word vectors is that word analogies can often be solved with vector arithmetic. However, it is unclear why arithmetic operators correspond to non-linear embedding models such as skip-gram with negative sampling (SGNS). We provide a formal explanation of this phenomenon without making the strong assumptions that past theories have made about the vector space and word distribution. Our theory has several implications. Past work has conjectured that linear substructures exist in vector spaces because relations can be represented as ratios; we prove that this holds for SGNS. We provide novel justification for the addition of SGNS word vectors by showing that it automatically down-weights the more frequent word, as weighting schemes do ad hoc. Lastly, we offer an information theoretic interpretation of Euclidean distance in vector spaces, justifying its use in capturing word dissimilarity.

Blog post: https://kawine.github.io/blog/nlp/2019/06/21/word-analogies.html

Paper: https://arxiv.org/abs/1810.04882