https://www.youtube.com/watch?v=ui4F_A46wN0

Speaker: Henrique Ponde de Oliveira Pinto

Talk about how OpenAI Five training was orchestrated, using Redis.

Cool stuff!

I want to make a CNN to classify first by race (black, white, Hispanic etc.) and then by gender (m/f) to give outputs like Black-Male, Hispanic-Female etc. How can I set up the structure? Is this even possible with 1 CNN?

Weight Agnostic Neural Networks

Abstract

Not all neural network architectures are created equal, some perform much better than others for certain tasks. But how important are the weight parameters of a neural network compared to its architecture? In this work, we question to what extent neural network architectures alone, without learning any weight parameters, can encode solutions for a given task. We propose a search method for neural network architectures that can already perform a task without any explicit weight training. To evaluate these networks, we populate the connections with a single shared weight parameter sampled from a uniform random distribution, and measure the expected performance. We demonstrate that our method can find minimal neural network architectures that can perform several reinforcement learning tasks without weight training. On a supervised learning domain, we find network architectures that achieve much higher than chance accuracy on MNIST using random weights.

interactive article: https://weightagnostic.github.io

arxiv version: https://arxiv.org/abs/1906.04358

I am a survey statistician by profession, but I am incorporating more and more machine learning techniques into my work and I am curious how survey sampling may affect the results of various machine learning models (e.g., DT, Random Forest, SVM, GBT, K-NN, etc.)

What I mean by survey sampling is that we often employ sampling techniques on a target population that on its own will return data that looks nothing like the population we are trying to study.

In the more simple situations, it could be an over-sample of a rare minority population that we would never get naturally through simple random samples. Or potentially more complex sampling, either sampling proportional to a variables of interest (e.g., revenue of a company) or sampling area clusters to reduce costs of in-person data collection (e.g., cheaper to survey 100 people in 3 states, rather than surveying 100 people in 50 states…) or a combination of any number of sampling techniques.

In survey statistics we usually use special procedures in combination with case weights and sample design variables to “fix” the known imbalances due to sampling so that the results look like the population of interest and we also take special care to properly inflate the variances (e.g., confidence intervals) of all of our estimates due to non-random sampling (e.g., a non random sample of 99 men & 1 woman, is not the same as a random sample that returns 50 men & 50 women, even though the total is n = 100 in both cases).

But I am not sure how that would influence various machine learning algorithms since their goals are slightly different and have less focus on p-values and confidence intervals.

I’m a grad student interested in the process described by Andrej at tesla’s autonomy day. Below I’ve put together some of my brief notes with links. So far the published research I could find that seemed to be related to their work was the NeurIPS 2017 paper Decoupling “when to update” from “how to update” and the Active Learning Survey

​

Does anyone else have other related papers to suggest? For example, I’m guessing measuring distance via the L2 is a bad idea.

​

Generic Object Detection improvement

If you know a specific problem you have: take that specific problem and use it to find similar examples to pull into a training set

I’m guessing they embed every image with a generic imagenet model and then find similar images based on L2 distance between embedded vectors

training pipeline

start training with a uniformly sampled dataset and select new images for training if:

1. detect uncertainties in the network predictions
   1. I’m guessing 2 networks disagreeing with each other(similar to: decoupling what to update from how to update)
2. driver intervention

Too fix either of (1) or (2) use the process described in generic object detection

Is there any way to automatically label objects when training an object detection model library from scratch? I’ve begun working through tutorials, eg TensorFlow’s post here, but common methods require labelled training data.

Perhaps there are established techniques to partially label training data at scale, or common methods to adapt existing libraries? Please forgive my ignorance, I just don’t know where to start googling.

I have been reading up on meta-learning the past few days and I read a study (sorry I lost the link) but in the study they were able to decrease computational resources needed to build machine learning models by 90-99% in some cases. I believe they used a simply genetic algorithm to design different ML models.

It seems that this could be the holy grail. If you can figure out how to effectively implement meta-learning that you could exponentially increase the desired results of a model.

I’m currently expirementing with a genetic algorithm for model design in keras on the fashion mnist dataset. But I have yet to come up with a more effective model than the one given on the tensorflow website for this task. Although I’m not finished my experiment yet… I can find sparse information on the topic other than in academic literature. But it seems like it should almost be the primary focus if you are designing model architectures from scratch no? Why aren’t more people doing this?

Here is a video on the topic if you are unaware https://www.youtube.com/watch?v=2z0ofe2lpz4