Hi, Reddit. I’m excited to share confident learning for characterizing, finding, and learning with label errors in datasets. To promote and standardize future research in learning with noisy labels and weak supervision, I’ve also open-sourced the cleanlab Python package: https://pypi.org/project/cleanlab/

Post: https://l7.curtisnorthcutt.com/confident-learning

Title: Confident Learning: Uncertainty Estimation for Dataset Labels

Abstract: Learning exists in the context of data, yet notions of confidence typically focus on model predictions, not label quality. Confident learning (CL) has emerged as an approach for characterizing, identifying, and learning with noisy labels in datasets, based on the principles of pruning noisy data, counting to estimate noise, and ranking examples to train with confidence. Here, we generalize CL, building on the assumption of a classification noise process, to directly estimate the joint distribution between noisy (given) labels and uncorrupted (unknown) labels. This generalized CL, open-sourced as cleanlab, is provably consistent under reasonable conditions, and experimentally performant on ImageNet and CIFAR, outperforming recent approaches, e.g. MentorNet, by 30% or more, when label noise is non-uniform. cleanlab also quantifies ontological class overlap, and can increase model accuracy (e.g. ResNet) by providing clean data for training.

Paper: https://arxiv.org/abs/1911.00068
Code: https://github.com/cgnorthcutt/cleanlab/

Hello everyone! I’m looking into building a machine learning Excel Add-In. Here is how it works (roughly):

1. Prepare a sheet of training data, one of the columns contain the target or label, other columns are features.
2. Open the add-in. Select relevant columns for training. For each column, choose whether it is categorical or numerical.
3. The data is submitted to a cluster of servers and the servers automatically try different types of models and hyperparameters to produce the most accurate results.
4. Then the user can use the add-in to make predictions on new data in another Excel sheet.

Would this be useful for people who don’t know how to train machine learning models?

Hello!

I have a bunch of video that I have annotated using CVAT. Then I exported the annotation. How can I train (and test) a model based on this using tensorflow?

I’ve done it with images before, but now I’m working with a video file and not a sequence of images, and there is only one annotation file, not one per image.

I was wondering if there is any work that studies the relation of learned weights between two neural nets.

For example, suppose we have a simple regression task, and we trained an MLP with one hidden layer with 20 neurons. If we train another MLP with 15 neurons in the hidden layer, what would the relation of the weight matrices be between these two networks?

I found some related works on neural network compression literature that start with the bigger model and use matrix pruning with factorization and/or decomposition to reach a smaller model. But, I’m not sure if the obtained parameters will be close to the weights a neural network(with the same parameters as the smaller model) will learn if trained from scratch. I mean, the fact that we can use pruning methods and get good accuracy doesn’t necessarily mean that that is the true relation between the bigger model and the smaller one. What do you think?