I created a discord server to organise open-ended, collaborative reading groups for theoretical machine learning. We are open to all topics and we discuss papers, books etc. I feel like I learn much better when working with other people. If you feel the same way come join us at https://discord.gg/DzWNtYJ

I have a very large dataset of shipment data where the company names are not normalized (e.g. companies that are supposed to be the same are treated different, like Walmart Inc., Walmart Incorporated, Wallmart, WalmartInc.). A simple string normalization like regex would not do good on this.

I have thought of TEXT SIMILARITY approach (Levenshtein Distance, Waro-Jinkler, etc.) which theoretically would work but would not do good in practice. One is that you should set a threshold and thresholds are different for each of them and various problems would arise.

Example:

1. Large and Short Company Names would skew the threshold: (Walmart Inc – Walmart Inc. vs ABC Co. – In behalf of ABC Group of Co.)
2. Almost similar company names that are supposed to be different (ABC Company Thailand vs ABC Company Taiwan)

The problem for #1 is that Thresholding for text similarity ratio is tricky.

The problem for #2 is that these companies have high ratio but are supposed to be different companies shipping different products (for example, ABC Thailand ships dresses while ABC Taiwan ships gadgets).

I have shipping data that looks like this

​

​

I am thinking of a solution that uses TEXT similarity metrics but across fields that could indicate that they are the same company (such as country, zip code, even products).

​

My proposed solution is

– a new entry is compared to a constructed table consisting of columns that are distinguishing features (company name, zip code, country for example)

– the new entry is only compared using the company name. the highest similarity is returned. And text similarity across different columns on new entry and selected data is produced.

– text similarity ratio/points of these two is fed to a classifier that tells if they are similar companies or not. Basically, the input for the classifier is the text similarity ratio of the new entry and the nearest company name from the list.

​

Any easier approach? The approach should be able to tackle both an existing large data and new entry (for example, deduplication does not seem to tackle addition of new entries). Thanks!

Does any one know if there exists popular benchmark in regression problem with image input such as ImageNet and MNIST for classification?

Hi all. I am very new to the ML world but I have been thinking about mapping a project out using a bunch of my own blood glucose data I have and did not know where to start with a sliding batch of input data? In order for a model to predict the next few values, it would need to know what happen in the last X number of minutes/ X number of data points. Does anyone have any links or textbooks to checkout on models like this? Anything is appreciated!

Brancher (brancher.org) is a new framework for deep probabilistic inferece based on PyTorch.

In this new tutorial, we show how to use Brancher to build stochastic deep learning models

​

https://colab.research.google.com/drive/1YNwZpJgrsicK3Pz8igAktbdocm-gA9mV

​

We hope you’ll enjoy it! Let us know if you have questions or suggestions for future improvements either here or on Twitter: @pybrancher

Hi! I have been wondering about how to build a model that learns to infer the physics of environments from its input.

More specifically: how to build a model that takes input videos of, let’s say, objects falling down ( a glass being pushed from a table, an apple falling from a tree, etc.) and then makes the inference that things tend to fall to the ground? The model does not have to figure out any mathematical formula for calculating the gravitational force. Learning the common sense physics as humans do is what I am primarily interested in.

Has anyone come across research that addresses this problem or a similar problem? I would greatly appreciate any help. Thanks in advance! 🙂

Hey, I’m interested in applying for ML roles at FAANG companies for non-research positions. I know FB has an applied ML team but I don’t know the equivalents in other companies. Can some one share he names of these teams and their experiences with them if possible?

Thanks!

Hi everyone. I just published a new blog post which talks about the evolution of GANs over the last few years. You can check it out here.

​

I think it’s fascinating to see sample images generated from these models side by side. It really does give a sense of how fast this field has progressed. In just five years, we’ve gone from blurry, grayscale pixel arrays that vaguely resemble human faces to thispersondoesnotexist, which can easily fool most people on first glance.

​

Apart from image samples, I’ve also included links to papers, code, and other learning resources for each model. So this article could be an excellent place to start if you’re a beginner looking to catch up with the latest GAN research.

​

I hope you enjoy it!

Hey, I’m an industry outsider/hobbyist and trying to use AutoML text as a ternary classifier to prioritize incoming service request. The thing I realized is that multi-label classification appears to be for unrelated labels, but I’m dealing with labels that sit on a line, “low priority”, “medium priority”, “high priority”.

​

I don’t think this is the same problem as classifying with categorical labels such as “car”, “boat”, “plane”. The distance between low and high priority is much greater than medium, but the distance between a car, boat and plane are likely arbitrary. Is there any way to capture this? The only thing I’ve thought of so far is using a binary decision tree to check if low priority or not. If not, then check high priority. If not, then it’s assumed to be medium priority. Or does even that not work?

​

Sorry, I’m still trying to learn terminology and more on this subject. I’m not even sure what the name of the property I’m referring to is called.