I know this is an old debate, but I was talking to one of my colleagues from work and something he said struck me as really odd. He said, Statistics is more concerned with inference than results. After that, I did a little bit of internet research and found the same narrative there too. How Statistics does not have train-test split, and not concerned with the performance on unseen data, etc.

But this led me to the line of thinking that, Statistics (if it’s not concerned with unseen data performance) is doing something wrong.

If you fit to your train set perfectly with an interpretable model, but the performance on unseen data is dismal, then should we really take the interpretations from such a model as the truth?

Looking towards all the statisticians out there, to tell me I’m wrong and why.

For those who have attended NeurIPS in the past, is there value in a data scientist attending the conference when your background is not in research/ academics?

I’ve been running into an issue lately trying to train a simple MLP.

I’m basically trying to get a network to map the XYZ position and RPY orientation of the end-effector of a robot arm (6-dimensional input) to the angle of every joint of the robot arm to reach that position (6-dimensional output), so this is a regression problem.

I’ve generated a dataset using the angles to compute the current position, and generated datasets with 5k, 500k and 500M sets of values.

My issue is the MLP I’m using doesn’t learn anything at all. Using Tensorboard (I’m using Keras), I’ve realized that the output of my very first layer is always zero (see Image1 ), no matter what I try.

Basically, my input is a shape (6,) vector and the output is also a shape (6,) vector.

Here is what I’ve tried so far, without success:

• I’ve tried MLPs with 2 layers of size 12, 24; 2 layers of size 48, 48; 4 layers of size 12, 24, 24, 48.
• Adam, SGD, RMSprop optimizers
• Learning rates ranging from 0.15 to 0.001, with and without decay
• Both Mean Squared Error (MSE) and Mean Absolute Error (MAE) as the loss function
• Normalizing the input data, and not normalizing it (the first 3 values are between -3 and +3, the last 3 are between -pi and pi)
• Batch sizes of 1, 10, 32
• Tested the MLP of all 3 datasets of 5k values, 500k values and 5M values.
• Tested with number of epoches ranging from 10 to 1000
• Tested multiple initializers for the bias and kernel.
• Tested both the Sequential model and the Keras functional API (to make sure the issue wasn’t how I called the model)
• All 3 of sigmoid, relu and tanh activation functions for the hidden layers (the last layer is a linear activation because its a regression)

Additionally, I’ve tried the very same MLP architecture on the basic Boston housing price regression dataset by Keras, and the net was definitely learning something, which leads me to believe that there may be some kind of issue with my data. However, I’m at a complete loss as to what it may be as the system in its current state does not learn anything at all, the loss function just stalls starting on the 1st epoch.

Any help or lead would be appreciated, and I will gladly provide code or data if needed!

Thank you

http://mitchgordon.me/machine/learning/2019/11/18/all-the-ways-to-compress-BERT.html

I’ve seen a lot of papers compressing BERT, so I compiled them all in one place. They all use pretty similar methods, so there’s also a brief taxonomy.

u/Luxonis-Brandon put together a video demonstrating the real-time speed of the DepthAI.

The device is something we’ve been working on that combines disparity depth and AI via Intel’s Myriad X VPU. We’ve developed a SoM that’s not much bigger than a US quarter which takes direct image inputs from 3 cameras (2x OV9282, 1x IMX378), processes it, and spits the result back to the host via USB3.1.

Our ultimate goal is to develop a rear-facing AI vision system that will alert cyclists of potential danger from distracted drivers, so we needed disparity + AI to get object localization outputs – an understanding of where and what objects are. This needs to happen fast and with as little latency as possible… and at the edge… and at low power!

There are some Myriad X solutions on the market already, but most use PCIe, so the data pipeline isn’t as direct as Sensor–>Myriad–>Host, and the existing solutions also don’t offer a three camera solution for RGBd. So, we built it!

If anyone has any questions or comments, we’d love to hear it!

Shameless plugs for our hackaday and crowdsupply 🙂

In the past 2 years I have been working as a Machine Learning developer, mostly with tabular data, and I’ve developed a tool to perform entity embeddings on categorical variables using CNN with Keras. I tried pretty much to make it easy to use and flexible to most of the existent scenarios (regression, binary and multi-class classification), but if you find any other need or issue to be fixed, do not hesitate to ask.

I tried to add some cool stuff on the project, such as unit tests, code coverage with Codacy, continuous integration with Travis CI and auto deployment to PyPi and auto-generated documentation with Sphinx and ReadTheDocs, so if any of you is interested in how to setup your project to have these features, feel free to use it as a base project.

Looking forward to any reviews about the source code. Any tip to improve the readability or even performance, its really welcome and well appreciated.

Github: https://github.com/bresan/entity_embeddings_categorical

PyPi: https://pypi.org/project/entity-embeddings-categorical/

Code coverage (nowadays reaching 97%): https://coveralls.io/github/bresan/entity_embeddings_categorical?branch=master

Thanks and I hope it can help somebody out there 🙂

Cortex just released V 0.10, which includes their new Predictor Interface for serving models. It lets you take models from any framework and implement them in simple Python, before deploying them with a single terminal command. V 0.10 also still includes out-of-the-box support for TensorFlow Serving and ONNX Runtime.

Repo link: https://github.com/cortexlabs/cortex

Examples:

– Deploying Hugging Face’s DistilGPT-2 – PyTorch

– Deploying a sentiment analyzer with BERT – TensorFlow

– The classic iris classifier – XGBoost/ONNX

In the recent update of GraphVite, we release a new large-scale knowledge graph dataset, along with new benchmarks of knowledge graph embedding methods. The dataset, Wikidata5m, contains 5 million entities and 21 million facts constructed from Wikidata and Wikipedia. Most of the entities come from the general domain or the scientific domain, such as celebrities, events, concepts and things.

To facilitate the usage of knowledge graph representations in semantic tasks, we provide a bunch of pre-trained embeddings from popular models, including TransE, DistMult, ComplEx, SimplE and RotatE. You can directly access these embeddings by natural language index, such as “machine learning”, “united states” or even abbreviations like “m.i.t.”. Check out these models here.

Here are the benchmarks of these models on Wikidata5m.

People familiar with NLP probably know about the ELIZA chatbot, which Joseph Weizenbaum created in 1966. ELIZA used a psychoanalyst’s tricks to keep a fairly natural conversation going, focusing in on keywords. (For example, if a human typed “I’m so angry with my sister,” the chatbot might reply, “Tell me why you’re so angry with your sister.”)

What I didn’t know was that Weizenbaum was thoroughly creeped out by people’s response to his creation. During their interactions with ELIZA, people developed emotional attachments to the program, and often confided in it.

From the article: “Even more surprising was that this sense of intimacy persisted even after Weizenbaum described how the machine worked and explained that it didn’t really understand anything that was being said. Weizenbaum was most troubled when his secretary, who had watched him build the program from scratch over many months, insisted that he leave the room so she could talk to Eliza in private.”

https://spectrum.ieee.org/tech-talk/robotics/artificial-intelligence/why-people-demanded-privacy-to-confide-in-the-worlds-first-chatbot

This is the third installment in a 6-part series on the history of NLP that I’m editing for IEEE Spectrum. I’m curious to know if people find it interesting.

Not mine, but I got permission from the creator to post it here on Reddit.

Blog post: https://littlemountainman.github.io/2019/11/27/selfdrivingfun/

GitHub: https://github.com/littlemountainman/selfdrive

Twitter: https://twitter.com/littlemtman/status/1196093540092272643