Link to Project

After finding myself repeatedly writing and rewriting the same code every time I started a new project, I decided to make a simple but efficient template that works for many RL algorithms. The template uses MPI to simulate environments on several cores in parallel to make the training process very efficient.

If you work with RL or are interested in getting into it, I would highly recommend doing this. Done correctly, you should notice a massive boost in training efficiency, especially when using complex environments.

The link on the sidebar is getting old. Was wondering if there were new more advanced ML courses (PhD level) which you’d recommend.

Excerpt from a post by Aki Matsukawa who worked at deepmind, google, and twitter before as an RSWE:

What does a research engineer do?

Responsibilities of a research engineer can differ quite a lot by company, and even by team within the same company. But generally, doing effective research means taking advantage of the work that others have done, be it algorithm implementations, model checkpoints, or evaluation tools. In order for this to happen at scale, a level of software engineering discipline is necessary. I define the primary goal of a research engineer as enabling, contributing to, and accelerating ML research by bringing engineering expertise to the projects. Some examples of a research engineer’s responsibilities are:

• Implementing algorithms and related baselines under a common API to allow for rapid experimentation.

• Setting up distributed training.

• Creating evaluation tools in Jupyter notebooks.

Often, a research engineer also makes contributions to the research itself, especially using insights and intuitions derived from implementing and iterating on experiments.

https://amatsukawa.github.io/re/

Hello, I compiled a list of papers from this year CVPR on the topic of NAS with short summaries here: https://drsleep.github.io/NAS-at-CVPR-2019/

We propose a method for training convolutional neural networks to learn similar systems of transformations as exist in the human brain. We recorded brain data from exposing a participant to plenty of visual data (a TV series) and use the same visual data as input to a network. We train the network by attaching brain activity observation models for different visual system regions to the layer activity tensors (through low-rank tensor decomposition).

​

The sole training signal is the error between the measured brain activity and the activity predicted by these observation models. We could verify that the model learns several known properties of the visual system.

https://www.biorxiv.org/content/10.1101/553255v2

It’s a bit older, but if you are interested in deep learning for pose estimation, you might find this interesting. The post includes a small introduction to deep learning and how to deal with such a problem by labeling with a Gaussian distribution, based on the work from 2016.

Project video here

Code and PCB/CAD design files here

I was looking for some kind of project that let me combine my love for building small electronics/3d-printing projects and machine learning, and this is what I came up with.

The machine learning side of this isn’t too interesting, just a small 3 layer vanilla RNN (all layers with 10-d state, trained in tensorflow/keras) that takes as input the moves of the two players and outputs a prediction for the opponents next move. The data comes from human games played on roshambo.me via this blog article. I added some simulated data of periodic sequences, because that seems like the sort of thing people might try out when playing against an “AI”.

Without the simulated data (which is easy to predict) the model gets something like 38% accuracy on the test set (compared to 33% playing randomly). One nice thing about rock-paper-scissors is that you can’t do much better than random, so there isn’t much pressure 😉

I had some problems getting training with larger batches to work (either with just padding or padding and masking the gradients) so I just trained it with batch_size=1, which wasn’t too bad for such a small network.

The probably more interesting part of this project is running that network on a small microcontroller. I quantized the network weights (to 8-bit integers) to save some space (it only has 2kiB RAM and 16kiB flash, although the network I ended up using would actually fit without that). The calculations are done using software floats, as performance isn’t really an issue. The C-code running on the microcontroller is all custom, not that it’s particularly complicated.

The finished device is about the size of a coin (but thicker). It uses a custom pcb (ordered from dirtyPCBs), is powered by a coin cell battery, and has a nice 3d-printed case. The form-factor is pretty close to my electronic dice so that wasn’t too much work.

Hey data scientists of Reddit, would like to seek some advice from the experts here.

I am trying to implement an assortment optimization problem, where you have to pick a subset of good from the master set which maximises the objective (i.e. revenue/profit). Eventually, I would like to work towards a tractable SKU-level sale forecast function (choice model) that accounts for cannibalization, substitution and seasonality. As I understand it, this function would have to be fed into an optimisation algorithm (i.e. linear integer programming).

From my readings, I gather that Multinomial Logit Model (MNL) should be the baseline/ starting point. However, I can’t seem to grasp how I could extend MMNL to account for cannibalization, substitution and seasonality.

Can any kind souls point me to a good place to start on optimization in the field of operations research for assortment optimization?

​

tldr: Where can I find resources on assortment optimization with cannibalization, substitution and seasonality effects?