Recently I’ve been thinking a lot about embeddings due to my silly question over in MLQuestions where I found on my data set that an embedding vastly improved my results even though the input was a discretized continuous variable.

So I have been looking for some intuition of what embeddings actually do apart from providing a vector space for one-hot encodings. In fact, what they do allow if imposed on a discretized continuous space, is to break up the ordering relationship, allowing each new interval to be completely independent in subsequent mapping relations. (Just as there is no ordering relation between words in a vocabulary.) In some cases where a continuous mapping is particular non-linear and convoluted, maybe this “breaking up” via discretization allows a more efficient expression of the mapping — each interval can have its own “starting point” in a space that better maps to the target space.

Well, thinking that maybe this is interesting, I starting wondering how a discretization + embedding layer could be inserted into a neural network, seeing as usually embedding is only the first layer because the table lookup (or one-hot encoding) is not differentiable.

I started thinking that a continuous approximation of a discretization would be to replace the embedding look-up with a logistic function that modulates the index of the embedding table. Similarly a one-hot would be such a look-up via a softmax function.

Then suddenly I realized, that is exactly a description of attention models — a differentiable table lookup. Is attention just a continuous version of discrete vector space embeddings? Is that why attention is so powerful, because it allows a conditioned “remapping” of a spatial transform? Similar to finding an optimal point in a vector space for each word in a vocabulary, an attention model finds an optimal transform for a given context (ie for a given distribution over a continuous space) to ease the work of the rest of the network.

Please tell me if I am out to lunch 🙂 I thought I might be onto some interesting ideas but I would be delighted to know if I simply stumbled onto a better understanding of something that is already known to work well!

Is it common practice to transform variables before training a mixture model? For example if a variable looks to be log-normal distributed, is there any harm in log-tranforming it before feeding it into a Gaussian Mixture Model? What about other transformations?

And for those of you who work with mixture models, how common is it to do things like this?

Sorry if this is a basic question, no one around me works with this stuff and I wanted to hear from people with experience.

Noisy Networks for Exploration

Parameter Space Noise for Exploration

​

While I was learning about various RL algorithms, I’ve successfully implemented noisy networks for my DQN project; I added another set of parameters used as the standard deviations for Gaussian noises added to the main parameters during a forward pass, and optimized through gradient descent along with the main parameters. I obtained satisfying results using my implementation and I finished the project with a decent agent.

​

Then, recently, I changed my DQN algorithm and turned it into a DDPG/D4PG algorithm. I used the same noisy network algorithm for exploration and it still gave me fine agents from time to time. However, it often did not perform significantly better than the ones that used action space noise with the Ornstein-Uhlenbeck process, sometimes performing worse, even.

​

Trying to find what I might have mistaken or misunderstood, I searched for the original paper again to read it once more thoroughly. Then, I found an article posted by OpenAI about parameter space noise. At first glance, I thought this was the same thing as the noisy networks—in fact, I still thought those were the same after reading through that article and skimming through the paper on parameter space noise.

​

Today, reading the paper on parameter space noise more carefully, I finally realized that these are two similar but different approaches to adding noise to the parameter space. I noticed how the method described in “Parameter Space Noise” samples random Gaussian noise values at the beginning of each episode and scales it according to the variation between the actions with and without noise; while noisy nets add additional noise parameters that are optimized through gradient descent rather than using a scalar standard deviation and a scaling factor.

​

So, I have tried noisy nets so far, but I haven’t used this alternative parameter space noise method.

​

Are there other differences between these two methods? Do they perform differently? Is one better than the other? What are the main applications for each approach and what would be the best option for DDPG/D4PG?

https://towardsdatascience.com/in-the-future-you-may-be-fired-by-an-algorithm-35aefd00481f

Photo by Adam Fossier on Unsplash

This work kinda began with the question: With all the freely available font data lying around, *is augmentation all you need* to learn to classify digits in scripts that are non-latin (Ex: Indic – scripts)

I also believe that the world of fonts (like https://fonts.google.com ) is a treasure trove waiting to be unwrapped.

Paper: https://github.com/unifyid-labs/DeepGenStruct-Notebooks/blob/master/ICLR_paper.pdf
Code: https://github.com/unifyid-labs/DeepGenStruct-Notebooks