I’ve seen a lot of theoretical studies of SGD that consider it in the limit as the step size goes to zero, which turns it into a stochastic differential equation of the form dx/dt = alpha*(-grad(loss)(x) + noise). It’s a lot easier to compute a lot of useful quantities using this form like e.g. stationary distributions.

However, one of the things that gets lost in this formalism is are intrinsic scales of the problem. In the continuous limit, rescaling the time variable t (or more generally, performing an arbitrary coordinate transformation) leaves the trajectory invariant, because the differential equation is formulated in a covariant fashion. This gives misleading results if you want to analyze something like the convergence rates. In this continuous formulation, you can just rescale your time parameter to ‘speed up’ your dynamics (which is equivalent to increasing alpha), whereas you obviously can’t do this in the discrete formulation, because if you rescale alpha arbitrarily you overshoot and you get bad convergence.

The first thing that came to mind when I started thinking about this was that you could amend your differential equation to include higher-order correction terms. Specifically, if we have a differential equation of the form x'(t) = f(x), we can Taylor expand to get x(t + delta) ≈ x(t) + f(x)*delta + 0.5*Df(x)*f(x)*delta^2 + O(delta^3). This tells us that the difference between the continuous trajectory solution x(t + delta) and the discrete trajectory x(t) + f(x)*delta after a time delta will be roughly 0.5*Df(x)*f(x)*delta^2. In order to get a more accurate model for the discrete-time process x(t+delta) = x(t) + f(x)*delta, we can introduce a correction term into our differential equation: x'(t) = f(x) – 0.5*Df(x)*f(x)*delta. When f is -alpha*grad(loss), this becomes x'(t) = -alpha*grad(loss)(x) + 0.5*alpha*Hessian(loss)(x)*grad(loss)(x)*delta. This correction term breaks covariance: when t is rescaled, both alpha and delta are necessarily rescaled, so the correction term transforms differently than the gradient term. It seems to me like this is a natural way to model the breakdown of covariance in the discrete dynamical system in the continuous setting and to study why certain timescales/learning rates are preferred.

tl;dr: Does know if this version of the continuous-time limit has been studied before? If so, can someone point me towards some references that I can read up on?

Intuitively, larger networks have higher capacity than smaller ones. However, the theoretical capacity of a huge network would never be reached in practice due to inefficient optimization procedure, limited dataset etc. So if we scale a network by 10 times, its actual capacity might only increase by eg. 5 times, and if we scale it by 100 times, the actual capacity could increase by only 20 times.

Is such a claim correct? Are there any papers that study the gap between the actual and theoretical network gap or relevant topic?

Hello there, I’m currently trying to train a GAN to produce Simpsons characters (0 for Homer, 1 for Lisa, 2 for Marge, etc…) but the output I am getting is a mixture of all the classes. For example, the output for the 1 label may be Lisa with Homer’s mouth, or the output for the 0 label may be Homer with Marge’s hair. Is this mode collapse or a different problem? Thanks!

This has been an idea I’ve been bouncing around in my head for the past year or so but I’ve been struggling to come up with a way to apply ML to it.

The idea is this. Imagine you had a group of agents who randomly match up and repeatedly play 2-player zero-sum games against one other. This game has a known structure, and each agent has a strategy for playing the game that has varied success depending on the opponent they face.

The goal is to represent each agent as a point in feature space such that these features maximize the ability to infer “skill level” for a player. Then given 2 agents’ representations, the outcome of a matchup between those agents can be predicted.

However, the idea is that the features have to be inferred from interactions between other players, so after each matchup’s outcome the features of the two agents are adjusted based on new information, and agents’ ratings converge to their “true” rating as more games are played.

The idea is inspired by the Elo rating system used widely in chess and competitive gaming, because it’s a special case of what I’m trying to do. Each player has an Elo rating that is a normally distributed measure of skill. After each seen game outcome, the winner gains points and the loser loses points, but the amount transferred depends on the disparity in score between players. The difference in Elo rating produces a predicted chance to win/lose for either player when a game is played between them.

In a more generalized case where you have an N-dimensional rating with more possible game outcomes, could you learn the optimal transition function that would need to be applied after each game that can maximize game outcome prediction accuracy?

Link: https://web.archive.org/web/20120312111546/http://yehar.com:80/blog/?p=167

​

2010-02-24

A method for training artificial neural networks to generate missing data within a variable context. As the idea is hard to put in a single sentence, I will use an example:

An image may have missing pixels (let’s say, under a smudge). How can one restore the missing pixels, knowing only the surrounding pixels? One approach would be a “generator” neural network that, given the surrounding pixels as input, generates the missing pixels.

But how to train such a network? One can’t expect the network to exactly produce the missing pixels. Imagine, for example, that the missing data is a patch of grass. One could teach the network with a bunch of images of lawns, with portions removed. The teacher knows the data that is missing, and could score the network according to the root mean square difference (RMSD) between the generated patch of grass and the original data. The problem is that if the generator encounters an image that is not part of the training set, it would be impossible for the neural network to put all the leaves, especially in the middle of the patch, in exactly the right places. The lowest RMSD error would probably be achieved by the network filling the middle area of the patch with a solid color that is the average of the color of pixels in typical images of grass. If the network tried to generate grass that looks convincing to a human and as such fulfills its purpose, there would be an unfortunate penalty by the RMSD metric.

My idea is this (see figure below): Train simultaneously with the generator a classifier network that is given, in random or alternating sequence, generated and original data. The classifier then has to guess, in the context of the surrounding image context, whether the input is original (1) or generated (0). The generator network is simultaneously trying to get a high score (1) from the classifier. The outcome, hopefully, is that both networks start out really simple, and progress towards generating and recognizing more and more advanced features, approaching and possibly defeating human’s ability to discern between the generated data and the original. If multiple training samples are considered for each score, then RMSD is the correct error metric to use, as this will encourage the classifier network to output probabilities.

Newest Gradient article on AI Planning by researchers at IBM!

https://www.reddit.com/r/MachineLearning/

How can I link Machine Learning to an android device? I want to create a reinforcement learning agent that uses an android application as its environment. Any procedure on how it can be done?

Hi,

Where I could find some good and correct material about the use of machine learning in weather forecast?

I would like some serious material to prove if this method can be, is or was used at some point, as an integral or support method.

I did a search on Google Scholar, but I have only found articles presenting scholar or personal projects, and I would not like to conclude that, from what I found, this method has not been officially used by any national meteorological agency.

Hi, I have a unusual background and I am looking at getting into a PhD in machine learning. Do you think it is possible to get into a ML PhD without an engineering background? I studied business in my bachelor in Europe, and since it had a relatively decent amount of math I got into a master specialising in financial engineering. The master was particularly quantitative. I am now working as a trader, doing a mix of quant/non-quant trading. And some of it is ML-based. For the last 2 years I really got into ML, taking classes on my free time (Learning from Data, reading Deep Learning book and other ML books, and now the David Silver RL class). And I found it so interesting that I want to spend more time on it and do research. Given how competitive PhD applications are, do I stand any chances, since my background is not pure engineering? If yes, any advices on how to start?

Note: While I am interested in many applications, I would be interested to research RL in trading too and I guess my profile would fit nicely there. But unfortunately it is not a developed area of research so it is difficult to point myself in that direction during applications.

The ‘mental’ picture I have in my head is that I have 2 models, a RNN and a CNN. These models take data (sensory and Image) from a self driving car simulator (CARLA). The output of both the models will be a set of instructions to keep the car in the center lane (and later on braking, speedingup etc)

My question is, what is the best approach to combine the outputs. so the RNN with output (lets assume) steer 40^o, the CNN with output steer 35^o. what approach should I take for finding out which is the best course to take.

Or should I be looking at other models (R-CNNs) or possible another approach for model building.

Cheers