When doing auto encoders on images L1 loss is the standard choice but it tend to produce blurry images. Is there any literature that have a learnable loss, maybe something like GANs or some other function that takes a target image and an output from the network but is also trainable?

I haven’t seen this in the VAE and AE papers I’ve read but I’m sure there are lots of examples. Have you found any learnable losses or similar approaches?

I’ve been looking at applying for some AI/ML Residencies and I wonder if anyone had any general opinions on them, bad or good.

Also I was curious as to how competitive these programs are if anyone knows.

Author: Lilian Weng

Summary: Gradient descent is not the only option when learning optimal model parameters. Evolution Strategies (ES) works out well in the cases where we don’t know the precise analytic form of an objective function or cannot compute the gradients directly. This post dives into several classic ES methods, as well as how ES can be used in deep reinforcement learning.

Link: https://lilianweng.github.io/lil-log/2019/09/05/evolution-strategies.html

TL/DR: Invertible generative models can be used to generate equilibrium states from high-dimensional multi-body systems such as proteins with hundreds of atoms. Training is a mixture of likelihood based training on biased trajectory data with subsequent fine-tuning using energy-based training (as done in parallel WaveNet). Such models allow rapid exploration using MC exploration in latent space and computing free energy differences between disconnected states.

Paper: https://science.sciencemag.org/content/365/6457/eaaw1147

Editorial putting the paper into context: https://science.sciencemag.org/content/365/6457/982

I have a project idea in IRL which can materialize into a paper. If it works out, I would like to submit it for ICLR, so there’s essentially 18 days of time from now. Though I have the basic idea chalked out, it would help me greatly to work with a collaborator.

I am looking for someone with decent Python programming skills (especially familiarity with OpenAI Gym and generic DRL algorithm implementations), good knowledge of Inverse Reinforcement Learning algorithms (up to, say, GAIL), and very basic understanding of GANs.

Please feel free to contact me via chat or PM if you are interested.

Let’s assume I have a polynomial regression but I’m not sure which polynomial degree d to use. In econometrics, one would probably try out the various regressions, each with different degrees and then check the significance of the individual terms, compare R^2 value and so on. Assuming I have enough data points, if I want to explore d = 1,..., 20, then I’ll need to do twenty individual regressions and compare them.

Would be wrong, if I start from the very largest model, say d=20, and then use lasso regression to select only the terms that have non-zero coefficients?

My question is absolutely related to the following old and unanswered question in stackexchange: https://stats.stackexchange.com/questions/224707/selecting-polynomial-terms-in-regression

Learning an effective similarity measure between image representations is key to the success of recent advances in visual search tasks (e.g. verification or zero-shot learning). Although the metric learning part is well addressed, this metric is usually computed over the average of the extracted deep features. This representation is then trained to be discriminative. However, these deep features tend to be scattered across the feature space. Consequently, the representations are not robust to outliers, object occlusions, background variations, etc. In this paper, we tackle this scattering problem with a distribution-aware regularization named HORDE. This regularizer enforces visually-close images to have deep features with the same distribution which are well localized in the feature space. We provide a theoretical analysis supporting this regularization effect. We also show the effectiveness of our approach by obtaining state-of-the-art results on 4 well-known datasets (Cub-200-2011, Cars-196, Stanford Online Products and Inshop Clothes Retrieval).

Paper: https://arxiv.org/abs/1908.02735

Code: https://github.com/pierre-jacob/ICCV2019-Horde