Hi, I have made an implementation of paper ‘vGraph: A Generative Model For Joint Community Detection and Node Representational Learning’ under NeurIPS Reproducibility Challenge 2019, which you can find here: https://github.com/aniket-agarwal1999/vGraph-Pytorch

Hope you all find it useful, feedback on the same would be appreciated.

Working on estimating position of minimum by modelling where linear trend of gradients interests zero, simple approximation (corr(g,theta)=1) leads to looking obvious:

learning rate ~ sqrt(var(theta)/var(g))

proportional to width of displacement of theta, and inversely proportional to width of displacement of gradients – assuming they are in line (corr(g,theta)=1), such learning rate would take us exactly to g=0 minimum of parabola in one step.

Adaptive variance estimation is just a matter of maintaining two exponential moving averages: of value and of value^2, hence we can e.g. cheaply do it coordinate-wise in SGD – getting 2nd order adaptation of learning rate independently for each coordinate (5th page here).

There is popular square root of mean gradient² in denominator (e.g. RMSprop, ADAM), but have anybody seen use of variance in SGD optimizers?

The deadline for meta-reviews is close (Dec 6th) and 1/3 reviewers hasn’t responded to our rebuttal yet. And others just updated their scores without any post-rebuttal comments 🙁

I understand that it is the job of AC to engage reviewers in active discussion, but I don’t see that happening.

We are releasing the new version of our MSG-GAN work. https://arxiv.org/abs/1903.06048 today.

Code at https://github.com/akanimax/msg-stylegan-tf.

We present much better experimental evaluation of the method and also incorporate the Multi-scale modifications in stylegan. We also experiment with our newly created (Indian Celebs) dataset (very small 3K) and get very nice results.

Please do check it out. Any feedback / suggestions are most welcome.

This paper. I’m not one of the authors or even familiar with the work but it certainly struck me by surprise reading the reviews.