Here is the link the paper I am referring to: https://arxiv.org/pdf/1611.02648.pdf

​

I am having trouble understanding how to implement this paper correctly. So I understand that instead of using an isotropic Gaussian as the prior for the latent space, they are using a mixture of Gaussians. And then I am really having trouble understanding how they are calculating their lower bound, or specifically the terms in it, which are reconstruction term, conditional prior term, w-prior term and z-prior term. The z-prior term is a direct probability of the class a data point would belong to, and I am not sure where they are getting this from. So if anybody could offer any help or point me to somewhere I could find some help, it would be greatly appreciated!

​

And in summary, here are my questions:

1. How are they generating a mixture of Gaussians for the latent space? Does this mean creating n distributions for the latent space (where n is the number of clusters), so basically having n sets of mean and variance layers (each with the size of the latent space), rather than just one set of these layers like in a normal variational autoencoder? Or is the latent space still representing all the data, and then Gaussian distributions are sampled from the latent space?
2. How are they reparameterizing the distributions of multiple distributions (assuming my understanding of how they are doing the multiple Gaussians correct, which it’s very likely not)?
3. How are they directly outputting the probability that a sample belongs to a certain distribution, which represents a cluster?

​

Any help is much appreciated, thank you!

​

Update: While thinking about it, I got this idea – they are generating n (number of clusters) mean and variance layers, and then averaging them out in order to do the reparametrization for the latent space? Or maybe averaging the reparametrization terms? I don’t know if that’s right, and it seems crazy expensive computationally if the value of n is large? And wouldn’t the mean and variance layers all be the same if done this way? I don’t know, I’m just confused, and it is very late.

100+ Machine Learning Trading Strategies

https://github.com/firmai/machine-learning-asset-management

​

– Deep Learning

– Reinforcement Learning

– Evolutionary Strategies

– Stacked Models

Here is an implementation of LSTMs backpropagation from scratch , I am not sure with the dfhs (derivative of hs state) am I doing it the right way ? Please do correct me where I am wrong in my code I tried for so many times still the LSTMs prediction is horrible does it take longest time to train LSTMs ?? Any Suggestions on Back Propagation is highly appericiated as I can’t figure out what is going wrong – Thank you in advance

I tried increasing the iterations to 10,000 from 5,000 which was intially and tried decreasing the learning rate and increasing the batch size

Forward propagation to store all the state necessary for back prop

for i in range(nw): xp = np.zeros(xl) xp[intx] = 1 x = np.hstack((hs[i-1],xp)) xs[i] = x fg[i] = sigmoid(np.dot(x,wf)) ig[i] = sigmoid(np.dot(x,wi)) cg[i] = tangent(np.dot(x,wc)) csc = (cs[i-1] * sigmoid(np.dot(x,wf))) + (sigmoid(np.dot(x,wi)) * sigmoid(np.dot(x,wc))) cs[i] = (cs[i-1] * sigmoid(np.dot(x,wf))) + (sigmoid(np.dot(x,wi)) * sigmoid(np.dot(x,wc))) og[i] = sigmoid(np.dot(x,wo)) hs[i] = sigmoid(np.dot(x,wo)) * tangent(csc) hsc = sigmoid(np.dot(x,wo)) * tangent(csc) ys[i] = sigmoid(np.dot(hsc,wy)) intx = np.argmax(vy[i-1]) dwy = np.zeros((yl,d)) dwf = np.zeros((xl+yl,yl)) dwi = np.zeros((xl+yl,yl)) dwc = np.zeros((xl+yl,yl)) dwo = np.zeros((xl+yl,yl)) dfhs = np.zeros(yl) dfcs = np.zeros(yl) totalError = 0 

Back Propagation

for i in reversed(range(nw)): merror = ys[i] - vy[i] dwy += np.dot(np.atleast_2d(hs[i]).T,np.atleast_2d((merror*dsigmoid(ys[i])))) error = np.dot(merror,wy.T) totalError += np.sum(error) e = np.clip(error+dfhs,-6,6) dho = tangent(cs[i]) * e dho = dsigmoid(og[i]) * dho dwo += np.dot(np.atleast_2d(xs[i]).T,np.atleast_2d(dho)) dc = og[i] * e * dtangent(cs[i]) dc = np.clip(dc + dfcs,-6,6) dhf = cs[i-1] * dc dhf = dsigmoid(fg[i]) * dhf dwf += np.dot(np.atleast_2d(xs[i]).T,np.atleast_2d(dhf)) dhi = cg[i] * dc dhi = dsigmoid(ig[i]) * dhi dwi += np.dot(np.atleast_2d(xs[i]).T,np.atleast_2d(dhi)) dhc = ig[i] * dc dhc = dsigmoid(cg[i]) * dhi dwc += np.dot(np.atleast_2d(xs[i]).T,np.atleast_2d(dhc)) dfhs = np.dot(dho,wo.T)[:yl]+np.dot(dhc,wc.T)[:yl]+np.dot(dhi,wi.T)[:yl]+np.dot(dhf,wf.T)[:yl] dfcs = fg[i] * dc 

Hi everyone,

I need some suggestions on projects to work where I would apply StyleGAN.

Here are some references of projects that has been implemented with this architecture:

http://www.thispersondoesnotexist.com/

https://thisrentaldoesnotexist.com/

https://thisresumedoesnotexist.com/

Please note that this project is just for learning purposes. Thanks in Advance!

Working on a Topic Modeling project and I came across pyLDAvis for the first time and decided to give it a go. While my corpus is relatively large, it is taking far long than the time it took to preprocess and train my Topic Modeling models to create the visualization. I am unsure if I am doing something wrong, but are there any alternatives to this package? I saw some demos on websites and it just looks amazing.

​

For reference, here is the code I am running (model, vectorized data, and vectorizer):

pyLDAvis.enable_notebook() viz = pyLDAvis.sklearn.prepare(lda_model, vectorized_data, count_vect) viz 

Any suggestions would be wonderful!

This question is more applicable to journals (esp. in non-CS fields) where the turnaround can be much longer than conferences. Basically how do you decide whether to cite related papers made available/published after you submitted your paper or put it on arxiv? Do you impose a temporal barrier on information based on some date or do you adjust your paper (esp. literature review) as things become available during the review process? Is it OK to cite a paper that cites another paper which in turn cites your arxiv preprint?

Code: fastai-serving repo

​

We’ve been experimenting with some Fast AI models recently for our remote sensing work. Unfortunately, we ran into a lot of issues when trying to deploy those models on large-scale inference jobs (specifically running land-classification on big satellite imagery datasets). This fastai-serving repo is meant to solve this in a way that mimics the TF Serving approach/API. Namely, it helps you package a trained model within a small Docker image (running a mini server) so you can make prediction requests via REST POST requests.

​

We’re working on expanding the functionality (and are very receptive to any help!). For anyone who’s running inference on large image sets, we usually spin up multiple of these these inference-ready images and run large batch predictions with our open chip-n-scale pipeline.

I made available some slides about a presentation from PyData Lisbon called “Uncertainty Estimation in Deep Learning”, for those who are interested in an intro to the subject, here you are the slides:

https://www.slideshare.net/perone/uncertainty-estimation-in-deep-learning