I was wondering if there’s any book of similar level , insight and beauty on other topics in machine learning apart from deep learning.. like reinforcement learning etc. ?
This is my first post here. Sorry , in case of any mistakes. Thank you , all !
submitted by /u/Jeevesh88
[link] [comments]
I know that adding a TimeDistributed(Dense) applies the same dense layer over all the timesteps but I wanted to know how to apply different dense layers for each timestep. The number of timesteps is not variable.
P.S.: I have seen the following link and can’t seem to find an answer
submitted by /u/atif_hassan
[link] [comments]
Has anyone (who understands Chinese) seen the Zhihu thread about the worst/bad CVPR 2019 accepted papers? Here it is – https://www.zhihu.com/question/327139341. I find it quite controversial and interesting. There are allegations that people use all kinds of unscrupulous ways to publish or get their papers accepted at CVPR. These include having big-name researchers to “say hello” to the supposedly anonymous reviewers (i.e., coaxing them into giving good paper ratings), blatantly plagiarizing ideas/contributions from previous papers, deliberately lowering baseline results, and so on.
submitted by /u/TreeNetworks
[link] [comments]
https://arxiv.org/abs/1907.02052 In this work, we focus on fine-tuning an OpenAI GPT-2 pre-trained model for generating patent claims. GPT-2 has demonstrated impressive efficacy of pre-trained language models on various tasks, particularly coherent text generation. Patent claim language itself has rarely been explored in the past and poses a unique challenge. We are motivated to generate coherent patent claims automatically so that augmented inventing might be viable someday. In our implementation, we identified a unique language structure in patent claims and leveraged its implicit human annotations. We investigated the fine-tuning process by probing the first 100 steps and observing the generated text at each step. Based on both conditional and unconditional random sampling, we analyze the overall quality of generated patent claims. Our contributions include: (1) being the first to generate patent claims by machines and being the first to apply GPT-2 to patent claim generation, (2) providing various experiment results for qualitative analysis and future research, (3) proposing a new sampling approach for text generation, and (4) building an e-mail bot for future researchers to explore the fine-tuned GPT-2 model further.
submitted by /u/js_lee
[link] [comments]
I usually prefer not to post articles in the popular press, but since this topic about the recent Deep Learning Earthquake publication on Nature has gathered considerable upvotes and discussion only on this subreddit (not anywhere else AFAIK), I’m glad some of the press has also picked up on it to make the broader community aware of our concerns.
The Register has published an article “Was this quake AI a little too artificial? Nature-published research accused of boosting accuracy by mixing training, testing data” (link) summarizing the ordeal. The author, Katyanna Quach IMO is a well-informed reporter to cover machine learning, so I believe she probably used this subreddit to source some of her leads.
Some previous discussion on this subreddit:
Misuse of Deep Learning in Nature Journal’s Earthquake Aftershock Paper
and follow up thread:
One neuron is more informative than a deep neural network for aftershock pattern forecasting
I would like to thank everyone who contributed to the discussion since I think it is important to call these papers out, and make what is wrong known to the wider community outside of research.
submitted by /u/milaworld
[link] [comments]
I just found this really nice tutorial about generative networks. It offers a really intuitive overview and look at the difference between memorizing and generalizing on a dataset.
Link: https://www.kaggle.com/cdeotte/supervised-generative-dog-net
submitted by /u/MediumInterview
[link] [comments]
![[D] Maximizing Likelihood results in a Degenerate VAE?, aka More Variational Autoencoder Confusion](https://b.thumbs.redditmedia.com/zQEP9ufClQB_0fBvnkDXbVXDOcGAF1XQi9VUpk-8DrI.jpg "[D] Maximizing Likelihood results in a Degenerate VAE?, aka More Variational Autoencoder Confusion") |
TL;DR I’m still confused about VAEs. My experience is indicating if I take a Beta VAE formulation and try to maximize the likelihood of some hold-out data while varying Beta, my model collapses. Please bare with me: With Variational Autoencoders, we estimate an approximate likelihood by maximizing the ELBO, aka data likelihood: log(p(x)) >= L(x) = E(log p(x|z)) – KL(q(z|x)||p(z)) aka: ELBO = L(x) = – (Distortion + Rate) where E(logo(x|z)) is equivalent to the reconstruction loss (up to scale) and KL divergence is determined via the reparameterization trick. One interpretation of this is that the model cares about two things: Good reconstruction up to a highly compressed representation. These sort of things are good for interpreting data with respect to the latent space, and whatnot. We can add a coefficient, B, to explore the trade off between Rate (KLD) and Distortion (reconstruction). That gives us the BetaVAE formulation: E(log p(x|z)) – B * KL(q(z|x)||p(z)). (e.g. https://openreview.net/forum?id=Sy2fzU9gl ) or you can do other things to “pin” the rate: E(log p(x|z)) + | C – KL(q(z|x)||p(z)) | (e.g. https://arxiv.org/abs/1804.03599 or https://arxiv.org/pdf/1711.00464.pdf ) There are many papers that use VAEs, often by training the VAE and picking the point in the training that maximizes the ELBO on some hold out set (e.g. https://www.nature.com/articles/s41592-018-0229-2.pdf ) My problem: Let’s say I take a VAE and one of the R vs D formulations and scan over Beta and plot the rate vs distortion of the held-out data for different models. I often get something like this (this is a screenshot from the ELBO pape, but I also get approximately these results): The dotted line is the R vs D tradeoff at the maximum likelihood model, and this occurs when the rate drops to zero. In the case of a “vanilla” VAE, this means a degenerate latent space where all points represent the same thing. All points in Z are the same (i.e. N(0,1)) and I have experienced the (grotesquely named) posterior collapse. In this case the model (usually) only emits the “average” input unless there is some side-channel of information. This is often considered a bad thing, and L(x) is therefore only determined by the reconstruction loss from the input and the “average” emission. But this is the maximum likelihood model! Let’s go back to the Lopez et al paper above: If I were to scan over Beta to find the maximum likelihood model and I get a collapsed latent space, I wouldn’t have a model that is particularly useful in that it would not provide “interpretable” latents. In the context where I am performing a conditional prediction task (e.g. https://scholar.google.com/scholar?q=variational+autoencoder+prediction), the VAE would emit the same value no matter the condition. Is maximum likelihood/ELBO right? One could imagine that there are alternate ways to evaluate this result (i.e. Frechet Inception Distance on generated examples). Should I use those? Should I just be using exact inference methods instead? submitted by /u/idioth |
