I remember reading a paper that showed that data samples that were used to train a GAN can be recovered almost perfectly, thus creating privacy concerns in GANs when used in medical patient data, for example.

​

Does anyone recall such a paper? Or what are your informed opinions on this topic, and could you provide some references?

​

Thanks!

Why is it that in tensorflow, the batchnorm results during inference time (e.g. using model.predict()) are dependent on the data in the batch? I would expect the network computation graph to be completely frozen.

Hi in my research paper I recently came up with a methodology to measure semantic narrowing and semantic broadening, but these definitions are based on intuition and their definitions from past literature (Blank 1999). If anyone has any experience in research in NLP, or semantic shift – I would love some feedback.

I haven’t made my paper public

Code: https://github.com/suyash/mlt

Implemented transformers with One-to-Many, Many-to-One, Many-to-Many translation, with another demo for fine-tuning, by replacing regular layer norm in both encoder and decoder with Conditional Norm.

Conditional Norm is a technique from Style Transfer, used to train a single network with multiple styles, as well as mix styles, for example

https://i.imgur.com/U0L1x8u.png

Broader set of demos at https://github.com/suyash/stylizer/blob/master/image_stylization_demo.ipynb

The goal here is similar, make the rest of the network learn a common representation, while making the normalization parameters learn language specific semantics.

The One-to-Many and Many-to-One models are trained for English to French, German, Italian and Spanish Translation and Vice Versa.

The Many to Many model is trained on English-French, French-English, English-German and German-English.

The image stylization paper specifies how a N-style network can pick up an N+1th style through fine-tuning an existing model. Similarly, I fine-tune my Many-to-Many model to pick up Portuguese.

I also provide demos for Few-Shot and Zero-Shot translation.

Have provided SavedModels, as well as standalone notebooks that run in Google Colaboratory.

One-to-Many: https://colab.research.google.com/github/suyash/mlt/blob/master/one_to_many_demo.ipynb

Many-to-One: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_one_demo.ipynb

Many-to-Many: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_many_demo.ipynb

Many-to-Many fine-tuned: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_many_fine_tune_demo.ipynb

Hi!

I have a Variational autoencoder model created in Keras.Encoder is built with three 3D Convolutional layers + Flatten + Dense layer. Decoder is built with three 3D Transposed Convolutional layers to reconstruct the input 3D images.

My goal is to replace Flatten and Dense layer in Encoder with 1x1x1 Convolutional layer. Any ideas how to do that?

My model structure looks like this:

original_dim = np.prod(np.array(original_shape)) input_shape = ( original_shape[0], original_shape[1], original_shape[2], 1) latent_dim = 100 def sampling(args): """Reparameterization trick by sampling fr an isotropic unit Gaussian. Arguments: args {tensor} -- mean and log of variance of Q(z|X) Returns: z {tensor} -- sampled latent vector """ z_mean, z_log_var = args batch = K.shape(z_mean)[0] dim = K.int_shape(z_mean)[1] epsilon = K.random_normal(shape=(batch, dim)) return z_mean + K.exp(0.5 * z_log_var) * epsilon # Encoder inputs = Input(shape=input_shape, name='encoder_input') x = inputs x = Conv3D(filters=32, kernel_size=3, activation='relu', strides=1, padding='same')(x) x = Conv3D(filters=64, kernel_size=3, activation='relu', strides=2, padding='same')(x) x = Conv3D(filters=128, kernel_size=3, activation='relu', strides=2, padding='same')(x) shape = K.int_shape(x) x = Flatten()(x) x = Dense(latent_dim, activation='relu')(x) z_mean = Dense(latent_dim, name='z_mean')(x) z_log_var = Dense(latent_dim, name='z_log_var')(x) z = Lambda(sampling, output_shape=(latent_dim,), name='z')([z_mean, z_log_var]) encoder = Model(inputs, [z_mean, z_log_var, z], name='encoder') # Decoder latent_inputs = Input(shape=(latent_dim,), name='z_sampling') x = Dense(shape[1] * shape[2] * shape[3] * shape[4], activation='relu')(latent_inputs) x = Reshape((shape[1], shape[2], shape[3], shape[4]))(x) x = Conv3DTranspose(filters=128, kernel_size=3, activation='relu', strides=2, padding='same')(x) x = Conv3DTranspose(filters=64, kernel_size=3, activation='relu', strides=2, padding='same')(x) x = Conv3DTranspose(filters=32, kernel_size=3, activation='relu', strides=1, padding='same')(x) outputs = Conv3DTranspose(filters=1, kernel_size=3, strides=1, activation='sigmoid', padding='same', name='decoder_output')(x) decoder = Model(latent_inputs, outputs, name='decoder') decoder.summary() outputs = decoder(encoder(inputs)[2]) vae = Model(inputs, outputs, name='vae') reconstruction_loss = mse(K.flatten(inputs), K.flatten(outputs)) reconstruction_loss *= original_dim kl_loss = 1 + z_log_var - K.square(z_mean) - K.exp(z_log_var) kl_loss = K.sum(kl_loss, axis=-1) kl_loss *= -0.5 vae_loss = K.mean(reconstruction_loss + kl_loss) vae.add_loss(vae_loss) vae.compile(optimizer='rmsprop') 

Thank you for any help

Companies have different roles for ML engineers/scientists, I have seen Data Scientist (at Flipkart (comes under Walmart)), Applied Scientist at Amazon and Software Engineer at Google.

As a fresh out of college, I wanted to know how much do these role matter? Is Applied Scientist a better role than Data Scientist? Are there another roles?

There’s been many popular posts on this subreddit regarding patents on well known ML techniques. Here is your chance to voice your concerns to the US patent office.

Request for Comments on Patenting Artificial Intelligence Inventions

https://www.federalregister.gov/documents/2019/08/27/2019-18443/request-for-comments-on-patenting-artificial-intelligence-inventions

AGENCY:

United States Patent and Trademark Office, Department of Commerce.

ACTION:

Request for comments.

SUMMARY:

The United States Patent and Trademark Office (USPTO) is interested in gathering information on patent-related issues regarding artificial intelligence inventions for purposes of evaluating whether further examination guidance is needed to promote the reliability and predictability of patenting artificial intelligence inventions. To assist in gathering this information, the USPTO is publishing questions on artificial intelligence inventions to obtain written comments from the public. The questions are designed to cover a variety of topics from patent examination policy to whether new forms of intellectual property protection are needed.

DATES:

Written comments must be received on or before October 11, 2019.

Hi r/ml. I hope this isn’t the kind of post that is unwelcome here but I thought maybe you guys could help give me some pointers. I am an undergraduate senior and I will be spending this year writing a capstone math thesis. I need to identify topics that I can propose to my department. They need to be topics that are in the scope of classical statistics, algorithmic analysis, or probability. It can’t be something that is more the domain of computer science, like network architecture.

I was wondering if there are any topics you guys think are important or foundational to other concepts that are important in ML. I spent the last summer researching variational autoencoders and became familiar with variational expectation maximization. This is something I’m excited about, but I’m looking for other ideas as well.

I guess another way of asking this question is: what are the parts of papers that you wished you knew more about? What math concepts come up a lot and are hard to understand?

Thanks for the help, and again sorry if this is the wrong place to ask.

We are looking for full-time data scientists for a ML/DL user study. You’ll be participating in a calibrated user research experiment for 45 minutes. The study will be done over a video call. We’ve got plenty of funny tees that you can show-off to your teammates. We’ll ship you a different one every month for a year.

Click here to learn more.

P.S: We love the reddit vibe and this community. Give us your best ML/DL/Data Science quote. We’ll make one and ship it you if it’s quirky and fun even if you aren’t part of the study group.

Any one here encountered ML models that you tried to implement but were junk/not reproducible? Or in general had a bad experience with open-source ML models?