Author: torontoai

[R] https://arxiv.org/abs/1811.07519 Higher-order Neural Networks for Action Recognition

Written on July 3, 2019. Posted in Reddit MachineLearning.

I am delighted to announce that I have submitted our recent work to arXiv. Any feedback would be highly appreciated. https://arxiv.org/abs/1811.07519

In this paper, we proposed a new architecture: the higher-order operation. The term “higher order” comes from higher order functions. A higher order function is a function that takes a function as an argument, or returns a function. Similarly, the outputs of the higher-order operation are not feature maps, but a bank of filters for extracting features. Then the network use the filters to extract features.

The intuition comes from the complexity of action recognition. It is much harder to recognize an action in a video than objects in still images. An effective architecture should not only recognize the appearance of target objects associated with the action, but also understand how they relate to other objects in the scene, in both space and time.

https://i.redd.it/ieidqv3rk6831.png

In the figure, we have 4 categories of actions: a) pull something from left to right, b) push something from right to left, c) push something from left to right and d) pull something from right to left. Only understanding the appearance info is not enough since we have only the hand and the object “something” in all four actions. It is also insufficient with temporal information. Figure b is the reverse of figure a “pull something from left to right”, but figure b is not simply the opposite: “pull something from right to left”. It is important to understand the object-in-context pattern to classify the actions.

As scenes become more complicated and the number of objects whose relations need to be tracked increases, the complexity of the learning task faced by the architecture increases rapidly. The vanilla convolutions use fixed filters to recognize every object-in-context pattern required to recognize one category of action, potentially leading to a blow up of the number parameters required for effective recognition of the actions.

In such settings, we do not want to have a huge number of filters to cover all possible object-in-context patterns. It is best if the model can propose/derive a filter given a certain context —- different filters for different contexts. The model does not need to store all the filters, but needs to learn how to propose a filter. Then the model can capture motions in the contexts better with reasonable parameters.

submitted by /u/KaiiHu
[link] [comments]

Explain NLP models with LIME & SHAP

Written on July 3, 2019. Posted in Susan Li.

Interpretation for Text Classification

Last week, I gave a talk on “Hands-on Feature Engineering for NLP” at QCon New York. As a very small part of the presentation, I gave a brief demo on how LIME & SHAP work in terms of text classification explainability.

I decided to write a blog post about them because they are fun, easy to use and visually compelling.

All machine learning models that operate in higher dimensions than what can be directly visualized by the human mind can be referred as black box models which come down to the interpretability of the models. In particular in the field of NLP, it’s always the case that the dimension of the features are very huge, explaining feature importance is getting much more complicated.

LIME & SHAP help us provide an explanation not only to end users but also ourselves about how a NLP model works.

Using the Stack Overflow questions tags classification data set, we are going to build a multi-class text classification model, then applying LIME & SHAP separately to explain the model. Because we have done text classification many times before, we will quickly build the NLP models and focus on the models interpretability.

Data Pre-processing, Feature Engineering and Logistic Regression

<a href="https://medium.com/media/ab22a2d62c16dcaac3702aa0f35e403c/href">https://medium.com/media/ab22a2d62c16dcaac3702aa0f35e403c/href</a>

Our objective here is not to produce the highest results. I wanted to dive into LIME & SHAP as soon as possible and that’s what happened next.

Interpreting text predictions with LIME

From now on, it’s the fun part. The following code snippets were largely borrowed from LIME tutorial.

<a href="https://medium.com/media/732f8dfa670d5df756fd450e3e9c972d/href">https://medium.com/media/732f8dfa670d5df756fd450e3e9c972d/href</a>

We randomly select a document in test set, it happens to be a document that labeled as sql, and our model predicts it as sql as well. Using this document, we generate explanations for label 4 which is sql and label 8 which is python.

print ('Explanation for class %s' % class_names[4])
print ('n'.join(map(str, exp.as_list(label=4))))

print ('Explanation for class %s' % class_names[8])
print ('n'.join(map(str, exp.as_list(label=8))))

It is obvious that this document has the highest explanation for label sql. We also notice that the positive and negative signs are with respect to a particular label, such as word “sql” is positive towards class sql while negative towards class python, and vice versa.

We are going to generate labels for the top 2 classes for this document.

exp = explainer.explain_instance(X_test[idx], c.predict_proba, num_features=6, top_labels=2)
print(exp.available_labels())

It gives us sql and python.

exp.show_in_notebook(text=False)

Let me try to explain this visualization:

For this document, word “sql” has the highest positive score for class sql.
Our model predicts this document should be labeled as sql with the probability of 100%.
If we remove word “sql” from the document, we would expect the model to predict label sql with the probability at 100% — 65% = 35%.
On the other hand, word “sql” is negative for class python, and our model has learned that word “range” has a small positive score for class python.

We may want to zoom in and study the explanations for class sql, as well as the document itself.

exp.show_in_notebook(text=y_test[idx], labels=(4,))

Interpreting text predictions with SHAP

The following process were learned from this tutorial.

<a href="https://medium.com/media/812fb43dae8c46ba8322804a2afe40ea/href">https://medium.com/media/812fb43dae8c46ba8322804a2afe40ea/href</a>

After model is trained, we use the first 200 training documents as our background data set to integrate over, and to create a SHAP explainer object.
We get the attribution values for individual predictions on a subset of the test set.
Transform the index to words.
Use SHAP’s summary_plot method to show the top features impacting model predictions.

attrib_data = X_train[:200]
explainer = shap.DeepExplainer(model, attrib_data)
num_explanations = 20
shap_vals = explainer.shap_values(X_test[:num_explanations])

words = processor._tokenizer.word_index
word_lookup = list()
for i in words.keys():
  word_lookup.append(i)

word_lookup = [''] + word_lookup
shap.summary_plot(shap_vals, feature_names=word_lookup, class_names=tag_encoder.classes_)

Word “want” is the biggest signal word used by our model, contribute most to class jquery predictions.
Word “php” is the 4th biggest signal word used by our model, contributing most to class php of course.
On the other hand, word “php” is likely to have a negative signal to the other class because it is unlikely to see word “php” to appear in a python document.

There are a lot to learn in terms of machine learning interpretability with LIME & SHAP. I have only covered a tiny piece for NLP. Jupyter notebook can be found on Github. Enjoy the fun!

Explain NLP models with LIME & SHAP was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

[D] Maximumizing Likelihood results in a Degenerate VAE?, aka More Variational Autoencoder Confusion

Written on July 3, 2019. Posted in Reddit MachineLearning.

TL;DR I’m still confused about VAEs. My experience is indicating if I take a Beta VAE formulation and try to maximize the likelihood of some hold-out data while varying Beta, my model collapses. Please bare with me:

With Variational Autoencoders, we estimate an approximate likelihood by maximizing the ELBO, aka data likelihood:

log(p(x)) >= L(x) = E(log p(x|z)) – KL(q(z|x)||p(z))

aka:

ELBO = L(x) = – (Distortion + Rate)

where E(logo(x|z)) is equivalent to the reconstruction loss (up to scale) and KL divergence is determined via the reparameterization trick.

One interpretation of this is that the model cares about two things: Good reconstruction up to a highly compressed representation. These sort of things are good for interpreting data with respect to the latent space, and whatnot.

We can add a coefficient, B, to explore the trade off between Rate (KLD) and Distortion (reconstruction). That gives us the BetaVAE formulation:

E(log p(x|z)) – B * KL(q(z|x)||p(z)). (e.g. https://openreview.net/forum?id=Sy2fzU9gl )

or you can do other things to “pin” the rate:

E(log p(x|z)) + | C – KL(q(z|x)||p(z)) | (e.g. https://arxiv.org/abs/1804.03599 or https://arxiv.org/pdf/1711.00464.pdf )

There are many papers that use VAEs, often by training the VAE and picking the point in the training that maximizes the ELBO on some hold out set (e.g. https://www.nature.com/articles/s41592-018-0229-2.pdf )

My problem:

Let’s say I take a VAE and one of the R vs D formulations and scan over Beta and plot the rate vs distortion of the held-out data for different models. I often get something like this (this is a screenshot from the ELBO pape, but I also get approximately these results):

Fixing a Broken ELBO, fig 3a

The dotted line is the R vs D tradeoff at the maximum likelihood model, and this occurs when the rate drops to zero. In the case of a “vanilla” VAE, this means a degenerate latent space where all points represent the same thing. All points in Z are the same (i.e. N(0,1)) and I have experienced the (grotesquely named) posterior collapse. In this case the model (usually) only emits the “average” input unless there is some side-channel of information. This is often considered a bad thing, and L(x) is therefore only determined by the reconstruction loss from the input and the “average” emission.

But this is the maximum likelihood model!

Let’s go back to the Lopez et al paper above: If I were to scan over Beta to find the maximum likelihood model and I get a collapsed latent space, I wouldn’t have a model that is particularly useful in that it would not provide “interpretable” latents. In the context where I am performing a conditional prediction task (e.g. https://scholar.google.com/scholar?q=variational+autoencoder+prediction), the VAE would emit the same value no matter the condition.

Is maximum likelihood/ELBO right? One could imagine that there are alternate ways to evaluate this result (i.e. Frechet Inception Distance on generated examples). Should I use those? Should I just be using exact inference methods instead?

submitted by /u/idioth
[link] [comments]

[D] Any book recommendation for applied linear modeling that is NOT R-based

Written on July 2, 2019. Posted in Reddit MachineLearning.

Looking for a book that sheds lights on practical issues of using linear modeling for prediction in real production/project scenario (without losing much mathematical rigor). Preferably the book is language agnostic or Python-based. (I found this book that seems nice but has R code: Applied Predictive Modeling)

submitted by /u/this_nicholas
[link] [comments]

[D] Techniques to sample unbalanced multi-label datasets?

Written on July 2, 2019. Posted in Reddit MachineLearning.

I have a dataset with multi-label classification outputs. These are (fortunately) binary, so a typical output is basically a binary vector like [1, 0, 0, 1, 1]. The length of this vector is fixed.

The dataset is pretty biased towards all 0’s, and since it is a multi-label output (and not like a one-hot encoding), I’m not sure what is the best way to undersample or oversample the dataset for my training epochs, since traditional stratification cannot work here.

Edit: All kinds of suggestions are welcome, be it simple beginner methods or ICLR papers 😉

submitted by /u/parekhnish
[link] [comments]

[R] Do deep neural networks learn shallow learnable examples first? (ICML 2019 Workshop paper)

Written on July 2, 2019. Posted in Reddit MachineLearning.

Dear Reddit-ML community,
At the ICML 2019 Workshop Deep Phenomena workshop, we presented our initial results addressing the question:
Do deep neural networks learn shallow learnable examples first?

In case you have addressed this question from different perspectives or have any feedback in this regard, we’d love to hear these!
Here are the pertinent links: (We’ll be updating the repo shortly with more results)

[Video] https://youtu.be/aXB7PbJ8osQ
[Code] https://github.com/karttikeya/Shallow_to_Deep/
[ Paper] https://openreview.net/forum?id=HkxHv4rn24

submitted by /u/VinayUPrabhu
[link] [comments]

[D] LidarView and Deeplearning

Written on July 2, 2019. Posted in Reddit MachineLearning.

https://youtu.be/7pQGYHgvvrY

Visualize your multi-sensor data with #LidarView, the open source #ParaView Lidar application. And thanks to our in-house Lidar SLAM, map them in 3D, be it indoor or outdoor.Here is a sequence in Lyon, nearby our Kitware Europe office: Only one VLP-16 from Velodyne Lidar, Inc. and a Gopro camera, no IMU or GPS. Automatic computation of the position/orientation of the camera relatively to the Lidar, as well as the time-sync. Showing our preferred semantic segmentation, from #bonnetal. Stay tuned as the 3D semantic segmentation should be available soon too.

submitted by /u/cmillionaire9
[link] [comments]

[D] Word embeddings for categorical variables?

Written on July 2, 2019. Posted in Reddit MachineLearning.

I am working on a classification problem with a data set containing numerical as well as categorical data. A colleague of mine said that instead of encoding the categorical variables in a “primitive way” (label encoding, creating dummy variables etc.) he would use word2vec to get some kind of word embeddings. This would be a more realistic way of representing these variables. To me this makes no sense. If I understood correctly, for word2vec to work the words we want to embed need neighbors for there to be some kind of context. In a column of a DataFrame containing one string in each row and maybe 3 – 10 unique categories there isn’t any context. Each entry is independent from the entry in the next row. Am I missing something?

I hope I posed the question in a somewhat understandable way.

Thanks, guys.

submitted by /u/aeppelsaeft
[link] [comments]

Turning Microsoft Word documents into audio playlists using Amazon Polly

Written on July 2, 2019. Posted in Amazon.

Listening to your Microsoft Word documents as audio is a great way to save time or to be productive on a long commute. You can easily convert an entire block of text into MP3 format with Amazon Polly. But you can vastly improve your listening experience with just a few simple steps.

In this blog post, I show how you can use a serverless workflow to convert your word documents into MP3 playlists using AWS Lambda and Amazon Polly.

To review a Word document that I needed to listen to, I converted the whole document to one block of text, then converted it to MP3 using Amazon Polly. After listening, I realized that a long, single-voice MP3 file results in a monotonous stream of audio.

Next, I split the document into small parts and processed each part with a different voice and cadence. This process added audio cues to keep me engaged while listening. I came up with the following serverless architecture that takes in a Microsoft Word document and generates MP3 files and an ordered M3U playlist file. I can download my list and listen to the Word document as an audio playlist anywhere!

Solution overview

The following diagram shows the architecture of this solution.

The following steps generate the MP3 files and playlist:

Upload the Word document to the Project bucket at /src.
On upload, a PUT object event triggers the Word to SSML AWS Lambda function.
The Lambda function splits the document into multiple SSML files, assigns a VoiceId tag to each file, and saves them to the project bucket at /ssml.
Several PUT object events in the /ssml key trigger the Amazon Polly SSML to MP3 Lambda function, which starts an Amazon Polly task to convert the SSML document into an MP3 file. The Amazon Polly task then saves the MP3 file in Amazon S3 and the file metadata to the Mp3 metadata table in Amazon DynamoDB.
After Amazon Polly completes its tasks, invoke the m3u builder Lambda function to generate the m3u playlist file and save it to the Project bucket.

The following table shows the solution components and describes how they are used.

Resource	Type	Description
Project bucket	S3 bucket	S3 bucket used for storing the Word document before processing, the generated SSML files, the generated MP3 files, and the M3U playlist file. Event notifications on the bucket trigger various Lambda functions.
Word to SSML	Lambda function	A Lambda function that uses the Java 8 runtime to take in a Word document and split it into several SSML documents based on the contained sections, topics, and paragraphs in the document. The S3 bucket stores the SSML documents, with each file assigned a VoiceId tag used later by the Amazon Polly SSML to MP3 Lambda function.
Amazon Polly SSML to MP3	Lambda function	A Lambda function that takes one SSML file in S3 and converts it to MP3 using an Amazon Polly voice that matches the assigned VoiceId. It then stores the MP3 files in the Project bucket. It also saves the metadata of processed files and the corresponding Amazon Polly tasks to a DynamoDB table.
MP3 metadata	DynamoDB table	A DynamoDB table that stores the metadata of processed SSML files and corresponding Amazon Polly tasks.
M3U builder	Lambda function	A Lambda function that processes the metadata in the MP3 metadata table database, generates a correctly ordered M3U playlist file, and stores it in the Project bucket.

Building the Word to SSML Lambda function

I used Apache POI to read the Word document and split it into several small SSML files. I provide an extensible implementation that works for any three-level document that contains a set of sections, each containing a set of topics, and each of those topics containing a set of paragraphs.

I used the public Amazon Polly FAQs as an example document, which uses categories of the FAQ (for example, general, billing, data privacy) as the sections. Those sections divide into individual questions for the topics, and into individual answers for the paragraphs.

This same model generally applies to any three-level document: The user supplies a way to identify sections and topics. The default implementation extracts the sections from text with the Heading 1 Word style and identifies topics by recognizing the question mark character in the sentence.

Prerequisites

You need a few tools to follow the steps in this post:

OpenJDK 8 and Apache Maven 3.5: The Word to SSML Lambda function uses the Java 8 runtime and uses Apache Maven for packaging. Install OpenJDK version 8 or higher and Maven version 3.5 or higher. I tested this solution with Maven version 3.5.0 and OpenJDK Runtime Environment Corretto-8.202.08.2.
AWS Command Line Interface: Some of the instructions assume that you have a working AWS CLI version to execute the test steps.
S3 bucket: Lambda functions can only use artifacts from an S3 bucket in the Region in which you choose to deploy your solution. Choose a bucket to reuse, or create a bucket by running the following command:
```
aws s3 mb s3://<PROJECT-BUCKET> --region <REGION>
```

Deployment steps

Follow these steps to deploy your tool.

Clone the GitHub repository for the project.

git clone https://github.com/aws-samples/amazon-polly-mp3-for-microsoft-word.git

Export the AWS Region, project S3 bucket, and AWS CloudFormation stack name as environment variables for convenience.
```
export PROJECT_BUCKET=<your-project-bucket>
export REGION=<your-region> 
export STACK_NAME=polly-stack
```
Change to the project directory and execute the deploy_lambda_cloudformation.sh script to provide your chosen AWS Region, S3 bucket, and name for your CloudFormation stack. This script performs the following actions:
1. Packages the three Lambda functions and copies it to your S3 bucket.
2. Copies the CloudFormation template to your S3 bucket.
3. Deploys the stack with the chosen name.
4. Waits until the Lambda function successfully creates the stack. This should take approximately two minutes.
5. Updates the bucket notifications template (scripts/bucket_lambda_notification.json) with values from the stack output.
6. Adds event notifications to the S3 bucket.
```
cd Amazon-Polly-Microsoft-Word-to-MP3
bash scripts/deploy_lambda_cloudformation.sh $REGION $PROJECT_BUCKET $STACK_NAME
```
[Optional] After the script executes, in the AWS CloudFormation console, verify that the stack deployed and is in CREATE_COMPLETE status.
In the S3 console, verify that the bucket contains your event notifications. The first notification, on the polly-faq-reader/src/ path, invokes the Word to SSML Lambda function when a new DOCX file uploads to this path. This Lambda function generates several SSML text files and uploads them to the polly-faq-reader/ssml/ A notification set up on this path then invokes the Amazon Polly SSML to MP3 Lambda function. The following screenshot shows sample events.
Now you’re ready to test the MP3 conversion. Copy the demo/src/polly-faq.docx to the Project bucket at polly-faq-reader/src/. This triggers the Lambda functions to generate SSML and MP3 files.
```
aws s3 cp demo/src/polly-faq.docx s3://${PROJECT_BUCKET}/polly-faq-reader/src/
```

List the polly-faq-reader/ prefix in the S3 bucket and verify that it generates new SSML and MP3 directories.

aws s3 ls s3://$PROJECT_BUCKET/polly-faq-reader/
                           PRE mp3/
                           PRE src/
                           PRE ssml/

Wait about two minutes for the Amazon Polly tasks to complete. To verify when MP3 conversion completes, you can verify that the number of files in the /ssml directory matches the number of /mp3 files.
```
aws s3 ls s3://$PROJECT_BUCKET/polly-faq-reader/mp3/ | wc -l
     62
aws s3 ls s3://$PROJECT_BUCKET/polly-faq-reader/ssml/ | wc -l
     62
```
The tool builds an M3U playlist file to play all the generated MP3 files in the correct order. In your terminal, in the scripts directory, execute the invoke_m3u_builder.sh script providing your Region, bucket name, and name of your AWS CloudFormation stack.
```
bash scripts/invoke_m3u_builder.sh $REGION ${PROJECT_BUCKET} ${STACK_NAME}
```
Verify that a new polly-faq.m3u file is present in the S3 bucket at polly-faq-reader/mp3/.
```
aws s3 ls s3://$PROJECT_BUCKET/polly-faq-reader/mp3/polly-faq.m3u
```

Download the mp3 files and m3u playlist to your computer.

cd <your-chosen-mp3-directory>
aws s3 sync s3://$PROJECT_BUCKET/polly-faq-reader/mp3/

Open the m3u playlist file in your preferred media player and listen to the files.

Clean up

To clean up the deployment and avoid incurring future costs, follow these steps:

In the S3 console, select your bucket and delete the two event notifications.
In the AWS CloudFormation console, and delete the polly-stack.

If you no longer need the SSML or MP3 files, delete them. Run the following commands:

aws s3 rm --recursive s3://$PROJECT_BUCKET/polly-faq-reader/ssml/
aws s3 rm --recursive s3://$PROJECT_BUCKET/polly-faq-reader/mp3/

Conclusion

In this post, I demonstrated a serverless workflow to convert Microsoft Word documents into an MP3 audio playlist using Amazon Polly and AWS Lambda.

To dig deeper into the code, check out the GitHub repository and create issues for providing feedback or suggesting enhancements. Open-source code contributions are welcome as pull requests.

About the Author

Vinod Shukla is a Partner Solutions Architect at Amazon Web Services. As part of the AWS Quick Starts team, he enjoys working with partners providing technical guidance and assistance in building gold-standard reference deployments.

[D] Lawyer: “even if the patent is granted, you can still use it as long as Google does not sue you. It is possible that Google can wait until your company grows up and come back to sue you”

Written on July 2, 2019. Posted in Reddit MachineLearning.

Update on the Google Dropout Patent discussion from last week:

https://www.reddit.com/r/MachineLearning/comments/c5mdm5/d_googles_patent_on_dropout_just_went_active_today/

From this article: https://medium.com/syncedreview/concerns-on-social-media-over-google-ml-patents-fadeb5a0b2e9

An attorney with a focus on corporate and securities law who asked not to be identified told Synced that “even if the patent is granted, you can still use it as long as Google does not sue you. It is possible that Google can wait until your company grows up and come back to sue you, which could potentially make you or your institution liable for consequences and damages, not Google.”

submitted by /u/SilentTheme
[link] [comments]

Blog

Learn About Our Meetup

5000+ Members

MEETUPS

JOB POSTINGS

CONTACT

Author: torontoai

[R] https://arxiv.org/abs/1811.07519 Higher-order Neural Networks for Action Recognition

Explain NLP models with LIME & SHAP

Interpretation for Text Classification

Data Pre-processing, Feature Engineering and Logistic Regression

Interpreting text predictions with LIME

Interpreting text predictions with SHAP

[D] Maximumizing Likelihood results in a Degenerate VAE?, aka More Variational Autoencoder Confusion

[D] Any book recommendation for applied linear modeling that is NOT R-based

[D] Techniques to sample unbalanced multi-label datasets?

[R] Do deep neural networks learn shallow learnable examples first? (ICML 2019 Workshop paper)

[D] LidarView and Deeplearning

[D] Word embeddings for categorical variables?

Turning Microsoft Word documents into audio playlists using Amazon Polly

Solution overview

Building the Word to SSML Lambda function

Prerequisites

Deployment steps

Clean up

Conclusion

About the Author

[D] Lawyer: “even if the patent is granted, you can still use it as long as Google does not sue you. It is possible that Google can wait until your company grows up and come back to sue you”