Skip to main content

Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

Author: torontoai

[D] Do you have to cite a dishonest paper?

If you come across a paper that is directly relevant to the problem you’re trying to solve but also makes claims that are unsupported by its own data (and contradict your own results), do you have to cite it? I feel that no credit should be given where none is due, and that debunking the claims is a waste of space.

For reference, this is the paper I’m talking about. The paper investigates adding attention layers on top of the standard CNN used by DQN on Atari. There are 6 variants considered, which differ in the location where the attention layers are added to the CNN.

Claim 1 (figure 2): “Here we can see the clear advantage of self-attention is able to provide with respect to sample efficiency.”

The baseline PPO learns faster or on-par with their 6 variants on 6 of the 10 games benchmarked.

Claim 2 (table 1): “These results clearly demonstrate the improved performance of multiple self-attention variants.”

Their best variant, SAN, performs better on 5 of the 10 games compared to the baseline PPO. Most of the other variants perform worse.

Claim 3 (conclusion): “We can also see clear improvements over baseline results with respect to 60% of environments tested”.

That is to say, the baseline PPO performs best on 4 games, and the proposed method performs best on the 6 other games if you cherry-pick the variant used on a per-game basis.

submitted by /u/Seerdecker
[link] [comments]

[D] Keras (RNNs) for multiple time series?

The two Keras subs seem pretty dead, hoping I can find some help here.

I was able to use Keras to create a prediction from a time series, with input data looking like so:

Date Volume
1/1/2017 5
1/2/2017 7
12/31/2018 12

But we would like to expand on this.

We have multiple products, which are each sold to multiple stores. I would like to create a tool where we throw data for each product at each store into a model, and then call “At this store, for this product, for this date, Keras predicts X”

I cannot seem to find an example that deals with having multiple rows for each date. An example of the full dataset is below. Seems like one way would be to make a column for each store/product combo and loop those through with a model for each, but I’m not sure if that would be ideal.

Date Volume Store Product
1/1/2017 5 A a
1/1/2017 6 A b
1/1/2017 1 A c
1/2/2017 8 A a
1/2/2017 9 A b
1/2/2017 2 A c
1/1/2017 6 B a
1/1/2017 6 B b
1/1/2017 1 B c
1/2/2017 7 B a
1/2/2017 8 B b
1/2/2017 2 B c
12/31/2018 5 A a
12/31/2018 6 A b
12/31/2018 1 A c
12/31/2018 8 A a
12/31/2018 9 A b
12/31/2018 2 A c
12/31/2018 6 B a
12/31/2018 6 B b
12/31/2018 1 B c
12/31/2018 7 B a
12/31/2018 8 B b
12/31/2018 2 B c

submitted by /u/heloderma1
[link] [comments]

[P] How to encode time information in LSTMs and perform weighted averaging where the weights are learned

[P] How to encode time information in LSTMs and perform weighted averaging where the weights are learned

I am trying to build a model based on a recent google paper called “Scalable and Accurate Deep Learning for Electronic Health Records”. The paper describes how to embed medical data for downstream prediction tasks.

Paper: https://arxiv.org/abs/1801.07860

Supplementary: https://static-content.springer.com/esm/art%3A10.1038%2Fs41746-018-0029-1/MediaObjects/41746_2018_29_MOESM1_ESM.pdf

I am using Pytorch and trying to modify an existing template I found online (https://github.com/yuchenlin/lstm_sentence_classifier/blob/master/LSTM_sentence_classifier.py)

In the supplementary material, they have a section describing how to embed medical data by doing the following (pic attached below too):

1) Take the raw medical data per patient per time step and embed them (like how its done in word embeddings)

2) Concatenate all history of that patient for that timestep together into a long vector

2a) For multiple information of the same type, average embeddings using a learned weighting. The weighted averaging is done by associating each feature with a non-negative weight that is trained jointly with the model.

3) Create another vector that contains time information (seconds)

https://i.redd.it/b41lbaylotw21.png

Finally, in the supplementary material, it states that:

The sequence of embeddings were further reduced down to a shorter sequence. Typically, the shorter sequences were split into time-steps of 12 hours where the embeddings for all features within a category in the same day were combined using weighted averaging. The weighted averaging is done by associating each feature with a non-negative weight that is trained jointly with the model. These weights are also used for prediction attribution. The log of the average time-delta divided by a factor (controlled by a hyperparameter) at each time-step is also embedded into a small floating-point vector (which is also randomly initialized) and concatenated to the input embedding at each time-step.

My first question is: How do I actually go about inserting this time vector?

The supplementary states that the sequences were split into 12-hour time steps. If that’s the case, why the need for a time vector? It then later states that you can just simply concatenate the time vector to the input embedding. Why would a simple concatenation like that work and how would the model know to associate that time information with the appropriate part of the input?

My second question is: For Step 2a), how do I go about getting a weighted average for the same-category features? That part is glossed over without much explanation and it is lost on me.

I’m sorry if these are a lot of questions, but even pointing to some tutorials or any readings would go a long way in the right direction!

submitted by /u/somethingstrang
[link] [comments]

[Project] Facebook post generator based on a facebook group

Hello everyone,

I would like to mine data from a facebook group to make a content generator for a facebook page, but I am quite clueless as to how to proceed. The use would be purely recreational, but google searchs did not yield anything interesting aside from Quora q&as.

Has anyone had any experience with making facebook page bots ? How can I mine data from a facebook group ?

Thank you for your replies.

submitted by /u/neoncolour
[link] [comments]

[R] FAIR: Advancing self-supervision, CV, NLP

> Our recent advances in natural language processing (NLP) and computer vision (CV) show how work in content understanding is producing benefits. In NLP, we’ve developed a shared multilingual embedding space that can act as a sort of lingua franca to help take action on harmful content even in low-resource languages. In CV, we’ve built on our industry-leading research to identify content in more parts of an image and achieve record-setting accuracy using hashtags for video understanding.

Post: https://ai.facebook.com/blog/advances-in-content-understanding-self-supervision-to-protect-people

submitted by /u/downtownslim
[link] [comments]