In the original Bert paper, it is stated on page 4 (bottom, first column) that:

Unfortunately, standard conditional language models can only be trained left-to-right or right-to-left, since bidirectional conditioning would allow each word to indirectly “see itself”, and the model could trivially predict the target word in a multi-layered context.

It’s not at all obvious to me why, if you have the sentence “I like funny cats”, predicting the word “funny” while conditioning on the fact that it’s preceded “I”, “like” and succeeded by “cats” would be trivial and how the model could “indirectly see the target word”.

I saw this question asked on a number of online platforms but it never got a response. It would be great if someone with a good understanding of this could give an explanation

https://blog.janestreet.com/deep-learning-the-hardest-go-problem-in-the-world/

Here’s a recent post about one of our experiments using AlphaZero-like selfplay learning to explore a fun little microcosm within Go that prior Go bots, including superhumanly strong AlphaZero-based bots, completely fail at.

Obviously, this is not an attempt at tackling any grand open challenge, but hopefully is still a useful case study and that touches on what seem to be some of the remaining weaknesses of modern deep RL in games.

Hope you find it interesting!

If given a dataset (e.g. InsuranceQA) with a list of questions and a list of passages of candidate answers (each question can be matched with multiple candidate answers) and the task is to rank the top 10 (the ordering is not labeled) most relevant answers (passages) given a query.

I have been reading about possible ways to solve this problem and came across both Answer Selection and Answer Re-ranking techniques.

I would like to know in terms of a NLP tasks, what is the difference between Answer Selection and Answer Re-Ranking?

I am thinking that Answer Re-ranking is where you first use something like BM25 to retrieve a list of say 1000 candidate answers then ranking the relevant answers whereas Answer Selection can use re-ranking to achieve this or not.

Hi everyone,

Would like to share the latest episode of our Podcast “the Humans of Ai”. The latest episode features Sunil Mallya, who is the Principal Deep Learning Scientist at Amazon Web Services Machine Learning Lab & is one of the key people behind AWS Deepracer & Sagemaker RL.

In this episode we explore the rich interaction that Sunil has had with machine learning over the course of his career including talking about what drew him to the field, his experience as an early phase machine learning entrepreneur, his knowledge in building distributed machine learning systems and finally the two biggest projects Sunil has taken on recently – Sagemaker RL & AWS Deepracer!

We also discuss many other topics including what the Machine Learning scene is like in San Francisco, some real world applications of Reinforcement Learning & what it takes to be a part of the Amazon’s ML Lab.

You can find The Humans of Ai on:

iTunes here:

https://podcasts.apple.com/au/podcast/the-humans-of-ai/id1464995550

Spotify here:

https://open.spotify.com/show/2RY5mcNl0iAs8HUTNbwT0J

Sticher here:

https://www.stitcher.com/s?fid=414486&refid=stpr

Moderators, if you feel that this is not the right place to be posting, feel free to remove & completely understand, but we do feel that this episode in particular has a lot of information that could benefit the Machine Learning community. Cheers & best