This offbeat essay argues that the roots of NLP go way back. In the 17th century, Gottfried Leibniz imagined a “great instrument of reason” that wouldn’t just generate text, it would generate entire ideas.

https://spectrum.ieee.org/tech-talk/robotics/artificial-intelligence/in-the-17th-century-leibniz-dreamed-of-a-machine-that-could-calculate-ideas

“Leibniz … embarked on a project to create his own method of idea generation through symbolic combination. He wanted to use his machine not for theological debate, but for philosophical reasoning. He proposed that such a system would require three things: an ‘alphabet of human thoughts’; a list of logical rules for their valid combination and re-combination; and a mechanism that could carry out the logical operations on the symbols quickly and accurately…”

I’m working on a project where I’m required to train T2T on a custom dataset.

Now, I’ve looked at tensor2tensor’s own vaguely brief documentation (https://tensorflow.github.io/tensor2tensor/new_problem.html) on how to do this. I have also browsed the very short amount of implementations available online which are either unclear or incomplete.

And as you can infer by now, I’m frustrated. Why is there a general lack of discussion about this? Have any of you trained t2t on your own dataset?

Good time to discuss scores and plans for the rebuttal/revision process.

Hi, I was wondering if anyone knows of a GPU Cloud Compute platform that accepts cryptocurrency?

I know Vultr does, but unfortunately they only offer VPS/storage services.

I’m not sure the recent trend of larger and larger models is going to help make deep learning more useful or applicable. Mulit-billion parameter models might add a few percentage points of accuracy, but they don’t make it easier to build DL-powered applications or help other people start using the technology.

At the same time, there are some incredible results out there applying techniques like distillation, pruning, and quantization. I’d love for it to be standard practice to apply these techniques to more projects to see just how small and efficient we can make models.

For anyone interested in the topic, I wrote up a brief primer on the problem and some research into solutions. I’d love to hear of any success or failures people here have had with these techniques in production settings.

https://heartbeat.fritz.ai/deep-learning-has-a-size-problem-ea601304cd8

Text Modulation Audio book(TMA)

​

Many people don’t enjoy the traditional reading style to know the story of a book; and have replaced this monotonous style with audio books. Actors and Authors have to take out a lot of time to make an audio book. This consumes a lot of time which could be used for other fruitful activities. Some people have tried to use the text to speech tool, but that bland, robotic sound is equivalent to the reading method. TMA will not only convert text into speech, but with the help of AI it will detect the genre and accordingly add appropriate voice modulation and also some background music. It will detect the genre through some key words, for e.g if the program finds ‘crime’, ‘murder’, ‘evidence’ etc. in the text, it will take it as a thriller genre and if it finds the author as ‘Agatha Christie’ then it will know the mystery genre through the data fed into the program. This will not only save time, but will also allow readers to read books consisting of 1000+ pages(e.g Mein Kampf, IT, etc.). This will help in the field of education, by attracting more and more students towards books and help them by building their vocabulary, IQ and EQ. This will change the current scenario and make reading books a lot more interesting.

Short summary: We extend the state of the art of using neural networks to learn similarity measures (learning distance between pairs of data points in a dataset) with a new method called Extended Siamese Neural Networks (eSNN). This new method is put into the context of a framework that is used to describe different types of similarity measures. eSNN outperforms all current methods, while at the same time requiring the least training of all methods (down to half of the training of some comparable methods).
Paper link (Open Access): https://link.springer.com/article/10.1007/s13748-019-00201-2
PDF (Open Access): https://link.springer.com/content/pdf/10.1007%2Fs13748-019-00201-2.pdf
Code: https://github.com/ntnu-ai-lab/eSNN/

Authors: Bjørn Magnus Mathisen, Agnar Aamodt, Kerstin Bach and Helge Langseth

Abstract:

Defining similarity measures is a requirement for some machine learning methods. One such method is case-based reasoning (CBR) where the similarity measure is used to retrieve the stored case or a set of cases most similar to the query case. Describing a similarity measure analytically is challenging, even for domain experts working with CBR experts. However, datasets are typically gathered as part of constructing a CBR or machine learning system. These datasets are assumed to contain the features that correctly identify the solution from the problem features; thus, they may also contain the knowledge to construct or learn such a similarity measure. The main motivation for this work is to automate the construction of similarity measures using machine learning. Additionally, we would like to do this while keeping training time as low as possible. Working toward this, our objective is to investigate how to apply machine learning to effectively learn a similarity measure. Such a learned similarity measure could be used for CBR systems, but also for clustering data in semi-supervised learning, or one-shot learning tasks. Recent work has advanced toward this goal which relies on either very long training times or manually modeling parts of the similarity measure. We created a framework to help us analyze the current methods for learning similarity measures. This analysis resulted in two novel similarity measure designs: The first design uses a pre-trained classifier as basis for a similarity measure, and the second design uses as little modeling as possible while learning the similarity measure from data and keeping training time low. Both similarity measures were evaluated on 14 different datasets. The evaluation shows that using a classifier as basis for a similarity measure gives state-of-the-art performance. Finally, the evaluation shows that our fully data-driven similarity measure design outperforms state-of-the-art methods while keeping training time low.

Hope you like my work. I will try to answer questions if you have any.

• Facebook AI Residency Program [Link]. Application Deadline: January 31, 2020, 05:00pm PST.
• Google AI Residency [Link]. Application Deadline: December 19th, 2019.
• Google X AI Residency [Link]
• Google AI Resident (Health), 2020 Start – London, UK [Application Closed]
• Google AI Resident (Health), 2020 – Start Palo Alto, CA, USA [Application Closed]
• OpenAI 2020 Winter Scholars [Link]. Application Deadline: Nov 15, 2019.

Thought it would be helpful to have a discussion thread for 2020 Residencies applicants to share the updates, info, resources to prepare etc.

Below are some useful discussion threads :

https://www.reddit.com/r/MachineLearning/comments/9uyzc1/d_google_ai_residency_2019_applicants_discussion/

https://www.reddit.com/r/MachineLearning/comments/7rajic/d_anyone_heard_back_from_google_ai_residency/

https://www.reddit.com/r/MachineLearning/comments/7wst07/d_study_guides_for_interview_at_ai_research/

https://www.reddit.com/r/MachineLearning/comments/690ixs/d_google_brain_residency_requirements_and/

ICLR 2020 reviews are supposed to be released on 4th November 2019. Creating a discussion thread for this year’s reviews.