Hi, there are 10 news/reflexions which showed up on September I summed up them, as a recap. Did I forgot an important one? What are the most important for you in October?

There is the list :

• Documentary aired on how AI was seen in 1960
• Tensorflow 2.0 released, patch note
• Reflexion on how to make technology work WITH society?
• MIT is helping doctors detecting patient pains
• Launch of the Facebook & Microsoft Deepfake challenge
• How US government overcomes European GDPR law
• Google finned for targeting children
• Report published on the world wide expansion of AI
• Google published a new Multilingual Speech Recognition model
• Report on the pollution associated to artificial intelligence

https://www.sicara.ai/blog/2019-10-21-best-of-ai-september-2019

I’m doing a study on inference latency. What are different ways of optimizing your model for this? Let’s say the goal is to get your inference latency as low as possible. I’ve heard of ONNX runtime (apparently used by Microsoft in production), compilers such as Intel nGraph, TVM, Intel OpenVINO and so on. Are these kind of tools used in production, or do most companies just use PyTorch and TF inference mode? If anyone here has experience from unique deployments I’d love to hear about it!

Transformers are currently beating the state of the art on different NLP tasks.

Some examples are:

• Machine translation: Transformer Big + BT
• Named entity recognition: BERT large
• Natural language inference: RoBERTa

Something I noticed is that in all of the papers, the models are massive with maybe 20 layers and 100s of millions of parameters.

Of course, using larger models is a general trend in NLP but it begs the question if small transformers are any good. I recently had to train a sequence to sequence model from scratch and I was unable to get better results with a transformer than with LSTMs.

I am wondering if someone here has had similar experiences or knows of any papers on this topic.

I am looking for biomedical databases similar to the Wisconsin breast cancer database (available at https://archive.ics.uci.edu/ml/datasets/breast+cancer+wisconsin+(original)) ). This database has 9 features (each feature values being integers ranging from 1 to 10) and two classes – benign and malignant. Defining characteristic of this dataset is that the higher feature values generally indicate higher chance of abnormality (malignancy). I am looking for other biomedical datasets having features with this property (not necessarily integer valued, can also be real valued; preferably with low number of features also – less than 30 or so)

Inspired by this post. I am curious, what tool would you choose and why? 😀

Hello everybody,

Are there any good unsupervised methods of retrieving top-k documents from corpus based on a rather short query?

I did a bit of googling but couldn’t find anything that isn’t tf-idf based.

Maybe it would be possible to somehow retrieve similarities between docs and query by utilising contextual embeddings (such as from BERT) and use some sort of scoring function to evaluate it.

​

Anyway, thank you in advance for your answers.

I have noticed a small discrepancy between theory and the implementation of AdamW and in general Adam. The epsilon in the denominator of the following Adam update should not be scaled by the bias correction (Algorithm 2, L9-12). Only the running average of the gradient (m) and squared gradients (v) should be scaled by their corresponding bias corrections.

In the current implementation, the epsilon is scaled by the square root of bias_correction2
. I have plotted this ratio as a function of step given beta2 = 0.999
and eps = 1e-8
. In the early steps of optimization, this ratio slightly deviates from theory (denoted by the horizontal red line)

See more here: https://github.com/pytorch/pytorch/pull/22628