This is a place to share machine learning research papers, journals, and articles that you’re reading this week. If it relates to what you’re researching, by all means elaborate and give us your insight, otherwise it could just be an interesting paper you’ve read.

Please try to provide some insight from your understanding and please don’t post things which are present in wiki.

Preferably you should link the arxiv page (not the PDF, you can easily access the PDF from the summary page but not the other way around) or any other pertinent links.

Previous weeks :

Most upvoted papers two weeks ago:

/u/PaintedOnCanvas: Realistic Evaluation of Deep Semi-Supervised Learning Algorithms

/u/mesayousa: These guys

/u/_swish_: Neural-symbolic learning

Besides that, there are no rules, have fun.

Hi, guys!

I’m going to introduce deep metric learning techniques used in various places such as image retrieval application recently.

You can easily find many researches of deep metric learning from past to present in this repository.

​

Thanks!

​

Refer below link:

https://github.com/kdhht2334/Survey_of_Deep_Metric_Learning

I am currently attempting to learn ML specifically object detection.

Usually I am a JS developer, but wanted to extend my knowledge and incorporate the two at some point.

I hope there are not any stupid questions here.

​

I am looking to create a project that recognises different elements drawn on a piece of paper, for example rectangles and squares of different dimensions.

I am currently having to create the images manually by drawing them myself, I am worried this data will become too bias as I know what I want the end result to be.

I then feed that data into CV2 to add a randomly generate background to the image as the first few runs I processed with TensorFlow seemed to pick up mostly the white areas and gave me false readings.

Once I have the background image I use labelImg.py to draw borders and label the areas which contain the elements.

My data it then ready to be trained below are the results I received:-

(I allowed the training to run through 20,000 steps before stopping and testing)

​

​

As you can see from the table above when testing against the same testing images, the more datasets I introduce the lower the success rate of recognising the boxes becomes.

​

Question Time:-

​

Is there a relation between the amount of steps the trainer should run multiplied by the amount of data sets provided?

eg.

​

Is it possible to over train making the machine only recognise the data sets you have provided?

​

Is there a place I can request data sets, I do not mind doing this manually, but for bias reasons, I am wondering if it is possible?

​

Is there an optimised configuration file for this particular task I am trying to achieve?

I am currently using the ssd_mobilenet_v2_coco config file with some edits to image and batch size.

​

Is there any advice you would offer a ML noob to help progress their knowledge?

I have been watching quite a few videos on YouTube (most notably Gilbert Tanner and Sentdex) and they are getting amazing results with only a small dataset and 20,000 steps.

Also just to note, the loss rate sits under 0.8 whilst training around the 20,000 step mark.

​

I am hoping the question above are not too targeted and could also help other people who are starting out with ML.

I see this YouTube channel is posting videos with some sort of avatar doing motion capture not just of face but the whole body in real time which seems impressive, is this based on some research like paGAN http://www.hao-li.com/publications/papers/siggraphAsia2018PAGAN.pdf and how is it being done? https://youtu.be/dfJZe5i4Ah0

Really inspiring article about why neuromorphic hardware will be the future for intelligence research.

But the fact that computers “think” very differently than our brains do actually gives them an advantage when it comes to tasks like number crunching, while making them decidedly primitive in other areas, such as understanding human speech or learning from experience. If scientists want to simulate a brain that can match human intelligence, let alone eclipse it, they may have to start with better building blocks—computer chips inspired by our brains.

Current artificial intelligence systems still trail the brain when it comes to flexibility and learning ability. “Google’s networks became very good at recognizing images of cats once they were shown 10 million images of cats, but if you show my two-year-old grandson one cat he will recognize cats for the rest of his life,” says Furber.

https://www.the-scientist.com/features/building-a-silicon-brain-65738

I’m working on a img2img.

my model works when i use batch size 1. However it doesn’t work with batch size > 1.

there’s no normalization layer.

​

Do you have any idea about is phenomenon??

It seems to me that Visdom can be considered dead for most of the users. Anyone has any idea?

It seems to me that RNNs are becoming less popular architectures. In the most recent papers on sequential data I’ve been reading (video, TTS, language modeling) I’ve more frequently seen convolutions over the time axis or transformers being implemented. Are RNNs still achieving high performance in any particular area?

I was reading this article by Jay Alammar The Illustrated Transformer which explains the transformer model in simple english. There is a section where he is talking about self attention and how its calculated. Here he introduced the concepts of query, key, value and the self-attention equation. I understood how query, key, value and the self-attention equation are calculated but how the researches came up with the idea of these vectors and equation.

I know that we are using hidden states in rnn because it has all the previous informations inside it and in a rnn model we pass this hidden state and new input to get new hidden state. In the article he says that

query, kery and value are the abstractions that are useful for calculating and thinking about attention.

But why? where is the concept of query, key and value come by? why can’t it be just one vector say only k or two vector(q,k)? What is the significance of these three vectors? Why this vector and the equation improve attention?

The next question; is the embedding vector necessary for a self-attention model. If the input dimension is small can’t we use the input vector to calcualte query,key and value and remove the embedding layer completely?