I’m curious about what the current state of conversational AI. To be more specific, by “conversation” I’m not talking about something that can take orders to schedule appointments or buy tickets or something like that. I mean something like discussing movies or TV shows or current events. I think remember Amazon holding a contest for this kind of thing but I haven’t really seen something like this implemented anywhere I can access it. Does anyone have any examples of what this kind of technology can do? Or better yet, anything I can play and experiment with on my own?

Is there any principled reason for doing softmax and cross entropy for the loss in for example transformers, rather than doing L2 over the target embeddings and the output from the model?

When the output from your model by necessity is a dot product such as in shallow models I understand why you need to do cross entropy loss. But for models such as rnns and some variants of transformers wouldn’t L2 loss directly on the desired embedding and output work as well or better?

Hi, during summer I worked on a project with the objective of removing people or objects from photos. Person-remover uses a pretrained YOLO to detect them and then feeds the resulting bounding boxes to the generator of a pix2pix which I trained from zero on Paris dataset. Even though the generator wasn’t trained with the purpose of filling person-shaped objects, the results are pretty great and seems to generalize well to unseen photos or video.

Any ideas on how to improve the results even further?

Repo: https://github.com/javirk/Person_remover

Is it domain specific? Does Ruders’ blog/paper still hold up: https://ruder.io/multi-task/?

Hi all,

I did a toy project for Korean NER tagger(in progress). If you are interested in Korean Named Entity Recognition, try it. (This NER tagger is implemented in PyTorch)

If you want to apply it to other languages, you don’t have to change the model architecture, you just change vocab, pretrained BERT(from huggingface), and training dataset.

https://github.com/eagle705/pytorch-bert-crf-ner

I often times work with people (medical studies) with a huge “knowledge” on statistical methods but none of the required basics or understanding what goes on inside some algorithms. That’s perfectly fine because after all that’s not their job but mine.

​

But over time, I’ve come across a few problems where (due to not finding the “needed significance”) some really basic over-sampling was applied. I’ve thrown together a really simple example, that anyone should be able to follow (without any deep statistical knowledge) to showcase what could happen – maybe it helps you or you can use it to your help:

​

https://stroemer.cc/resample-imbalanced-data/

Something that seems to be under-discussed in machine learning is why we bother with momentum method in the first place.

Suppose we are training a classifier and the loss function has two local minima, one of which is global. Suppose by sheer unluck, the gradient descent gets stuck in the worse local minima. If you ask around as to what can be done, you will hear answers like “oh just use the momentum method, it gets you out of the local minima”.

First, there is no guarantee you will be out of the local minima (only if the difference between the current and previous iterate is large enough do you have a chance), and more importantly,

Second, great, you have found the global mimina and….you have just potentially overfitted your classifier.

In other words, we are looking for local minima (or even just some point associated with the loss function) with good generalization properties, and I don’t think momentum methods guarantees that.

Has there been any research on the generalization properties of the minima that you find and what algorithm get you the best minima, not in terms of how small the loss is, but how well it achieves generalization?