Abstract:

Machine translation of ancient languages faces a low-resource problem, caused by the limited amount of available textual source data and their translations. We present a multi-task modeling approach to translating Middle Egyptian that is inspired by recent successful approaches to multi-task learning in end-to-end speech translation. We leverage the phonographic aspect of the hieroglyphic writing system, and show that similar to multi-task learning of speech recognition and translation, joint learning and sharing of structural information between hieroglyph transcriptions, translations, and POS tagging can improve direct translation of hieroglyphs by several BLEU points, using a minimal amount of manual transcriptions.

paper blog post

We’re presenting this at IWSLT 2019 in Hong Kong in a poster session, if you’re there please stop by and ask questions!

Can skip links still be used by the model in the absence of regularization?

Empirical or theoretical responses are invited!

Hey, just wanted to post some of our recent work on Bayesian optimization for neural architecture search. We show how to use a neural network to predict the accuracy of cell-based neural networks, and how this can be used for neural architecture search.

BANANAS: Bayesian optimization with neural architectures for neural architecture search

Arxiv: https://arxiv.org/abs/1910.11858

code: https://github.com/naszilla/bananas

blog: https://medium.com/reality-engines/bananas-a-new-method-for-neural-architecture-search-192d21959c0c

There has been a lot of discussion with regards to choosing an appropriate performance metric to use for model training and evaluation for classification problems with a moderate to large amount of class imbalance present (e.g. 1, 2, 3, 4, 5).

In these cases, it is generally suggest that one uses a metric like cohen’s kappa or PR AUC in place of accuracy or AUROC, perhaps along with up-sampling or SMOTE.

What are people’s thoughts on cases where there is only a minor class imbalance present in the data? For example, something like a 3:1, 2:1, or even 1.5:1 ratio of major to minor class members? Is it still beneficial (and what would be the cost?) or using a metric geared at addressing larger imbalances in these cases?

Also, somewhat tangential, but are there ever any scenarios where you might want to use one metric for an outer CV loop / model performance evaluation, but a different metric for inner CV (e.g. feature selection / hyperparameter optimization)?

Hi r /ML,

Four months ago we open-sourced our experiment management platform. My relevant post on r /ML eventually gave us valuable feedback from actual researchers. One of the recurring themes was that we were lacking a definitive differentiation from other solutions.Well, I am happy to announce that we took that to heart and are now adding… <drumroll.wav>

All the devops your research is going to need^{and then some}

Now live at (star/forks are appreciated): https://github.com/allegroai/trains (now with trains-agent!)

​We are eager to know if the readme conveys just how cool this thing is!

tl;dr:

• An order of magnitude easier maintenance than K8S for research
• Schedule GPU resources across any cloud and/or on-prem (and only use containers if you want to)
• Straightforward, automagical configuration
• User friendly UI
• Roll your own autoML

The fine print: your agents work in conjunction with your trains platform.

FAQ:

1. Why are we releasing this as FOSS – We feel that automagical devops are necessary for research 😉 Our formal answer is still valid: “We acknowledge that deep learning R&D and operations are not well-established yet, and we want TRAINS to remain relevant as paradigms shift in the field”
2. … counting on you guys here. AMA?

PS: If anyone is at ODSC West on 10/31, Come meet Gregory from our team, he will be talking about Integrating data as part of autoML

We have analyzed abstracts and keywords of ICLR submissions over the last three years and observed interesting patterns, such as decreasing fraction of papers about GANs and increasing interest in attention/transformer and neural architecture search.

For more details see: https://deepsense.ai/key-findings-from-the-international-conference-on-learning-representations-iclr/

Hello,

What services are you using to deploy your deep learning model to production? GCP AI Platform, Amazon Sagemaker, etc.?