In one of my projects I need to train ELMo embeddings. AllenNLP has an implementation of this but I thought I’ll take this opportunity to implement it from scratch.

I always wanted to develop the skill to replicate the result of research papers and experiment with them. So I think implementing this from scratch will give me a kick start. Also, I’ll be able to learn a lot about PyTorch.

I already read the paper of ELMo, along with Character-Aware Neural Language Models, Highway Networks, really cool papers!

I’m pretty sure you passed the stage where I am at right now. So it would be tremendously helpful if you could share your opinion, experience and suggestions.

TIA