[P] ELMo from scratch?
In one of my projects I need to train ELMo embeddings. AllenNLP has an implementation of this but I thought I’ll take this opportunity to implement it from scratch.
I always wanted to develop the skill to replicate the result of research papers and experiment with them. So I think implementing this from scratch will give me a kick start. Also, I’ll be able to learn a lot about PyTorch.
I already read the paper of ELMo, along with Character-Aware Neural Language Models, Highway Networks, really cool papers!
I’m pretty sure you passed the stage where I am at right now. So it would be tremendously helpful if you could share your opinion, experience and suggestions.