[P] Pytorch Implementation of Autoregressive Language Model
A step-by-step tutorial on how to implement and adapt Autoregressive language model to Wikipedia text.
A pre-trained BERT, XLNET is publicly available ! But, for NLP beginners, It could be hard to use/adapt after full understanding. For them, I covered whole, end-to-end implementation process for language modeling, using unidirectional/bidirectional LSTM network, we already know.
- – do not use torchtext library !
- + include trained model file, training logs
I hope that this repo can be a good solution for people who want to have their own language model 🙂
https://github.com/lyeoni/pretraining-for-language-understanding
submitted by /u/lyeoni
[link] [comments]