[D] History of NLP: How Andrey Markov and Claude Shannon Made Language Models for Text Generation
These were painstaking procedures. Markov counted vowels and consonants in a Russian novel to make probability rules for which letter would appear next. 30 years later, Shannon took the idea a step further by creating statistical models of language and generating text according to the rules he devised. Shannon’s first output was “OCRO HLI RGWR,” but his model did get a bit better.