[D] Average Perplexity Values
What are average Perplexity values for sentences?
I am using Flair to calculate perplexity for sentences I generate (on the One Billion Words language model) and get very low values like 4 or 5.
I thought typical perplexity values are 20+ all the way to 100+ for less fluent sentences. Is there something I am overlooking? I assume the Flair code is correct lol