I’m relatively new to ML and have been running my own finetuning of GPT2 for a while. I’m not sure if this is the best or most appropriate place to ask this, so please forgive me if not (and feel free to redirect me elsewhere). With many prompts, GPT2 seems to get stuck in an infinite loop repeating the input phrase or part of the input phrase, particularly when I pass it titles. It will also randomly spit out the endtext token, or say things like Join my mailing list! Read the comments blow at the start of an “article”.

​

Am I doing something wrong with finetuning, or is this just a thing that happens and that people discard?