Let’s say I have a chatbot that can generate random sentences (and hence we have no ground truth to compare to) like:

1. “I like the weather today because it is very sunny and bright”.

2. “I like the cat today because it is rainy”.

3. “I like cat yesterday it is because”.

What are the best methods to evaluate if each sentence makes sense based on both its grammar and/or content? Example 1. above would make the most sense in terms of content and grammar, 2. makes grammatical sense but not in terms of content (makes no sense to like a cat because it is rainy), and 3. above has horrible grammar.

The issue here is these are randomly generated sentences with no ground truth to compare to (unlike, for example, machine translation where you have an expected translation and hence can use BLEU or perplexity).