I compared accuracies on small dataset, got 2 classes, which are negative and positive, each class has 5k of sentences. You can check it here english-polarity. All notebooks finetune on single Tesla V100, 100 max length of sentences. 80% to train, 20% to test. All notebooks do early stopping with 3 patients, and I will pick highest test accuracy as the benchmark.

1. BERT English Base, test accuracy 81%.
2. GPT2 345M, test accuracy 79%.
3. XLnet Large, test accuracy 51%.

I am not sure what is wrong here maybe not follow an exact batch size and max length?, I doubled check all the code to transform into notebooks based, everything looks same, just a bit tweak to make it notebook-able. Great model and repository, https://github.com/zihangdai/xlnet!