[R] Low accuracy on small English dataset using XLNet
I compared accuracies on small dataset, got 2 classes, which are
positive, each class has 5k of sentences. You can check it here english-polarity. All notebooks finetune on single Tesla V100, 100 max length of sentences. 80% to train, 20% to test. All notebooks do early stopping with 3 patients, and I will pick highest test accuracy as the benchmark.
I am not sure what is wrong here maybe not follow an exact batch size and max length?, I doubled check all the code to transform into notebooks based, everything looks same, just a bit tweak to make it notebook-able. Great model and repository, https://github.com/zihangdai/xlnet!