Does anyone have experience with finetuning a resnet / VGG model pretrained on imagenet but using a higher resolution for the inputs?

Any advice, papers or slides are most welcome!

Background:

I am currently working on binary classification of images and have had pretty good success by finetuning a resnet50 pretrained on imagenet (this is a pretty good guide https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html ). I have only a few hundred images of each class, so overfitting is a massive concern and thus unfreezing the lower parts of the resnet would not work.

However, my images are much higher resolution than imagenet – most tutorials I have found seem to rescale images to 244×244 which is what I am doing right now, but I think that downscaling the images a bit less might make the task easier to solve.

Keras resnet50 implementation accepts higher resolution images, but the model doesn’t improve during training at all.