We apply simple corruptions to MNIST to create a new dataset for the purpose of measuring non-adversarial robustness in computer vision models. We then evaluate various models on MNIST-C and find that a simple CNN outperforms various adversarial defenses and alternative architectures by a wide margin.

​

Paper: https://arxiv.org/abs/1906.02337

Code: https://github.com/google-research/mnist-c