CIFAR-10H is a new dataset of soft labels reflecting human perceptual uncertainty for the 10,000-image CIFAR-10 test set, which we are releasing today. It first appears in the paper:

Joshua C. Peterson, Ruairidh M. Battleday, Thomas L. Griffiths, & Olga Russakovsky (2019). Human uncertainty makes classification more robust. In Proceedings of the IEEE International Conference on Computer Vision. https://arxiv.org/abs/1908.07086

Dataset Link: https://github.com/jcpeterson/cifar-10h/